2024-03-16T10:23:47.283 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T10:23:47.283 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376 -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376/orig.config.yaml 2024-03-16T10:23:47.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T10:23:51.571 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376 branch: wip-batrick-testing-20240314.183243 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7602376' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTP5ffQoRaQWg2M7qz4kbZA1CctnYZjlxe2eMtw0CIrI4WN9rM6sBriyjphVKwGLr5jWsjUJD9OUfFFia5X5qM= smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBi1Dd9A+cCFtgVisZOc7XJ8tr0N0mScbsEA4uyKj9Wxzsy6nLgHer/SejJjek/uTw+c9/ZL0+vXMj+6gVUJc/Q= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/pdonnell/src/git.ceph.com_ceph-c_wip-batrick-testing-20240314.183243/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_13:00:41 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-16T10:23:51.571 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa; will attempt to use it 2024-03-16T10:23:51.572 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks 2024-03-16T10:23:51.572 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T10:23:51.574 INFO:teuthology.task.internal:Checking packages... 2024-03-16T10:23:51.596 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-16T10:23:51.597 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T10:23:51.597 INFO:teuthology.packaging:ref: None 2024-03-16T10:23:51.597 INFO:teuthology.packaging:tag: None 2024-03-16T10:23:51.597 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-16T10:23:51.597 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:23:51.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-16T10:23:51.811 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162.g3d78f5ba 2024-03-16T10:23:51.812 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T10:23:51.819 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T10:23:51.819 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T10:23:51.852 INFO:teuthology.task.internal:Saving configuration 2024-03-16T10:23:51.871 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T10:23:51.877 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T10:23:51.900 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 10:19:04.656286', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTP5ffQoRaQWg2M7qz4kbZA1CctnYZjlxe2eMtw0CIrI4WN9rM6sBriyjphVKwGLr5jWsjUJD9OUfFFia5X5qM='} 2024-03-16T10:23:51.923 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 10:19:04.658306', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBi1Dd9A+cCFtgVisZOc7XJ8tr0N0mScbsEA4uyKj9Wxzsy6nLgHer/SejJjek/uTw+c9/ZL0+vXMj+6gVUJc/Q='} 2024-03-16T10:23:51.923 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T10:23:51.929 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-16T10:23:51.930 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-16T10:23:51.930 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T10:23:52.006 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1729116160>, signals=[15]) 2024-03-16T10:23:52.007 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T10:23:52.013 INFO:teuthology.task.internal:Opening connections... 2024-03-16T10:23:52.013 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:23:52.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:23:52.120 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-03-16T10:23:52.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:23:52.222 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T10:23:52.228 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-03-16T10:23:52.245 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-03-16T10:23:52.245 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-03-16T10:23:52.301 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-03-16T10:23:52.302 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T10:23:52.302 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-03-16T10:23:52.302 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T10:23:52.302 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T10:23:52.302 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T10:23:52.302 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-03-16T10:23:52.325 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-03-16T10:23:52.342 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-03-16T10:23:52.342 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-03-16T10:23:52.399 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-03-16T10:23:52.399 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-03-16T10:23:52.399 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-03-16T10:23:52.399 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-03-16T10:23:52.399 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T10:23:52.400 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T10:23:52.400 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-03-16T10:23:52.416 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T10:23:52.426 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T10:23:52.434 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T10:23:52.434 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-03-16T10:23:52.437 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-03-16T10:23:52.455 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T10:23:52.460 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T10:23:52.460 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-03-16T10:23:52.493 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-03-16T10:23:52.523 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T10:23:52.713 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T10:23:52.729 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-16T10:23:52.729 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-16T10:23:52.729 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T10:23:52.730 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T10:23:52.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:23:52.747 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-03-16T10:23:52.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:23:52.748 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-03-16T10:23:52.804 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-361.el9.x86_64 2024-03-16T10:23:52.804 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.14.0-361.el9.x86_64 2024-03-16T10:23:52.805 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-03-16T10:23:52.806 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-361.el9.x86_64 2024-03-16T10:23:52.806 INFO:teuthology.task.kernel:Running kernel on smithi002: 5.14.0-361.el9.x86_64 2024-03-16T10:23:52.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2024-03-16T10:23:54.888 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-16T10:23:54.973 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-16T10:23:56.380 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-16T10:23:56.499 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-16T10:23:57.414 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-16T10:23:57.923 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.0 MB 00:01 2024-03-16T10:24:01.607 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-03-16T10:24:05.957 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.1 MB 00:00 2024-03-16T10:24:07.411 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:08 2024-03-16T10:24:07.551 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-03-16T10:24:10.418 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 21 MB 00:02 2024-03-16T10:24:12.479 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.1 MB 00:01 2024-03-16T10:24:13.930 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 15 kB 00:00 2024-03-16T10:24:15.551 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-03-16T10:24:15.692 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-16T10:24:17.816 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-16T10:24:17.816 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-16T10:24:17.917 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout:Install 4 Packages 2024-03-16T10:24:17.927 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:24:17.928 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 100 M 2024-03-16T10:24:17.928 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 123 M 2024-03-16T10:24:17.928 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:24:18.277 INFO:teuthology.orchestra.run.smithi026.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 21 MB/s | 6.0 MB 00:00 2024-03-16T10:24:19.027 INFO:teuthology.orchestra.run.smithi026.stdout:(2/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 38 MB/s | 39 MB 00:01 2024-03-16T10:24:19.236 INFO:teuthology.orchestra.run.smithi026.stdout:(3/4): kernel-core-5.14.0-427.el9.x86_64.rpm 17 MB/s | 21 MB 00:01 2024-03-16T10:24:19.803 INFO:teuthology.orchestra.run.smithi026.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 22 MB/s | 33 MB 00:01 2024-03-16T10:24:19.803 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:24:19.803 INFO:teuthology.orchestra.run.smithi026.stdout:Total 53 MB/s | 100 MB 00:01 2024-03-16T10:24:20.725 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:24:20.856 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-16T10:24:21.162 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:24:21.162 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:24:22.538 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:24:22.539 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:24:23.146 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-16T10:24:23.146 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-16T10:24:23.238 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-16T10:24:23.245 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-16T10:24:23.246 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-16T10:24:23.246 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:24:23.246 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:24:23.246 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:24:23.246 INFO:teuthology.orchestra.run.smithi002.stdout:Install 4 Packages 2024-03-16T10:24:23.246 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:24:23.246 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 100 M 2024-03-16T10:24:23.247 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 123 M 2024-03-16T10:24:23.247 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:24:24.074 INFO:teuthology.orchestra.run.smithi002.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 10 MB/s | 6.0 MB 00:00 2024-03-16T10:24:24.782 INFO:teuthology.orchestra.run.smithi002.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 16 MB/s | 21 MB 00:01 2024-03-16T10:24:25.421 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:24:26.102 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-16T10:24:26.199 INFO:teuthology.orchestra.run.smithi002.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 14 MB/s | 39 MB 00:02 2024-03-16T10:24:26.253 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T10:24:27.233 INFO:teuthology.orchestra.run.smithi002.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 11 MB/s | 33 MB 00:03 2024-03-16T10:24:27.233 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:24:27.233 INFO:teuthology.orchestra.run.smithi002.stdout:Total 25 MB/s | 100 MB 00:03 2024-03-16T10:24:28.153 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:24:28.589 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:24:28.589 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:24:29.192 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T10:24:29.379 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T10:24:30.035 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:24:30.035 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:24:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:24:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-16T10:24:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T10:24:34.481 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T10:24:34.608 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:24:36.625 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T10:24:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T10:24:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:24:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T10:24:42.031 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:24:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:10.609 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:10.609 INFO:teuthology.orchestra.run.smithi026.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T10:25:10.609 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:25:10.698 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:11.189 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:11.189 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-16T10:25:11.189 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T10:25:11.189 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T10:25:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:11.617 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:25:11.617 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:25:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-16T10:25:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-16T10:25:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-16T10:25:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-16T10:25:11.618 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:25:11.618 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:25:12.134 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T10:25:12.330 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T10:25:12.330 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T10:25:12.330 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-16T10:25:12.330 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T10:25:12.330 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T10:25:12.330 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-16T10:25:12.331 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T10:25:12.331 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T10:25:12.457 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T10:25:12.457 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T10:25:12.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-03-16T10:25:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-03-16T10:25:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-03-16T10:25:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-03-16T10:25:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 68.el9 2024-03-16T10:25:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-03-16T10:25:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8264254 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T10:25:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T10:25:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T10:25:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-03-16T10:25:12.501 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:25:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T10:25:12.502 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T10:25:12.502 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 5.14.0-427.el9.x86_64 2024-03-16T10:25:12.502 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T10:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-03-16T10:25:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T10:25:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-03-16T10:25:14.590 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-03-16T10:25:14.605 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.4v2hPv4HH4 2024-03-16T10:25:14.605 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4v2hPv4HH4 2024-03-16T10:25:14.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.4v2hPv4HH4 2024-03-16T10:25:14.801 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.4v2hPv4HH4 is 6KB 2024-03-16T10:25:14.849 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.4v2hPv4HH4 2024-03-16T10:25:14.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T10:25:14.932 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T10:25:14.932 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T10:25:14.932 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T10:25:14.932 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-16T10:25:14.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-16T10:25:15.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-03-16T10:25:17.887 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:17.887 INFO:teuthology.orchestra.run.smithi002.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T10:25:17.887 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:25:17.975 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:18.357 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:18.357 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-16T10:25:18.357 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T10:25:18.357 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:25:18.812 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:25:19.289 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T10:25:19.440 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T10:25:19.440 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T10:25:19.440 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-16T10:25:19.440 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T10:25:19.440 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T10:25:19.440 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-16T10:25:19.441 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T10:25:19.441 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T10:25:19.566 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T10:25:19.566 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T10:25:19.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.06 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 68.el9 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Group : Unspecified 2024-03-16T10:25:19.615 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 8264254 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : builder@centos.org 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : CentOS 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T10:25:19.616 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T10:25:19.617 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2024-03-16T10:25:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:25:19.617 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T10:25:19.617 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T10:25:19.618 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 5.14.0-427.el9.x86_64 2024-03-16T10:25:19.618 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T10:25:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-03-16T10:25:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T10:25:21.642 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-03-16T10:25:21.645 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-03-16T10:25:21.660 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.nuB4vVeJBX 2024-03-16T10:25:21.660 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nuB4vVeJBX 2024-03-16T10:25:21.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.nuB4vVeJBX 2024-03-16T10:25:21.870 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.nuB4vVeJBX is 6KB 2024-03-16T10:25:21.918 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.nuB4vVeJBX 2024-03-16T10:25:21.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T10:25:22.000 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T10:25:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T10:25:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T10:25:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-16T10:25:22.002 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-16T10:25:22.149 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-03-16T10:25:45.102 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-16T10:25:45.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-16T10:25:45.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:25:52.158 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-16T10:25:52.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-16T10:25:52.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:26:03.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-03-16T10:26:10.591 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-16T10:26:12.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-16T10:26:12.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:26:19.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-16T10:26:19.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:26:30.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-03-16T10:26:42.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-16T10:26:42.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:26:43.178 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-03-16T10:26:43.743 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-16T10:26:43.743 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-427.el9.x86_64"... 2024-03-16T10:26:43.743 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-03-16T10:26:43.757 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-427.el9.x86_64 2024-03-16T10:26:43.757 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-16T10:26:43.757 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T10:26:43.757 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T10:26:44.758 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-16T10:26:44.759 DEBUG:teuthology.orchestra.run.smithi026:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T10:26:44.843 INFO:teuthology.orchestra.run.smithi026.stdout:ttyS1 2024-03-16T10:26:44.892 DEBUG:teuthology.parallel:result is None 2024-03-16T10:27:19.602 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T10:27:31.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-16T10:27:31.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:27:31.938 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-03-16T10:27:32.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-16T10:27:32.350 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-427.el9.x86_64"... 2024-03-16T10:27:32.351 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-03-16T10:27:32.367 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-427.el9.x86_64 2024-03-16T10:27:32.367 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-16T10:27:32.367 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T10:27:32.367 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T10:27:33.368 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-16T10:27:33.368 DEBUG:teuthology.orchestra.run.smithi002:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T10:27:33.436 INFO:teuthology.orchestra.run.smithi002.stdout:ttyS1 2024-03-16T10:27:33.468 DEBUG:teuthology.parallel:result is None 2024-03-16T10:27:33.468 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T10:27:33.476 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T10:27:33.476 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T10:27:33.479 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T10:27:33.499 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T10:27:33.505 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T10:27:33.512 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T10:27:33.512 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T10:27:33.536 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T10:27:33.563 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T10:27:33.572 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T10:27:33.573 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T10:27:33.598 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T10:27:33.630 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:27:33.637 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:27:33.644 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:27:33.650 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:27:33.652 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T10:27:33.659 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T10:27:33.659 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T10:27:33.688 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T10:27:33.723 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T10:27:33.735 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T10:27:33.735 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T10:27:33.758 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T10:27:33.779 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T10:27:37.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T10:27:37.327 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T10:27:37.409 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T10:27:37.457 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:27:37.457 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T10:27:37.528 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T10:27:37.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T10:27:37.624 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T10:27:37.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T10:27:37.754 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:27:37.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T10:27:37.825 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-03-16T10:27:37.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-03-16T10:27:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T10:27:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T10:27:38.334 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T10:27:38.342 INFO:teuthology.task.internal:Starting timer... 2024-03-16T10:27:38.342 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T10:27:38.353 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T10:27:38.360 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-16T10:27:38.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2024-03-16T10:27:38.414 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2024-03-16T10:27:38.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-03-16T10:27:38.486 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-03-16T10:27:38.488 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T10:27:38.488 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2024-03-16T10:27:38.510 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2024-03-16T10:27:38.510 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-03-16T10:27:38.531 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-03-16T10:27:38.531 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi026.front.sepia.ceph.com': 'permissive'} 2024-03-16T10:27:38.532 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T10:27:38.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:27:38.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T10:27:38.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:27:38.608 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T10:27:38.608 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2024-03-16T10:27:38.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-03-16T10:27:38.683 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T10:27:38.692 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-16T10:27:38.698 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-16T10:27:38.829 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T10:27:38.846 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T10:27:38.847 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi026.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T10:31:53.753 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi026.front.sepia.ceph.com')] 2024-03-16T10:31:53.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-16T10:31:53.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:31:53.842 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-03-16T10:31:53.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-16T10:31:53.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-16T10:31:53.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:31:54.015 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-03-16T10:31:54.100 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-03-16T10:31:54.101 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T10:31:54.111 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T10:31:54.111 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T10:31:54.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:31:54.114 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T10:31:54.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:31:54.142 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T10:31:54.159 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T10:31:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T10:31:54.188 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2024-03-16T10:31:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T10:31:54.201 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2024-03-16T10:31:54.218 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T10:31:54.218 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-03-16T10:31:54.231 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-03-16T10:31:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T10:31:54.248 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T10:31:54.264 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T10:31:54.290 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-03-16T10:31:54.294 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T10:31:54.294 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-03-16T10:31:54.294 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.294 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.294 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.294 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-03-16T10:31:54.325 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T10:31:54.325 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-03-16T10:31:54.325 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.325 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.325 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.325 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T10:31:54.326 INFO:teuthology.run_tasks:Running task install... 2024-03-16T10:31:54.333 DEBUG:teuthology.task.install:project ceph 2024-03-16T10:31:54.333 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-16T10:31:54.333 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T10:31:54.333 INFO:teuthology.task.install:Using flavor: default 2024-03-16T10:31:54.338 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T10:31:54.339 INFO:teuthology.task.install:extra packages: [] 2024-03-16T10:31:54.339 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T10:31:54.339 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T10:31:54.339 INFO:teuthology.packaging:ref: None 2024-03-16T10:31:54.339 INFO:teuthology.packaging:tag: None 2024-03-16T10:31:54.339 INFO:teuthology.packaging:branch: reef 2024-03-16T10:31:54.339 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:31:54.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-16T10:31:54.341 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T10:31:54.342 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T10:31:54.342 INFO:teuthology.packaging:ref: None 2024-03-16T10:31:54.342 INFO:teuthology.packaging:tag: None 2024-03-16T10:31:54.342 INFO:teuthology.packaging:branch: reef 2024-03-16T10:31:54.342 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:31:54.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-16T10:31:54.571 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/ 2024-03-16T10:31:54.571 INFO:teuthology.task.install.rpm:Package version is 18.2.2-878.ga752c2f7 2024-03-16T10:31:54.672 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/ 2024-03-16T10:31:54.672 INFO:teuthology.task.install.rpm:Package version is 18.2.2-878.ga752c2f7 2024-03-16T10:31:54.745 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T10:31:54.745 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:31:54.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T10:31:54.774 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-16T10:31:54.774 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T10:31:54.774 INFO:teuthology.packaging:ref: None 2024-03-16T10:31:54.774 INFO:teuthology.packaging:tag: None 2024-03-16T10:31:54.774 INFO:teuthology.packaging:branch: reef 2024-03-16T10:31:54.774 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:31:54.774 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T10:31:54.841 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T10:31:54.841 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:31:54.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T10:31:54.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T10:31:54.870 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-16T10:31:54.871 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T10:31:54.871 INFO:teuthology.packaging:ref: None 2024-03-16T10:31:54.871 INFO:teuthology.packaging:tag: None 2024-03-16T10:31:54.871 INFO:teuthology.packaging:branch: reef 2024-03-16T10:31:54.871 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:31:54.871 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T10:31:54.922 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T10:31:54.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T10:31:54.989 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-03-16T10:31:54.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-03-16T10:31:55.019 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T10:31:55.085 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2024-03-16T10:31:55.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2024-03-16T10:31:55.242 INFO:teuthology.orchestra.run.smithi026.stdout:74 files removed 2024-03-16T10:31:55.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw 2024-03-16T10:31:55.341 INFO:teuthology.orchestra.run.smithi002.stdout:74 files removed 2024-03-16T10:31:55.368 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw 2024-03-16T10:31:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 208 kB/s | 77 kB 00:00 2024-03-16T10:31:56.039 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 221 kB/s | 77 kB 00:00 2024-03-16T10:31:56.354 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-03-16T10:31:56.356 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 37 kB/s | 9.9 kB 00:00 2024-03-16T10:31:56.707 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-03-16T10:31:56.771 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-03-16T10:31:57.149 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-16T10:31:57.207 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-16T10:31:58.657 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-16T10:31:58.718 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-03-16T10:31:59.634 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-16T10:31:59.777 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-03-16T10:32:01.785 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-03-16T10:32:01.815 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-16T10:32:05.810 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-16T10:32:05.983 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-16T10:32:07.278 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 15 kB 00:00 2024-03-16T10:32:07.475 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2024-03-16T10:32:08.732 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-16T10:32:08.963 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-16T10:32:13.788 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-03-16T10:32:14.010 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-03-16T10:32:16.408 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:32:16.411 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:32:16.411 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:32:16.411 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:32:16.411 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:32:16.411 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 7.7 M 2024-03-16T10:32:16.411 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2024-03-16T10:32:16.411 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 3.3 M 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 3.0 M 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 5.2 M 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 18 M 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 25 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 703 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T10:32:16.412 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 474 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 4.5 M 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 45 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 127 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 161 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 320 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 299 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 99 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T10:32:16.413 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T10:32:16.414 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:16.414 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:32:16.414 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:32:16.414 INFO:teuthology.orchestra.run.smithi002.stdout:Install 28 Packages 2024-03-16T10:32:16.414 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 2 Packages 2024-03-16T10:32:16.414 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:16.414 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 53 M 2024-03-16T10:32:16.415 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:32:16.634 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 7.7 M 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 3.3 M 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 3.0 M 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T10:32:16.638 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 5.2 M 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 18 M 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 25 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 703 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 474 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 4.5 M 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T10:32:16.639 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 45 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 127 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 161 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 320 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 299 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 99 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout:Install 28 Packages 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 2 Packages 2024-03-16T10:32:16.640 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:16.641 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 53 M 2024-03-16T10:32:16.641 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:32:17.864 INFO:teuthology.orchestra.run.smithi002.stdout:(1/30): ceph-base-18.2.2-878.ga752c2f7.el9.x86_ 4.5 MB/s | 5.2 MB 00:01 2024-03-16T10:32:17.905 INFO:teuthology.orchestra.run.smithi002.stdout:(2/30): ceph-selinux-18.2.2-878.ga752c2f7.el9.x 605 kB/s | 25 kB 00:00 2024-03-16T10:32:17.911 INFO:teuthology.orchestra.run.smithi026.stdout:(1/30): ceph-base-18.2.2-878.ga752c2f7.el9.x86_ 4.4 MB/s | 5.2 MB 00:01 2024-03-16T10:32:17.986 INFO:teuthology.orchestra.run.smithi026.stdout:(2/30): ceph-selinux-18.2.2-878.ga752c2f7.el9.x 336 kB/s | 25 kB 00:00 2024-03-16T10:32:18.047 INFO:teuthology.orchestra.run.smithi002.stdout:(3/30): libcephfs2-18.2.2-878.ga752c2f7.el9.x86 4.8 MB/s | 703 kB 00:00 2024-03-16T10:32:18.162 INFO:teuthology.orchestra.run.smithi026.stdout:(3/30): libcephfs2-18.2.2-878.ga752c2f7.el9.x86 3.9 MB/s | 703 kB 00:00 2024-03-16T10:32:18.239 INFO:teuthology.orchestra.run.smithi002.stdout:(4/30): ceph-radosgw-18.2.2-878.ga752c2f7.el9.x 5.0 MB/s | 7.7 MB 00:01 2024-03-16T10:32:18.264 INFO:teuthology.orchestra.run.smithi002.stdout:(5/30): libradosstriper1-18.2.2-878.ga752c2f7.e 2.1 MB/s | 474 kB 00:00 2024-03-16T10:32:18.270 INFO:teuthology.orchestra.run.smithi026.stdout:(4/30): libradosstriper1-18.2.2-878.ga752c2f7.e 4.3 MB/s | 474 kB 00:00 2024-03-16T10:32:18.323 INFO:teuthology.orchestra.run.smithi002.stdout:(6/30): python3-ceph-argparse-18.2.2-878.ga752c 771 kB/s | 45 kB 00:00 2024-03-16T10:32:18.423 INFO:teuthology.orchestra.run.smithi002.stdout:(7/30): python3-ceph-common-18.2.2-878.ga752c2f 1.2 MB/s | 127 kB 00:00 2024-03-16T10:32:18.532 INFO:teuthology.orchestra.run.smithi002.stdout:(8/30): python3-cephfs-18.2.2-878.ga752c2f7.el9 1.5 MB/s | 161 kB 00:00 2024-03-16T10:32:18.632 INFO:teuthology.orchestra.run.smithi002.stdout:(9/30): python3-rados-18.2.2-878.ga752c2f7.el9. 3.1 MB/s | 320 kB 00:00 2024-03-16T10:32:18.724 INFO:teuthology.orchestra.run.smithi002.stdout:(10/30): python3-rbd-18.2.2-878.ga752c2f7.el9.x 3.2 MB/s | 299 kB 00:00 2024-03-16T10:32:18.808 INFO:teuthology.orchestra.run.smithi002.stdout:(11/30): python3-rgw-18.2.2-878.ga752c2f7.el9.x 1.2 MB/s | 99 kB 00:00 2024-03-16T10:32:18.933 INFO:teuthology.orchestra.run.smithi002.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 847 kB/s | 106 kB 00:00 2024-03-16T10:32:18.992 INFO:teuthology.orchestra.run.smithi002.stdout:(13/30): librgw2-18.2.2-878.ga752c2f7.el9.x86_6 5.9 MB/s | 4.5 MB 00:00 2024-03-16T10:32:19.017 INFO:teuthology.orchestra.run.smithi002.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-16T10:32:19.034 INFO:teuthology.orchestra.run.smithi002.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-16T10:32:19.084 INFO:teuthology.orchestra.run.smithi002.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-16T10:32:19.109 INFO:teuthology.orchestra.run.smithi002.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-16T10:32:19.120 INFO:teuthology.orchestra.run.smithi026.stdout:(5/30): librgw2-18.2.2-878.ga752c2f7.el9.x86_64 5.2 MB/s | 4.5 MB 00:00 2024-03-16T10:32:19.126 INFO:teuthology.orchestra.run.smithi002.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 992 kB/s | 41 kB 00:00 2024-03-16T10:32:19.204 INFO:teuthology.orchestra.run.smithi026.stdout:(6/30): ceph-radosgw-18.2.2-878.ga752c2f7.el9.x 3.1 MB/s | 7.7 MB 00:02 2024-03-16T10:32:19.221 INFO:teuthology.orchestra.run.smithi026.stdout:(7/30): python3-ceph-argparse-18.2.2-878.ga752c 450 kB/s | 45 kB 00:00 2024-03-16T10:32:19.413 INFO:teuthology.orchestra.run.smithi026.stdout:(8/30): ceph-common-18.2.2-878.ga752c2f7.el9.x8 6.8 MB/s | 18 MB 00:02 2024-03-16T10:32:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:(9/30): python3-ceph-common-18.2.2-878.ga752c2f 542 kB/s | 127 kB 00:00 2024-03-16T10:32:19.463 INFO:teuthology.orchestra.run.smithi026.stdout:(10/30): python3-cephfs-18.2.2-878.ga752c2f7.el 666 kB/s | 161 kB 00:00 2024-03-16T10:32:19.538 INFO:teuthology.orchestra.run.smithi026.stdout:(11/30): python3-rados-18.2.2-878.ga752c2f7.el9 2.5 MB/s | 320 kB 00:00 2024-03-16T10:32:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:(12/30): python3-rgw-18.2.2-878.ga752c2f7.el9.x 992 kB/s | 99 kB 00:00 2024-03-16T10:32:19.614 INFO:teuthology.orchestra.run.smithi026.stdout:(13/30): python3-rbd-18.2.2-878.ga752c2f7.el9.x 1.7 MB/s | 299 kB 00:00 2024-03-16T10:32:19.677 INFO:teuthology.orchestra.run.smithi002.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 195 kB/s | 111 kB 00:00 2024-03-16T10:32:19.681 INFO:teuthology.orchestra.run.smithi026.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 745 kB/s | 106 kB 00:00 2024-03-16T10:32:19.706 INFO:teuthology.orchestra.run.smithi026.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 683 kB/s | 44 kB 00:00 2024-03-16T10:32:19.748 INFO:teuthology.orchestra.run.smithi026.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-03-16T10:32:19.815 INFO:teuthology.orchestra.run.smithi026.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 661 kB 00:00 2024-03-16T10:32:19.818 INFO:teuthology.orchestra.run.smithi002.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 446 kB/s | 309 kB 00:00 2024-03-16T10:32:19.840 INFO:teuthology.orchestra.run.smithi026.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 282 kB 00:00 2024-03-16T10:32:19.866 INFO:teuthology.orchestra.run.smithi026.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 352 kB/s | 41 kB 00:00 2024-03-16T10:32:19.927 INFO:teuthology.orchestra.run.smithi002.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-03-16T10:32:20.024 INFO:teuthology.orchestra.run.smithi026.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 532 kB/s | 111 kB 00:00 2024-03-16T10:32:20.044 INFO:teuthology.orchestra.run.smithi002.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-03-16T10:32:20.052 INFO:teuthology.orchestra.run.smithi026.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.6 MB/s | 309 kB 00:00 2024-03-16T10:32:20.068 INFO:teuthology.orchestra.run.smithi026.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 605 kB/s | 26 kB 00:00 2024-03-16T10:32:20.086 INFO:teuthology.orchestra.run.smithi026.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-03-16T10:32:20.111 INFO:teuthology.orchestra.run.smithi026.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-03-16T10:32:20.178 INFO:teuthology.orchestra.run.smithi026.stdout:(25/30): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-03-16T10:32:20.178 INFO:teuthology.orchestra.run.smithi002.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-03-16T10:32:20.237 INFO:teuthology.orchestra.run.smithi026.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 838 kB 00:00 2024-03-16T10:32:20.261 INFO:teuthology.orchestra.run.smithi026.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-16T10:32:20.278 INFO:teuthology.orchestra.run.smithi002.stdout:(24/30): libarrow-9.0.0-10.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-03-16T10:32:20.295 INFO:teuthology.orchestra.run.smithi026.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-03-16T10:32:20.387 INFO:teuthology.orchestra.run.smithi002.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-16T10:32:20.445 INFO:teuthology.orchestra.run.smithi002.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-03-16T10:32:20.529 INFO:teuthology.orchestra.run.smithi002.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-16T10:32:20.587 INFO:teuthology.orchestra.run.smithi026.stdout:(29/30): librbd1-18.2.2-878.ga752c2f7.el9.x86_6 9.2 MB/s | 3.0 MB 00:00 2024-03-16T10:32:20.671 INFO:teuthology.orchestra.run.smithi026.stdout:(30/30): librados2-18.2.2-878.ga752c2f7.el9.x86 7.6 MB/s | 3.3 MB 00:00 2024-03-16T10:32:20.678 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:32:20.678 INFO:teuthology.orchestra.run.smithi026.stdout:Total 13 MB/s | 53 MB 00:04 2024-03-16T10:32:20.679 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:32:20.717 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:32:20.717 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:32:20.737 INFO:teuthology.orchestra.run.smithi002.stdout:(28/30): ceph-common-18.2.2-878.ga752c2f7.el9.x 4.6 MB/s | 18 MB 00:04 2024-03-16T10:32:20.954 INFO:teuthology.orchestra.run.smithi002.stdout:(29/30): librbd1-18.2.2-878.ga752c2f7.el9.x86_6 7.1 MB/s | 3.0 MB 00:00 2024-03-16T10:32:21.005 INFO:teuthology.orchestra.run.smithi002.stdout:(30/30): librados2-18.2.2-878.ga752c2f7.el9.x86 5.9 MB/s | 3.3 MB 00:00 2024-03-16T10:32:21.010 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:32:21.010 INFO:teuthology.orchestra.run.smithi002.stdout:Total 12 MB/s | 53 MB 00:04 2024-03-16T10:32:21.010 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:32:21.051 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:32:21.051 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:32:21.197 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:32:21.197 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:32:21.524 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:32:21.524 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:32:22.727 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:32:22.820 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-16T10:32:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-16T10:32:22.918 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-16T10:32:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:32:23.132 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-16T10:32:23.159 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-16T10:32:23.210 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-16T10:32:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-16T10:32:23.349 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 5/32 2024-03-16T10:32:23.425 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 5/32 2024-03-16T10:32:23.511 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 6/32 2024-03-16T10:32:23.534 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 7/32 2024-03-16T10:32:23.580 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 7/32 2024-03-16T10:32:23.600 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-16T10:32:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-16T10:32:23.636 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:18.2.2-878.ga752c2f7.el9.x 9/32 2024-03-16T10:32:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 5/32 2024-03-16T10:32:23.704 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 10/32 2024-03-16T10:32:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 11/32 2024-03-16T10:32:23.758 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 5/32 2024-03-16T10:32:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 11/32 2024-03-16T10:32:23.844 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 6/32 2024-03-16T10:32:23.885 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 7/32 2024-03-16T10:32:23.921 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-16T10:32:23.939 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 7/32 2024-03-16T10:32:23.952 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-16T10:32:23.967 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-16T10:32:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-16T10:32:24.012 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:18.2.2-878.ga752c2f7.el9.x 9/32 2024-03-16T10:32:24.033 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-16T10:32:24.061 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-16T10:32:24.079 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 10/32 2024-03-16T10:32:24.103 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 11/32 2024-03-16T10:32:24.105 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-16T10:32:24.138 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-16T10:32:24.169 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 11/32 2024-03-16T10:32:24.278 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-16T10:32:24.310 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-16T10:32:24.350 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-16T10:32:24.375 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-16T10:32:24.402 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-16T10:32:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-16T10:32:24.456 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-16T10:32:24.470 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-16T10:32:24.483 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 20/32 2024-03-16T10:32:24.567 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 20/32 2024-03-16T10:32:24.607 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 21/32 2024-03-16T10:32:24.659 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-16T10:32:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-16T10:32:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 20/32 2024-03-16T10:32:24.908 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 20/32 2024-03-16T10:32:24.938 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 21/32 2024-03-16T10:32:24.983 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-16T10:32:25.090 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-16T10:32:25.128 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 24/32 2024-03-16T10:32:25.199 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 24/32 2024-03-16T10:32:25.302 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 25/32 2024-03-16T10:32:25.330 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:18.2.2-878.ga752c2f7.el9.x86 26/32 2024-03-16T10:32:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-16T10:32:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 24/32 2024-03-16T10:32:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 24/32 2024-03-16T10:32:25.560 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 25/32 2024-03-16T10:32:25.588 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:18.2.2-878.ga752c2f7.el9.x86 26/32 2024-03-16T10:32:27.436 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 27/32 2024-03-16T10:32:27.468 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 27/32 2024-03-16T10:32:27.677 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 27/32 2024-03-16T10:32:27.718 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 27/32 2024-03-16T10:32:28.058 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 27/32 2024-03-16T10:32:28.105 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 28/32 2024-03-16T10:32:28.187 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 28/32 2024-03-16T10:32:28.187 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T10:32:28.188 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T10:32:28.188 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:28.222 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 29/32 2024-03-16T10:32:28.302 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 27/32 2024-03-16T10:32:28.338 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 28/32 2024-03-16T10:32:28.464 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 28/32 2024-03-16T10:32:28.464 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T10:32:28.464 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T10:32:28.465 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:28.505 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 29/32 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 29/32 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-03-16T10:32:38.310 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:38.553 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 29/32 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /root 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2024-03-16T10:32:38.554 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:39.108 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 30/32 2024-03-16T10:32:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 30/32 2024-03-16T10:32:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:32:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T10:32:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:32:39.132 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:32:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-16T10:32:39.219 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-16T10:32:39.219 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-16T10:32:39.321 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 30/32 2024-03-16T10:32:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 30/32 2024-03-16T10:32:39.346 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:32:39.346 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T10:32:39.346 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:32:39.346 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:32:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-16T10:32:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-16T10:32:39.457 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-16T10:32:40.362 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 1/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 2/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 3/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 4/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 5/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 6/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 7/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:18.2.2-878.ga752c2f7.el9.x 8/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:18.2.2-878.ga752c2f7.el9.x86 9/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 10/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 11/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 12/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 13/32 2024-03-16T10:32:40.363 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-16T10:32:40.364 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 29/32 2024-03-16T10:32:40.365 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-16T10:32:40.365 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 31/32 2024-03-16T10:32:40.682 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 1/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 2/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 3/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 4/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 5/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 6/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 7/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:18.2.2-878.ga752c2f7.el9.x 8/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:18.2.2-878.ga752c2f7.el9.x86 9/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 10/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 11/32 2024-03-16T10:32:40.683 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 12/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 13/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-16T10:32:40.684 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-16T10:32:40.685 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-16T10:32:40.685 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 29/32 2024-03-16T10:32:40.685 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-16T10:32:40.685 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 31/32 2024-03-16T10:32:41.369 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-16T10:32:41.369 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:41.369 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-03-16T10:32:41.369 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.369 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.369 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:32:41.369 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.370 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:41.371 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:32:41.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-test 2024-03-16T10:32:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-16T10:32:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:41.770 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2024-03-16T10:32:41.770 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T10:32:41.771 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:41.772 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:32:42.105 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-test 2024-03-16T10:32:42.283 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:29 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:32:42.448 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 40 M 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 166 k 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T10:32:42.449 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout:Install 6 Packages 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 41 M 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 168 M 2024-03-16T10:32:42.450 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:32:42.688 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:29 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:32:42.824 INFO:teuthology.orchestra.run.smithi026.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-16T10:32:42.849 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 40 M 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 166 k 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T10:32:42.850 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T10:32:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T10:32:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:42.851 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:32:42.851 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:32:42.851 INFO:teuthology.orchestra.run.smithi002.stdout:Install 6 Packages 2024-03-16T10:32:42.851 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:42.851 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 41 M 2024-03-16T10:32:42.852 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 168 M 2024-03-16T10:32:42.852 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:32:42.874 INFO:teuthology.orchestra.run.smithi026.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-16T10:32:42.899 INFO:teuthology.orchestra.run.smithi026.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 63 kB 00:00 2024-03-16T10:32:42.925 INFO:teuthology.orchestra.run.smithi026.stdout:(4/6): libcephsqlite-18.2.2-878.ga752c2f7.el9.x 601 kB/s | 166 kB 00:00 2024-03-16T10:32:43.053 INFO:teuthology.orchestra.run.smithi002.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-16T10:32:43.095 INFO:teuthology.orchestra.run.smithi002.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-16T10:32:43.120 INFO:teuthology.orchestra.run.smithi002.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.4 MB/s | 63 kB 00:00 2024-03-16T10:32:43.287 INFO:teuthology.orchestra.run.smithi002.stdout:(4/6): libcephsqlite-18.2.2-878.ga752c2f7.el9.x 422 kB/s | 166 kB 00:00 2024-03-16T10:32:43.341 INFO:teuthology.orchestra.run.smithi026.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 424 kB/s | 187 kB 00:00 2024-03-16T10:32:43.546 INFO:teuthology.orchestra.run.smithi002.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 440 kB/s | 187 kB 00:00 2024-03-16T10:32:44.917 INFO:teuthology.orchestra.run.smithi026.stdout:(6/6): ceph-test-18.2.2-878.ga752c2f7.el9.x86_6 17 MB/s | 40 MB 00:02 2024-03-16T10:32:44.920 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:32:44.920 INFO:teuthology.orchestra.run.smithi026.stdout:Total 16 MB/s | 41 MB 00:02 2024-03-16T10:32:44.934 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:32:44.956 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:32:44.956 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:32:45.310 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:32:45.310 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:32:45.857 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:32:45.927 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-16T10:32:45.975 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-16T10:32:46.015 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T10:32:46.067 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-16T10:32:46.094 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 5/6 2024-03-16T10:32:46.155 INFO:teuthology.orchestra.run.smithi002.stdout:(6/6): ceph-test-18.2.2-878.ga752c2f7.el9.x86_6 12 MB/s | 40 MB 00:03 2024-03-16T10:32:46.158 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:32:46.158 INFO:teuthology.orchestra.run.smithi002.stdout:Total 12 MB/s | 41 MB 00:03 2024-03-16T10:32:46.172 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:32:46.195 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:32:46.195 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:32:46.558 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:32:46.559 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:32:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:32:47.190 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-16T10:32:47.226 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-16T10:32:47.267 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T10:32:47.322 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-16T10:32:47.349 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 5/6 2024-03-16T10:32:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 5/6 2024-03-16T10:32:50.146 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 6/6 2024-03-16T10:32:51.291 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 5/6 2024-03-16T10:32:51.383 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 6/6 2024-03-16T10:32:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 6/6 2024-03-16T10:32:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 1/6 2024-03-16T10:32:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 2/6 2024-03-16T10:32:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T10:32:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-16T10:32:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-16T10:32:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-16T10:32:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: jq-1.6-16.el9.x86_64 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:52.696 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:32:53.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph 2024-03-16T10:32:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 6/6 2024-03-16T10:32:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 1/6 2024-03-16T10:32:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 2/6 2024-03-16T10:32:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-16T10:32:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-16T10:32:53.519 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-16T10:32:53.580 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:40 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:32:53.737 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:32:53.739 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================== 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================== 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 6.5 k 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 2.1 M 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 1.4 M 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 245 k 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 4.4 M 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 18 M 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T10:32:53.740 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T10:32:53.741 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================== 2024-03-16T10:32:53.742 INFO:teuthology.orchestra.run.smithi026.stdout:Install 33 Packages 2024-03-16T10:32:53.743 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:53.743 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 30 M 2024-03-16T10:32:53.743 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 110 M 2024-03-16T10:32:53.743 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:32:53.933 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-16T10:32:53.933 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:53.933 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout: jq-1.6-16.el9.x86_64 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:53.934 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:32:54.202 INFO:teuthology.orchestra.run.smithi026.stdout:(1/33): ceph-18.2.2-878.ga752c2f7.el9.x86_64.rp 32 kB/s | 6.5 kB 00:00 2024-03-16T10:32:54.242 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph 2024-03-16T10:32:54.411 INFO:teuthology.orchestra.run.smithi026.stdout:(2/33): ceph-mds-18.2.2-878.ga752c2f7.el9.x86_6 5.1 MB/s | 2.1 MB 00:00 2024-03-16T10:32:54.452 INFO:teuthology.orchestra.run.smithi026.stdout:(3/33): ceph-mgr-18.2.2-878.ga752c2f7.el9.x86_6 3.2 MB/s | 1.4 MB 00:00 2024-03-16T10:32:54.545 INFO:teuthology.orchestra.run.smithi026.stdout:(4/33): ceph-mgr-modules-core-18.2.2-878.ga752c 2.6 MB/s | 245 kB 00:00 2024-03-16T10:32:54.670 INFO:teuthology.orchestra.run.smithi026.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2024-03-16T10:32:54.703 INFO:teuthology.orchestra.run.smithi026.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 950 kB/s | 32 kB 00:00 2024-03-16T10:32:54.746 INFO:teuthology.orchestra.run.smithi026.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-16T10:32:54.779 INFO:teuthology.orchestra.run.smithi026.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-16T10:32:54.813 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:41 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:32:54.829 INFO:teuthology.orchestra.run.smithi026.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-16T10:32:54.896 INFO:teuthology.orchestra.run.smithi026.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-16T10:32:54.930 INFO:teuthology.orchestra.run.smithi026.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-16T10:32:54.964 INFO:teuthology.orchestra.run.smithi026.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-03-16T10:32:54.975 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:32:54.978 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================== 2024-03-16T10:32:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:32:54.978 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================== 2024-03-16T10:32:54.978 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:32:54.978 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 6.5 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 2.1 M 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 1.4 M 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 245 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 4.4 M 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 18 M 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T10:32:54.979 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T10:32:54.980 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================== 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout:Install 33 Packages 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 30 M 2024-03-16T10:32:54.981 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 110 M 2024-03-16T10:32:54.982 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:32:54.997 INFO:teuthology.orchestra.run.smithi026.stdout:(13/33): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-03-16T10:32:55.306 INFO:teuthology.orchestra.run.smithi026.stdout:(14/33): ceph-osd-18.2.2-878.ga752c2f7.el9.x86_ 20 MB/s | 18 MB 00:00 2024-03-16T10:32:55.373 INFO:teuthology.orchestra.run.smithi026.stdout:(15/33): ceph-mon-18.2.2-878.ga752c2f7.el9.x86_ 3.8 MB/s | 4.4 MB 00:01 2024-03-16T10:32:55.398 INFO:teuthology.orchestra.run.smithi026.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 312 kB/s | 125 kB 00:00 2024-03-16T10:32:55.540 INFO:teuthology.orchestra.run.smithi026.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 186 kB/s | 43 kB 00:00 2024-03-16T10:32:55.639 INFO:teuthology.orchestra.run.smithi002.stdout:(1/33): ceph-18.2.2-878.ga752c2f7.el9.x86_64.rp 37 kB/s | 6.5 kB 00:00 2024-03-16T10:32:55.682 INFO:teuthology.orchestra.run.smithi026.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 556 kB/s | 172 kB 00:00 2024-03-16T10:32:55.757 INFO:teuthology.orchestra.run.smithi026.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-03-16T10:32:55.782 INFO:teuthology.orchestra.run.smithi026.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 44 kB/s | 11 kB 00:00 2024-03-16T10:32:55.799 INFO:teuthology.orchestra.run.smithi026.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-03-16T10:32:55.824 INFO:teuthology.orchestra.run.smithi026.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 345 kB/s | 23 kB 00:00 2024-03-16T10:32:55.850 INFO:teuthology.orchestra.run.smithi026.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 289 kB/s | 19 kB 00:00 2024-03-16T10:32:55.867 INFO:teuthology.orchestra.run.smithi026.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 293 kB/s | 20 kB 00:00 2024-03-16T10:32:55.881 INFO:teuthology.orchestra.run.smithi002.stdout:(2/33): ceph-mds-18.2.2-878.ga752c2f7.el9.x86_6 5.1 MB/s | 2.1 MB 00:00 2024-03-16T10:32:55.892 INFO:teuthology.orchestra.run.smithi026.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 688 kB/s | 46 kB 00:00 2024-03-16T10:32:55.923 INFO:teuthology.orchestra.run.smithi002.stdout:(3/33): ceph-mgr-18.2.2-878.ga752c2f7.el9.x86_6 3.2 MB/s | 1.4 MB 00:00 2024-03-16T10:32:55.967 INFO:teuthology.orchestra.run.smithi026.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 219 kB/s | 16 kB 00:00 2024-03-16T10:32:55.992 INFO:teuthology.orchestra.run.smithi026.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 553 kB/s | 79 kB 00:00 2024-03-16T10:32:56.014 INFO:teuthology.orchestra.run.smithi002.stdout:(4/33): ceph-mgr-modules-core-18.2.2-878.ga752c 2.6 MB/s | 245 kB 00:00 2024-03-16T10:32:56.018 INFO:teuthology.orchestra.run.smithi026.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-03-16T10:32:56.043 INFO:teuthology.orchestra.run.smithi026.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-03-16T10:32:56.068 INFO:teuthology.orchestra.run.smithi026.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-03-16T10:32:56.102 INFO:teuthology.orchestra.run.smithi026.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-16T10:32:56.123 INFO:teuthology.orchestra.run.smithi002.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-16T10:32:56.135 INFO:teuthology.orchestra.run.smithi026.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 300 kB/s | 20 kB 00:00 2024-03-16T10:32:56.148 INFO:teuthology.orchestra.run.smithi002.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-03-16T10:32:56.160 INFO:teuthology.orchestra.run.smithi026.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-03-16T10:32:56.163 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:32:56.164 INFO:teuthology.orchestra.run.smithi026.stdout:Total 12 MB/s | 30 MB 00:02 2024-03-16T10:32:56.174 INFO:teuthology.orchestra.run.smithi002.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.9 MB/s | 47 kB 00:00 2024-03-16T10:32:56.178 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:32:56.199 INFO:teuthology.orchestra.run.smithi002.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2024-03-16T10:32:56.222 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:32:56.223 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:32:56.249 INFO:teuthology.orchestra.run.smithi002.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-16T10:32:56.316 INFO:teuthology.orchestra.run.smithi002.stdout:(10/33): ceph-mon-18.2.2-878.ga752c2f7.el9.x86_ 6.5 MB/s | 4.4 MB 00:00 2024-03-16T10:32:56.350 INFO:teuthology.orchestra.run.smithi002.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 11 MB/s | 1.1 MB 00:00 2024-03-16T10:32:56.367 INFO:teuthology.orchestra.run.smithi002.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2024-03-16T10:32:56.392 INFO:teuthology.orchestra.run.smithi002.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2024-03-16T10:32:56.417 INFO:teuthology.orchestra.run.smithi002.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-16T10:32:56.509 INFO:teuthology.orchestra.run.smithi002.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 475 kB/s | 43 kB 00:00 2024-03-16T10:32:56.596 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:32:56.597 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:32:56.868 INFO:teuthology.orchestra.run.smithi002.stdout:(16/33): ceph-osd-18.2.2-878.ga752c2f7.el9.x86_ 18 MB/s | 18 MB 00:00 2024-03-16T10:32:56.893 INFO:teuthology.orchestra.run.smithi002.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 448 kB/s | 172 kB 00:00 2024-03-16T10:32:56.952 INFO:teuthology.orchestra.run.smithi002.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-03-16T10:32:57.002 INFO:teuthology.orchestra.run.smithi002.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.6 MB/s | 358 kB 00:00 2024-03-16T10:32:57.027 INFO:teuthology.orchestra.run.smithi002.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 235 kB/s | 18 kB 00:00 2024-03-16T10:32:57.052 INFO:teuthology.orchestra.run.smithi002.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-03-16T10:32:57.078 INFO:teuthology.orchestra.run.smithi002.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-03-16T10:32:57.103 INFO:teuthology.orchestra.run.smithi002.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-03-16T10:32:57.162 INFO:teuthology.orchestra.run.smithi002.stdout:(24/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-03-16T10:32:57.187 INFO:teuthology.orchestra.run.smithi002.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 424 kB/s | 46 kB 00:00 2024-03-16T10:32:57.229 INFO:teuthology.orchestra.run.smithi002.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 395 kB/s | 16 kB 00:00 2024-03-16T10:32:57.271 INFO:teuthology.orchestra.run.smithi002.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-16T10:32:57.296 INFO:teuthology.orchestra.run.smithi002.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-03-16T10:32:57.330 INFO:teuthology.orchestra.run.smithi002.stdout:(29/33): python3-requests-2.25.1-8.el9.noarch.r 133 kB/s | 125 kB 00:00 2024-03-16T10:32:57.355 INFO:teuthology.orchestra.run.smithi002.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 426 kB/s | 36 kB 00:00 2024-03-16T10:32:57.397 INFO:teuthology.orchestra.run.smithi002.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-16T10:32:57.423 INFO:teuthology.orchestra.run.smithi002.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 300 kB/s | 20 kB 00:00 2024-03-16T10:32:57.472 INFO:teuthology.orchestra.run.smithi002.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.9 MB/s | 427 kB 00:00 2024-03-16T10:32:57.476 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:32:57.476 INFO:teuthology.orchestra.run.smithi002.stdout:Total 12 MB/s | 30 MB 00:02 2024-03-16T10:32:57.490 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:32:57.534 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:32:57.534 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:32:57.885 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:32:57.908 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:32:57.908 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:32:57.947 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-16T10:32:57.979 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-16T10:32:58.012 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-16T10:32:58.108 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-16T10:32:58.209 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-16T10:32:58.257 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-16T10:32:58.339 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-16T10:32:58.392 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-16T10:32:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-16T10:32:58.487 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-16T10:32:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-16T10:32:58.751 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-16T10:32:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-16T10:32:58.849 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-16T10:32:58.883 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-16T10:32:58.913 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-16T10:32:58.993 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-16T10:32:59.032 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T10:32:59.065 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-16T10:32:59.106 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-16T10:32:59.141 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-16T10:32:59.175 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-16T10:32:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:32:59.268 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-16T10:32:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-16T10:32:59.306 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-16T10:32:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-16T10:32:59.366 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-16T10:32:59.380 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-16T10:32:59.462 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-16T10:32:59.530 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-16T10:32:59.539 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-16T10:32:59.586 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-16T10:32:59.653 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T10:32:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-16T10:32:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-16T10:32:59.751 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-16T10:32:59.801 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-16T10:32:59.814 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-878.ga752c2f7.el9.n 28/33 2024-03-16T10:32:59.844 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 29/33 2024-03-16T10:32:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 29/33 2024-03-16T10:32:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:32:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T10:32:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T10:32:59.869 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T10:32:59.869 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:32:59.872 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-16T10:33:00.066 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-16T10:33:00.130 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-16T10:33:00.169 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-16T10:33:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-16T10:33:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-16T10:33:00.313 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-16T10:33:00.352 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T10:33:00.385 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-16T10:33:00.417 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-16T10:33:00.461 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-16T10:33:00.495 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-16T10:33:00.589 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-16T10:33:00.643 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-16T10:33:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-16T10:33:00.876 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-16T10:33:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T10:33:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-878.ga752c2f7.el9.n 28/33 2024-03-16T10:33:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 29/33 2024-03-16T10:33:01.204 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 29/33 2024-03-16T10:33:01.204 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:01.204 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T10:33:01.204 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T10:33:01.205 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T10:33:01.205 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:01.470 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 30/33 2024-03-16T10:33:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 30/33 2024-03-16T10:33:01.496 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:01.496 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T10:33:01.496 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T10:33:01.497 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T10:33:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:01.970 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 31/33 2024-03-16T10:33:01.997 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 31/33 2024-03-16T10:33:01.997 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:01.997 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T10:33:01.997 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T10:33:01.998 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T10:33:01.998 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:02.213 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 32/33 2024-03-16T10:33:02.235 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 32/33 2024-03-16T10:33:02.235 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:02.235 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T10:33:02.236 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T10:33:02.236 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T10:33:02.236 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:02.305 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 33/33 2024-03-16T10:33:02.823 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 30/33 2024-03-16T10:33:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 30/33 2024-03-16T10:33:02.852 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:02.852 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T10:33:02.852 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T10:33:02.852 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T10:33:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:03.324 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 31/33 2024-03-16T10:33:03.347 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 31/33 2024-03-16T10:33:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T10:33:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T10:33:03.347 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T10:33:03.347 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:03.566 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 32/33 2024-03-16T10:33:03.589 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 32/33 2024-03-16T10:33:03.589 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:03.589 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T10:33:03.589 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T10:33:03.589 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T10:33:03.589 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:03.657 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 33/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 33/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 1/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 2/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 3/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 4/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 5/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-878.ga752c2f7.el9.n 6/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-16T10:33:04.036 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-16T10:33:04.037 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-16T10:33:04.038 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T10:33:04.882 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T10:33:04.883 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:04.884 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:05.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-base 2024-03-16T10:33:05.358 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 33/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 1/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 2/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 3/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 4/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 5/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-878.ga752c2f7.el9.n 6/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-16T10:33:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-16T10:33:05.360 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-16T10:33:05.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-16T10:33:05.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-16T10:33:05.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-16T10:33:05.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-16T10:33:05.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-16T10:33:05.750 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:52 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:05.850 INFO:teuthology.orchestra.run.smithi026.stdout:Package ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:05.904 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:05.905 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:05.905 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:05.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install cephadm 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T10:33:06.311 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T10:33:06.312 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:06.313 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:06.532 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:53 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:06.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-base 2024-03-16T10:33:06.687 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 224 k 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:06.688 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 224 k 2024-03-16T10:33:06.689 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 215 k 2024-03-16T10:33:06.689 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:06.945 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm-18.2.2-878.ga752c2f7.el9.noarch.rpm 876 kB/s | 224 kB 00:00 2024-03-16T10:33:06.945 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:06.945 INFO:teuthology.orchestra.run.smithi026.stdout:Total 871 kB/s | 224 kB 00:00 2024-03-16T10:33:06.945 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:06.951 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:06.951 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:06.954 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:06.954 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:07.028 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:07.123 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:54 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:07.222 INFO:teuthology.orchestra.run.smithi002.stdout:Package ceph-base-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:07.276 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:07.277 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:07.277 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:07.335 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install cephadm 2024-03-16T10:33:07.473 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:07.903 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:54 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:07.986 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:08.056 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 224 k 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:08.057 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 224 k 2024-03-16T10:33:08.058 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 215 k 2024-03-16T10:33:08.058 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:08.294 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:08.295 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:08.406 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm-18.2.2-878.ga752c2f7.el9.noarch.rpm 643 kB/s | 224 kB 00:00 2024-03-16T10:33:08.407 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:08.407 INFO:teuthology.orchestra.run.smithi002.stdout:Total 640 kB/s | 224 kB 00:00 2024-03-16T10:33:08.407 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:08.412 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:08.413 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:08.415 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:08.416 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:08.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-immutable-object-cache 2024-03-16T10:33:08.487 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:08.829 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:08.930 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:09.052 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:56 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:09.207 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:09.207 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:09.207 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 143 k 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 143 k 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 395 k 2024-03-16T10:33:09.208 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:09.296 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:09.450 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-immutable-object-cache-18.2.2-878.ga752c2f 589 kB/s | 143 kB 00:00 2024-03-16T10:33:09.451 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:09.451 INFO:teuthology.orchestra.run.smithi026.stdout:Total 586 kB/s | 143 kB 00:00 2024-03-16T10:33:09.451 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:09.459 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:09.459 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:09.514 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:09.515 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:09.589 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/1 2024-03-16T10:33:09.589 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:09.589 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:09.589 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:09.590 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:09.590 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:09.664 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:09.726 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.e 1/1 2024-03-16T10:33:09.750 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.e 1/1 2024-03-16T10:33:09.750 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:09.750 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T10:33:09.750 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:09.781 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-immutable-object-cache 2024-03-16T10:33:10.357 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:57 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 143 k 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-03-16T10:33:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 143 k 2024-03-16T10:33:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 395 k 2024-03-16T10:33:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:10.717 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.e 1/1 2024-03-16T10:33:10.717 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:10.717 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:10.717 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:10.717 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:10.717 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:10.745 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-immutable-object-cache-18.2.2-878.ga752c2f 620 kB/s | 143 kB 00:00 2024-03-16T10:33:10.745 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:10.745 INFO:teuthology.orchestra.run.smithi002.stdout:Total 616 kB/s | 143 kB 00:00 2024-03-16T10:33:10.746 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:10.753 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:10.753 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:10.807 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:10.808 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:10.947 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:10.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr 2024-03-16T10:33:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.e 1/1 2024-03-16T10:33:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.e 1/1 2024-03-16T10:33:11.036 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:11.036 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T10:33:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:11.555 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:58 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:11.654 INFO:teuthology.orchestra.run.smithi026.stdout:Package ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:11.708 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:11.709 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:11.709 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:11.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-dashboard 2024-03-16T10:33:12.106 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.e 1/1 2024-03-16T10:33:12.107 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:12.107 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:12.107 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:12.107 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:12.107 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:12.345 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:59 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:12.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr 2024-03-16T10:33:12.498 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:12.499 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:33:12.499 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 1.7 M 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 23 k 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 15 k 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout:Install 5 Packages 2024-03-16T10:33:12.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 2.0 M 2024-03-16T10:33:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 8.3 M 2024-03-16T10:33:12.501 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:12.746 INFO:teuthology.orchestra.run.smithi026.stdout:(1/5): ceph-prometheus-alerts-18.2.2-878.ga752c 82 kB/s | 15 kB 00:00 2024-03-16T10:33:12.763 INFO:teuthology.orchestra.run.smithi026.stdout:(2/5): ceph-grafana-dashboards-18.2.2-878.ga752 117 kB/s | 23 kB 00:00 2024-03-16T10:33:12.941 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:59 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 123 kB/s | 31 kB 00:00 2024-03-16T10:33:13.041 INFO:teuthology.orchestra.run.smithi002.stdout:Package ceph-mgr-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:(4/5): ceph-mgr-dashboard-18.2.2-878.ga752c2f7. 3.4 MB/s | 1.7 MB 00:00 2024-03-16T10:33:13.095 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:13.096 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:13.096 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:13.161 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-dashboard 2024-03-16T10:33:13.230 INFO:teuthology.orchestra.run.smithi026.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 403 kB/s | 188 kB 00:00 2024-03-16T10:33:13.232 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:13.232 INFO:teuthology.orchestra.run.smithi026.stdout:Total 2.7 MB/s | 2.0 MB 00:00 2024-03-16T10:33:13.232 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:13.240 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:13.240 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:13.302 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:13.303 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:13.674 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:13.727 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:00 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:13.745 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-03-16T10:33:13.794 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-03-16T10:33:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-878.ga752c2f7.el9.no 3/5 2024-03-16T10:33:13.883 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 1.7 M 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 23 k 2024-03-16T10:33:13.884 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 15 k 2024-03-16T10:33:13.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T10:33:13.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T10:33:13.885 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:13.885 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:13.885 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:13.885 INFO:teuthology.orchestra.run.smithi002.stdout:Install 5 Packages 2024-03-16T10:33:13.885 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:13.886 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 2.0 M 2024-03-16T10:33:13.886 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 8.3 M 2024-03-16T10:33:13.886 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:14.068 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-878.ga752c2f7.el9.n 4/5 2024-03-16T10:33:14.100 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 5/5 2024-03-16T10:33:14.241 INFO:teuthology.orchestra.run.smithi002.stdout:(1/5): ceph-grafana-dashboards-18.2.2-878.ga752 117 kB/s | 23 kB 00:00 2024-03-16T10:33:14.341 INFO:teuthology.orchestra.run.smithi002.stdout:(2/5): ceph-prometheus-alerts-18.2.2-878.ga752c 49 kB/s | 15 kB 00:00 2024-03-16T10:33:14.516 INFO:teuthology.orchestra.run.smithi002.stdout:(3/5): ceph-mgr-dashboard-18.2.2-878.ga752c2f7. 3.7 MB/s | 1.7 MB 00:00 2024-03-16T10:33:14.628 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 5/5 2024-03-16T10:33:14.628 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-878.ga752c2f7.el9.n 1/5 2024-03-16T10:33:14.628 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 2/5 2024-03-16T10:33:14.628 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-878.ga752c2f7.el9.no 3/5 2024-03-16T10:33:14.628 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-03-16T10:33:14.633 INFO:teuthology.orchestra.run.smithi002.stdout:(4/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 78 kB/s | 31 kB 00:00 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T10:33:14.927 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:14.928 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:14.967 INFO:teuthology.orchestra.run.smithi002.stdout:(5/5): python3-routes-2.5.1-5.el9.noarch.rpm 314 kB/s | 188 kB 00:00 2024-03-16T10:33:14.970 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:14.970 INFO:teuthology.orchestra.run.smithi002.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2024-03-16T10:33:14.970 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:14.978 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:14.978 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:15.053 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:15.053 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:15.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-16T10:33:15.297 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-03-16T10:33:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-03-16T10:33:15.441 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-878.ga752c2f7.el9.no 3/5 2024-03-16T10:33:15.686 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-878.ga752c2f7.el9.n 4/5 2024-03-16T10:33:15.703 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:02 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:15.727 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 5/5 2024-03-16T10:33:15.859 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:15.860 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================== 2024-03-16T10:33:15.860 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:33:15.860 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================== 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 7.4 M 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T10:33:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T10:33:15.862 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T10:33:15.862 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T10:33:15.862 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:15.862 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:15.862 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================== 2024-03-16T10:33:15.862 INFO:teuthology.orchestra.run.smithi026.stdout:Install 12 Packages 2024-03-16T10:33:15.862 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:15.863 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 37 M 2024-03-16T10:33:15.863 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 209 M 2024-03-16T10:33:15.863 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:16.288 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 5/5 2024-03-16T10:33:16.288 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-878.ga752c2f7.el9.n 1/5 2024-03-16T10:33:16.288 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 2/5 2024-03-16T10:33:16.288 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-878.ga752c2f7.el9.no 3/5 2024-03-16T10:33:16.288 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-03-16T10:33:16.353 INFO:teuthology.orchestra.run.smithi026.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 213 kB/s | 32 kB 00:00 2024-03-16T10:33:16.386 INFO:teuthology.orchestra.run.smithi026.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 539 kB/s | 17 kB 00:00 2024-03-16T10:33:16.420 INFO:teuthology.orchestra.run.smithi026.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.1 MB/s | 34 kB 00:00 2024-03-16T10:33:16.487 INFO:teuthology.orchestra.run.smithi026.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:16.597 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:16.687 INFO:teuthology.orchestra.run.smithi026.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2024-03-16T10:33:16.754 INFO:teuthology.orchestra.run.smithi026.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-03-16T10:33:16.780 INFO:teuthology.orchestra.run.smithi026.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 217 kB 00:00 2024-03-16T10:33:16.833 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-16T10:33:16.947 INFO:teuthology.orchestra.run.smithi026.stdout:(8/12): ceph-mgr-diskprediction-local-18.2.2-87 10 MB/s | 7.4 MB 00:00 2024-03-16T10:33:17.180 INFO:teuthology.orchestra.run.smithi026.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-16T10:33:17.297 INFO:teuthology.orchestra.run.smithi026.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-03-16T10:33:17.322 INFO:teuthology.orchestra.run.smithi026.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 524 kB/s | 195 kB 00:00 2024-03-16T10:33:17.389 INFO:teuthology.orchestra.run.smithi026.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-16T10:33:17.393 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:17.393 INFO:teuthology.orchestra.run.smithi026.stdout:Total 24 MB/s | 37 MB 00:01 2024-03-16T10:33:17.403 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:04 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:17.432 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:17.457 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:17.457 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:17.561 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================== 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================== 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 7.4 M 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T10:33:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================== 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout:Install 12 Packages 2024-03-16T10:33:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:17.564 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 37 M 2024-03-16T10:33:17.564 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 209 M 2024-03-16T10:33:17.564 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:17.789 INFO:teuthology.orchestra.run.smithi002.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 228 kB/s | 32 kB 00:00 2024-03-16T10:33:17.839 INFO:teuthology.orchestra.run.smithi002.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 353 kB/s | 17 kB 00:00 2024-03-16T10:33:17.906 INFO:teuthology.orchestra.run.smithi002.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-03-16T10:33:17.918 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:17.919 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:17.940 INFO:teuthology.orchestra.run.smithi002.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 350 kB/s | 34 kB 00:00 2024-03-16T10:33:18.115 INFO:teuthology.orchestra.run.smithi002.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2024-03-16T10:33:18.182 INFO:teuthology.orchestra.run.smithi002.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-16T10:33:18.207 INFO:teuthology.orchestra.run.smithi002.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-16T10:33:18.383 INFO:teuthology.orchestra.run.smithi002.stdout:(8/12): ceph-mgr-diskprediction-local-18.2.2-87 10 MB/s | 7.4 MB 00:00 2024-03-16T10:33:18.600 INFO:teuthology.orchestra.run.smithi002.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-16T10:33:18.644 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:18.649 INFO:teuthology.orchestra.run.smithi002.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-03-16T10:33:18.692 INFO:teuthology.orchestra.run.smithi002.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 638 kB/s | 195 kB 00:00 2024-03-16T10:33:18.712 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-16T10:33:18.752 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-16T10:33:18.817 INFO:teuthology.orchestra.run.smithi002.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-16T10:33:18.818 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:18.819 INFO:teuthology.orchestra.run.smithi002.stdout:Total 30 MB/s | 37 MB 00:01 2024-03-16T10:33:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-16T10:33:18.859 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:18.884 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:18.884 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:18.889 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-16T10:33:19.073 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T10:33:19.096 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T10:33:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-16T10:33:19.325 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:19.326 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:19.872 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-16T10:33:19.985 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-16T10:33:20.047 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-16T10:33:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-16T10:33:20.246 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-16T10:33:20.291 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-16T10:33:20.477 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T10:33:20.506 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T10:33:20.594 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-16T10:33:21.277 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-16T10:33:21.296 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-16T10:33:21.391 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-16T10:33:22.363 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-16T10:33:22.395 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2 12/12 2024-03-16T10:33:22.705 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-16T10:33:23.785 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-16T10:33:23.814 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2 12/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2 12/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2 1/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T10:33:24.909 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-16T10:33:24.910 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-16T10:33:24.910 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-16T10:33:24.910 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-16T10:33:24.910 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-16T10:33:25.342 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T10:33:25.343 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T10:33:25.344 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T10:33:25.344 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:25.344 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:25.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-rook 2024-03-16T10:33:26.252 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:13 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2 12/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2 1/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-16T10:33:26.304 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-16T10:33:26.305 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-16T10:33:26.410 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 49 k 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T10:33:26.411 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout:Install 13 Packages 2024-03-16T10:33:26.412 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:26.413 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 2.0 M 2024-03-16T10:33:26.413 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 26 M 2024-03-16T10:33:26.413 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:26.720 INFO:teuthology.orchestra.run.smithi026.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 180 kB/s | 24 kB 00:00 2024-03-16T10:33:26.741 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-16T10:33:26.741 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:26.741 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:26.741 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:26.741 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T10:33:26.741 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:26.742 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:26.745 INFO:teuthology.orchestra.run.smithi026.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 112 kB/s | 18 kB 00:00 2024-03-16T10:33:26.804 INFO:teuthology.orchestra.run.smithi026.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 3.0 MB/s | 174 kB 00:00 2024-03-16T10:33:26.829 INFO:teuthology.orchestra.run.smithi026.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-16T10:33:26.854 INFO:teuthology.orchestra.run.smithi026.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-16T10:33:26.871 INFO:teuthology.orchestra.run.smithi026.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 43 kB 00:00 2024-03-16T10:33:26.896 INFO:teuthology.orchestra.run.smithi026.stdout:(7/13): ceph-mgr-rook-18.2.2-878.ga752c2f7.el9. 158 kB/s | 49 kB 00:00 2024-03-16T10:33:26.938 INFO:teuthology.orchestra.run.smithi026.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 225 kB/s | 15 kB 00:00 2024-03-16T10:33:26.964 INFO:teuthology.orchestra.run.smithi026.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 295 kB/s | 32 kB 00:00 2024-03-16T10:33:27.014 INFO:teuthology.orchestra.run.smithi026.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-16T10:33:27.039 INFO:teuthology.orchestra.run.smithi026.stdout:(11/13): python3-google-auth-2.28.0-1.el9.noarc 1.4 MB/s | 203 kB 00:00 2024-03-16T10:33:27.070 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-rook 2024-03-16T10:33:27.081 INFO:teuthology.orchestra.run.smithi026.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-03-16T10:33:27.107 INFO:teuthology.orchestra.run.smithi026.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 970 kB/s | 90 kB 00:00 2024-03-16T10:33:27.108 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:27.108 INFO:teuthology.orchestra.run.smithi026.stdout:Total 2.9 MB/s | 2.0 MB 00:00 2024-03-16T10:33:27.109 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:27.122 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:27.123 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:27.225 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:27.226 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:27.642 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:14 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:27.761 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:27.798 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 49 k 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T10:33:27.799 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout:Install 13 Packages 2024-03-16T10:33:27.800 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:27.801 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 2.0 M 2024-03-16T10:33:27.801 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 26 M 2024-03-16T10:33:27.801 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-16T10:33:27.984 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-16T10:33:28.040 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-16T10:33:28.076 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-16T10:33:28.118 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-16T10:33:28.215 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-16T10:33:28.280 INFO:teuthology.orchestra.run.smithi002.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 203 kB/s | 24 kB 00:00 2024-03-16T10:33:28.306 INFO:teuthology.orchestra.run.smithi002.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 124 kB/s | 18 kB 00:00 2024-03-16T10:33:28.333 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-16T10:33:28.356 INFO:teuthology.orchestra.run.smithi002.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-16T10:33:28.381 INFO:teuthology.orchestra.run.smithi002.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-16T10:33:28.385 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-16T10:33:28.406 INFO:teuthology.orchestra.run.smithi002.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-16T10:33:28.432 INFO:teuthology.orchestra.run.smithi002.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 858 kB/s | 43 kB 00:00 2024-03-16T10:33:28.457 INFO:teuthology.orchestra.run.smithi002.stdout:(7/13): ceph-mgr-rook-18.2.2-878.ga752c2f7.el9. 166 kB/s | 49 kB 00:00 2024-03-16T10:33:28.832 INFO:teuthology.orchestra.run.smithi002.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 37 kB/s | 15 kB 00:00 2024-03-16T10:33:28.858 INFO:teuthology.orchestra.run.smithi002.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 71 kB/s | 32 kB 00:00 2024-03-16T10:33:29.058 INFO:teuthology.orchestra.run.smithi002.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 296 kB/s | 59 kB 00:00 2024-03-16T10:33:29.075 INFO:teuthology.orchestra.run.smithi002.stdout:(11/13): python3-google-auth-2.28.0-1.el9.noarc 328 kB/s | 203 kB 00:00 2024-03-16T10:33:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-16T10:33:29.190 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-16T10:33:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-16T10:33:29.225 INFO:teuthology.orchestra.run.smithi002.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 538 kB/s | 90 kB 00:00 2024-03-16T10:33:29.272 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-16T10:33:29.302 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 13/13 2024-03-16T10:33:29.415 INFO:teuthology.orchestra.run.smithi002.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.8 MB/s | 1.0 MB 00:00 2024-03-16T10:33:29.419 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:29.420 INFO:teuthology.orchestra.run.smithi002.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2024-03-16T10:33:29.420 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:29.436 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:29.436 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:29.546 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:29.546 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:30.084 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 13/13 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 1/13 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-16T10:33:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-16T10:33:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-16T10:33:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-16T10:33:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-16T10:33:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-16T10:33:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-16T10:33:30.271 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-16T10:33:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-16T10:33:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-16T10:33:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-16T10:33:30.459 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-16T10:33:30.459 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:30.459 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:30.459 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:30.459 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T10:33:30.459 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:30.460 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-16T10:33:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-16T10:33:30.666 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-16T10:33:30.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-mgr-cephadm 2024-03-16T10:33:30.721 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-16T10:33:31.292 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:18 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:31.449 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:31.449 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-03-16T10:33:31.449 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:33:31.449 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 138 k 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout:Install 6 Packages 2024-03-16T10:33:31.450 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:31.451 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 6.8 M 2024-03-16T10:33:31.451 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 33 M 2024-03-16T10:33:31.451 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-16T10:33:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-16T10:33:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-16T10:33:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-16T10:33:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 13/13 2024-03-16T10:33:31.892 INFO:teuthology.orchestra.run.smithi026.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-16T10:33:31.917 INFO:teuthology.orchestra.run.smithi026.stdout:(2/6): ceph-mgr-cephadm-18.2.2-878.ga752c2f7.el 466 kB/s | 138 kB 00:00 2024-03-16T10:33:32.192 INFO:teuthology.orchestra.run.smithi026.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 210 kB/s | 58 kB 00:00 2024-03-16T10:33:32.284 INFO:teuthology.orchestra.run.smithi026.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 557 kB/s | 51 kB 00:00 2024-03-16T10:33:32.359 INFO:teuthology.orchestra.run.smithi026.stdout:(5/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-03-16T10:33:32.406 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 13/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 1/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-16T10:33:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-16T10:33:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T10:33:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T10:33:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T10:33:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T10:33:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:32.900 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:32.909 INFO:teuthology.orchestra.run.smithi026.stdout:(6/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 192 kB/s | 248 kB 00:01 2024-03-16T10:33:32.913 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:32.913 INFO:teuthology.orchestra.run.smithi026.stdout:Total 4.6 MB/s | 6.8 MB 00:01 2024-03-16T10:33:32.927 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:32.939 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:32.939 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:33.040 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:33.040 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:33.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-cephadm 2024-03-16T10:33:33.359 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:33.533 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-16T10:33:33.587 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-16T10:33:33.735 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:20 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:33.890 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:18.2.2-878.ga752c2f7.el9 ceph-noarch 138 k 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T10:33:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T10:33:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:33.892 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:33.892 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-03-16T10:33:33.892 INFO:teuthology.orchestra.run.smithi002.stdout:Install 6 Packages 2024-03-16T10:33:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:33.892 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 6.8 M 2024-03-16T10:33:33.892 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 33 M 2024-03-16T10:33:33.893 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:34.141 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-16T10:33:34.225 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-16T10:33:34.294 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-16T10:33:34.302 INFO:teuthology.orchestra.run.smithi002.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-03-16T10:33:34.323 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 6/6 2024-03-16T10:33:34.428 INFO:teuthology.orchestra.run.smithi002.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-16T10:33:34.545 INFO:teuthology.orchestra.run.smithi002.stdout:(3/6): ceph-mgr-cephadm-18.2.2-878.ga752c2f7.el 323 kB/s | 138 kB 00:00 2024-03-16T10:33:34.720 INFO:teuthology.orchestra.run.smithi002.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 216 kB/s | 58 kB 00:00 2024-03-16T10:33:34.812 INFO:teuthology.orchestra.run.smithi002.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 191 kB/s | 51 kB 00:00 2024-03-16T10:33:34.837 INFO:teuthology.orchestra.run.smithi002.stdout:(6/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-03-16T10:33:34.840 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:34.840 INFO:teuthology.orchestra.run.smithi002.stdout:Total 7.2 MB/s | 6.8 MB 00:00 2024-03-16T10:33:34.853 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:34.863 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:34.863 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:34.958 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:34.959 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:35.111 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 6/6 2024-03-16T10:33:35.112 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/6 2024-03-16T10:33:35.112 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-16T10:33:35.112 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-16T10:33:35.112 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-16T10:33:35.112 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-16T10:33:35.266 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:35.426 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-16T10:33:35.431 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-16T10:33:35.431 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:35.431 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:35.431 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:35.431 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T10:33:35.432 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T10:33:35.432 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T10:33:35.432 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T10:33:35.432 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T10:33:35.432 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:35.432 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:35.473 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-16T10:33:35.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-fuse 2024-03-16T10:33:36.028 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-16T10:33:36.110 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-16T10:33:36.179 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-16T10:33:36.209 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 6/6 2024-03-16T10:33:36.263 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:23 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:36.418 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 844 k 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T10:33:36.419 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:36.420 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:36.420 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:36.420 INFO:teuthology.orchestra.run.smithi026.stdout:Install 2 Packages 2024-03-16T10:33:36.420 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:36.420 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 922 k 2024-03-16T10:33:36.420 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 2.7 M 2024-03-16T10:33:36.420 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:36.728 INFO:teuthology.orchestra.run.smithi026.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 256 kB/s | 79 kB 00:00 2024-03-16T10:33:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 6/6 2024-03-16T10:33:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 1/6 2024-03-16T10:33:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-16T10:33:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-16T10:33:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-16T10:33:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-16T10:33:37.278 INFO:teuthology.orchestra.run.smithi026.stdout:(2/2): ceph-fuse-18.2.2-878.ga752c2f7.el9.x86_6 984 kB/s | 844 kB 00:00 2024-03-16T10:33:37.279 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:37.279 INFO:teuthology.orchestra.run.smithi026.stdout:Total 1.0 MB/s | 922 kB 00:00 2024-03-16T10:33:37.280 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:37.302 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:37.303 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:18.2.2-878.ga752c2f7.el9.noarch 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:37.370 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:37.383 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:37.383 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:37.569 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:37.642 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-fuse 2024-03-16T10:33:37.705 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-16T10:33:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 2/2 2024-03-16T10:33:38.216 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:25 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-03-16T10:33:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:38.372 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 844 k 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:Install 2 Packages 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 922 k 2024-03-16T10:33:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 2.7 M 2024-03-16T10:33:38.374 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:38.505 INFO:teuthology.orchestra.run.smithi002.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 597 kB/s | 79 kB 00:00 2024-03-16T10:33:38.519 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 2/2 2024-03-16T10:33:38.519 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/2 2024-03-16T10:33:38.830 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-16T10:33:38.830 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:38.830 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:38.830 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-16T10:33:38.830 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:38.830 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:39.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install librados-devel 2024-03-16T10:33:39.389 INFO:teuthology.orchestra.run.smithi002.stdout:(2/2): ceph-fuse-18.2.2-878.ga752c2f7.el9.x86_6 831 kB/s | 844 kB 00:01 2024-03-16T10:33:39.419 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:39.419 INFO:teuthology.orchestra.run.smithi002.stdout:Total 906 kB/s | 922 kB 00:01 2024-03-16T10:33:39.419 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:39.419 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:39.419 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:39.499 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:39.499 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:39.688 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:39.717 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:26 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:39.824 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-16T10:33:39.872 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:39.872 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 127 k 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-03-16T10:33:39.873 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:39.874 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 127 k 2024-03-16T10:33:39.874 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 456 k 2024-03-16T10:33:39.874 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:39.890 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 2/2 2024-03-16T10:33:40.207 INFO:teuthology.orchestra.run.smithi026.stdout:librados-devel-18.2.2-878.ga752c2f7.el9.x86_64. 380 kB/s | 127 kB 00:00 2024-03-16T10:33:40.208 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:40.208 INFO:teuthology.orchestra.run.smithi026.stdout:Total 379 kB/s | 127 kB 00:00 2024-03-16T10:33:40.208 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:40.213 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:40.213 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:40.270 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:40.271 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:40.414 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:40.516 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:40.546 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 2/2 2024-03-16T10:33:40.546 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/2 2024-03-16T10:33:40.885 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-16T10:33:40.886 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:40.886 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:40.886 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-16T10:33:40.886 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:40.886 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:40.928 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:41.159 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install librados-devel 2024-03-16T10:33:41.197 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:41.197 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:41.197 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:41.197 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:41.198 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:41.198 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:41.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install libcephfs2 2024-03-16T10:33:41.737 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:28 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:41.892 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 127 k 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:41.893 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:41.894 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-03-16T10:33:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:41.894 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 127 k 2024-03-16T10:33:41.894 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 456 k 2024-03-16T10:33:41.894 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:42.034 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:29 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:42.135 INFO:teuthology.orchestra.run.smithi026.stdout:Package libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:42.191 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:42.192 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:42.193 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:42.201 INFO:teuthology.orchestra.run.smithi002.stdout:librados-devel-18.2.2-878.ga752c2f7.el9.x86_64. 413 kB/s | 127 kB 00:00 2024-03-16T10:33:42.201 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:42.202 INFO:teuthology.orchestra.run.smithi002.stdout:Total 411 kB/s | 127 kB 00:00 2024-03-16T10:33:42.202 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:42.208 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:42.208 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:42.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install libcephfs-devel 2024-03-16T10:33:42.266 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:42.266 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:42.418 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:42.536 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:42.823 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:29 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:42.966 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:42.978 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:42.979 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-03-16T10:33:42.979 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:42.979 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 31 k 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 31 k 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 139 k 2024-03-16T10:33:42.980 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:43.244 INFO:teuthology.orchestra.run.smithi026.stdout:libcephfs-devel-18.2.2-878.ga752c2f7.el9.x86_64 118 kB/s | 31 kB 00:00 2024-03-16T10:33:43.244 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:43.244 INFO:teuthology.orchestra.run.smithi026.stdout:Total 117 kB/s | 31 kB 00:00 2024-03-16T10:33:43.245 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:43.248 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:43.249 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:43.268 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:43.268 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:43.268 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:43.268 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:43.268 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:43.268 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:43.276 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:43.276 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:43.369 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:43.487 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install libcephfs2 2024-03-16T10:33:43.494 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:43.866 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:44.059 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:31 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:44.144 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:44.144 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:44.144 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:44.144 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:44.144 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:44.144 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:44.160 INFO:teuthology.orchestra.run.smithi002.stdout:Package libcephfs2-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:44.215 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:44.216 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:44.216 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:44.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install libcephfs-devel 2024-03-16T10:33:44.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install librados2 2024-03-16T10:33:44.854 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:31 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:44.900 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:31 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:45.001 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:45.008 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 31 k 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-03-16T10:33:45.009 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:45.010 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 31 k 2024-03-16T10:33:45.010 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 139 k 2024-03-16T10:33:45.010 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:45.057 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:45.058 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:45.058 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:45.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install librbd1 2024-03-16T10:33:45.263 INFO:teuthology.orchestra.run.smithi002.stdout:libcephfs-devel-18.2.2-878.ga752c2f7.el9.x86_64 123 kB/s | 31 kB 00:00 2024-03-16T10:33:45.266 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:45.266 INFO:teuthology.orchestra.run.smithi002.stdout:Total 122 kB/s | 31 kB 00:00 2024-03-16T10:33:45.266 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:45.267 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:45.267 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:45.284 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:45.284 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:45.379 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:45.497 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:45.695 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:32 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:45.795 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:45.850 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:45.851 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:45.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-rados 2024-03-16T10:33:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:46.144 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:46.144 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:46.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install librados2 2024-03-16T10:33:46.488 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:33 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:46.589 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:46.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-rgw 2024-03-16T10:33:46.903 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:33 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:47.000 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:47.055 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:47.057 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:47.057 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:47.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install librbd1 2024-03-16T10:33:47.280 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:34 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:47.381 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:47.437 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:47.438 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:47.438 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:47.500 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-cephfs 2024-03-16T10:33:47.696 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:34 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:47.796 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:47.853 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:47.853 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:47.919 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-rados 2024-03-16T10:33:48.076 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:35 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:48.178 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:48.234 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:48.235 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:48.235 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:48.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install python3-rbd 2024-03-16T10:33:48.491 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:35 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:48.592 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-rados-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:48.647 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:48.648 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:48.648 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:48.741 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-rgw 2024-03-16T10:33:48.871 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:35 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:48.972 INFO:teuthology.orchestra.run.smithi026.stdout:Package python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:49.027 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:49.028 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-03-16T10:33:49.028 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:49.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install rbd-fuse 2024-03-16T10:33:49.310 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:36 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:49.410 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-rgw-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:49.466 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:49.468 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:49.468 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:49.535 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-cephfs 2024-03-16T10:33:49.665 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:36 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:49.824 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:49.824 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:49.824 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 87 k 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 87 k 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 227 k 2024-03-16T10:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:50.108 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:37 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:50.206 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-cephfs-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:50.230 INFO:teuthology.orchestra.run.smithi026.stdout:rbd-fuse-18.2.2-878.ga752c2f7.el9.x86_64.rpm 214 kB/s | 87 kB 00:00 2024-03-16T10:33:50.230 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:50.231 INFO:teuthology.orchestra.run.smithi026.stdout:Total 214 kB/s | 87 kB 00:00 2024-03-16T10:33:50.231 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:50.238 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:50.238 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:50.262 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:50.263 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:50.263 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:50.295 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:50.296 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:50.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-rbd 2024-03-16T10:33:50.445 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:50.548 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:50.900 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:37 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:51.001 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-rbd-2:18.2.2-878.ga752c2f7.el9.x86_64 is already installed. 2024-03-16T10:33:51.057 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:51.058 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-03-16T10:33:51.059 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:51.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install rbd-fuse 2024-03-16T10:33:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:51.203 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:51.203 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:51.431 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install rbd-mirror 2024-03-16T10:33:51.703 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:38 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:51.858 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 87 k 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:51.859 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 87 k 2024-03-16T10:33:51.860 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 227 k 2024-03-16T10:33:51.860 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:52.003 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:39 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:52.161 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:52.161 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repo Size 2024-03-16T10:33:52.161 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 3.0 M 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 3.0 M 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 12 M 2024-03-16T10:33:52.162 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:52.182 INFO:teuthology.orchestra.run.smithi002.stdout:rbd-fuse-18.2.2-878.ga752c2f7.el9.x86_64.rpm 269 kB/s | 87 kB 00:00 2024-03-16T10:33:52.183 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:52.183 INFO:teuthology.orchestra.run.smithi002.stdout:Total 268 kB/s | 87 kB 00:00 2024-03-16T10:33:52.183 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:52.191 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:52.191 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:52.248 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:52.248 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:52.882 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:53.225 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:53.225 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:53.391 INFO:teuthology.orchestra.run.smithi026.stdout:rbd-mirror-18.2.2-878.ga752c2f7.el9.x86_64.rpm 2.4 MB/s | 3.0 MB 00:01 2024-03-16T10:33:53.392 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:53.392 INFO:teuthology.orchestra.run.smithi026.stdout:Total 2.4 MB/s | 3.0 MB 00:01 2024-03-16T10:33:53.392 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:53.403 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:53.403 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:53.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install rbd-mirror 2024-03-16T10:33:53.480 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:53.481 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:53.917 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:54.003 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:54.004 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:54.004 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T10:33:54.004 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T10:33:54.004 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T10:33:54.004 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:54.026 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:41 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:54.184 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:54.185 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-03-16T10:33:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:54.185 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:54.185 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 3.0 M 2024-03-16T10:33:54.185 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-03-16T10:33:54.186 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 3.0 M 2024-03-16T10:33:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 12 M 2024-03-16T10:33:54.186 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:54.810 INFO:teuthology.orchestra.run.smithi002.stdout:rbd-mirror-18.2.2-878.ga752c2f7.el9.x86_64.rpm 4.8 MB/s | 3.0 MB 00:00 2024-03-16T10:33:54.811 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:54.811 INFO:teuthology.orchestra.run.smithi002.stdout:Total 4.8 MB/s | 3.0 MB 00:00 2024-03-16T10:33:54.812 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:54.822 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:54.822 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:54.898 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:54.898 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:55.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:55.168 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:55.168 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:55.168 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:55.169 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:55.169 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:55.328 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:55.386 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:55.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install rbd-nbd 2024-03-16T10:33:55.409 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:55.409 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T10:33:55.409 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T10:33:55.409 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T10:33:55.409 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T10:33:55.410 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:55.981 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:42 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:56.137 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 172 k 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout:Install 1 Package 2024-03-16T10:33:56.138 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:56.139 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 172 k 2024-03-16T10:33:56.139 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 494 k 2024-03-16T10:33:56.139 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-03-16T10:33:56.494 INFO:teuthology.orchestra.run.smithi026.stdout:rbd-nbd-18.2.2-878.ga752c2f7.el9.x86_64.rpm 484 kB/s | 172 kB 00:00 2024-03-16T10:33:56.495 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:56.495 INFO:teuthology.orchestra.run.smithi026.stdout:Total 482 kB/s | 172 kB 00:00 2024-03-16T10:33:56.495 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-03-16T10:33:56.504 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-03-16T10:33:56.504 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-03-16T10:33:56.508 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:56.508 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:56.508 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:56.508 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:56.508 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:56.508 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:56.561 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-03-16T10:33:56.561 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-03-16T10:33:56.707 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-03-16T10:33:56.756 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install rbd-nbd 2024-03-16T10:33:56.803 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:57.273 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:57.335 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:44 ago on Sat 16 Mar 2024 10:32:13 AM UTC. 2024-03-16T10:33:57.492 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:18.2.2-878.ga752c2f7.el9 ceph 172 k 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-03-16T10:33:57.493 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-03-16T10:33:57.494 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-03-16T10:33:57.494 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:57.494 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 172 k 2024-03-16T10:33:57.494 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 494 k 2024-03-16T10:33:57.494 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-03-16T10:33:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:57.584 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-03-16T10:33:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:57.584 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:33:57.584 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-03-16T10:33:57.707 INFO:teuthology.orchestra.run.smithi002.stdout:rbd-nbd-18.2.2-878.ga752c2f7.el9.x86_64.rpm 810 kB/s | 172 kB 00:00 2024-03-16T10:33:57.707 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:33:57.707 INFO:teuthology.orchestra.run.smithi002.stdout:Total 804 kB/s | 172 kB 00:00 2024-03-16T10:33:57.708 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-03-16T10:33:57.716 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-03-16T10:33:57.716 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-03-16T10:33:57.774 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-03-16T10:33:57.775 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-03-16T10:33:57.811 DEBUG:teuthology.parallel:result is None 2024-03-16T10:33:57.927 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-03-16T10:33:58.016 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 1/1 2024-03-16T10:33:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:58.749 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-03-16T10:33:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:18.2.2-878.ga752c2f7.el9.x86_64 2024-03-16T10:33:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:33:58.750 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-03-16T10:33:59.002 DEBUG:teuthology.parallel:result is None 2024-03-16T10:33:59.002 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T10:33:59.002 INFO:teuthology.packaging:ref: None 2024-03-16T10:33:59.002 INFO:teuthology.packaging:tag: None 2024-03-16T10:33:59.002 INFO:teuthology.packaging:branch: reef 2024-03-16T10:33:59.002 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:33:59.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-16T10:33:59.337 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T10:33:59.358 INFO:teuthology.orchestra.run.smithi002.stdout:18.2.2-878.ga752c2f7.el9 2024-03-16T10:33:59.358 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-878.ga752c2f7.el9 2024-03-16T10:33:59.358 INFO:teuthology.task.install:The correct ceph version 18.2.2-878.ga752c2f7 is installed. 2024-03-16T10:33:59.360 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T10:33:59.360 INFO:teuthology.packaging:ref: None 2024-03-16T10:33:59.360 INFO:teuthology.packaging:tag: None 2024-03-16T10:33:59.360 INFO:teuthology.packaging:branch: reef 2024-03-16T10:33:59.360 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:33:59.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-03-16T10:33:59.704 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T10:33:59.729 INFO:teuthology.orchestra.run.smithi026.stdout:18.2.2-878.ga752c2f7.el9 2024-03-16T10:33:59.729 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-878.ga752c2f7.el9 2024-03-16T10:33:59.729 INFO:teuthology.task.install:The correct ceph version 18.2.2-878.ga752c2f7 is installed. 2024-03-16T10:33:59.731 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T10:33:59.731 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:33:59.731 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T10:33:59.758 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:33:59.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T10:33:59.800 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T10:33:59.801 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:33:59.801 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T10:33:59.827 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T10:33:59.894 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:33:59.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T10:33:59.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T10:33:59.995 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T10:33:59.995 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:33:59.996 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T10:34:00.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T10:34:00.089 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:34:00.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T10:34:00.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T10:34:00.188 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T10:34:00.189 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:34:00.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T10:34:00.215 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T10:34:00.282 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:34:00.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T10:34:00.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T10:34:00.383 INFO:teuthology.run_tasks:Running task print... 2024-03-16T10:34:00.395 INFO:teuthology.task.print:**** done install task... 2024-03-16T10:34:00.395 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-16T10:34:00.495 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T10:34:00.495 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-03-16T10:34:00.496 INFO:tasks.cephadm:Cluster fsid is b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:00.496 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-16T10:34:00.496 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-16T10:34:00.496 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi002': '172.21.15.2', 'mon.smithi026': '172.21.15.26'} 2024-03-16T10:34:00.496 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-16T10:34:00.497 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname $(hostname -s) 2024-03-16T10:34:00.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2024-03-16T10:34:00.554 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-03-16T10:34:00.555 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:34:00.802 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77658/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2162-g3d78f5ba', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2162.g3d78f5ba'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-15 01:33:04.210510', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240314.183243', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-16T10:34:01.143 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 a752c2f7898f53056d3000f2350a5865d1d67887 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-03-16T10:34:01.144 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/x86_64/flavors/default/cephadm 2024-03-16T10:34:01.145 DEBUG:teuthology.orchestra.run.smithi002:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T10:34:01.789 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-r--r--. 1 ubuntu ubuntu 215083 Mar 16 10:34 /home/ubuntu/cephtest/cephadm 2024-03-16T10:34:01.790 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/a752c2f7898f53056d3000f2350a5865d1d67887/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T10:34:02.427 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r--. 1 ubuntu ubuntu 215083 Mar 16 10:34 /home/ubuntu/cephtest/cephadm 2024-03-16T10:34:02.427 DEBUG:teuthology.orchestra.run.smithi002:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T10:34:02.445 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T10:34:02.475 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-03-16T10:34:02.475 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-03-16T10:34:02.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-03-16T10:34:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-16T10:34:03.426 INFO:teuthology.orchestra.run.smithi002.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-16T10:34:28.531 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-16T10:34:28.531 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)", 2024-03-16T10:34:28.531 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "f692adb9a2659e2d754990b1827eb3b2c942ca2ee277bfd6dd253d312951c845", 2024-03-16T10:34:28.532 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2024-03-16T10:34:28.532 INFO:teuthology.orchestra.run.smithi026.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704" 2024-03-16T10:34:28.532 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-03-16T10:34:28.532 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-16T10:34:32.536 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:34:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)", 2024-03-16T10:34:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "image_id": "f692adb9a2659e2d754990b1827eb3b2c942ca2ee277bfd6dd253d312951c845", 2024-03-16T10:34:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "repo_digests": [ 2024-03-16T10:34:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704" 2024-03-16T10:34:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-03-16T10:34:32.536 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:34:32.555 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph 2024-03-16T10:34:32.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2024-03-16T10:34:32.651 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /etc/ceph 2024-03-16T10:34:32.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2024-03-16T10:34:32.723 INFO:tasks.cephadm:Writing seed config... 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-16T10:34:32.724 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-16T10:34:32.725 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-16T10:34:32.726 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-16T10:34:32.727 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:34:32.727 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-16T10:34:32.742 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b3aed5e6-e380-11ee-95c9-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-16T10:34:32.742 DEBUG:teuthology.orchestra.run.smithi002:mon.smithi002> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service 2024-03-16T10:34:32.785 INFO:tasks.cephadm:Bootstrapping... 2024-03-16T10:34:32.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.2 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:34:32.944 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-03-16T10:34:32.944 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', 'b3aed5e6-e380-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.2', '--skip-admin-label'] 2024-03-16T10:34:32.969 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T10:34:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-16T10:34:32.970 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying podman|docker is present... 2024-03-16T10:34:32.994 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T10:34:32.994 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying lvm2 is present... 2024-03-16T10:34:32.994 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying time synchronization is in place... 2024-03-16T10:34:33.001 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T10:34:33.001 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T10:34:33.007 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T10:34:33.007 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-16T10:34:33.013 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-03-16T10:34:33.019 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-03-16T10:34:33.019 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-03-16T10:34:33.019 INFO:teuthology.orchestra.run.smithi002.stdout:Repeating the final host check... 2024-03-16T10:34:33.042 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T10:34:33.042 INFO:teuthology.orchestra.run.smithi002.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-16T10:34:33.042 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl is present 2024-03-16T10:34:33.042 INFO:teuthology.orchestra.run.smithi002.stdout:lvcreate is present 2024-03-16T10:34:33.048 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T10:34:33.048 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T10:34:33.054 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T10:34:33.054 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-16T10:34:33.060 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-03-16T10:34:33.066 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-03-16T10:34:33.066 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-03-16T10:34:33.066 INFO:teuthology.orchestra.run.smithi002.stdout:Host looks OK 2024-03-16T10:34:33.066 INFO:teuthology.orchestra.run.smithi002.stdout:Cluster fsid: b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:33.066 INFO:teuthology.orchestra.run.smithi002.stdout:Acquiring lock 140308566530752 on /run/cephadm/b3aed5e6-e380-11ee-95c9-87774f69a715.lock 2024-03-16T10:34:33.066 INFO:teuthology.orchestra.run.smithi002.stdout:Lock 140308566530752 acquired on /run/cephadm/b3aed5e6-e380-11ee-95c9-87774f69a715.lock 2024-03-16T10:34:33.067 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 3300 ... 2024-03-16T10:34:33.067 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 6789 ... 2024-03-16T10:34:33.067 INFO:teuthology.orchestra.run.smithi002.stdout:Base mon IP(s) is [172.21.15.2:3300, 172.21.15.2:6789], mon addrv is [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-03-16T10:34:33.070 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.2 metric 100 2024-03-16T10:34:33.070 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.2 metric 100 2024-03-16T10:34:33.072 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-16T10:34:33.073 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-16T10:34:33.073 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-16T10:34:33.075 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-16T10:34:33.075 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-16T10:34:33.075 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T10:34:33.075 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-16T10:34:33.075 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-03-16T10:34:33.075 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T10:34:33.076 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-03-16T10:34:33.076 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-03-16T10:34:33.076 INFO:teuthology.orchestra.run.smithi002.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-16T10:34:33.076 INFO:teuthology.orchestra.run.smithi002.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-16T10:34:33.077 INFO:teuthology.orchestra.run.smithi002.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-03-16T10:34:33.514 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout f692adb9a2659e2d754990b1827eb3b2c942ca2ee277bfd6dd253d312951c845 2024-03-16T10:34:33.514 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-03-16T10:34:33.514 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Getting image source signatures 2024-03-16T10:34:33.514 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:e3738fc78f1ad519f77fc92a77487f7a64f201502cabe3fb88b8b71a4e507d59 2024-03-16T10:34:33.514 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-16T10:34:33.514 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying config sha256:f692adb9a2659e2d754990b1827eb3b2c942ca2ee277bfd6dd253d312951c845 2024-03-16T10:34:33.514 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-16T10:34:34.401 INFO:teuthology.orchestra.run.smithi002.stdout:ceph: stdout ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable) 2024-03-16T10:34:34.402 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph version: ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable) 2024-03-16T10:34:34.402 INFO:teuthology.orchestra.run.smithi002.stdout:Extracting ceph user uid/gid from container image... 2024-03-16T10:34:35.232 INFO:teuthology.orchestra.run.smithi002.stdout:stat: stdout 167 167 2024-03-16T10:34:35.233 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial keys... 2024-03-16T10:34:36.238 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQC7dfVlO//UHxAAPX1VzonuX/1oGYiCctGOWA== 2024-03-16T10:34:37.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQC8dfVleNUYIhAA4a8W7l2qJS3U60Mkym//ng== 2024-03-16T10:34:38.041 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQC9dfVlsPgAGhAAcaYpmIUsttTNCECckMZ/jQ== 2024-03-16T10:34:38.041 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial monmap... 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool for smithi002 [v2:172.21.15.2:3300,v1:172.21.15.2:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = pacific 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: set fsid to b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:34:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mon... 2024-03-16T10:34:39.845 INFO:teuthology.orchestra.run.smithi002.stdout:create mon.smithi002 on 2024-03-16T10:34:40.050 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-16T10:34:40.203 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-16T10:34:40.374 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715.target. 2024-03-16T10:34:40.374 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715.target. 2024-03-16T10:34:40.632 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002 2024-03-16T10:34:40.632 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service: Unit ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service not loaded. 2024-03-16T10:34:40.807 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715.target.wants/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service → /etc/systemd/system/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@.service. 2024-03-16T10:34:41.326 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T10:34:41.326 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-16T10:34:41.332 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T10:34:41.332 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-16T10:34:41.333 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-16T10:34:41.333 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T10:34:41.333 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon to start... 2024-03-16T10:34:41.333 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon... 2024-03-16T10:34:42.403 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:42 smithi002 ceph-mon[28877]: from='client.? 172.21.15.2:0/4162100778' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-16T10:34:42.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout cluster: 2024-03-16T10:34:42.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout id: b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout services: 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi002 (age 0.518302s) 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout data: 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pgs: 2024-03-16T10:34:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.739+0000 7f4b15cb5700 1 Processor -- start 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.739+0000 7f4b15cb5700 1 -- start start 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.739+0000 7f4b15cb5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10109240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.739+0000 7f4b15cb5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b10079da0 con 0x7f4b10106e50 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.739+0000 7f4b0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.740+0000 7f4b0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50238/0 (socket says 172.21.15.2:50238) 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.740+0000 7f4b0f7fe700 1 -- 172.21.15.2:0/3993290522 learned_addr learned my addr 172.21.15.2:0/3993290522 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.740+0000 7f4b0f7fe700 1 -- 172.21.15.2:0/3993290522 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10109780 con 0x7f4b10106e50 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.740+0000 7f4b0f7fe700 1 --2- 172.21.15.2:0/3993290522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10109240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4b000098a0 tx=0x7f4b00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8671a457042e91d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.740+0000 7f4b0e7fc700 1 -- 172.21.15.2:0/3993290522 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b00004180 con 0x7f4b10106e50 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.740+0000 7f4b0e7fc700 1 -- 172.21.15.2:0/3993290522 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000042e0 con 0x7f4b10106e50 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.740+0000 7f4b0e7fc700 1 -- 172.21.15.2:0/3993290522 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b000036a0 con 0x7f4b10106e50 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 -- 172.21.15.2:0/3993290522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 msgr2=0x7f4b10109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 --2- 172.21.15.2:0/3993290522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10109240 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4b000098a0 tx=0x7f4b00009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 -- 172.21.15.2:0/3993290522 shutdown_connections 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 --2- 172.21.15.2:0/3993290522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10109240 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 -- 172.21.15.2:0/3993290522 >> 172.21.15.2:0/3993290522 conn(0x7f4b10100820 msgr2=0x7f4b10102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 -- 172.21.15.2:0/3993290522 shutdown_connections 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 -- 172.21.15.2:0/3993290522 wait complete. 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.741+0000 7f4b15cb5700 1 Processor -- start 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b15cb5700 1 -- start start 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b15cb5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b15cb5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b10199d90 con 0x7f4b10106e50 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50244/0 (socket says 172.21.15.2:50244) 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0f7fe700 1 -- 172.21.15.2:0/4162100778 learned_addr learned my addr 172.21.15.2:0/4162100778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:42.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0f7fe700 1 -- 172.21.15.2:0/4162100778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00009550 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0f7fe700 1 --2- 172.21.15.2:0/4162100778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10199850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b100791e0 tx=0x7f4b00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0cff9700 1 -- 172.21.15.2:0/4162100778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b00003ce0 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0cff9700 1 -- 172.21.15.2:0/4162100778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b00020070 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b10199f90 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.742+0000 7f4b0cff9700 1 -- 172.21.15.2:0/4162100778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b00011630 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.743+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1019a430 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.743+0000 7f4b0cff9700 1 -- 172.21.15.2:0/4162100778 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4b00022030 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.743+0000 7f4b0cff9700 1 -- 172.21.15.2:0/4162100778 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4b00025d40 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.744+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b10193660 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.746+0000 7f4b0cff9700 1 -- 172.21.15.2:0/4162100778 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f4b00025480 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.819+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4b1002cf10 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.819+0000 7f4b0cff9700 1 -- 172.21.15.2:0/4162100778 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f4b00011c20 con 0x7f4b10106e50 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.820+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 msgr2=0x7f4b10199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:42.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.820+0000 7f4b15cb5700 1 --2- 172.21.15.2:0/4162100778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10199850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b100791e0 tx=0x7f4b00004750 comp rx=0 tx=0).stop 2024-03-16T10:34:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.820+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 shutdown_connections 2024-03-16T10:34:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.820+0000 7f4b15cb5700 1 --2- 172.21.15.2:0/4162100778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b10106e50 0x7f4b10199850 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.820+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 >> 172.21.15.2:0/4162100778 conn(0x7f4b10100820 msgr2=0x7f4b10102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.820+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 shutdown_connections 2024-03-16T10:34:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:41.820+0000 7f4b15cb5700 1 -- 172.21.15.2:0/4162100778 wait complete. 2024-03-16T10:34:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:mon is available 2024-03-16T10:34:42.416 INFO:teuthology.orchestra.run.smithi002.stdout:Assimilating anything we can from ceph.conf... 2024-03-16T10:34:43.501 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:43 smithi002 ceph-mon[28877]: from='client.? 172.21.15.2:0/3235017248' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T10:34:43.501 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:43 smithi002 ceph-mon[28877]: from='client.? 172.21.15.2:0/3235017248' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T10:34:43.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.840+0000 7fe603aa8700 1 Processor -- start 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.840+0000 7fe603aa8700 1 -- start start 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.840+0000 7fe603aa8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc105680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.840+0000 7fe603aa8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc105c50 con 0x7fe5fc105260 2024-03-16T10:34:43.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe601844700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc105680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe601844700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc105680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39038/0 (socket says 172.21.15.2:39038) 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe601844700 1 -- 172.21.15.2:0/1102434102 learned_addr learned my addr 172.21.15.2:0/1102434102 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe601844700 1 -- 172.21.15.2:0/1102434102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc106460 con 0x7fe5fc105260 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe601844700 1 --2- 172.21.15.2:0/1102434102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc105680 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40098a0 tx=0x7fe5f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91837b7d58354b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe600842700 1 -- 172.21.15.2:0/1102434102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5f4004180 con 0x7fe5fc105260 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe600842700 1 -- 172.21.15.2:0/1102434102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f40042e0 con 0x7fe5fc105260 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.841+0000 7fe600842700 1 -- 172.21.15.2:0/1102434102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5f40036a0 con 0x7fe5fc105260 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 -- 172.21.15.2:0/1102434102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 msgr2=0x7fe5fc105680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 --2- 172.21.15.2:0/1102434102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc105680 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40098a0 tx=0x7fe5f4009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 -- 172.21.15.2:0/1102434102 shutdown_connections 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 --2- 172.21.15.2:0/1102434102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc105680 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 -- 172.21.15.2:0/1102434102 >> 172.21.15.2:0/1102434102 conn(0x7fe5fc100820 msgr2=0x7fe5fc102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 -- 172.21.15.2:0/1102434102 shutdown_connections 2024-03-16T10:34:43.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 -- 172.21.15.2:0/1102434102 wait complete. 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 Processor -- start 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 -- start start 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.842+0000 7fe603aa8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc079ed0 con 0x7fe5fc105260 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe601844700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe601844700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc079990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39048/0 (socket says 172.21.15.2:39048) 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe601844700 1 -- 172.21.15.2:0/3235017248 learned_addr learned my addr 172.21.15.2:0/3235017248 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe601844700 1 -- 172.21.15.2:0/3235017248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4009550 con 0x7fe5fc105260 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe601844700 1 --2- 172.21.15.2:0/3235017248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc079990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4004000 tx=0x7fe5f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5f4003ce0 con 0x7fe5fc105260 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f4020070 con 0x7fe5fc105260 2024-03-16T10:34:43.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc19c570 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5f4011630 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc19ca10 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.843+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe5f4022030 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.844+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe5f4025d40 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.844+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc192470 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.846+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fe5f4025480 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.916+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe5fc02c300 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.919+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 7 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe5f4011bd0 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.919+0000 7fe5faffd700 1 -- 172.21.15.2:0/3235017248 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fe5f4036020 con 0x7fe5fc105260 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.920+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 msgr2=0x7fe5fc079990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.920+0000 7fe603aa8700 1 --2- 172.21.15.2:0/3235017248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc079990 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4004000 tx=0x7fe5f4004750 comp rx=0 tx=0).stop 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.920+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 shutdown_connections 2024-03-16T10:34:43.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.920+0000 7fe603aa8700 1 --2- 172.21.15.2:0/3235017248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5fc105260 0x7fe5fc079990 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:43.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.920+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 >> 172.21.15.2:0/3235017248 conn(0x7fe5fc100820 msgr2=0x7fe5fc18f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:43.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.920+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 shutdown_connections 2024-03-16T10:34:43.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:42.920+0000 7fe603aa8700 1 -- 172.21.15.2:0/3235017248 wait complete. 2024-03-16T10:34:43.516 INFO:teuthology.orchestra.run.smithi002.stdout:Generating new minimal ceph.conf... 2024-03-16T10:34:44.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:44 smithi002 ceph-mon[28877]: from='client.? 172.21.15.2:0/55720485' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.885+0000 7f21ae20f700 1 Processor -- start 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21ae20f700 1 -- start start 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21ae20f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a8106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21ae20f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a8106d40 con 0x7f21a8106350 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a8106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a8106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39060/0 (socket says 172.21.15.2:39060) 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a77fe700 1 -- 172.21.15.2:0/1646986915 learned_addr learned my addr 172.21.15.2:0/1646986915 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a77fe700 1 -- 172.21.15.2:0/1646986915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8107550 con 0x7f21a8106350 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a77fe700 1 --2- 172.21.15.2:0/1646986915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a8106770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21980098a0 tx=0x7f2198009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfde1ab21e180a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a67fc700 1 -- 172.21.15.2:0/1646986915 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2198004180 con 0x7f21a8106350 2024-03-16T10:34:44.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a67fc700 1 -- 172.21.15.2:0/1646986915 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f21980042e0 con 0x7f21a8106350 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.886+0000 7f21a67fc700 1 -- 172.21.15.2:0/1646986915 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2198011a70 con 0x7f21a8106350 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.887+0000 7f21ae20f700 1 -- 172.21.15.2:0/1646986915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 msgr2=0x7f21a8106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.887+0000 7f21ae20f700 1 --2- 172.21.15.2:0/1646986915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a8106770 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21980098a0 tx=0x7f2198009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.887+0000 7f21ae20f700 1 -- 172.21.15.2:0/1646986915 shutdown_connections 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.887+0000 7f21ae20f700 1 --2- 172.21.15.2:0/1646986915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a8106770 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.887+0000 7f21ae20f700 1 -- 172.21.15.2:0/1646986915 >> 172.21.15.2:0/1646986915 conn(0x7f21a81018f0 msgr2=0x7f21a8103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.887+0000 7f21ae20f700 1 -- 172.21.15.2:0/1646986915 shutdown_connections 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.887+0000 7f21ae20f700 1 -- 172.21.15.2:0/1646986915 wait complete. 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21ae20f700 1 Processor -- start 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21ae20f700 1 -- start start 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21ae20f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21ae20f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a819a600 con 0x7f21a8106350 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:44.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21a77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39070/0 (socket says 172.21.15.2:39070) 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21a77fe700 1 -- 172.21.15.2:0/55720485 learned_addr learned my addr 172.21.15.2:0/55720485 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.888+0000 7f21a77fe700 1 -- 172.21.15.2:0/55720485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2198009550 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21a77fe700 1 --2- 172.21.15.2:0/55720485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a819a0c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2198011710 tx=0x7f21980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21a4ff9700 1 -- 172.21.15.2:0/55720485 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2198003b10 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21a4ff9700 1 -- 172.21.15.2:0/55720485 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f219801b8c0 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a819a800 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21a4ff9700 1 -- 172.21.15.2:0/55720485 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2198019710 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a819aca0 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21a4ff9700 1 -- 172.21.15.2:0/55720485 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2198019bc0 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.889+0000 7f21a4ff9700 1 -- 172.21.15.2:0/55720485 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f219801a710 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.890+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a80623e0 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.892+0000 7f21a4ff9700 1 -- 172.21.15.2:0/55720485 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f219801ba30 con 0x7f21a8106350 2024-03-16T10:34:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.961+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f21a810a780 con 0x7f21a8106350 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.962+0000 7f21a4ff9700 1 -- 172.21.15.2:0/55720485 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f219801bbd0 con 0x7f21a8106350 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.962+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 msgr2=0x7f21a819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.962+0000 7f21ae20f700 1 --2- 172.21.15.2:0/55720485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a819a0c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2198011710 tx=0x7f21980047c0 comp rx=0 tx=0).stop 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.962+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 shutdown_connections 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.962+0000 7f21ae20f700 1 --2- 172.21.15.2:0/55720485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21a8106350 0x7f21a819a0c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.962+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 >> 172.21.15.2:0/55720485 conn(0x7f21a81018f0 msgr2=0x7f21a81024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.963+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 shutdown_connections 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:43.963+0000 7f21ae20f700 1 -- 172.21.15.2:0/55720485 wait complete. 2024-03-16T10:34:44.568 INFO:teuthology.orchestra.run.smithi002.stdout:Restarting the monitor... 2024-03-16T10:34:44.836 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:44 smithi002 systemd[1]: Stopping Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:34:45.219 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:44 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[28873]: 2024-03-16T10:34:44.835+0000 7fdb42228700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi002 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:34:45.220 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:44 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[28873]: 2024-03-16T10:34:44.835+0000 7fdb42228700 -1 mon.smithi002@0(leader) e1 *** Got Signal Terminated *** 2024-03-16T10:34:45.220 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:44 smithi002 podman[29123]: 2024-03-16 10:34:44.94851694 +0000 UTC m=+0.224451412 container died 8fc38362f202b853c14689b0bb5d2766f474860678c807db5c396f6d62997351 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, org.label-schema.license=GPLv2, RELEASE=reef-a752c2f, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-16T10:34:45.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 podman[29123]: 2024-03-16 10:34:45.318538805 +0000 UTC m=+0.594473293 container cleanup 8fc38362f202b853c14689b0bb5d2766f474860678c807db5c396f6d62997351 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=reef-a752c2f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:34:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 bash[29123]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002 2024-03-16T10:34:45.960 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 podman[29137]: 2024-03-16 10:34:45.573385631 +0000 UTC m=+0.621939142 container remove 8fc38362f202b853c14689b0bb5d2766f474860678c807db5c396f6d62997351 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=reef-a752c2f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-03-16T10:34:45.960 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service: Deactivated successfully. 2024-03-16T10:34:45.960 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 systemd[1]: Stopped Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:34:45.961 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 systemd[1]: Starting Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:34:46.248 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 podman[29228]: 2024-03-16 10:34:45.862795657 +0000 UTC m=+0.018235649 image pull f692adb9a2659e2d754990b1827eb3b2c942ca2ee277bfd6dd253d312951c845 quay.ceph.io/ceph-ci/ceph:reef 2024-03-16T10:34:46.248 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:45 smithi002 podman[29228]: 2024-03-16 10:34:45.961914395 +0000 UTC m=+0.117354370 container create 3e28a22ab682e8d4a3b13e1d1f76800a8e811c140725e40a15d1aea73f6e1183 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-a752c2f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:34:46.248 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 podman[29228]: 2024-03-16 10:34:46.164233528 +0000 UTC m=+0.319673506 container init 3e28a22ab682e8d4a3b13e1d1f76800a8e811c140725e40a15d1aea73f6e1183 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=reef-a752c2f, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-16T10:34:46.248 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 podman[29228]: 2024-03-16 10:34:46.167310926 +0000 UTC m=+0.322750902 container start 3e28a22ab682e8d4a3b13e1d1f76800a8e811c140725e40a15d1aea73f6e1183 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=reef-a752c2f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T10:34:46.248 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable), process ceph-mon, pid 2 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: pidfile_write: ignore empty --pid-file 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: load: jerasure load: lrc 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: RocksDB version: 7.9.2 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Git sha 0 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Compile date 2024-03-15 17:44:22 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: DB SUMMARY 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: DB Session ID: SZ2PSBR13MAD1I8HOW6T 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: CURRENT file: CURRENT 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi002/store.db dir, Total Num: 1, files: 000008.sst 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi002/store.db: 000009.log size: 97558 ; 2024-03-16T10:34:46.249 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.error_if_exists: 0 2024-03-16T10:34:46.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.create_if_missing: 0 2024-03-16T10:34:46.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.paranoid_checks: 1 2024-03-16T10:34:46.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T10:34:46.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T10:34:46.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.env: 0x55c02cef1720 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.info_log: 0x55c030425380 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.statistics: (nil) 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.use_fsync: 0 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_log_file_size: 0 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.allow_fallocate: 1 2024-03-16T10:34:46.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.use_direct_reads: 0 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.db_log_dir: 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.wal_dir: 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T10:34:46.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.write_buffer_manager: 0x55c02f69e140 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.unordered_write: 0 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T10:34:46.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.row_cache: None 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.wal_filter: None 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.two_write_queues: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.wal_compression: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.atomic_flush: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.log_readahead_size: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T10:34:46.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_background_jobs: 2 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_background_compactions: -1 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_subcompactions: 1 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T10:34:46.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_open_files: -1 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_background_flushes: -1 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Compression algorithms supported: 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kZSTD supported: 0 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kXpressCompression supported: 0 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kZlibCompression supported: 1 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kSnappyCompression supported: 1 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kLZ4Compression supported: 1 2024-03-16T10:34:46.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: kBZip2Compression supported: 0 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi002/store.db/MANIFEST-000010 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.merge_operator: 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_filter: None 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c0304254a0) 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: cache_index_and_filter_blocks: 1 2024-03-16T10:34:46.257 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: pin_top_level_index_and_filter: 1 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: index_type: 0 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: data_block_index_type: 0 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: index_shortening: 1 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: checksum: 4 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: no_block_cache: 0 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache: 0x55c02f73f1f0 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache_name: BinnedLRUCache 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache_options: 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: capacity : 536870912 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: num_shard_bits : 4 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: strict_capacity_limit : 0 2024-03-16T10:34:46.258 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: high_pri_pool_ratio: 0.000 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache_compressed: (nil) 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: persistent_cache: (nil) 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_size: 4096 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_size_deviation: 10 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_restart_interval: 16 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: index_block_restart_interval: 1 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: metadata_block_size: 4096 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: partition_filters: 0 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: use_delta_encoding: 1 2024-03-16T10:34:46.259 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: filter_policy: bloomfilter 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: whole_key_filtering: 1 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: verify_compression: 0 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: read_amp_bytes_per_bit: 0 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: format_version: 5 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: enable_index_compression: 1 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_align: 0 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: max_auto_readahead_size: 262144 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: prepopulate_block_cache: 0 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: initial_auto_readahead_size: 8192 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression: NoCompression 2024-03-16T10:34:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.num_levels: 7 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T10:34:46.261 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T10:34:46.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T10:34:46.262 INFO:teuthology.orchestra.run.smithi002.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-16T10:34:46.263 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T10:34:46.263 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T10:34:46.263 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T10:34:46.263 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T10:34:46.263 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T10:34:46.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.table_properties_collectors: 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.inplace_update_support: 0 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T10:34:46.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.bloom_locality: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.max_successive_merges: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.ttl: 2592000 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.enable_blob_files: false 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.min_blob_size: 0 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T10:34:46.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi002/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4c5de884-776c-4b23-b701-ba39ad44b2cb 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585286197757, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585286198739, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 90925, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83640, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710585286, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4c5de884-776c-4b23-b701-ba39ad44b2cb", "db_session_id": "SZ2PSBR13MAD1I8HOW6T", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585286198821, "job": 1, "event": "recovery_finished"} 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-16T10:34:46.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi002/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c02f7dc000 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: DB pointer 0x55c02f7ca000 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** DB Stats ** 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** Compaction Stats [default] ** 2024-03-16T10:34:46.268 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: L0 2/0 92.60 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Sum 2/0 92.60 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 142.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** Compaction Stats [default] ** 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 142.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T10:34:46.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative compaction: 0.00 GB write, 30.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval compaction: 0.00 GB write, 30.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Block cache BinnedLRUCache@0x55c02f73f1f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: starting mon.smithi002 rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi002 fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???) e1 preinit fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???).mds e1 new map 2024-03-16T10:34:46.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???).mds e1 print_map 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: e1 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: legacy client fscid: -1 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: No filesystems configured 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 bash[29228]: 3e28a22ab682e8d4a3b13e1d1f76800a8e811c140725e40a15d1aea73f6e1183 2024-03-16T10:34:46.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 systemd[1]: Started Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:34:46.525 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mon.smithi002 is new leader, mons smithi002 in quorum (ranks 0) 2024-03-16T10:34:46.525 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: monmap e1: 1 mons at {smithi002=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-16T10:34:46.525 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: fsmap 2024-03-16T10:34:46.525 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T10:34:46.525 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:46 smithi002 ceph-mon[29248]: mgrmap e1: no daemons active 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.739+0000 7f4333a17700 1 Processor -- start 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.740+0000 7f4333a17700 1 -- start start 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.740+0000 7f4333a17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.740+0000 7f4333a17700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c106d40 con 0x7f432c106350 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.740+0000 7f43317b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.740+0000 7f43317b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39080/0 (socket says 172.21.15.2:39080) 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.740+0000 7f43317b3700 1 -- 172.21.15.2:0/2006309671 learned_addr learned my addr 172.21.15.2:0/2006309671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.740+0000 7f43317b3700 1 -- 172.21.15.2:0/2006309671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c107550 con 0x7f432c106350 2024-03-16T10:34:47.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f43317b3700 1 --2- 172.21.15.2:0/2006309671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c106770 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43280098a0 tx=0x7f4328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd47252e1eb3bd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4323fff700 1 -- 172.21.15.2:0/2006309671 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4328004180 con 0x7f432c106350 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4323fff700 1 -- 172.21.15.2:0/2006309671 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f43280042e0 con 0x7f432c106350 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4323fff700 1 -- 172.21.15.2:0/2006309671 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4328011a70 con 0x7f432c106350 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4333a17700 1 -- 172.21.15.2:0/2006309671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 msgr2=0x7f432c106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4333a17700 1 --2- 172.21.15.2:0/2006309671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c106770 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43280098a0 tx=0x7f4328009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4333a17700 1 -- 172.21.15.2:0/2006309671 shutdown_connections 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4333a17700 1 --2- 172.21.15.2:0/2006309671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c106770 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.741+0000 7f4333a17700 1 -- 172.21.15.2:0/2006309671 >> 172.21.15.2:0/2006309671 conn(0x7f432c101910 msgr2=0x7f432c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f4333a17700 1 -- 172.21.15.2:0/2006309671 shutdown_connections 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f4333a17700 1 -- 172.21.15.2:0/2006309671 wait complete. 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f4333a17700 1 Processor -- start 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f4333a17700 1 -- start start 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f4333a17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:47.306 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f4333a17700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c19a5e0 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f43317b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f43317b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39086/0 (socket says 172.21.15.2:39086) 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.742+0000 7f43317b3700 1 -- 172.21.15.2:0/177419654 learned_addr learned my addr 172.21.15.2:0/177419654 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f43317b3700 1 -- 172.21.15.2:0/177419654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328009550 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f43317b3700 1 --2- 172.21.15.2:0/177419654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c19a0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4328011710 tx=0x7f43280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f43227fc700 1 -- 172.21.15.2:0/177419654 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4328003b10 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f43227fc700 1 -- 172.21.15.2:0/177419654 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f432801b8c0 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c19a7e0 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f43227fc700 1 -- 172.21.15.2:0/177419654 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4328019710 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c19ac80 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f43227fc700 1 -- 172.21.15.2:0/177419654 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4328019bc0 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.743+0000 7f43227fc700 1 -- 172.21.15.2:0/177419654 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f432801a710 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.744+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c0623e0 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.746+0000 7f43227fc700 1 -- 172.21.15.2:0/177419654 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f432801ba30 con 0x7f432c106350 2024-03-16T10:34:47.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.815+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f432c19d930 con 0x7f432c106350 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.817+0000 7f43227fc700 1 -- 172.21.15.2:0/177419654 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f43280300c0 con 0x7f432c106350 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.818+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 msgr2=0x7f432c19a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.818+0000 7f4333a17700 1 --2- 172.21.15.2:0/177419654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c19a0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4328011710 tx=0x7f43280047c0 comp rx=0 tx=0).stop 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.818+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 shutdown_connections 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.818+0000 7f4333a17700 1 --2- 172.21.15.2:0/177419654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f432c106350 0x7f432c19a0a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.818+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 >> 172.21.15.2:0/177419654 conn(0x7f432c101910 msgr2=0x7f432c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.818+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 shutdown_connections 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:46.818+0000 7f4333a17700 1 -- 172.21.15.2:0/177419654 wait complete. 2024-03-16T10:34:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-16T10:34:47.330 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:34:47.330 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mgr... 2024-03-16T10:34:47.331 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-16T10:34:47.331 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-16T10:34:47.331 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:8443 ... 2024-03-16T10:34:47.580 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mgr.smithi002.zevrtm 2024-03-16T10:34:47.580 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mgr.smithi002.zevrtm.service: Unit ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mgr.smithi002.zevrtm.service not loaded. 2024-03-16T10:34:47.726 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715.target.wants/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mgr.smithi002.zevrtm.service → /etc/systemd/system/ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@.service. 2024-03-16T10:34:48.045 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:47 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/177419654' entity='client.admin' 2024-03-16T10:34:48.216 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T10:34:48.216 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-16T10:34:48.223 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T10:34:48.223 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-16T10:34:48.223 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-16T10:34:48.223 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T10:34:48.231 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T10:34:48.232 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-16T10:34:48.239 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T10:34:48.239 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-16T10:34:48.239 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-16T10:34:48.239 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-03-16T10:34:48.239 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr to start... 2024-03-16T10:34:48.239 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr... 2024-03-16T10:34:49.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:48 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2125804465' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T10:34:49.246 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:49.246 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "b3aed5e6-e380-11ee-95c9-87774f69a715", 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T10:34:49.247 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "smithi002" 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:49.248 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T10:34:49.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T10:34:49.249 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T10:34:49.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T10:34:49.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T10:34:49.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T10:34:49.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:49.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T10:34:49.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T10:34:49.257 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T10:34:49.258 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T10:34:41.303283+0000", 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T10:34:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.638+0000 7fe8b76a3700 1 Processor -- start 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.639+0000 7fe8b76a3700 1 -- start start 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.639+0000 7fe8b76a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b007a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.639+0000 7fe8b76a3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b007a840 con 0x7fe8b007be10 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.639+0000 7fe8b543f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.639+0000 7fe8b543f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b007a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39112/0 (socket says 172.21.15.2:39112) 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.639+0000 7fe8b543f700 1 -- 172.21.15.2:0/1022070908 learned_addr learned my addr 172.21.15.2:0/1022070908 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.639+0000 7fe8b543f700 1 -- 172.21.15.2:0/1022070908 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b007a980 con 0x7fe8b007be10 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.640+0000 7fe8b543f700 1 --2- 172.21.15.2:0/1022070908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b007a270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0098a0 tx=0x7fe8ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13955d571818122 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.640+0000 7fe8a3fff700 1 -- 172.21.15.2:0/1022070908 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac004180 con 0x7fe8b007be10 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.640+0000 7fe8a3fff700 1 -- 172.21.15.2:0/1022070908 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe8ac0042e0 con 0x7fe8b007be10 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.640+0000 7fe8a3fff700 1 -- 172.21.15.2:0/1022070908 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac011a70 con 0x7fe8b007be10 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 -- 172.21.15.2:0/1022070908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 msgr2=0x7fe8b007a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:49.260 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 --2- 172.21.15.2:0/1022070908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b007a270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0098a0 tx=0x7fe8ac009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 -- 172.21.15.2:0/1022070908 shutdown_connections 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 --2- 172.21.15.2:0/1022070908 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b007a270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 -- 172.21.15.2:0/1022070908 >> 172.21.15.2:0/1022070908 conn(0x7fe8b01018f0 msgr2=0x7fe8b0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 -- 172.21.15.2:0/1022070908 shutdown_connections 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 -- 172.21.15.2:0/1022070908 wait complete. 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 Processor -- start 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.641+0000 7fe8b76a3700 1 -- start start 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b76a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b76a3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b019c7e0 con 0x7fe8b007be10 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b543f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b543f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39124/0 (socket says 172.21.15.2:39124) 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b543f700 1 -- 172.21.15.2:0/2125804465 learned_addr learned my addr 172.21.15.2:0/2125804465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b543f700 1 -- 172.21.15.2:0/2125804465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac009550 con 0x7fe8b007be10 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b543f700 1 --2- 172.21.15.2:0/2125804465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b019c2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac011710 tx=0x7fe8ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:49.261 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8a27fc700 1 -- 172.21.15.2:0/2125804465 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac003b10 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8a27fc700 1 -- 172.21.15.2:0/2125804465 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe8ac01b8c0 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b019c9e0 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8a27fc700 1 -- 172.21.15.2:0/2125804465 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8ac019710 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.642+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b019ce80 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.643+0000 7fe8a27fc700 1 -- 172.21.15.2:0/2125804465 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe8ac019bc0 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.643+0000 7fe8a27fc700 1 -- 172.21.15.2:0/2125804465 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe8ac01a710 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.643+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b010c6b0 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.646+0000 7fe8a27fc700 1 -- 172.21.15.2:0/2125804465 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fe8ac01ba30 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.719+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe8b010c8a0 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.719+0000 7fe8a27fc700 1 -- 172.21.15.2:0/2125804465 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fe8ac01bbd0 con 0x7fe8b007be10 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.720+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 msgr2=0x7fe8b019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.720+0000 7fe8b76a3700 1 --2- 172.21.15.2:0/2125804465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b019c2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac011710 tx=0x7fe8ac0047c0 comp rx=0 tx=0).stop 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.720+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 shutdown_connections 2024-03-16T10:34:49.262 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.720+0000 7fe8b76a3700 1 --2- 172.21.15.2:0/2125804465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe8b007be10 0x7fe8b019c2a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:49.263 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.720+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 >> 172.21.15.2:0/2125804465 conn(0x7fe8b01018f0 msgr2=0x7fe8b01024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:49.263 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.721+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 shutdown_connections 2024-03-16T10:34:49.263 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:48.721+0000 7fe8b76a3700 1 -- 172.21.15.2:0/2125804465 wait complete. 2024-03-16T10:34:49.263 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (1/15)... 2024-03-16T10:34:52.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:52 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/838883399' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T10:34:52.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:52.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:34:52.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "b3aed5e6-e380-11ee-95c9-87774f69a715", 2024-03-16T10:34:52.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T10:34:52.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T10:34:52.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T10:34:52.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "smithi002" 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T10:34:52.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:52.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T10:34:52.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T10:34:52.421 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T10:34:52.422 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T10:34:41.303283+0000", 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T10:34:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.685+0000 7fb406eb6700 1 Processor -- start 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.685+0000 7fb406eb6700 1 -- start start 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.685+0000 7fb406eb6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb400109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.685+0000 7fb406eb6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb400074520 con 0x7fb400106e10 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.685+0000 7fb404c52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb400109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.685+0000 7fb404c52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb400109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39128/0 (socket says 172.21.15.2:39128) 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.685+0000 7fb404c52700 1 -- 172.21.15.2:0/3471340406 learned_addr learned my addr 172.21.15.2:0/3471340406 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.686+0000 7fb404c52700 1 -- 172.21.15.2:0/3471340406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400109740 con 0x7fb400106e10 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.686+0000 7fb404c52700 1 --2- 172.21.15.2:0/3471340406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb400109200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40098a0 tx=0x7fb3f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c5e9dd91e457b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.686+0000 7fb3ff7fe700 1 -- 172.21.15.2:0/3471340406 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3f4004180 con 0x7fb400106e10 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.686+0000 7fb3ff7fe700 1 -- 172.21.15.2:0/3471340406 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb3f40042e0 con 0x7fb400106e10 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.686+0000 7fb3ff7fe700 1 -- 172.21.15.2:0/3471340406 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3f4011a70 con 0x7fb400106e10 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 -- 172.21.15.2:0/3471340406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 msgr2=0x7fb400109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:52.424 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 --2- 172.21.15.2:0/3471340406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb400109200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40098a0 tx=0x7fb3f4009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 -- 172.21.15.2:0/3471340406 shutdown_connections 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 --2- 172.21.15.2:0/3471340406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb400109200 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 -- 172.21.15.2:0/3471340406 >> 172.21.15.2:0/3471340406 conn(0x7fb400100800 msgr2=0x7fb400102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 -- 172.21.15.2:0/3471340406 shutdown_connections 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 -- 172.21.15.2:0/3471340406 wait complete. 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.687+0000 7fb406eb6700 1 Processor -- start 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb406eb6700 1 -- start start 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb406eb6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb40019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb406eb6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40019a5e0 con 0x7fb400106e10 2024-03-16T10:34:52.425 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb404c52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb40019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb404c52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb40019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39134/0 (socket says 172.21.15.2:39134) 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb404c52700 1 -- 172.21.15.2:0/838883399 learned_addr learned my addr 172.21.15.2:0/838883399 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb404c52700 1 -- 172.21.15.2:0/838883399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009550 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb404c52700 1 --2- 172.21.15.2:0/838883399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb40019a0a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4011710 tx=0x7fb3f40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb3fdffb700 1 -- 172.21.15.2:0/838883399 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3f4003b10 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb3fdffb700 1 -- 172.21.15.2:0/838883399 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb3f401b8c0 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.688+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40019a7e0 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.689+0000 7fb3fdffb700 1 -- 172.21.15.2:0/838883399 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3f40197e0 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.689+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40019ac80 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.689+0000 7fb3fdffb700 1 -- 172.21.15.2:0/838883399 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb3f4019cd0 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.689+0000 7fb3fdffb700 1 -- 172.21.15.2:0/838883399 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb3f401a710 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.690+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40010a460 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.692+0000 7fb3fdffb700 1 -- 172.21.15.2:0/838883399 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fb3f4029060 con 0x7fb400106e10 2024-03-16T10:34:52.426 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.765+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb40010a680 con 0x7fb400106e10 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.765+0000 7fb3fdffb700 1 -- 172.21.15.2:0/838883399 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fb3f401f070 con 0x7fb400106e10 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.767+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 msgr2=0x7fb40019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.767+0000 7fb406eb6700 1 --2- 172.21.15.2:0/838883399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb40019a0a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4011710 tx=0x7fb3f40047c0 comp rx=0 tx=0).stop 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.767+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 shutdown_connections 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.767+0000 7fb406eb6700 1 --2- 172.21.15.2:0/838883399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb400106e10 0x7fb40019a0a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.767+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 >> 172.21.15.2:0/838883399 conn(0x7fb400100800 msgr2=0x7fb4001013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.767+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 shutdown_connections 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:51.767+0000 7fb406eb6700 1 -- 172.21.15.2:0/838883399 wait complete. 2024-03-16T10:34:52.427 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (2/15)... 2024-03-16T10:34:55.144 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:54 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/4076560189' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "b3aed5e6-e380-11ee-95c9-87774f69a715", 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:55.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "smithi002" 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T10:34:55.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T10:34:55.510 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T10:34:55.511 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T10:34:41.303283+0000", 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.840+0000 7fb111dc0700 1 Processor -- start 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb111dc0700 1 -- start start 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb111dc0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c105660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb111dc0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c105c30 con 0x7fb10c105240 2024-03-16T10:34:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb10b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb10b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c105660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54954/0 (socket says 172.21.15.2:54954) 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb10b7fe700 1 -- 172.21.15.2:0/1540410511 learned_addr learned my addr 172.21.15.2:0/1540410511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb10b7fe700 1 -- 172.21.15.2:0/1540410511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c106440 con 0x7fb10c105240 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.841+0000 7fb10b7fe700 1 --2- 172.21.15.2:0/1540410511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c105660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0098a0 tx=0x7fb0fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe485030820546a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb10a7fc700 1 -- 172.21.15.2:0/1540410511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0fc004180 con 0x7fb10c105240 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb10a7fc700 1 -- 172.21.15.2:0/1540410511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb0fc0042e0 con 0x7fb10c105240 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb10a7fc700 1 -- 172.21.15.2:0/1540410511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0fc011a70 con 0x7fb10c105240 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb111dc0700 1 -- 172.21.15.2:0/1540410511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 msgr2=0x7fb10c105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb111dc0700 1 --2- 172.21.15.2:0/1540410511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c105660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0098a0 tx=0x7fb0fc009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb111dc0700 1 -- 172.21.15.2:0/1540410511 shutdown_connections 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb111dc0700 1 --2- 172.21.15.2:0/1540410511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c105660 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb111dc0700 1 -- 172.21.15.2:0/1540410511 >> 172.21.15.2:0/1540410511 conn(0x7fb10c100800 msgr2=0x7fb10c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb111dc0700 1 -- 172.21.15.2:0/1540410511 shutdown_connections 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.842+0000 7fb111dc0700 1 -- 172.21.15.2:0/1540410511 wait complete. 2024-03-16T10:34:55.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.843+0000 7fb111dc0700 1 Processor -- start 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.843+0000 7fb111dc0700 1 -- start start 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.843+0000 7fb111dc0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.843+0000 7fb111dc0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c19a670 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.843+0000 7fb10b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.843+0000 7fb10b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54960/0 (socket says 172.21.15.2:54960) 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.843+0000 7fb10b7fe700 1 -- 172.21.15.2:0/4076560189 learned_addr learned my addr 172.21.15.2:0/4076560189 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb10b7fe700 1 -- 172.21.15.2:0/4076560189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc009550 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb10b7fe700 1 --2- 172.21.15.2:0/4076560189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c19a130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc011710 tx=0x7fb0fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb108ff9700 1 -- 172.21.15.2:0/4076560189 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0fc003b10 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c19a870 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb108ff9700 1 -- 172.21.15.2:0/4076560189 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb0fc01b8c0 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb108ff9700 1 -- 172.21.15.2:0/4076560189 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0fc019770 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c19ad10 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb108ff9700 1 -- 172.21.15.2:0/4076560189 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb0fc019cc0 con 0x7fb10c105240 2024-03-16T10:34:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.844+0000 7fb108ff9700 1 -- 172.21.15.2:0/4076560189 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb0fc01a710 con 0x7fb10c105240 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.845+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c194260 con 0x7fb10c105240 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.847+0000 7fb108ff9700 1 -- 172.21.15.2:0/4076560189 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fb0fc029060 con 0x7fb10c105240 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.919+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb10c0623e0 con 0x7fb10c105240 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.920+0000 7fb108ff9700 1 -- 172.21.15.2:0/4076560189 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fb0fc01f070 con 0x7fb10c105240 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.921+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 msgr2=0x7fb10c19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.921+0000 7fb111dc0700 1 --2- 172.21.15.2:0/4076560189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c19a130 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc011710 tx=0x7fb0fc0047c0 comp rx=0 tx=0).stop 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.921+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 shutdown_connections 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.921+0000 7fb111dc0700 1 --2- 172.21.15.2:0/4076560189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb10c105240 0x7fb10c19a130 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.921+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 >> 172.21.15.2:0/4076560189 conn(0x7fb10c100800 msgr2=0x7fb10c101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.921+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 shutdown_connections 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:54.921+0000 7fb111dc0700 1 -- 172.21.15.2:0/4076560189 wait complete. 2024-03-16T10:34:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (3/15)... 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: Activating manager daemon smithi002.zevrtm 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: mgrmap e2: smithi002.zevrtm(active, starting, since 0.00311962s) 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' 2024-03-16T10:34:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' 2024-03-16T10:34:56.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:55 smithi002 ceph-mon[29248]: from='mgr.14100 172.21.15.2:0/2000009891' entity='mgr.smithi002.zevrtm' 2024-03-16T10:34:57.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:57 smithi002 ceph-mon[29248]: mgrmap e3: smithi002.zevrtm(active, since 1.00739s) 2024-03-16T10:34:58.798 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "b3aed5e6-e380-11ee-95c9-87774f69a715", 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T10:34:58.799 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "smithi002" 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T10:34:58.800 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T10:34:58.809 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T10:34:58.809 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:58.810 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T10:34:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T10:34:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T10:34:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T10:34:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T10:34:58.811 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T10:34:58.812 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T10:34:41.303283+0000", 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.931+0000 7f797f906700 1 Processor -- start 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797f906700 1 -- start start 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797f906700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f7978106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797f906700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7978106d40 con 0x7f7978106350 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797d6a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f7978106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797d6a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f7978106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55028/0 (socket says 172.21.15.2:55028) 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797d6a2700 1 -- 172.21.15.2:0/259829123 learned_addr learned my addr 172.21.15.2:0/259829123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797d6a2700 1 -- 172.21.15.2:0/259829123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978107550 con 0x7f7978106350 2024-03-16T10:34:58.813 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f797d6a2700 1 --2- 172.21.15.2:0/259829123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f7978106770 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79700098a0 tx=0x7f7970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc50709898e2ac2d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.932+0000 7f7977fff700 1 -- 172.21.15.2:0/259829123 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7970004180 con 0x7f7978106350 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f7977fff700 1 -- 172.21.15.2:0/259829123 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f79700042e0 con 0x7f7978106350 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f7977fff700 1 -- 172.21.15.2:0/259829123 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7970011a70 con 0x7f7978106350 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f797f906700 1 -- 172.21.15.2:0/259829123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 msgr2=0x7f7978106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f797f906700 1 --2- 172.21.15.2:0/259829123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f7978106770 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79700098a0 tx=0x7f7970009bb0 comp rx=0 tx=0).stop 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f797f906700 1 -- 172.21.15.2:0/259829123 shutdown_connections 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f797f906700 1 --2- 172.21.15.2:0/259829123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f7978106770 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f797f906700 1 -- 172.21.15.2:0/259829123 >> 172.21.15.2:0/259829123 conn(0x7f79781018f0 msgr2=0x7f7978103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f797f906700 1 -- 172.21.15.2:0/259829123 shutdown_connections 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.933+0000 7f797f906700 1 -- 172.21.15.2:0/259829123 wait complete. 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797f906700 1 Processor -- start 2024-03-16T10:34:58.814 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797f906700 1 -- start start 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797f906700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f797819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797f906700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797819a5e0 con 0x7f7978106350 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797d6a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f797819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797d6a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f797819a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55034/0 (socket says 172.21.15.2:55034) 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797d6a2700 1 -- 172.21.15.2:0/1431160395 learned_addr learned my addr 172.21.15.2:0/1431160395 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.934+0000 7f797d6a2700 1 -- 172.21.15.2:0/1431160395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7970009550 con 0x7f7978106350 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.935+0000 7f797d6a2700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f797819a0a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7970011710 tx=0x7f79700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.935+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7970003b10 con 0x7f7978106350 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.935+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f797001b8c0 con 0x7f7978106350 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.935+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797819a7e0 con 0x7f7978106350 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.935+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7970019710 con 0x7f7978106350 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.935+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797819ac80 con 0x7f7978106350 2024-03-16T10:34:58.815 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.936+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 3) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f7970019910 con 0x7f7978106350 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.936+0000 7f79767fc700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f7960038340 0x7f796003a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.936+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f797004c5a0 con 0x7f7978106350 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.936+0000 7f797cea1700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f7960038340 0x7f796003a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.936+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79780623e0 con 0x7f7978106350 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.936+0000 7f797cea1700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f7960038340 0x7f796003a800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f796c006fd0 tx=0x7f796c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:57.941+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7970019bc0 con 0x7f7978106350 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.161+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 4) v1 ==== 45378+0+0 (secure 0 0 0) 0x7f797001a8f0 con 0x7f7978106350 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.226+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f797819dbc0 con 0x7f7978106350 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.227+0000 7f79767fc700 1 -- 172.21.15.2:0/1431160395 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f7970019bc0 con 0x7f7978106350 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f7960038340 msgr2=0x7f796003a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f7960038340 0x7f796003a800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f796c006fd0 tx=0x7f796c006e40 comp rx=0 tx=0).stop 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 msgr2=0x7f797819a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:34:58.816 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f797819a0a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7970011710 tx=0x7f79700047c0 comp rx=0 tx=0).stop 2024-03-16T10:34:58.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 shutdown_connections 2024-03-16T10:34:58.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f7960038340 0x7f796003a800 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:58.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 --2- 172.21.15.2:0/1431160395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7978106350 0x7f797819a0a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:34:58.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 >> 172.21.15.2:0/1431160395 conn(0x7f79781018f0 msgr2=0x7f79781024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:34:58.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 shutdown_connections 2024-03-16T10:34:58.817 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:34:58.229+0000 7f797f906700 1 -- 172.21.15.2:0/1431160395 wait complete. 2024-03-16T10:34:58.817 INFO:teuthology.orchestra.run.smithi002.stdout:mgr is available 2024-03-16T10:34:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:59 smithi002 ceph-mon[29248]: mgrmap e4: smithi002.zevrtm(active, since 3s) 2024-03-16T10:34:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:34:59 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1431160395' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T10:35:02.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:02 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1812280420' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-03-16T10:35:07.565 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd99433d700 1 Processor -- start 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd99433d700 1 -- start start 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd99433d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd99433d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c106d40 con 0x7fd98c106350 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd9920d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:07.566 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd9920d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55042/0 (socket says 172.21.15.2:55042) 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd9920d9700 1 -- 172.21.15.2:0/3405139163 learned_addr learned my addr 172.21.15.2:0/3405139163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.114+0000 7fd9920d9700 1 -- 172.21.15.2:0/3405139163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c107550 con 0x7fd98c106350 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd9920d9700 1 --2- 172.21.15.2:0/3405139163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c106770 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098a0 tx=0x7fd988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f5fa3c4628a4a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd9910d7700 1 -- 172.21.15.2:0/3405139163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd988004180 con 0x7fd98c106350 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd9910d7700 1 -- 172.21.15.2:0/3405139163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd9880042e0 con 0x7fd98c106350 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd9910d7700 1 -- 172.21.15.2:0/3405139163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd988011a70 con 0x7fd98c106350 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd99433d700 1 -- 172.21.15.2:0/3405139163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 msgr2=0x7fd98c106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd99433d700 1 --2- 172.21.15.2:0/3405139163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c106770 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd9880098a0 tx=0x7fd988009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd99433d700 1 -- 172.21.15.2:0/3405139163 shutdown_connections 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd99433d700 1 --2- 172.21.15.2:0/3405139163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c106770 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.115+0000 7fd99433d700 1 -- 172.21.15.2:0/3405139163 >> 172.21.15.2:0/3405139163 conn(0x7fd98c1018f0 msgr2=0x7fd98c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd99433d700 1 -- 172.21.15.2:0/3405139163 shutdown_connections 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd99433d700 1 -- 172.21.15.2:0/3405139163 wait complete. 2024-03-16T10:35:07.567 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd99433d700 1 Processor -- start 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd99433d700 1 -- start start 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd99433d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd99433d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c19c800 con 0x7fd98c106350 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd9920d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd9920d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55058/0 (socket says 172.21.15.2:55058) 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.116+0000 7fd9920d9700 1 -- 172.21.15.2:0/1812280420 learned_addr learned my addr 172.21.15.2:0/1812280420 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.117+0000 7fd9920d9700 1 -- 172.21.15.2:0/1812280420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988009550 con 0x7fd98c106350 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.117+0000 7fd9920d9700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c19c2c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd988011710 tx=0x7fd9880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.117+0000 7fd97f7fe700 1 -- 172.21.15.2:0/1812280420 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd988003b10 con 0x7fd98c106350 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.117+0000 7fd97f7fe700 1 -- 172.21.15.2:0/1812280420 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd98801b8c0 con 0x7fd98c106350 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.117+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98c19ca00 con 0x7fd98c106350 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.117+0000 7fd97f7fe700 1 -- 172.21.15.2:0/1812280420 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd988019770 con 0x7fd98c106350 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.117+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c19cea0 con 0x7fd98c106350 2024-03-16T10:35:07.568 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.118+0000 7fd97f7fe700 1 -- 172.21.15.2:0/1812280420 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45378+0+0 (secure 0 0 0) 0x7fd98801b440 con 0x7fd98c106350 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.118+0000 7fd97f7fe700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7fd9780384f0 0x7fd97803a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.118+0000 7fd97f7fe700 1 -- 172.21.15.2:0/1812280420 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd98804c5b0 con 0x7fd98c106350 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.118+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c10a780 con 0x7fd98c106350 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.118+0000 7fd9918d8700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7fd9780384f0 0x7fd97803a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.119+0000 7fd9918d8700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7fd9780384f0 0x7fd97803a9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd980006fd0 tx=0x7fd980006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.123+0000 7fd97f7fe700 1 -- 172.21.15.2:0/1812280420 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd98801f080 con 0x7fd98c106350 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.325+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd98c1a0ff0 con 0x7fd98c106350 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.326+0000 7fd97f7fe700 1 -- 172.21.15.2:0/1812280420 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd98801ba30 con 0x7fd98c106350 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7fd9780384f0 msgr2=0x7fd97803a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7fd9780384f0 0x7fd97803a9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd980006fd0 tx=0x7fd980006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 msgr2=0x7fd98c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c19c2c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd988011710 tx=0x7fd9880047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 shutdown_connections 2024-03-16T10:35:07.569 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7fd9780384f0 0x7fd97803a9b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:07.570 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 --2- 172.21.15.2:0/1812280420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd98c106350 0x7fd98c19c2c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:07.570 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 >> 172.21.15.2:0/1812280420 conn(0x7fd98c1018f0 msgr2=0x7fd98c102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:07.570 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 shutdown_connections 2024-03-16T10:35:07.570 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:01.328+0000 7fd99433d700 1 -- 172.21.15.2:0/1812280420 wait complete. 2024-03-16T10:35:07.570 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling cephadm module... 2024-03-16T10:35:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:10 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/946646240' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-16T10:35:11.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.275+0000 7f533cebb700 1 Processor -- start 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.275+0000 7f533cebb700 1 -- start start 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.276+0000 7f533cebb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533807a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.276+0000 7f533cebb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533807a820 con 0x7f533807bdf0 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.276+0000 7f533659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533807a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.276+0000 7f533659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533807a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41036/0 (socket says 172.21.15.2:41036) 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.276+0000 7f533659c700 1 -- 172.21.15.2:0/280688641 learned_addr learned my addr 172.21.15.2:0/280688641 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.276+0000 7f533659c700 1 -- 172.21.15.2:0/280688641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533807a960 con 0x7f533807bdf0 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.276+0000 7f533659c700 1 --2- 172.21.15.2:0/280688641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533807a250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f532c0098a0 tx=0x7f532c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0e62274c666ca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533559a700 1 -- 172.21.15.2:0/280688641 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f532c004180 con 0x7f533807bdf0 2024-03-16T10:35:11.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533559a700 1 -- 172.21.15.2:0/280688641 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f532c0042e0 con 0x7f533807bdf0 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533559a700 1 -- 172.21.15.2:0/280688641 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f532c011a70 con 0x7f533807bdf0 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533cebb700 1 -- 172.21.15.2:0/280688641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 msgr2=0x7f533807a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533cebb700 1 --2- 172.21.15.2:0/280688641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533807a250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f532c0098a0 tx=0x7f532c009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533cebb700 1 -- 172.21.15.2:0/280688641 shutdown_connections 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533cebb700 1 --2- 172.21.15.2:0/280688641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533807a250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533cebb700 1 -- 172.21.15.2:0/280688641 >> 172.21.15.2:0/280688641 conn(0x7f53381018e0 msgr2=0x7f5338103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533cebb700 1 -- 172.21.15.2:0/280688641 shutdown_connections 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.277+0000 7f533cebb700 1 -- 172.21.15.2:0/280688641 wait complete. 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533cebb700 1 Processor -- start 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533cebb700 1 -- start start 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533cebb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533cebb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533819c7e0 con 0x7f533807bdf0 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:11.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533819c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41044/0 (socket says 172.21.15.2:41044) 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533659c700 1 -- 172.21.15.2:0/946646240 learned_addr learned my addr 172.21.15.2:0/946646240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.278+0000 7f533659c700 1 -- 172.21.15.2:0/946646240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c009550 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.279+0000 7f533659c700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533819c2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f532c011710 tx=0x7f532c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.279+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f532c003b10 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.279+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f532c01b8c0 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.279+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533819c9e0 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.279+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f532c0197e0 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.279+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533819ce80 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.280+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45378+0+0 (secure 0 0 0) 0x7f532c019a20 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.280+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533810c700 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.280+0000 7f53277fe700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f53200384a0 0x7f532003a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.280+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f532c04c660 con 0x7f533807bdf0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.280+0000 7f5335d9b700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f53200384a0 0x7f532003a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:11.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.280+0000 7f5335d9b700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f53200384a0 0x7f532003a960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5328009940 tx=0x7f5328006e30 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.285+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f532c018db0 con 0x7f533807bdf0 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.544+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f533810c8f0 con 0x7f533807bdf0 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.597+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 5) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f532c02ad20 con 0x7f533807bdf0 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.598+0000 7f53277fe700 1 -- 172.21.15.2:0/946646240 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f532c04c080 con 0x7f533807bdf0 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f53200384a0 msgr2=0x7f532003a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f53200384a0 0x7f532003a960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5328009940 tx=0x7f5328006e30 comp rx=0 tx=0).stop 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 msgr2=0x7f533819c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533819c2a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f532c011710 tx=0x7f532c0047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 shutdown_connections 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f53200384a0 0x7f532003a960 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 --2- 172.21.15.2:0/946646240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533807bdf0 0x7f533819c2a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 >> 172.21.15.2:0/946646240 conn(0x7f53381018e0 msgr2=0x7f53381024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 shutdown_connections 2024-03-16T10:35:11.202 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:10.601+0000 7f533cebb700 1 -- 172.21.15.2:0/946646240 wait complete. 2024-03-16T10:35:11.858 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:11 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/946646240' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-16T10:35:11.858 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:11 smithi002 ceph-mon[29248]: mgrmap e5: smithi002.zevrtm(active, since 15s) 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "smithi002.zevrtm", 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.591+0000 7f3c0798d700 1 Processor -- start 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.592+0000 7f3c0798d700 1 -- start start 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.592+0000 7f3c0798d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0007a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.592+0000 7f3c0798d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0007a860 con 0x7f3c0007be30 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.592+0000 7f3c05729700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0007a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.592+0000 7f3c05729700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0007a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41064/0 (socket says 172.21.15.2:41064) 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.592+0000 7f3c05729700 1 -- 172.21.15.2:0/3897022668 learned_addr learned my addr 172.21.15.2:0/3897022668 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:12.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.592+0000 7f3c05729700 1 -- 172.21.15.2:0/3897022668 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0007a9a0 con 0x7f3c0007be30 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3c05729700 1 --2- 172.21.15.2:0/3897022668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0007a290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0098a0 tx=0x7f3bfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8091c54fc21d06e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3bf3fff700 1 -- 172.21.15.2:0/3897022668 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bfc004180 con 0x7f3c0007be30 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3bf3fff700 1 -- 172.21.15.2:0/3897022668 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3bfc0042e0 con 0x7f3c0007be30 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3bf3fff700 1 -- 172.21.15.2:0/3897022668 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bfc011a70 con 0x7f3c0007be30 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3c0798d700 1 -- 172.21.15.2:0/3897022668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 msgr2=0x7f3c0007a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3c0798d700 1 --2- 172.21.15.2:0/3897022668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0007a290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0098a0 tx=0x7f3bfc009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3c0798d700 1 -- 172.21.15.2:0/3897022668 shutdown_connections 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3c0798d700 1 --2- 172.21.15.2:0/3897022668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0007a290 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.593+0000 7f3c0798d700 1 -- 172.21.15.2:0/3897022668 >> 172.21.15.2:0/3897022668 conn(0x7f3c001018b0 msgr2=0x7f3c00103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.594+0000 7f3c0798d700 1 -- 172.21.15.2:0/3897022668 shutdown_connections 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.594+0000 7f3c0798d700 1 -- 172.21.15.2:0/3897022668 wait complete. 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.594+0000 7f3c0798d700 1 Processor -- start 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.594+0000 7f3c0798d700 1 -- start start 2024-03-16T10:35:12.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.594+0000 7f3c0798d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.594+0000 7f3c0798d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0019c800 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.594+0000 7f3c05729700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3c05729700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41066/0 (socket says 172.21.15.2:41066) 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3c05729700 1 -- 172.21.15.2:0/1311351073 learned_addr learned my addr 172.21.15.2:0/1311351073 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3c05729700 1 -- 172.21.15.2:0/1311351073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc009550 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3c05729700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0019c2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc011710 tx=0x7f3bfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3bf27fc700 1 -- 172.21.15.2:0/1311351073 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bfc003b10 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3bf27fc700 1 -- 172.21.15.2:0/1311351073 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3bfc01b8c0 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0019ca00 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3bf27fc700 1 -- 172.21.15.2:0/1311351073 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bfc019710 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.595+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0019cea0 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.596+0000 7f3bf27fc700 1 -- 172.21.15.2:0/1311351073 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f3bfc019910 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.596+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0010c690 con 0x7f3c0007be30 2024-03-16T10:35:12.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.596+0000 7f3bf27fc700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 0x7f3bec03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.596+0000 7f3bf27fc700 1 -- 172.21.15.2:0/1311351073 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3bfc04ca50 con 0x7f3c0007be30 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.596+0000 7f3c04f28700 1 -- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 msgr2=0x7f3bec03a970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3464791333 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.596+0000 7f3c04f28700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 0x7f3bec03a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.601+0000 7f3bf27fc700 1 -- 172.21.15.2:0/1311351073 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3bfc018db0 con 0x7f3c0007be30 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.797+0000 7f3c04f28700 1 -- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 msgr2=0x7f3bec03a970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3464791333 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.797+0000 7f3c04f28700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 0x7f3bec03a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.857+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3c0010c8b0 con 0x7f3c0007be30 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.857+0000 7f3bf27fc700 1 -- 172.21.15.2:0/1311351073 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f3bfc018350 con 0x7f3c0007be30 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 msgr2=0x7f3bec03a970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 0x7f3bec03a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 msgr2=0x7f3c0019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0019c2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc011710 tx=0x7f3bfc0047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 shutdown_connections 2024-03-16T10:35:12.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f3bec0384b0 0x7f3bec03a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:12.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 --2- 172.21.15.2:0/1311351073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c0007be30 0x7f3c0019c2c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:12.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 >> 172.21.15.2:0/1311351073 conn(0x7f3c001018b0 msgr2=0x7f3c001024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:12.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 shutdown_connections 2024-03-16T10:35:12.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:11.861+0000 7f3c0798d700 1 -- 172.21.15.2:0/1311351073 wait complete. 2024-03-16T10:35:12.459 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-03-16T10:35:12.459 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 5... 2024-03-16T10:35:12.715 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:12 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1311351073' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T10:35:17.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: Active manager daemon smithi002.zevrtm restarted 2024-03-16T10:35:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: Activating manager daemon smithi002.zevrtm 2024-03-16T10:35:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: osdmap e2: 0 total, 0 up, 0 in 2024-03-16T10:35:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: mgrmap e6: smithi002.zevrtm(active, starting, since 0.00325052s) 2024-03-16T10:35:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:35:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:35:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:35:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:35:17.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:35:17.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:35:17.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:17.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:17.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:17.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:17.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:17 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:18.877 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:18 smithi002 ceph-mon[29248]: Found migration_current of "None". Setting to last migration. 2024-03-16T10:35:18.877 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:18 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:35:18.877 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:18 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:35:18.877 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:18 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:18.877 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:18 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:18.877 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:18 smithi002 ceph-mon[29248]: mgrmap e7: smithi002.zevrtm(active, since 1.00496s) 2024-03-16T10:35:19.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:35:19.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-16T10:35:19.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T10:35:19.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:35:19.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.814+0000 7f76cbbd0700 1 Processor -- start 2024-03-16T10:35:19.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.814+0000 7f76cbbd0700 1 -- start start 2024-03-16T10:35:19.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76cbbd0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c4104a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76cbbd0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4105010 con 0x7f76c4104620 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c996c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c4104a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c996c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c4104a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38090/0 (socket says 172.21.15.2:38090) 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c996c700 1 -- 172.21.15.2:0/1122732784 learned_addr learned my addr 172.21.15.2:0/1122732784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c996c700 1 -- 172.21.15.2:0/1122732784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c41075b0 con 0x7f76c4104620 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c996c700 1 --2- 172.21.15.2:0/1122732784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c4104a40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76c00098a0 tx=0x7f76c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800901299e5f74 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c896a700 1 -- 172.21.15.2:0/1122732784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c0004180 con 0x7f76c4104620 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c896a700 1 -- 172.21.15.2:0/1122732784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f76c00042e0 con 0x7f76c4104620 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.815+0000 7f76c896a700 1 -- 172.21.15.2:0/1122732784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c0011a70 con 0x7f76c4104620 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.816+0000 7f76cbbd0700 1 -- 172.21.15.2:0/1122732784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 msgr2=0x7f76c4104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.816+0000 7f76cbbd0700 1 --2- 172.21.15.2:0/1122732784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c4104a40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76c00098a0 tx=0x7f76c0009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.816+0000 7f76cbbd0700 1 -- 172.21.15.2:0/1122732784 shutdown_connections 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.816+0000 7f76cbbd0700 1 --2- 172.21.15.2:0/1122732784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c4104a40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.816+0000 7f76cbbd0700 1 -- 172.21.15.2:0/1122732784 >> 172.21.15.2:0/1122732784 conn(0x7f76c40fffd0 msgr2=0x7f76c41023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.816+0000 7f76cbbd0700 1 -- 172.21.15.2:0/1122732784 shutdown_connections 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.816+0000 7f76cbbd0700 1 -- 172.21.15.2:0/1122732784 wait complete. 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.817+0000 7f76cbbd0700 1 Processor -- start 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.817+0000 7f76cbbd0700 1 -- start start 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.817+0000 7f76cbbd0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.817+0000 7f76cbbd0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c419a5f0 con 0x7f76c4104620 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76c996c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76c996c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38102/0 (socket says 172.21.15.2:38102) 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76c996c700 1 -- 172.21.15.2:0/592870689 learned_addr learned my addr 172.21.15.2:0/592870689 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76c996c700 1 -- 172.21.15.2:0/592870689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c0009550 con 0x7f76c4104620 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76c996c700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c419a0b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76c0011710 tx=0x7f76c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c0003b10 con 0x7f76c4104620 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f76c001b8c0 con 0x7f76c4104620 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c419a7f0 con 0x7f76c4104620 2024-03-16T10:35:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c419ac90 con 0x7f76c4104620 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.818+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76c0019710 con 0x7f76c4104620 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.819+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f76c00198c0 con 0x7f76c4104620 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.819+0000 7f76baffd700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.819+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 --> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76b003b0d0 con 0x7f76b0038500 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.819+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f76c004c790 con 0x7f76c4104620 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.819+0000 7f76c916b700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 msgr2=0x7f76b003a9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3464791333 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:12.819+0000 7f76c916b700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:13.020+0000 7f76c916b700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 msgr2=0x7f76b003a9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3464791333 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:13.020+0000 7f76c916b700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:13.421+0000 7f76c916b700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 msgr2=0x7f76b003a9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3464791333 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:13.421+0000 7f76c916b700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:14.222+0000 7f76c916b700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 msgr2=0x7f76b003a9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3464791333 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:14.222+0000 7f76c916b700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T10:35:19.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:15.823+0000 7f76c916b700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 msgr2=0x7f76b003a9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3464791333 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:15.823+0000 7f76c916b700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:17.553+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 6) v1 ==== 45156+0+0 (secure 0 0 0) 0x7f76c001a8f0 con 0x7f76c4104620 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:17.553+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 msgr2=0x7f76b003a9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:17.553+0000 7f76baffd700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.553+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 7) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f76c000ed40 con 0x7f76c4104620 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.554+0000 7f76baffd700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f76b003be30 0x7f76b003e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.554+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76b003b0d0 con 0x7f76b003be30 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.554+0000 7f76c916b700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f76b003be30 0x7f76b003e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.555+0000 7f76c916b700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f76b003be30 0x7f76b003e220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f76b4003a10 tx=0x7f76b40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.555+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f76b003b0d0 con 0x7f76b003be30 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.561+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f76c40623e0 con 0x7f76b003be30 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76baffd700 1 -- 172.21.15.2:0/592870689 <== mgr.14120 v2:172.21.15.2:6800/3739300318 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f76c40623e0 con 0x7f76b003be30 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f76b003be30 msgr2=0x7f76b003e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:19.107 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f76b003be30 0x7f76b003e220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f76b4003a10 tx=0x7f76b40092b0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 msgr2=0x7f76c419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c419a0b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76c0011710 tx=0x7f76c00047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 shutdown_connections 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f76b003be30 0x7f76b003e220 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76c4104620 0x7f76c419a0b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 --2- 172.21.15.2:0/592870689 >> [v2:172.21.15.2:6800/3464791333,v1:172.21.15.2:6801/3464791333] conn(0x7f76b0038500 0x7f76b003a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 >> 172.21.15.2:0/592870689 conn(0x7f76c40fffd0 msgr2=0x7f76c4100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 shutdown_connections 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:18.562+0000 7f76cbbd0700 1 -- 172.21.15.2:0/592870689 wait complete. 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 5 is available 2024-03-16T10:35:19.108 INFO:teuthology.orchestra.run.smithi002.stdout:Setting orchestrator backend to cephadm... 2024-03-16T10:35:19.744 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:19 smithi002 ceph-mon[29248]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T10:35:19.744 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:19 smithi002 ceph-mon[29248]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T10:35:20.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.495+0000 7f9088a20700 1 Processor -- start 2024-03-16T10:35:20.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.495+0000 7f9088a20700 1 -- start start 2024-03-16T10:35:20.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.495+0000 7f9088a20700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f9084079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.495+0000 7f9088a20700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9084079730 con 0x7f908407ad00 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f9084079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f9084079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38182/0 (socket says 172.21.15.2:38182) 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908259c700 1 -- 172.21.15.2:0/2804969869 learned_addr learned my addr 172.21.15.2:0/2804969869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908259c700 1 -- 172.21.15.2:0/2804969869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9084079870 con 0x7f908407ad00 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908259c700 1 --2- 172.21.15.2:0/2804969869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f9084079160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90780098a0 tx=0x7f9078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ecaf5e082a328d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908159a700 1 -- 172.21.15.2:0/2804969869 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9078004180 con 0x7f908407ad00 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908159a700 1 -- 172.21.15.2:0/2804969869 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f90780042e0 con 0x7f908407ad00 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.496+0000 7f908159a700 1 -- 172.21.15.2:0/2804969869 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9078011a70 con 0x7f908407ad00 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.497+0000 7f9088a20700 1 -- 172.21.15.2:0/2804969869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 msgr2=0x7f9084079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.497+0000 7f9088a20700 1 --2- 172.21.15.2:0/2804969869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f9084079160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f90780098a0 tx=0x7f9078009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.497+0000 7f9088a20700 1 -- 172.21.15.2:0/2804969869 shutdown_connections 2024-03-16T10:35:20.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.497+0000 7f9088a20700 1 --2- 172.21.15.2:0/2804969869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f9084079160 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.497+0000 7f9088a20700 1 -- 172.21.15.2:0/2804969869 >> 172.21.15.2:0/2804969869 conn(0x7f9084101930 msgr2=0x7f9084103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.497+0000 7f9088a20700 1 -- 172.21.15.2:0/2804969869 shutdown_connections 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.497+0000 7f9088a20700 1 -- 172.21.15.2:0/2804969869 wait complete. 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f9088a20700 1 Processor -- start 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f9088a20700 1 -- start start 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f9088a20700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f908419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f9088a20700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908419c820 con 0x7f908407ad00 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f908259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f908419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f908259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f908419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38192/0 (socket says 172.21.15.2:38192) 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f908259c700 1 -- 172.21.15.2:0/2746685954 learned_addr learned my addr 172.21.15.2:0/2746685954 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f908259c700 1 -- 172.21.15.2:0/2746685954 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9078009550 con 0x7f908407ad00 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.498+0000 7f908259c700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f908419c2e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9078011710 tx=0x7f90780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.499+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9078003b10 con 0x7f908407ad00 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.499+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f907801b8c0 con 0x7f908407ad00 2024-03-16T10:35:20.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.499+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908419ca20 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.499+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9078019710 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.499+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908419cec0 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.500+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 7) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f907801ba30 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.500+0000 7f90737fe700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f906c038390 0x7f906c03a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.500+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f907804c680 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.500+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908410c6b0 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.500+0000 7f9081d9b700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f906c038390 0x7f906c03a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.500+0000 7f9081d9b700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f906c038390 0x7f906c03a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.505+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f907801a3d0 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.607+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f90780198c0 con 0x7f908407ad00 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.743+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f9084102bd0 con 0x7f906c038390 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.750+0000 7f90737fe700 1 -- 172.21.15.2:0/2746685954 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9084102bd0 con 0x7f906c038390 2024-03-16T10:35:20.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f906c038390 msgr2=0x7f906c03a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f906c038390 0x7f906c03a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9074006fd0 tx=0x7f9074006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 msgr2=0x7f908419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f908419c2e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9078011710 tx=0x7f90780047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 shutdown_connections 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f906c038390 0x7f906c03a850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 --2- 172.21.15.2:0/2746685954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f908407ad00 0x7f908419c2e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.752+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 >> 172.21.15.2:0/2746685954 conn(0x7f9084101930 msgr2=0x7f90841024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.753+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 shutdown_connections 2024-03-16T10:35:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:19.753+0000 7f9088a20700 1 -- 172.21.15.2:0/2746685954 wait complete. 2024-03-16T10:35:20.963 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:20 smithi002 ceph-mon[29248]: mgrmap e8: smithi002.zevrtm(active, since 2s) 2024-03-16T10:35:20.964 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:20 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:20.964 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:20 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.750+0000 7f08e872e700 1 Processor -- start 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.750+0000 7f08e872e700 1 -- start start 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.750+0000 7f08e872e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e0106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.750+0000 7f08e872e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0106d40 con 0x7f08e0106350 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e64ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e0106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e64ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e0106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38198/0 (socket says 172.21.15.2:38198) 2024-03-16T10:35:21.489 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e64ca700 1 -- 172.21.15.2:0/2657857412 learned_addr learned my addr 172.21.15.2:0/2657857412 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e64ca700 1 -- 172.21.15.2:0/2657857412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e0107550 con 0x7f08e0106350 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e64ca700 1 --2- 172.21.15.2:0/2657857412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e0106770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08d40098a0 tx=0x7f08d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917f4e7233f363bd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e54c8700 1 -- 172.21.15.2:0/2657857412 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d4004180 con 0x7f08e0106350 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e54c8700 1 -- 172.21.15.2:0/2657857412 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f08d40042e0 con 0x7f08e0106350 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.751+0000 7f08e54c8700 1 -- 172.21.15.2:0/2657857412 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d4011a70 con 0x7f08e0106350 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.752+0000 7f08e872e700 1 -- 172.21.15.2:0/2657857412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 msgr2=0x7f08e0106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.752+0000 7f08e872e700 1 --2- 172.21.15.2:0/2657857412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e0106770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f08d40098a0 tx=0x7f08d4009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.752+0000 7f08e872e700 1 -- 172.21.15.2:0/2657857412 shutdown_connections 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.752+0000 7f08e872e700 1 --2- 172.21.15.2:0/2657857412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e0106770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.752+0000 7f08e872e700 1 -- 172.21.15.2:0/2657857412 >> 172.21.15.2:0/2657857412 conn(0x7f08e01018f0 msgr2=0x7f08e0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.752+0000 7f08e872e700 1 -- 172.21.15.2:0/2657857412 shutdown_connections 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.752+0000 7f08e872e700 1 -- 172.21.15.2:0/2657857412 wait complete. 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e872e700 1 Processor -- start 2024-03-16T10:35:21.490 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e872e700 1 -- start start 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e872e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e007ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e872e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e007bfd0 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e64ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e007ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e64ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e007ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38208/0 (socket says 172.21.15.2:38208) 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e64ca700 1 -- 172.21.15.2:0/3884019121 learned_addr learned my addr 172.21.15.2:0/3884019121 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e64ca700 1 -- 172.21.15.2:0/3884019121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d4009550 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08e64ca700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e007ba90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08d4011710 tx=0x7f08d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08d37fe700 1 -- 172.21.15.2:0/3884019121 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d4003b10 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.753+0000 7f08d37fe700 1 -- 172.21.15.2:0/3884019121 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f08d401b8c0 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.754+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e007a210 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.754+0000 7f08d37fe700 1 -- 172.21.15.2:0/3884019121 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d4019770 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.754+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e007a6b0 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.754+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e00623e0 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.755+0000 7f08d37fe700 1 -- 172.21.15.2:0/3884019121 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f08d401ba30 con 0x7f08e0106350 2024-03-16T10:35:21.491 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.755+0000 7f08d37fe700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f08cc0384f0 0x7f08cc03a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.755+0000 7f08d37fe700 1 -- 172.21.15.2:0/3884019121 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f08d404c610 con 0x7f08e0106350 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.755+0000 7f08e5cc9700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f08cc0384f0 0x7f08cc03a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.755+0000 7f08e5cc9700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f08cc0384f0 0x7f08cc03a9b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08dc006fd0 tx=0x7f08dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.760+0000 7f08d37fe700 1 -- 172.21.15.2:0/3884019121 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08d402cce0 con 0x7f08e0106350 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.962+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f08e007b100 con 0x7f08cc0384f0 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.963+0000 7f08d37fe700 1 -- 172.21.15.2:0/3884019121 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f08dc009e20 con 0x7f08cc0384f0 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f08cc0384f0 msgr2=0x7f08cc03a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f08cc0384f0 0x7f08cc03a9b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f08dc006fd0 tx=0x7f08dc006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 msgr2=0x7f08e007ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e007ba90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f08d4011710 tx=0x7f08d40047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 shutdown_connections 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f08cc0384f0 0x7f08cc03a9b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:21.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 --2- 172.21.15.2:0/3884019121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e0106350 0x7f08e007ba90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:21.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.965+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 >> 172.21.15.2:0/3884019121 conn(0x7f08e01018f0 msgr2=0x7f08e01024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:21.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.966+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 shutdown_connections 2024-03-16T10:35:21.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:20.966+0000 7f08e872e700 1 -- 172.21.15.2:0/3884019121 wait complete. 2024-03-16T10:35:21.493 INFO:teuthology.orchestra.run.smithi002.stdout:Generating ssh key... 2024-03-16T10:35:21.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:21 smithi002 ceph-mon[29248]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:21.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:21 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:22.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:22 smithi002 ceph-mon[29248]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:22.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:22 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:20] ENGINE Bus STARTING 2024-03-16T10:35:22.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:22 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:21] ENGINE Serving on https://172.21.15.2:7150 2024-03-16T10:35:22.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:22 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:21] ENGINE Serving on http://172.21.15.2:8765 2024-03-16T10:35:22.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:22 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:21] ENGINE Bus STARTED 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79ed364700 1 Processor -- start 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79ed364700 1 -- start start 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79ed364700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e8106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79ed364700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8106d40 con 0x7f79e8106350 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e8106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e8106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38214/0 (socket says 172.21.15.2:38214) 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79e6ffd700 1 -- 172.21.15.2:0/788454320 learned_addr learned my addr 172.21.15.2:0/788454320 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.924+0000 7f79e6ffd700 1 -- 172.21.15.2:0/788454320 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e8107550 con 0x7f79e8106350 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79e6ffd700 1 --2- 172.21.15.2:0/788454320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e8106770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f79d80098a0 tx=0x7f79d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6d36c8709e3315 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79e5ffb700 1 -- 172.21.15.2:0/788454320 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79d8004180 con 0x7f79e8106350 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79e5ffb700 1 -- 172.21.15.2:0/788454320 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f79d80042e0 con 0x7f79e8106350 2024-03-16T10:35:23.453 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79e5ffb700 1 -- 172.21.15.2:0/788454320 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79d8011a70 con 0x7f79e8106350 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79ed364700 1 -- 172.21.15.2:0/788454320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 msgr2=0x7f79e8106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79ed364700 1 --2- 172.21.15.2:0/788454320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e8106770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f79d80098a0 tx=0x7f79d8009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79ed364700 1 -- 172.21.15.2:0/788454320 shutdown_connections 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79ed364700 1 --2- 172.21.15.2:0/788454320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e8106770 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79ed364700 1 -- 172.21.15.2:0/788454320 >> 172.21.15.2:0/788454320 conn(0x7f79e81018f0 msgr2=0x7f79e8103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79ed364700 1 -- 172.21.15.2:0/788454320 shutdown_connections 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.925+0000 7f79ed364700 1 -- 172.21.15.2:0/788454320 wait complete. 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.926+0000 7f79ed364700 1 Processor -- start 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.926+0000 7f79ed364700 1 -- start start 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.926+0000 7f79ed364700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.926+0000 7f79ed364700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e819a600 con 0x7f79e8106350 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.926+0000 7f79e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.926+0000 7f79e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38230/0 (socket says 172.21.15.2:38230) 2024-03-16T10:35:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.926+0000 7f79e6ffd700 1 -- 172.21.15.2:0/3078093809 learned_addr learned my addr 172.21.15.2:0/3078093809 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.927+0000 7f79e6ffd700 1 -- 172.21.15.2:0/3078093809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d8009550 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.927+0000 7f79e6ffd700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e819a0c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79d8011710 tx=0x7f79d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.927+0000 7f79cffff700 1 -- 172.21.15.2:0/3078093809 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79d8003b10 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.927+0000 7f79cffff700 1 -- 172.21.15.2:0/3078093809 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f79d801b8c0 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.927+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e819a800 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.927+0000 7f79cffff700 1 -- 172.21.15.2:0/3078093809 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79d8019710 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.927+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e819aca0 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.928+0000 7f79cffff700 1 -- 172.21.15.2:0/3078093809 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f79d80198c0 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.928+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e810a780 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.928+0000 7f79cffff700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f79d0038500 0x7f79d003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.928+0000 7f79cffff700 1 -- 172.21.15.2:0/3078093809 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f79d804c7e0 con 0x7f79e8106350 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.928+0000 7f79e67fc700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f79d0038500 0x7f79d003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.929+0000 7f79e67fc700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f79d0038500 0x7f79d003a9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:21.933+0000 7f79cffff700 1 -- 172.21.15.2:0/3078093809 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f79d8018b80 con 0x7f79e8106350 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.134+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f79e8102bb0 con 0x7f79d0038500 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.905+0000 7f79cffff700 1 -- 172.21.15.2:0/3078093809 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f79e8102bb0 con 0x7f79d0038500 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.907+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f79d0038500 msgr2=0x7f79d003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.907+0000 7f79ed364700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f79d0038500 0x7f79d003a9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.907+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 msgr2=0x7f79e819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.907+0000 7f79ed364700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e819a0c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f79d8011710 tx=0x7f79d80047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.908+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 shutdown_connections 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.908+0000 7f79ed364700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f79d0038500 0x7f79d003a9c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.908+0000 7f79ed364700 1 --2- 172.21.15.2:0/3078093809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79e8106350 0x7f79e819a0c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.908+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 >> 172.21.15.2:0/3078093809 conn(0x7f79e81018f0 msgr2=0x7f79e81024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.908+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 shutdown_connections 2024-03-16T10:35:23.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:22.908+0000 7f79ed364700 1 -- 172.21.15.2:0/3078093809 wait complete. 2024-03-16T10:35:23.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:23 smithi002 ceph-mon[29248]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:23.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:23 smithi002 ceph-mon[29248]: Generating ssh key... 2024-03-16T10:35:23.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:23 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:23.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:23 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfD3ktJzskklXwp11K0/nBSlFpMprbzlonPgm5WWfzYfDqPcrcfwbqxhW4YxRsXG7jg0V0AccCz5fYSHJASt6KBhKzreozmga8NiFENVMiSIMseFpXBb0oTSFC8e/wttxVXMNYM45r2AwH+rezqyauBhvbRTLAJtjNKRaRjaopi4v3B2KXKuYqaSNiZNwiC4QBewcLWQJFoWIsNiPSiNmDueV8WiTXjOQT4KIW9lp0Ye+iUPeUzKdOzTkgim7kqFWyUXcDt9zfpvQaf6N4PRJ1U5qDQd3FxHhaUxXA8mnZ8m7HmJyRYAiiMQ6FJiXfRc5xnZjgTS3UA7Cs2HEn10N22x9oQI0ONrhldxfnCb1ChrGhM3sqaM54JBqXMMlj8iyb4w3yYfZ9xlXlib20d+XMrO7nSCySXtO0bweJNxv2MZfhqRuaMDks6W8HKezodmtsqB7WN1ROP6su+1lW7dEjUG/jkOJ2SgNNSyDVFxbQK97AjBbE0BxSjv8kdCGYQHM= ceph-b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.864+0000 7f1e38b13700 1 Processor -- start 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.864+0000 7f1e38b13700 1 -- start start 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.864+0000 7f1e38b13700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e34106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.864+0000 7f1e38b13700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e34106d40 con 0x7f1e34106350 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e34106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e34106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32832/0 (socket says 172.21.15.2:32832) 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3259c700 1 -- 172.21.15.2:0/3137508498 learned_addr learned my addr 172.21.15.2:0/3137508498 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3259c700 1 -- 172.21.15.2:0/3137508498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34107550 con 0x7f1e34106350 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3259c700 1 --2- 172.21.15.2:0/3137508498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e34106770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e240098a0 tx=0x7f1e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeb393c4e4bf623 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:24.636 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3159a700 1 -- 172.21.15.2:0/3137508498 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e24004180 con 0x7f1e34106350 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3159a700 1 -- 172.21.15.2:0/3137508498 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1e240042e0 con 0x7f1e34106350 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.865+0000 7f1e3159a700 1 -- 172.21.15.2:0/3137508498 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e24011a70 con 0x7f1e34106350 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.866+0000 7f1e38b13700 1 -- 172.21.15.2:0/3137508498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 msgr2=0x7f1e34106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.866+0000 7f1e38b13700 1 --2- 172.21.15.2:0/3137508498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e34106770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e240098a0 tx=0x7f1e24009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.866+0000 7f1e38b13700 1 -- 172.21.15.2:0/3137508498 shutdown_connections 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.866+0000 7f1e38b13700 1 --2- 172.21.15.2:0/3137508498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e34106770 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.866+0000 7f1e38b13700 1 -- 172.21.15.2:0/3137508498 >> 172.21.15.2:0/3137508498 conn(0x7f1e341018f0 msgr2=0x7f1e34103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.866+0000 7f1e38b13700 1 -- 172.21.15.2:0/3137508498 shutdown_connections 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.866+0000 7f1e38b13700 1 -- 172.21.15.2:0/3137508498 wait complete. 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e38b13700 1 Processor -- start 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e38b13700 1 -- start start 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e38b13700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e3407ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e38b13700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3407bfd0 con 0x7f1e34106350 2024-03-16T10:35:24.637 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e3259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e3407ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e3259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e3407ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32840/0 (socket says 172.21.15.2:32840) 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e3259c700 1 -- 172.21.15.2:0/2377895323 learned_addr learned my addr 172.21.15.2:0/2377895323 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.867+0000 7f1e3259c700 1 -- 172.21.15.2:0/2377895323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e24009550 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.868+0000 7f1e3259c700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e3407ba90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1e24011710 tx=0x7f1e240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.868+0000 7f1e237fe700 1 -- 172.21.15.2:0/2377895323 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e24003b10 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.868+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3407a210 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.868+0000 7f1e237fe700 1 -- 172.21.15.2:0/2377895323 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1e2401b8c0 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.868+0000 7f1e237fe700 1 -- 172.21.15.2:0/2377895323 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e24019770 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.868+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3407a6b0 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.869+0000 7f1e237fe700 1 -- 172.21.15.2:0/2377895323 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f1e240199f0 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.869+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e340623e0 con 0x7f1e34106350 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.869+0000 7f1e237fe700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f1e1c0384f0 0x7f1e1c03a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:24.638 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.869+0000 7f1e237fe700 1 -- 172.21.15.2:0/2377895323 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1e2404c7c0 con 0x7f1e34106350 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.869+0000 7f1e31d9b700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f1e1c0384f0 0x7f1e1c03a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.869+0000 7f1e31d9b700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f1e1c0384f0 0x7f1e1c03a9b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e28006fd0 tx=0x7f1e28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:23.874+0000 7f1e237fe700 1 -- 172.21.15.2:0/2377895323 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1e24018db0 con 0x7f1e34106350 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.075+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e3407b110 con 0x7f1e1c0384f0 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.076+0000 7f1e237fe700 1 -- 172.21.15.2:0/2377895323 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1e3407b110 con 0x7f1e1c0384f0 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f1e1c0384f0 msgr2=0x7f1e1c03a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f1e1c0384f0 0x7f1e1c03a9b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1e28006fd0 tx=0x7f1e28006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 msgr2=0x7f1e3407ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e3407ba90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1e24011710 tx=0x7f1e240047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 shutdown_connections 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f1e1c0384f0 0x7f1e1c03a9b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 --2- 172.21.15.2:0/2377895323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1e34106350 0x7f1e3407ba90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 >> 172.21.15.2:0/2377895323 conn(0x7f1e341018f0 msgr2=0x7f1e341024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 shutdown_connections 2024-03-16T10:35:24.639 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:24.078+0000 7f1e38b13700 1 -- 172.21.15.2:0/2377895323 wait complete. 2024-03-16T10:35:24.640 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-16T10:35:24.640 INFO:teuthology.orchestra.run.smithi002.stdout:Adding key to root@localhost authorized_keys... 2024-03-16T10:35:24.640 INFO:teuthology.orchestra.run.smithi002.stdout:Adding host smithi002... 2024-03-16T10:35:24.931 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:24 smithi002 ceph-mon[29248]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:25.928 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:25 smithi002 ceph-mon[29248]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:27.484 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:27 smithi002 ceph-mon[29248]: Deploying cephadm binary to smithi002 2024-03-16T10:35:28.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Added host 'smithi002' with addr '172.21.15.2' 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.039+0000 7f283740e700 1 Processor -- start 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.039+0000 7f283740e700 1 -- start start 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.039+0000 7f283740e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283007a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.039+0000 7f283740e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283007a800 con 0x7f283007bdd0 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.039+0000 7f28351aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283007a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.039+0000 7f28351aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283007a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32856/0 (socket says 172.21.15.2:32856) 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.039+0000 7f28351aa700 1 -- 172.21.15.2:0/1811873467 learned_addr learned my addr 172.21.15.2:0/1811873467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.040+0000 7f28351aa700 1 -- 172.21.15.2:0/1811873467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283007a940 con 0x7f283007bdd0 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.040+0000 7f28351aa700 1 --2- 172.21.15.2:0/1811873467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283007a230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098a0 tx=0x7f282c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e298d6d1b61f07a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.040+0000 7f2827fff700 1 -- 172.21.15.2:0/1811873467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f282c004180 con 0x7f283007bdd0 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.040+0000 7f2827fff700 1 -- 172.21.15.2:0/1811873467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f282c0042e0 con 0x7f283007bdd0 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.040+0000 7f2827fff700 1 -- 172.21.15.2:0/1811873467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f282c011a70 con 0x7f283007bdd0 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.040+0000 7f283740e700 1 -- 172.21.15.2:0/1811873467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 msgr2=0x7f283007a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.040+0000 7f283740e700 1 --2- 172.21.15.2:0/1811873467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283007a230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f282c0098a0 tx=0x7f282c009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 -- 172.21.15.2:0/1811873467 shutdown_connections 2024-03-16T10:35:28.460 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 --2- 172.21.15.2:0/1811873467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283007a230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 -- 172.21.15.2:0/1811873467 >> 172.21.15.2:0/1811873467 conn(0x7f28301018e0 msgr2=0x7f2830103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 -- 172.21.15.2:0/1811873467 shutdown_connections 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 -- 172.21.15.2:0/1811873467 wait complete. 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 Processor -- start 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 -- start start 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.041+0000 7f283740e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283019c820 con 0x7f283007bdd0 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28351aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28351aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32858/0 (socket says 172.21.15.2:32858) 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28351aa700 1 -- 172.21.15.2:0/639856195 learned_addr learned my addr 172.21.15.2:0/639856195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28351aa700 1 -- 172.21.15.2:0/639856195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c009550 con 0x7f283007bdd0 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28351aa700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283019c2e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f282c011a20 tx=0x7f282c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28267fc700 1 -- 172.21.15.2:0/639856195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f282c003b10 con 0x7f283007bdd0 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28267fc700 1 -- 172.21.15.2:0/639856195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f282c01b8c0 con 0x7f283007bdd0 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283019ca20 con 0x7f283007bdd0 2024-03-16T10:35:28.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f28267fc700 1 -- 172.21.15.2:0/639856195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f282c0237b0 con 0x7f283007bdd0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.042+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283019cec0 con 0x7f283007bdd0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.043+0000 7f28267fc700 1 -- 172.21.15.2:0/639856195 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f282c01b440 con 0x7f283007bdd0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.043+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283010c6c0 con 0x7f283007bdd0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.043+0000 7f28267fc700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f281c038500 0x7f281c03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.043+0000 7f28267fc700 1 -- 172.21.15.2:0/639856195 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f282c04c750 con 0x7f283007bdd0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.043+0000 7f28349a9700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f281c038500 0x7f281c03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.044+0000 7f28349a9700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f281c038500 0x7f281c03a9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2820006fd0 tx=0x7f2820006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.048+0000 7f28267fc700 1 -- 172.21.15.2:0/639856195 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f282c01b6f0 con 0x7f283007bdd0 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:25.254+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2830102be0 con 0x7f281c038500 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.908+0000 7f28267fc700 1 -- 172.21.15.2:0/639856195 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f2830102be0 con 0x7f281c038500 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f281c038500 msgr2=0x7f281c03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f281c038500 0x7f281c03a9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2820006fd0 tx=0x7f2820006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 msgr2=0x7f283019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:28.462 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283019c2e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f282c011a20 tx=0x7f282c0047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:28.463 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 shutdown_connections 2024-03-16T10:35:28.463 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f281c038500 0x7f281c03a9c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:28.463 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 --2- 172.21.15.2:0/639856195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f283007bdd0 0x7f283019c2e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:28.463 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 >> 172.21.15.2:0/639856195 conn(0x7f28301018e0 msgr2=0x7f28301024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:28.463 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 shutdown_connections 2024-03-16T10:35:28.463 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:27.911+0000 7f283740e700 1 -- 172.21.15.2:0/639856195 wait complete. 2024-03-16T10:35:28.463 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying mon service with default placement... 2024-03-16T10:35:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:28 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:28 smithi002 ceph-mon[29248]: Added host smithi002 2024-03-16T10:35:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:28 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:30.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-16T10:35:30.347 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.883+0000 7fdccd5b7700 1 Processor -- start 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.883+0000 7fdccd5b7700 1 -- start start 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.883+0000 7fdccd5b7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8105660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.883+0000 7fdccd5b7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc8105c30 con 0x7fdcc8105240 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.883+0000 7fdcc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.883+0000 7fdcc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8105660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32860/0 (socket says 172.21.15.2:32860) 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.883+0000 7fdcc6ffd700 1 -- 172.21.15.2:0/451731766 learned_addr learned my addr 172.21.15.2:0/451731766 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdcc6ffd700 1 -- 172.21.15.2:0/451731766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc8106440 con 0x7fdcc8105240 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdcc6ffd700 1 --2- 172.21.15.2:0/451731766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8105660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80098a0 tx=0x7fdcb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06a7ad7bba72f10 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdcc5ffb700 1 -- 172.21.15.2:0/451731766 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb8004180 con 0x7fdcc8105240 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdcc5ffb700 1 -- 172.21.15.2:0/451731766 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdcb80042e0 con 0x7fdcc8105240 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdcc5ffb700 1 -- 172.21.15.2:0/451731766 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb8011a70 con 0x7fdcc8105240 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdccd5b7700 1 -- 172.21.15.2:0/451731766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 msgr2=0x7fdcc8105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdccd5b7700 1 --2- 172.21.15.2:0/451731766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8105660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80098a0 tx=0x7fdcb8009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdccd5b7700 1 -- 172.21.15.2:0/451731766 shutdown_connections 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdccd5b7700 1 --2- 172.21.15.2:0/451731766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8105660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:30.348 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdccd5b7700 1 -- 172.21.15.2:0/451731766 >> 172.21.15.2:0/451731766 conn(0x7fdcc8100820 msgr2=0x7fdcc8102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdccd5b7700 1 -- 172.21.15.2:0/451731766 shutdown_connections 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.884+0000 7fdccd5b7700 1 -- 172.21.15.2:0/451731766 wait complete. 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.885+0000 7fdccd5b7700 1 Processor -- start 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.885+0000 7fdccd5b7700 1 -- start start 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.885+0000 7fdccd5b7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8190290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.885+0000 7fdccd5b7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc81907d0 con 0x7fdcc8105240 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.885+0000 7fdcc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8190290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.885+0000 7fdcc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8190290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32864/0 (socket says 172.21.15.2:32864) 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.885+0000 7fdcc6ffd700 1 -- 172.21.15.2:0/2277469607 learned_addr learned my addr 172.21.15.2:0/2277469607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.886+0000 7fdcc6ffd700 1 -- 172.21.15.2:0/2277469607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb8009550 con 0x7fdcc8105240 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.886+0000 7fdcc6ffd700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8190290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011710 tx=0x7fdcb80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.886+0000 7fdcaffff700 1 -- 172.21.15.2:0/2277469607 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb8003b10 con 0x7fdcc8105240 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.886+0000 7fdcaffff700 1 -- 172.21.15.2:0/2277469607 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdcb801b8c0 con 0x7fdcc8105240 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.886+0000 7fdcaffff700 1 -- 172.21.15.2:0/2277469607 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcb8019770 con 0x7fdcc8105240 2024-03-16T10:35:30.349 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.886+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc818ea60 con 0x7fdcc8105240 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.886+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc818ef00 con 0x7fdcc8105240 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.887+0000 7fdcaffff700 1 -- 172.21.15.2:0/2277469607 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7fdcb80199f0 con 0x7fdcc8105240 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.887+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc818f910 con 0x7fdcc8105240 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.887+0000 7fdcaffff700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fdcb00384f0 0x7fdcb003a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.887+0000 7fdcaffff700 1 -- 172.21.15.2:0/2277469607 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdcb804c7c0 con 0x7fdcc8105240 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.887+0000 7fdcc67fc700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fdcb00384f0 0x7fdcb003a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.888+0000 7fdcc67fc700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fdcb00384f0 0x7fdcb003a9b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc006fd0 tx=0x7fdcbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:28.892+0000 7fdcaffff700 1 -- 172.21.15.2:0/2277469607 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdcb8018db0 con 0x7fdcc8105240 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.108+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcc8194210 con 0x7fdcb00384f0 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.113+0000 7fdcaffff700 1 -- 172.21.15.2:0/2277469607 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdcc8194210 con 0x7fdcb00384f0 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fdcb00384f0 msgr2=0x7fdcb003a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fdcb00384f0 0x7fdcb003a9b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc006fd0 tx=0x7fdcbc006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 msgr2=0x7fdcc8190290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:30.350 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8190290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011710 tx=0x7fdcb80047c0 comp rx=0 tx=0).stop 2024-03-16T10:35:30.351 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 shutdown_connections 2024-03-16T10:35:30.351 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fdcb00384f0 0x7fdcb003a9b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:30.351 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 --2- 172.21.15.2:0/2277469607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdcc8105240 0x7fdcc8190290 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:30.351 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 >> 172.21.15.2:0/2277469607 conn(0x7fdcc8100820 msgr2=0x7fdcc8101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:30.351 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 shutdown_connections 2024-03-16T10:35:30.351 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:29.115+0000 7fdccd5b7700 1 -- 172.21.15.2:0/2277469607 wait complete. 2024-03-16T10:35:30.351 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying mgr service with default placement... 2024-03-16T10:35:30.467 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:30 smithi002 ceph-mon[29248]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:30.467 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:30 smithi002 ceph-mon[29248]: Saving service mon spec with placement count:5 2024-03-16T10:35:30.467 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:30 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:30.467 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:30 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:31.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:31 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:31.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:31 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.815+0000 7fa2efd0d700 1 Processor -- start 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.816+0000 7fa2efd0d700 1 -- start start 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.816+0000 7fa2efd0d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e8106790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.816+0000 7fa2efd0d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8106d60 con 0x7fa2e8106370 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.816+0000 7fa2edaa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e8106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.816+0000 7fa2edaa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e8106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32868/0 (socket says 172.21.15.2:32868) 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.816+0000 7fa2edaa9700 1 -- 172.21.15.2:0/376770372 learned_addr learned my addr 172.21.15.2:0/376770372 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.816+0000 7fa2edaa9700 1 -- 172.21.15.2:0/376770372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8107570 con 0x7fa2e8106370 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.817+0000 7fa2edaa9700 1 --2- 172.21.15.2:0/376770372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e8106790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009e80 tx=0x7fa2e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aef9ad3a5b5e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:32.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.817+0000 7fa2ecaa7700 1 -- 172.21.15.2:0/376770372 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2e400e070 con 0x7fa2e8106370 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.817+0000 7fa2ecaa7700 1 -- 172.21.15.2:0/376770372 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2e400b8c0 con 0x7fa2e8106370 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.817+0000 7fa2ecaa7700 1 -- 172.21.15.2:0/376770372 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2e40044d0 con 0x7fa2e8106370 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.817+0000 7fa2efd0d700 1 -- 172.21.15.2:0/376770372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 msgr2=0x7fa2e8106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.817+0000 7fa2efd0d700 1 --2- 172.21.15.2:0/376770372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e8106790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009e80 tx=0x7fa2e400b330 comp rx=0 tx=0).stop 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 -- 172.21.15.2:0/376770372 shutdown_connections 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 --2- 172.21.15.2:0/376770372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e8106790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 -- 172.21.15.2:0/376770372 >> 172.21.15.2:0/376770372 conn(0x7fa2e81018f0 msgr2=0x7fa2e8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 -- 172.21.15.2:0/376770372 shutdown_connections 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 -- 172.21.15.2:0/376770372 wait complete. 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 Processor -- start 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 -- start start 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.818+0000 7fa2efd0d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2efd0d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e819a600 con 0x7fa2e8106370 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2edaa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:32.167 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2edaa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32870/0 (socket says 172.21.15.2:32870) 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2edaa9700 1 -- 172.21.15.2:0/3806520658 learned_addr learned my addr 172.21.15.2:0/3806520658 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2edaa9700 1 -- 172.21.15.2:0/3806520658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4009b30 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2edaa9700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e819a0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40072a0 tx=0x7fa2e400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2deffd700 1 -- 172.21.15.2:0/3806520658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2e400e040 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2deffd700 1 -- 172.21.15.2:0/3806520658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2e4003dc0 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2deffd700 1 -- 172.21.15.2:0/3806520658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2e4012600 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e819a800 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.819+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e819ad00 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.820+0000 7fa2deffd700 1 -- 172.21.15.2:0/3806520658 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7fa2e4019040 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.820+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e810a7a0 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.821+0000 7fa2deffd700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa2d40384e0 0x7fa2d403a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.821+0000 7fa2deffd700 1 -- 172.21.15.2:0/3806520658 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa2e404bf20 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.821+0000 7fa2ed2a8700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa2d40384e0 0x7fa2d403a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.821+0000 7fa2ed2a8700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa2d40384e0 0x7fa2d403a9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:30.826+0000 7fa2deffd700 1 -- 172.21.15.2:0/3806520658 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa2e4011750 con 0x7fa2e8106370 2024-03-16T10:35:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.046+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2e819d950 con 0x7fa2d40384e0 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.050+0000 7fa2deffd700 1 -- 172.21.15.2:0/3806520658 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa2e819d950 con 0x7fa2d40384e0 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa2d40384e0 msgr2=0x7fa2d403a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa2d40384e0 0x7fa2d403a9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 msgr2=0x7fa2e819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e819a0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40072a0 tx=0x7fa2e400bf70 comp rx=0 tx=0).stop 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 shutdown_connections 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa2d40384e0 0x7fa2d403a9a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 --2- 172.21.15.2:0/3806520658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa2e8106370 0x7fa2e819a0c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 >> 172.21.15.2:0/3806520658 conn(0x7fa2e81018f0 msgr2=0x7fa2e8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 shutdown_connections 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:31.052+0000 7fa2efd0d700 1 -- 172.21.15.2:0/3806520658 wait complete. 2024-03-16T10:35:32.169 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying crash service with default placement... 2024-03-16T10:35:32.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:32 smithi002 ceph-mon[29248]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:32.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:32 smithi002 ceph-mon[29248]: Saving service mgr spec with placement count:2 2024-03-16T10:35:32.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:32 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.631+0000 7f13889bd700 1 Processor -- start 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.631+0000 7f13889bd700 1 -- start start 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.631+0000 7f13889bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f1384106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.631+0000 7f13889bd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13841071f0 con 0x7f1384106800 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.631+0000 7f138259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f1384106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.631+0000 7f138259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f1384106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48942/0 (socket says 172.21.15.2:48942) 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.631+0000 7f138259c700 1 -- 172.21.15.2:0/1498717618 learned_addr learned my addr 172.21.15.2:0/1498717618 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.632+0000 7f138259c700 1 -- 172.21.15.2:0/1498717618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384105460 con 0x7f1384106800 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.632+0000 7f138259c700 1 --2- 172.21.15.2:0/1498717618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f1384106c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f13740098a0 tx=0x7f1374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d36a9000a7d5f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.632+0000 7f138159a700 1 -- 172.21.15.2:0/1498717618 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1374004180 con 0x7f1384106800 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.632+0000 7f138159a700 1 -- 172.21.15.2:0/1498717618 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13740042e0 con 0x7f1384106800 2024-03-16T10:35:33.431 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.632+0000 7f138159a700 1 -- 172.21.15.2:0/1498717618 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1374011b10 con 0x7f1384106800 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.633+0000 7f13889bd700 1 -- 172.21.15.2:0/1498717618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 msgr2=0x7f1384106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.633+0000 7f13889bd700 1 --2- 172.21.15.2:0/1498717618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f1384106c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f13740098a0 tx=0x7f1374009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.633+0000 7f13889bd700 1 -- 172.21.15.2:0/1498717618 shutdown_connections 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.633+0000 7f13889bd700 1 --2- 172.21.15.2:0/1498717618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f1384106c20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.633+0000 7f13889bd700 1 -- 172.21.15.2:0/1498717618 >> 172.21.15.2:0/1498717618 conn(0x7f13840fffd0 msgr2=0x7f13841023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.633+0000 7f13889bd700 1 -- 172.21.15.2:0/1498717618 shutdown_connections 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.633+0000 7f13889bd700 1 -- 172.21.15.2:0/1498717618 wait complete. 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f13889bd700 1 Processor -- start 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f13889bd700 1 -- start start 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f13889bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f138419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f13889bd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138419a690 con 0x7f1384106800 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f138259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f138419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f138259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f138419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48952/0 (socket says 172.21.15.2:48952) 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f138259c700 1 -- 172.21.15.2:0/631752970 learned_addr learned my addr 172.21.15.2:0/631752970 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f138259c700 1 -- 172.21.15.2:0/631752970 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374009550 con 0x7f1384106800 2024-03-16T10:35:33.432 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f138259c700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f138419a150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1374009fb0 tx=0x7f1374011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f13737fe700 1 -- 172.21.15.2:0/631752970 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13740038b0 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f13737fe700 1 -- 172.21.15.2:0/631752970 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1374003a10 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.634+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138419a890 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.635+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138419ad30 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.635+0000 7f13737fe700 1 -- 172.21.15.2:0/631752970 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1374019770 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.636+0000 7f13737fe700 1 -- 172.21.15.2:0/631752970 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f13740198d0 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.636+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13640052f0 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.636+0000 7f13737fe700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f136c038500 0x7f136c03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.636+0000 7f13737fe700 1 -- 172.21.15.2:0/631752970 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f137404c4b0 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.636+0000 7f1381d9b700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f136c038500 0x7f136c03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.636+0000 7f1381d9b700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f136c038500 0x7f136c03a9c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1378006fd0 tx=0x7f1378006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.641+0000 7f13737fe700 1 -- 172.21.15.2:0/631752970 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1374022b40 con 0x7f1384106800 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.865+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f1364000bc0 con 0x7f136c038500 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.870+0000 7f13737fe700 1 -- 172.21.15.2:0/631752970 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f1364000bc0 con 0x7f136c038500 2024-03-16T10:35:33.433 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f136c038500 msgr2=0x7f136c03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f136c038500 0x7f136c03a9c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1378006fd0 tx=0x7f1378006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 msgr2=0x7f138419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f138419a150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1374009fb0 tx=0x7f1374011890 comp rx=0 tx=0).stop 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 shutdown_connections 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f136c038500 0x7f136c03a9c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 --2- 172.21.15.2:0/631752970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1384106800 0x7f138419a150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 >> 172.21.15.2:0/631752970 conn(0x7f13840fffd0 msgr2=0x7f1384100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 shutdown_connections 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:32.872+0000 7f13889bd700 1 -- 172.21.15.2:0/631752970 wait complete. 2024-03-16T10:35:33.434 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying ceph-exporter service with default placement... 2024-03-16T10:35:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:33 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:33 smithi002 ceph-mon[29248]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:33 smithi002 ceph-mon[29248]: Saving service crash spec with placement * 2024-03-16T10:35:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:33 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:35.335 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:35 smithi002 ceph-mon[29248]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:35.335 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:35 smithi002 ceph-mon[29248]: Saving service ceph-exporter spec with placement * 2024-03-16T10:35:35.335 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:35 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:35.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.826+0000 7f3a3978f700 1 Processor -- start 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.826+0000 7f3a3978f700 1 -- start start 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.826+0000 7f3a3978f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a341091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.826+0000 7f3a3978f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a34079de0 con 0x7f3a34106df0 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.826+0000 7f3a32ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a341091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.826+0000 7f3a32ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a341091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48956/0 (socket says 172.21.15.2:48956) 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.826+0000 7f3a32ffd700 1 -- 172.21.15.2:0/3604232375 learned_addr learned my addr 172.21.15.2:0/3604232375 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.827+0000 7f3a32ffd700 1 -- 172.21.15.2:0/3604232375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a34109720 con 0x7f3a34106df0 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.827+0000 7f3a32ffd700 1 --2- 172.21.15.2:0/3604232375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a341091e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a240098a0 tx=0x7f3a24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c225438131b1a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.827+0000 7f3a31ffb700 1 -- 172.21.15.2:0/3604232375 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a24004180 con 0x7f3a34106df0 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.827+0000 7f3a31ffb700 1 -- 172.21.15.2:0/3604232375 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a240042e0 con 0x7f3a34106df0 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.827+0000 7f3a31ffb700 1 -- 172.21.15.2:0/3604232375 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a24011b10 con 0x7f3a34106df0 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.828+0000 7f3a3978f700 1 -- 172.21.15.2:0/3604232375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 msgr2=0x7f3a341091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:35.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.828+0000 7f3a3978f700 1 --2- 172.21.15.2:0/3604232375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a341091e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a240098a0 tx=0x7f3a24009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.828+0000 7f3a3978f700 1 -- 172.21.15.2:0/3604232375 shutdown_connections 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.828+0000 7f3a3978f700 1 --2- 172.21.15.2:0/3604232375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a341091e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.828+0000 7f3a3978f700 1 -- 172.21.15.2:0/3604232375 >> 172.21.15.2:0/3604232375 conn(0x7f3a341007e0 msgr2=0x7f3a34102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.828+0000 7f3a3978f700 1 -- 172.21.15.2:0/3604232375 shutdown_connections 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.828+0000 7f3a3978f700 1 -- 172.21.15.2:0/3604232375 wait complete. 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.829+0000 7f3a3978f700 1 Processor -- start 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.829+0000 7f3a3978f700 1 -- start start 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.829+0000 7f3a3978f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a3419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.829+0000 7f3a3978f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3419a600 con 0x7f3a34106df0 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.829+0000 7f3a32ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a3419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.829+0000 7f3a32ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a3419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48960/0 (socket says 172.21.15.2:48960) 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.829+0000 7f3a32ffd700 1 -- 172.21.15.2:0/4102511869 learned_addr learned my addr 172.21.15.2:0/4102511869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.830+0000 7f3a32ffd700 1 -- 172.21.15.2:0/4102511869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a24009550 con 0x7f3a34106df0 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.830+0000 7f3a32ffd700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a3419a0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009fb0 tx=0x7f3a24011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:35.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.830+0000 7f3a1bfff700 1 -- 172.21.15.2:0/4102511869 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a240038b0 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.830+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3419a800 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.830+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3419aca0 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.830+0000 7f3a1bfff700 1 -- 172.21.15.2:0/4102511869 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a24003a10 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.830+0000 7f3a1bfff700 1 -- 172.21.15.2:0/4102511869 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a24019750 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.831+0000 7f3a1bfff700 1 -- 172.21.15.2:0/4102511869 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f3a240198b0 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.831+0000 7f3a1bfff700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f3a1c038500 0x7f3a1c03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.832+0000 7f3a1bfff700 1 -- 172.21.15.2:0/4102511869 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3a2404d540 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.832+0000 7f3a327fc700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f3a1c038500 0x7f3a1c03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.832+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a20005320 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.832+0000 7f3a327fc700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f3a1c038500 0x7f3a1c03a9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:33.837+0000 7f3a1bfff700 1 -- 172.21.15.2:0/4102511869 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3a24022d70 con 0x7f3a34106df0 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.069+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a20000bf0 con 0x7f3a1c038500 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.073+0000 7f3a1bfff700 1 -- 172.21.15.2:0/4102511869 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f3a20000bf0 con 0x7f3a1c038500 2024-03-16T10:35:35.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f3a1c038500 msgr2=0x7f3a1c03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f3a1c038500 0x7f3a1c03a9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 msgr2=0x7f3a3419a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a3419a0c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009fb0 tx=0x7f3a24011890 comp rx=0 tx=0).stop 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 shutdown_connections 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f3a1c038500 0x7f3a1c03a9c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 --2- 172.21.15.2:0/4102511869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a34106df0 0x7f3a3419a0c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.075+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 >> 172.21.15.2:0/4102511869 conn(0x7f3a341007e0 msgr2=0x7f3a34101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.076+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 shutdown_connections 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:34.076+0000 7f3a3978f700 1 -- 172.21.15.2:0/4102511869 wait complete. 2024-03-16T10:35:35.346 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying prometheus service with default placement... 2024-03-16T10:35:36.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-16T10:35:36.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.753+0000 7fbef6660700 1 Processor -- start 2024-03-16T10:35:36.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.753+0000 7fbef6660700 1 -- start start 2024-03-16T10:35:36.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.753+0000 7fbef6660700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef0106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.753+0000 7fbef6660700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0106d30 con 0x7fbef0106340 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef0106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef0106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48966/0 (socket says 172.21.15.2:48966) 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeffff700 1 -- 172.21.15.2:0/533356120 learned_addr learned my addr 172.21.15.2:0/533356120 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeffff700 1 -- 172.21.15.2:0/533356120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0107540 con 0x7fbef0106340 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeffff700 1 --2- 172.21.15.2:0/533356120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef0106760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbee00098a0 tx=0x7fbee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a20b42c29d615 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeeffd700 1 -- 172.21.15.2:0/533356120 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbee0004180 con 0x7fbef0106340 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeeffd700 1 -- 172.21.15.2:0/533356120 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbee00042e0 con 0x7fbef0106340 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.754+0000 7fbeeeffd700 1 -- 172.21.15.2:0/533356120 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbee0011b10 con 0x7fbef0106340 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.755+0000 7fbef6660700 1 -- 172.21.15.2:0/533356120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 msgr2=0x7fbef0106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.755+0000 7fbef6660700 1 --2- 172.21.15.2:0/533356120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef0106760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbee00098a0 tx=0x7fbee0009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.755+0000 7fbef6660700 1 -- 172.21.15.2:0/533356120 shutdown_connections 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.755+0000 7fbef6660700 1 --2- 172.21.15.2:0/533356120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef0106760 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.755+0000 7fbef6660700 1 -- 172.21.15.2:0/533356120 >> 172.21.15.2:0/533356120 conn(0x7fbef01018e0 msgr2=0x7fbef0103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.755+0000 7fbef6660700 1 -- 172.21.15.2:0/533356120 shutdown_connections 2024-03-16T10:35:36.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.755+0000 7fbef6660700 1 -- 172.21.15.2:0/533356120 wait complete. 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbef6660700 1 Processor -- start 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbef6660700 1 -- start start 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbef6660700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef007ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbef6660700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef007bfd0 con 0x7fbef0106340 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbeeffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef007ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbeeffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef007ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48976/0 (socket says 172.21.15.2:48976) 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbeeffff700 1 -- 172.21.15.2:0/164295773 learned_addr learned my addr 172.21.15.2:0/164295773 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbeeffff700 1 -- 172.21.15.2:0/164295773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0009550 con 0x7fbef0106340 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.756+0000 7fbeeffff700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef007ba90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbee00117b0 tx=0x7fbee0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.757+0000 7fbeed7fa700 1 -- 172.21.15.2:0/164295773 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbee00038b0 con 0x7fbef0106340 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.757+0000 7fbeed7fa700 1 -- 172.21.15.2:0/164295773 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbee0003a10 con 0x7fbef0106340 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.757+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef007a210 con 0x7fbef0106340 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.757+0000 7fbeed7fa700 1 -- 172.21.15.2:0/164295773 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbee00246e0 con 0x7fbef0106340 2024-03-16T10:35:36.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.757+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef007a6b0 con 0x7fbef0106340 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.757+0000 7fbeed7fa700 1 -- 172.21.15.2:0/164295773 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7fbee002d3f0 con 0x7fbef0106340 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.758+0000 7fbeed7fa700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fbed8038540 0x7fbed803aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.758+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef00623e0 con 0x7fbef0106340 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.758+0000 7fbeed7fa700 1 -- 172.21.15.2:0/164295773 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbee004c520 con 0x7fbef0106340 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.758+0000 7fbeef7fe700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fbed8038540 0x7fbed803aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.758+0000 7fbeef7fe700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fbed8038540 0x7fbed803aa00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006fd0 tx=0x7fbee4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.763+0000 7fbeed7fa700 1 -- 172.21.15.2:0/164295773 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbee002d6f0 con 0x7fbef0106340 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.977+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef010a710 con 0x7fbed8038540 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.981+0000 7fbeed7fa700 1 -- 172.21.15.2:0/164295773 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fbef010a710 con 0x7fbed8038540 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.983+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fbed8038540 msgr2=0x7fbed803aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.983+0000 7fbef6660700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fbed8038540 0x7fbed803aa00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006fd0 tx=0x7fbee4006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.983+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 msgr2=0x7fbef007ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.983+0000 7fbef6660700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef007ba90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbee00117b0 tx=0x7fbee0011890 comp rx=0 tx=0).stop 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.984+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 shutdown_connections 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.984+0000 7fbef6660700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fbed8038540 0x7fbed803aa00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:36.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.984+0000 7fbef6660700 1 --2- 172.21.15.2:0/164295773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbef0106340 0x7fbef007ba90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:36.544 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.984+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 >> 172.21.15.2:0/164295773 conn(0x7fbef01018e0 msgr2=0x7fbef0102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:36.544 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.984+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 shutdown_connections 2024-03-16T10:35:36.544 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:35.984+0000 7fbef6660700 1 -- 172.21.15.2:0/164295773 wait complete. 2024-03-16T10:35:36.544 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying grafana service with default placement... 2024-03-16T10:35:37.205 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:36 smithi002 ceph-mon[29248]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:37.206 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:36 smithi002 ceph-mon[29248]: Saving service prometheus spec with placement count:1 2024-03-16T10:35:37.206 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:36 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a6f26700 1 Processor -- start 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a6f26700 1 -- start start 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a6f26700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0104a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a6f26700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0105060 con 0x7fc2a0104670 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a4cc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a4cc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0104a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48980/0 (socket says 172.21.15.2:48980) 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a4cc2700 1 -- 172.21.15.2:0/1993170388 learned_addr learned my addr 172.21.15.2:0/1993170388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a4cc2700 1 -- 172.21.15.2:0/1993170388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0107600 con 0x7fc2a0104670 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.980+0000 7fc2a4cc2700 1 --2- 172.21.15.2:0/1993170388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0104a90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2940098a0 tx=0x7fc294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df4d5b8f223d2db server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.982+0000 7fc29f7fe700 1 -- 172.21.15.2:0/1993170388 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc294004180 con 0x7fc2a0104670 2024-03-16T10:35:37.922 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.982+0000 7fc29f7fe700 1 -- 172.21.15.2:0/1993170388 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2940042e0 con 0x7fc2a0104670 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.982+0000 7fc29f7fe700 1 -- 172.21.15.2:0/1993170388 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc294011b10 con 0x7fc2a0104670 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1993170388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 msgr2=0x7fc2a0104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 --2- 172.21.15.2:0/1993170388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0104a90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2940098a0 tx=0x7fc294009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1993170388 shutdown_connections 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 --2- 172.21.15.2:0/1993170388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0104a90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1993170388 >> 172.21.15.2:0/1993170388 conn(0x7fc2a0100000 msgr2=0x7fc2a0102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1993170388 shutdown_connections 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1993170388 wait complete. 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 Processor -- start 2024-03-16T10:35:37.923 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 -- start start 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.983+0000 7fc2a6f26700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0107370 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc2a4cc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc2a4cc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0106e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48986/0 (socket says 172.21.15.2:48986) 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc2a4cc2700 1 -- 172.21.15.2:0/1509957487 learned_addr learned my addr 172.21.15.2:0/1509957487 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc2a4cc2700 1 -- 172.21.15.2:0/1509957487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294009550 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc2a4cc2700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0106e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2940117b0 tx=0x7fc294011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc29dffb700 1 -- 172.21.15.2:0/1509957487 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2940038b0 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc29dffb700 1 -- 172.21.15.2:0/1509957487 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc294003a10 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc29dffb700 1 -- 172.21.15.2:0/1509957487 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2940246e0 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a01055b0 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.984+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a0105a50 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.985+0000 7fc29dffb700 1 -- 172.21.15.2:0/1509957487 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7fc29401ab80 con 0x7fc2a0104670 2024-03-16T10:35:37.924 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.985+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a00623e0 con 0x7fc2a0104670 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.985+0000 7fc29dffb700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fc2900384f0 0x7fc29003a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.985+0000 7fc29dffb700 1 -- 172.21.15.2:0/1509957487 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc29404c580 con 0x7fc2a0104670 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.985+0000 7fc29ffff700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fc2900384f0 0x7fc29003a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.986+0000 7fc29ffff700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fc2900384f0 0x7fc29003a9b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006fd0 tx=0x7fc28c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:36.990+0000 7fc29dffb700 1 -- 172.21.15.2:0/1509957487 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc294018ba0 con 0x7fc2a0104670 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.204+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2a0106470 con 0x7fc2900384f0 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.209+0000 7fc29dffb700 1 -- 172.21.15.2:0/1509957487 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc2a0106470 con 0x7fc2900384f0 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fc2900384f0 msgr2=0x7fc29003a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fc2900384f0 0x7fc29003a9b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006fd0 tx=0x7fc28c006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 msgr2=0x7fc2a0106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0106e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2940117b0 tx=0x7fc294011890 comp rx=0 tx=0).stop 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 shutdown_connections 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fc2900384f0 0x7fc29003a9b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 --2- 172.21.15.2:0/1509957487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc2a0104670 0x7fc2a0106e30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 >> 172.21.15.2:0/1509957487 conn(0x7fc2a0100000 msgr2=0x7fc2a0100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:37.926 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 shutdown_connections 2024-03-16T10:35:37.926 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:37.211+0000 7fc2a6f26700 1 -- 172.21.15.2:0/1509957487 wait complete. 2024-03-16T10:35:37.926 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying node-exporter service with default placement... 2024-03-16T10:35:38.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:38 smithi002 ceph-mon[29248]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:38.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:38 smithi002 ceph-mon[29248]: Saving service grafana spec with placement count:1 2024-03-16T10:35:38.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:38 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:38.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:38 smithi002 ceph-mon[29248]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:35:39.161 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.343+0000 7fe102f83700 1 Processor -- start 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.343+0000 7fe102f83700 1 -- start start 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.343+0000 7fe102f83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.343+0000 7fe102f83700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc079dc0 con 0x7fe0fc106e40 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.343+0000 7fe100d1f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.343+0000 7fe100d1f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48992/0 (socket says 172.21.15.2:48992) 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.343+0000 7fe100d1f700 1 -- 172.21.15.2:0/159860775 learned_addr learned my addr 172.21.15.2:0/159860775 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:39.162 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.344+0000 7fe100d1f700 1 -- 172.21.15.2:0/159860775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc109770 con 0x7fe0fc106e40 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.344+0000 7fe100d1f700 1 --2- 172.21.15.2:0/159860775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc109230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00098a0 tx=0x7fe0f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3169cacdeb5cbf88 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.344+0000 7fe0fb7fe700 1 -- 172.21.15.2:0/159860775 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0f0004180 con 0x7fe0fc106e40 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.344+0000 7fe0fb7fe700 1 -- 172.21.15.2:0/159860775 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0f00042e0 con 0x7fe0fc106e40 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.344+0000 7fe0fb7fe700 1 -- 172.21.15.2:0/159860775 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0f0011b10 con 0x7fe0fc106e40 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.344+0000 7fe102f83700 1 -- 172.21.15.2:0/159860775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 msgr2=0x7fe0fc109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.344+0000 7fe102f83700 1 --2- 172.21.15.2:0/159860775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc109230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00098a0 tx=0x7fe0f0009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 -- 172.21.15.2:0/159860775 shutdown_connections 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 --2- 172.21.15.2:0/159860775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc109230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 -- 172.21.15.2:0/159860775 >> 172.21.15.2:0/159860775 conn(0x7fe0fc100870 msgr2=0x7fe0fc102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 -- 172.21.15.2:0/159860775 shutdown_connections 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 -- 172.21.15.2:0/159860775 wait complete. 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 Processor -- start 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 -- start start 2024-03-16T10:35:39.163 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.345+0000 7fe102f83700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc19a660 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe100d1f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe100d1f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49008/0 (socket says 172.21.15.2:49008) 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe100d1f700 1 -- 172.21.15.2:0/1909138882 learned_addr learned my addr 172.21.15.2:0/1909138882 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe100d1f700 1 -- 172.21.15.2:0/1909138882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f0009550 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe100d1f700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc19a120 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00117b0 tx=0x7fe0f0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe0f9ffb700 1 -- 172.21.15.2:0/1909138882 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0f00038b0 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe0f9ffb700 1 -- 172.21.15.2:0/1909138882 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0f0003a10 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc19a860 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe0f9ffb700 1 -- 172.21.15.2:0/1909138882 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0f0024780 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.346+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc19ad00 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.347+0000 7fe0f9ffb700 1 -- 172.21.15.2:0/1909138882 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7fe0f00248e0 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.347+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc10a500 con 0x7fe0fc106e40 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.347+0000 7fe0f9ffb700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe0ec038540 0x7fe0ec03aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:39.164 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.348+0000 7fe0f9ffb700 1 -- 172.21.15.2:0/1909138882 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe0f004c5b0 con 0x7fe0fc106e40 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.348+0000 7fe0fbfff700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe0ec038540 0x7fe0ec03aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.348+0000 7fe0fbfff700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe0ec038540 0x7fe0ec03aa00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8006fd0 tx=0x7fe0e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.352+0000 7fe0f9ffb700 1 -- 172.21.15.2:0/1909138882 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe0f0018db0 con 0x7fe0fc106e40 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.569+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0fc101aa0 con 0x7fe0ec038540 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.573+0000 7fe0f9ffb700 1 -- 172.21.15.2:0/1909138882 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fe0fc101aa0 con 0x7fe0ec038540 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe0ec038540 msgr2=0x7fe0ec03aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe0ec038540 0x7fe0ec03aa00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8006fd0 tx=0x7fe0e8006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 msgr2=0x7fe0fc19a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc19a120 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00117b0 tx=0x7fe0f0011890 comp rx=0 tx=0).stop 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 shutdown_connections 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe0ec038540 0x7fe0ec03aa00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:39.165 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 --2- 172.21.15.2:0/1909138882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0fc106e40 0x7fe0fc19a120 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:39.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 >> 172.21.15.2:0/1909138882 conn(0x7fe0fc100870 msgr2=0x7fe0fc101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:39.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.575+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 shutdown_connections 2024-03-16T10:35:39.166 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:38.576+0000 7fe102f83700 1 -- 172.21.15.2:0/1909138882 wait complete. 2024-03-16T10:35:39.166 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying alertmanager service with default placement... 2024-03-16T10:35:39.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:39 smithi002 ceph-mon[29248]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:39.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:39 smithi002 ceph-mon[29248]: Saving service node-exporter spec with placement * 2024-03-16T10:35:39.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:39 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.591+0000 7fe204d3e700 1 Processor -- start 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.591+0000 7fe204d3e700 1 -- start start 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.591+0000 7fe204d3e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe200106c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.591+0000 7fe204d3e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200107240 con 0x7fe200106850 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe200106c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe200106c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49010/0 (socket says 172.21.15.2:49010) 2024-03-16T10:35:40.393 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fe59c700 1 -- 172.21.15.2:0/931805547 learned_addr learned my addr 172.21.15.2:0/931805547 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fe59c700 1 -- 172.21.15.2:0/931805547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe200107380 con 0x7fe200106850 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fe59c700 1 --2- 172.21.15.2:0/931805547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe200106c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009e80 tx=0x7fe1f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be6be47c4134e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fd59a700 1 -- 172.21.15.2:0/931805547 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1f000e070 con 0x7fe200106850 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fd59a700 1 -- 172.21.15.2:0/931805547 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1f000b8c0 con 0x7fe200106850 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.592+0000 7fe1fd59a700 1 -- 172.21.15.2:0/931805547 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1f00044d0 con 0x7fe200106850 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.593+0000 7fe204d3e700 1 -- 172.21.15.2:0/931805547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 msgr2=0x7fe200106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.593+0000 7fe204d3e700 1 --2- 172.21.15.2:0/931805547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe200106c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009e80 tx=0x7fe1f000b330 comp rx=0 tx=0).stop 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.593+0000 7fe204d3e700 1 -- 172.21.15.2:0/931805547 shutdown_connections 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.593+0000 7fe204d3e700 1 --2- 172.21.15.2:0/931805547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe200106c70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.593+0000 7fe204d3e700 1 -- 172.21.15.2:0/931805547 >> 172.21.15.2:0/931805547 conn(0x7fe200100000 msgr2=0x7fe200102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.593+0000 7fe204d3e700 1 -- 172.21.15.2:0/931805547 shutdown_connections 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.593+0000 7fe204d3e700 1 -- 172.21.15.2:0/931805547 wait complete. 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.594+0000 7fe204d3e700 1 Processor -- start 2024-03-16T10:35:40.394 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.594+0000 7fe204d3e700 1 -- start start 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.594+0000 7fe204d3e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe20019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.594+0000 7fe204d3e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20019c7f0 con 0x7fe200106850 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.594+0000 7fe1fe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe20019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.594+0000 7fe1fe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe20019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49016/0 (socket says 172.21.15.2:49016) 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.594+0000 7fe1fe59c700 1 -- 172.21.15.2:0/2901582288 learned_addr learned my addr 172.21.15.2:0/2901582288 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.595+0000 7fe1fe59c700 1 -- 172.21.15.2:0/2901582288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0009b30 con 0x7fe200106850 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.595+0000 7fe1fe59c700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe20019c2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00072a0 tx=0x7fe1f000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.595+0000 7fe1f77fe700 1 -- 172.21.15.2:0/2901582288 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1f000e040 con 0x7fe200106850 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.595+0000 7fe1f77fe700 1 -- 172.21.15.2:0/2901582288 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1f0003dc0 con 0x7fe200106850 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.595+0000 7fe1f77fe700 1 -- 172.21.15.2:0/2901582288 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1f0012620 con 0x7fe200106850 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.595+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20019c9f0 con 0x7fe200106850 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.595+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20019cef0 con 0x7fe200106850 2024-03-16T10:35:40.395 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.596+0000 7fe1f77fe700 1 -- 172.21.15.2:0/2901582288 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7fe1f0019040 con 0x7fe200106850 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.596+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe200104fe0 con 0x7fe200106850 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.596+0000 7fe1f77fe700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe1ec0384e0 0x7fe1ec03a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.596+0000 7fe1f77fe700 1 -- 172.21.15.2:0/2901582288 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe1f004c080 con 0x7fe200106850 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.596+0000 7fe1fdd9b700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe1ec0384e0 0x7fe1ec03a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.597+0000 7fe1fdd9b700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe1ec0384e0 0x7fe1ec03a9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.601+0000 7fe1f77fe700 1 -- 172.21.15.2:0/2901582288 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe1f002cb40 con 0x7fe200106850 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.817+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2000611f0 con 0x7fe1ec0384e0 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.822+0000 7fe1f77fe700 1 -- 172.21.15.2:0/2901582288 <== mgr.14120 v2:172.21.15.2:6800/3739300318 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fe2000611f0 con 0x7fe1ec0384e0 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe1ec0384e0 msgr2=0x7fe1ec03a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe1ec0384e0 0x7fe1ec03a9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8006fd0 tx=0x7fe1e8006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 msgr2=0x7fe20019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe20019c2b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00072a0 tx=0x7fe1f000bf70 comp rx=0 tx=0).stop 2024-03-16T10:35:40.396 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 shutdown_connections 2024-03-16T10:35:40.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe1ec0384e0 0x7fe1ec03a9a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:40.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 --2- 172.21.15.2:0/2901582288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe200106850 0x7fe20019c2b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:40.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 >> 172.21.15.2:0/2901582288 conn(0x7fe200100000 msgr2=0x7fe200192ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:40.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 shutdown_connections 2024-03-16T10:35:40.397 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:39.824+0000 7fe204d3e700 1 -- 172.21.15.2:0/2901582288 wait complete. 2024-03-16T10:35:40.729 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:40 smithi002 ceph-mon[29248]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:35:40.729 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:40 smithi002 ceph-mon[29248]: from='mgr.14120 172.21.15.2:0/1034049435' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:41.783 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:41 smithi002 ceph-mon[29248]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:41.783 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:41 smithi002 ceph-mon[29248]: Saving service alertmanager spec with placement count:1 2024-03-16T10:35:41.783 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:41 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1519329742' entity='client.admin' 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa4552b6700 1 Processor -- start 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa4552b6700 1 -- start start 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa4552b6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa450106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa4552b6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450106d50 con 0x7fa450106360 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa44effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa450106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa44effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa450106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49032/0 (socket says 172.21.15.2:49032) 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa44effd700 1 -- 172.21.15.2:0/2454887783 learned_addr learned my addr 172.21.15.2:0/2454887783 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.838+0000 7fa44effd700 1 -- 172.21.15.2:0/2454887783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa450107560 con 0x7fa450106360 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa44effd700 1 --2- 172.21.15.2:0/2454887783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa450106780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa4400098a0 tx=0x7fa440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698a0589a2bcaabf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa44dffb700 1 -- 172.21.15.2:0/2454887783 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa440004180 con 0x7fa450106360 2024-03-16T10:35:41.791 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa44dffb700 1 -- 172.21.15.2:0/2454887783 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4400042e0 con 0x7fa450106360 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa44dffb700 1 -- 172.21.15.2:0/2454887783 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa440011b10 con 0x7fa450106360 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa4552b6700 1 -- 172.21.15.2:0/2454887783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 msgr2=0x7fa450106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa4552b6700 1 --2- 172.21.15.2:0/2454887783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa450106780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa4400098a0 tx=0x7fa440009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa4552b6700 1 -- 172.21.15.2:0/2454887783 shutdown_connections 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa4552b6700 1 --2- 172.21.15.2:0/2454887783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa450106780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa4552b6700 1 -- 172.21.15.2:0/2454887783 >> 172.21.15.2:0/2454887783 conn(0x7fa450101940 msgr2=0x7fa450103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa4552b6700 1 -- 172.21.15.2:0/2454887783 shutdown_connections 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.839+0000 7fa4552b6700 1 -- 172.21.15.2:0/2454887783 wait complete. 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.840+0000 7fa4552b6700 1 Processor -- start 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.840+0000 7fa4552b6700 1 -- start start 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.840+0000 7fa4552b6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa45019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.840+0000 7fa4552b6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45019a630 con 0x7fa450106360 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.840+0000 7fa44effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa45019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.840+0000 7fa44effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa45019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49038/0 (socket says 172.21.15.2:49038) 2024-03-16T10:35:41.792 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.840+0000 7fa44effd700 1 -- 172.21.15.2:0/1519329742 learned_addr learned my addr 172.21.15.2:0/1519329742 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.841+0000 7fa44effd700 1 -- 172.21.15.2:0/1519329742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440009550 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.841+0000 7fa44effd700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa45019a0f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa440009fb0 tx=0x7fa440011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.841+0000 7fa437fff700 1 -- 172.21.15.2:0/1519329742 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4400038b0 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.841+0000 7fa437fff700 1 -- 172.21.15.2:0/1519329742 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa440003a10 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.841+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45019a830 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.841+0000 7fa437fff700 1 -- 172.21.15.2:0/1519329742 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa440019770 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.841+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45019acd0 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.842+0000 7fa437fff700 1 -- 172.21.15.2:0/1519329742 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7fa4400198d0 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.842+0000 7fa437fff700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa438038500 0x7fa43803a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.842+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45010a790 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.842+0000 7fa437fff700 1 -- 172.21.15.2:0/1519329742 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa44004c560 con 0x7fa450106360 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.842+0000 7fa44e7fc700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa438038500 0x7fa43803a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.843+0000 7fa44e7fc700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa438038500 0x7fa43803a9c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa444006fd0 tx=0x7fa444006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:41.793 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:40.847+0000 7fa437fff700 1 -- 172.21.15.2:0/1519329742 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa440019b80 con 0x7fa450106360 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.047+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa45019dd50 con 0x7fa450106360 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.050+0000 7fa437fff700 1 -- 172.21.15.2:0/1519329742 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa440022310 con 0x7fa450106360 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.052+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa438038500 msgr2=0x7fa43803a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.052+0000 7fa4552b6700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa438038500 0x7fa43803a9c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa444006fd0 tx=0x7fa444006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.052+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 msgr2=0x7fa45019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.052+0000 7fa4552b6700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa45019a0f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa440009fb0 tx=0x7fa440011890 comp rx=0 tx=0).stop 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.053+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 shutdown_connections 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.053+0000 7fa4552b6700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fa438038500 0x7fa43803a9c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.053+0000 7fa4552b6700 1 --2- 172.21.15.2:0/1519329742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa450106360 0x7fa45019a0f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.053+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 >> 172.21.15.2:0/1519329742 conn(0x7fa450101940 msgr2=0x7fa4501024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.053+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 shutdown_connections 2024-03-16T10:35:41.794 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:41.053+0000 7fa4552b6700 1 -- 172.21.15.2:0/1519329742 wait complete. 2024-03-16T10:35:42.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:42 smithi002 ceph-mon[29248]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:35:42.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:42 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/452859119' entity='client.admin' 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.249+0000 7f5086f4f700 1 Processor -- start 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.249+0000 7f5086f4f700 1 -- start start 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.249+0000 7f5086f4f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f5080105670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.249+0000 7f5086f4f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5080105c40 con 0x7f5080105250 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f5084ceb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f5080105670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f5084ceb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f5080105670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49040/0 (socket says 172.21.15.2:49040) 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f5084ceb700 1 -- 172.21.15.2:0/920875039 learned_addr learned my addr 172.21.15.2:0/920875039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:43.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f5084ceb700 1 -- 172.21.15.2:0/920875039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5080106450 con 0x7f5080105250 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f5084ceb700 1 --2- 172.21.15.2:0/920875039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f5080105670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5078009e80 tx=0x7f507800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7306f8d6d15bb162 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f507f7fe700 1 -- 172.21.15.2:0/920875039 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f507800e070 con 0x7f5080105250 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f507f7fe700 1 -- 172.21.15.2:0/920875039 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f507800b8c0 con 0x7f5080105250 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.250+0000 7f507f7fe700 1 -- 172.21.15.2:0/920875039 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50780044d0 con 0x7f5080105250 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.251+0000 7f5086f4f700 1 -- 172.21.15.2:0/920875039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 msgr2=0x7f5080105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.251+0000 7f5086f4f700 1 --2- 172.21.15.2:0/920875039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f5080105670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5078009e80 tx=0x7f507800b330 comp rx=0 tx=0).stop 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.251+0000 7f5086f4f700 1 -- 172.21.15.2:0/920875039 shutdown_connections 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.251+0000 7f5086f4f700 1 --2- 172.21.15.2:0/920875039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f5080105670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.251+0000 7f5086f4f700 1 -- 172.21.15.2:0/920875039 >> 172.21.15.2:0/920875039 conn(0x7f5080100810 msgr2=0x7f5080102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.251+0000 7f5086f4f700 1 -- 172.21.15.2:0/920875039 shutdown_connections 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.251+0000 7f5086f4f700 1 -- 172.21.15.2:0/920875039 wait complete. 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5086f4f700 1 Processor -- start 2024-03-16T10:35:43.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5086f4f700 1 -- start start 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5086f4f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f508019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5086f4f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508019a660 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5084ceb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f508019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5084ceb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f508019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49046/0 (socket says 172.21.15.2:49046) 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5084ceb700 1 -- 172.21.15.2:0/452859119 learned_addr learned my addr 172.21.15.2:0/452859119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.252+0000 7f5084ceb700 1 -- 172.21.15.2:0/452859119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078009b30 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.253+0000 7f5084ceb700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f508019a120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5078000f80 tx=0x7f507800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.253+0000 7f507dffb700 1 -- 172.21.15.2:0/452859119 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f507800e040 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.253+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508019a860 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.253+0000 7f507dffb700 1 -- 172.21.15.2:0/452859119 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5078003c30 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.253+0000 7f507dffb700 1 -- 172.21.15.2:0/452859119 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5078012670 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.253+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508019ad60 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.254+0000 7f507dffb700 1 -- 172.21.15.2:0/452859119 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7f5078019040 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.254+0000 7f507dffb700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f506c0384f0 0x7f506c03a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.254+0000 7f507dffb700 1 -- 172.21.15.2:0/452859119 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f507804ceb0 con 0x7f5080105250 2024-03-16T10:35:43.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.254+0000 7f507ffff700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f506c0384f0 0x7f506c03a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.254+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5080194290 con 0x7f5080105250 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.255+0000 7f507ffff700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f506c0384f0 0x7f506c03a9b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5070006fd0 tx=0x7f5070006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.259+0000 7f507dffb700 1 -- 172.21.15.2:0/452859119 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f507801f080 con 0x7f5080105250 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.461+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f508002d0d0 con 0x7f5080105250 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.463+0000 7f507dffb700 1 -- 172.21.15.2:0/452859119 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f507804b0c0 con 0x7f5080105250 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f506c0384f0 msgr2=0x7f506c03a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f506c0384f0 0x7f506c03a9b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5070006fd0 tx=0x7f5070006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 msgr2=0x7f508019a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f508019a120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5078000f80 tx=0x7f507800bd10 comp rx=0 tx=0).stop 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 shutdown_connections 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7f506c0384f0 0x7f506c03a9b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 --2- 172.21.15.2:0/452859119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5080105250 0x7f508019a120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 >> 172.21.15.2:0/452859119 conn(0x7f5080100810 msgr2=0x7f50801016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:43.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 shutdown_connections 2024-03-16T10:35:43.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:42.466+0000 7f5086f4f700 1 -- 172.21.15.2:0/452859119 wait complete. 2024-03-16T10:35:43.017 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling the dashboard module... 2024-03-16T10:35:44.598 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:44 smithi002 ceph-mon[29248]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:35:44.599 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:44 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2141865722' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.465+0000 7faa41f73700 1 Processor -- start 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.465+0000 7faa41f73700 1 -- start start 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.465+0000 7faa41f73700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c1091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.465+0000 7faa41f73700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c079e20 con 0x7faa3c106dc0 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c1091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c1091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49732/0 (socket says 172.21.15.2:49732) 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3b7fe700 1 -- 172.21.15.2:0/4212847534 learned_addr learned my addr 172.21.15.2:0/4212847534 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3b7fe700 1 -- 172.21.15.2:0/4212847534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c1096f0 con 0x7faa3c106dc0 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3b7fe700 1 --2- 172.21.15.2:0/4212847534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c1091b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009e80 tx=0x7faa2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a2d5b94a700af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3a7fc700 1 -- 172.21.15.2:0/4212847534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa2c00e070 con 0x7faa3c106dc0 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3a7fc700 1 -- 172.21.15.2:0/4212847534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa2c00b8c0 con 0x7faa3c106dc0 2024-03-16T10:35:45.196 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.466+0000 7faa3a7fc700 1 -- 172.21.15.2:0/4212847534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa2c0044d0 con 0x7faa3c106dc0 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.467+0000 7faa41f73700 1 -- 172.21.15.2:0/4212847534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 msgr2=0x7faa3c1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.467+0000 7faa41f73700 1 --2- 172.21.15.2:0/4212847534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c1091b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009e80 tx=0x7faa2c00b330 comp rx=0 tx=0).stop 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.467+0000 7faa41f73700 1 -- 172.21.15.2:0/4212847534 shutdown_connections 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.467+0000 7faa41f73700 1 --2- 172.21.15.2:0/4212847534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c1091b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.467+0000 7faa41f73700 1 -- 172.21.15.2:0/4212847534 >> 172.21.15.2:0/4212847534 conn(0x7faa3c100810 msgr2=0x7faa3c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.467+0000 7faa41f73700 1 -- 172.21.15.2:0/4212847534 shutdown_connections 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.467+0000 7faa41f73700 1 -- 172.21.15.2:0/4212847534 wait complete. 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa41f73700 1 Processor -- start 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa41f73700 1 -- start start 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa41f73700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa41f73700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c19a5d0 con 0x7faa3c106dc0 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49748/0 (socket says 172.21.15.2:49748) 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa3b7fe700 1 -- 172.21.15.2:0/2141865722 learned_addr learned my addr 172.21.15.2:0/2141865722 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa3b7fe700 1 -- 172.21.15.2:0/2141865722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c009b30 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.468+0000 7faa3b7fe700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c19a090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0072a0 tx=0x7faa2c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.469+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa2c00e040 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.469+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa2c003dc0 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.469+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa2c012750 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.469+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c19a7d0 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.469+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c19acd0 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.470+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45389+0+0 (secure 0 0 0) 0x7faa2c019040 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.470+0000 7faa38ff9700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7faa28038540 0x7faa2803aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.470+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c10a4e0 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.470+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faa2c04bdc0 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.470+0000 7faa3affd700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7faa28038540 0x7faa2803aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.470+0000 7faa3affd700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7faa28038540 0x7faa2803aa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa24006fd0 tx=0x7faa24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.475+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa2c01b430 con 0x7faa3c106dc0 2024-03-16T10:35:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:43.734+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7faa3c10a700 con 0x7faa3c106dc0 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.579+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 9) v1 ==== 45402+0+0 (secure 0 0 0) 0x7faa2c01bbf0 con 0x7faa3c106dc0 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.580+0000 7faa38ff9700 1 -- 172.21.15.2:0/2141865722 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7faa2c01b7b0 con 0x7faa3c106dc0 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7faa28038540 msgr2=0x7faa2803aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7faa28038540 0x7faa2803aa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa24006fd0 tx=0x7faa24006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 msgr2=0x7faa3c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c19a090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0072a0 tx=0x7faa2c00be00 comp rx=0 tx=0).stop 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 shutdown_connections 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7faa28038540 0x7faa2803aa00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 --2- 172.21.15.2:0/2141865722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa3c106dc0 0x7faa3c19a090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.583+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 >> 172.21.15.2:0/2141865722 conn(0x7faa3c100810 msgr2=0x7faa3c101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.584+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 shutdown_connections 2024-03-16T10:35:45.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:44.584+0000 7faa41f73700 1 -- 172.21.15.2:0/2141865722 wait complete. 2024-03-16T10:35:45.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:45 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2141865722' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-16T10:35:45.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:45 smithi002 ceph-mon[29248]: mgrmap e9: smithi002.zevrtm(active, since 27s) 2024-03-16T10:35:46.478 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:35:46.478 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-16T10:35:46.478 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "smithi002.zevrtm", 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.623+0000 7ff909359700 1 Processor -- start 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.623+0000 7ff909359700 1 -- start start 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.623+0000 7ff909359700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff904106790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.623+0000 7ff909359700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff904106d60 con 0x7ff904106370 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.623+0000 7ff902ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff904106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.624+0000 7ff902ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff904106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49772/0 (socket says 172.21.15.2:49772) 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.624+0000 7ff902ffd700 1 -- 172.21.15.2:0/3083179811 learned_addr learned my addr 172.21.15.2:0/3083179811 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.624+0000 7ff902ffd700 1 -- 172.21.15.2:0/3083179811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff904107570 con 0x7ff904106370 2024-03-16T10:35:46.479 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.624+0000 7ff902ffd700 1 --2- 172.21.15.2:0/3083179811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff904106790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80098a0 tx=0x7ff8f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d69c759a1a0079 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.624+0000 7ff901ffb700 1 -- 172.21.15.2:0/3083179811 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8f8004180 con 0x7ff904106370 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.624+0000 7ff901ffb700 1 -- 172.21.15.2:0/3083179811 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8f80042e0 con 0x7ff904106370 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.624+0000 7ff901ffb700 1 -- 172.21.15.2:0/3083179811 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8f8011b10 con 0x7ff904106370 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 -- 172.21.15.2:0/3083179811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 msgr2=0x7ff904106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 --2- 172.21.15.2:0/3083179811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff904106790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80098a0 tx=0x7ff8f8009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 -- 172.21.15.2:0/3083179811 shutdown_connections 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 --2- 172.21.15.2:0/3083179811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff904106790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 -- 172.21.15.2:0/3083179811 >> 172.21.15.2:0/3083179811 conn(0x7ff9041018f0 msgr2=0x7ff904103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 -- 172.21.15.2:0/3083179811 shutdown_connections 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 -- 172.21.15.2:0/3083179811 wait complete. 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.625+0000 7ff909359700 1 Processor -- start 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff909359700 1 -- start start 2024-03-16T10:35:46.480 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff909359700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff90419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff909359700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90419a660 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff902ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff90419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff902ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff90419a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49786/0 (socket says 172.21.15.2:49786) 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff902ffd700 1 -- 172.21.15.2:0/51049240 learned_addr learned my addr 172.21.15.2:0/51049240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff902ffd700 1 -- 172.21.15.2:0/51049240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f8009550 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.626+0000 7ff902ffd700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff90419a120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009fb0 tx=0x7ff8f8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.627+0000 7ff8ebfff700 1 -- 172.21.15.2:0/51049240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8f80038b0 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.627+0000 7ff8ebfff700 1 -- 172.21.15.2:0/51049240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8f8003a10 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.627+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90419a860 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.627+0000 7ff8ebfff700 1 -- 172.21.15.2:0/51049240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8f8019770 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.627+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90419ad00 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.627+0000 7ff8ebfff700 1 -- 172.21.15.2:0/51049240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45402+0+0 (secure 0 0 0) 0x7ff8f80198d0 con 0x7ff904106370 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.628+0000 7ff8ebfff700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 0x7ff8ec03aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:46.481 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.628+0000 7ff8ebfff700 1 -- 172.21.15.2:0/51049240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff8f804d5b0 con 0x7ff904106370 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.628+0000 7ff9027fc700 1 -- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 msgr2=0x7ff8ec03aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3739300318 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.628+0000 7ff9027fc700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 0x7ff8ec03aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.628+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9040623e0 con 0x7ff904106370 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.633+0000 7ff8ebfff700 1 -- 172.21.15.2:0/51049240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff8f8019bd0 con 0x7ff904106370 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.829+0000 7ff9027fc700 1 -- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 msgr2=0x7ff8ec03aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3739300318 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.829+0000 7ff9027fc700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 0x7ff8ec03aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.895+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff90419dc10 con 0x7ff904106370 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.896+0000 7ff8ebfff700 1 -- 172.21.15.2:0/51049240 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7ff8f8022310 con 0x7ff904106370 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 msgr2=0x7ff8ec03aa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 0x7ff8ec03aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 msgr2=0x7ff90419a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:46.482 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff90419a120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009fb0 tx=0x7ff8f8011890 comp rx=0 tx=0).stop 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 shutdown_connections 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7ff8ec038550 0x7ff8ec03aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 --2- 172.21.15.2:0/51049240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff904106370 0x7ff90419a120 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 >> 172.21.15.2:0/51049240 conn(0x7ff9041018f0 msgr2=0x7ff9041024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 shutdown_connections 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:45.898+0000 7ff909359700 1 -- 172.21.15.2:0/51049240 wait complete. 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-03-16T10:35:46.483 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 9... 2024-03-16T10:35:46.753 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:46 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/51049240' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: Active manager daemon smithi002.zevrtm restarted 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: Activating manager daemon smithi002.zevrtm 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: osdmap e3: 0 total, 0 up, 0 in 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: mgrmap e10: smithi002.zevrtm(active, starting, since 0.00343216s) 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:35:52.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:35:52.531 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:35:52.531 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:52.531 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:35:52.531 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:35:52.531 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:35:53.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-03-16T10:35:53.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-16T10:35:53.413 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.880+0000 7fe813dca700 1 Processor -- start 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.880+0000 7fe813dca700 1 -- start start 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.880+0000 7fe813dca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c105660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.880+0000 7fe813dca700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c105c30 con 0x7fe80c105240 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe811b66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe811b66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c105660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49790/0 (socket says 172.21.15.2:49790) 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe811b66700 1 -- 172.21.15.2:0/1380673158 learned_addr learned my addr 172.21.15.2:0/1380673158 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe811b66700 1 -- 172.21.15.2:0/1380673158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c106440 con 0x7fe80c105240 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe811b66700 1 --2- 172.21.15.2:0/1380673158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c105660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00bb40 tx=0x7fe7fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9593fab1f64a9ec server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe810b64700 1 -- 172.21.15.2:0/1380673158 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7fc00dc00 con 0x7fe80c105240 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe810b64700 1 -- 172.21.15.2:0/1380673158 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7fc0044f0 con 0x7fe80c105240 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.881+0000 7fe810b64700 1 -- 172.21.15.2:0/1380673158 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7fc010470 con 0x7fe80c105240 2024-03-16T10:35:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.882+0000 7fe813dca700 1 -- 172.21.15.2:0/1380673158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 msgr2=0x7fe80c105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.882+0000 7fe813dca700 1 --2- 172.21.15.2:0/1380673158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c105660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00bb40 tx=0x7fe7fc00be50 comp rx=0 tx=0).stop 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.882+0000 7fe813dca700 1 -- 172.21.15.2:0/1380673158 shutdown_connections 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.882+0000 7fe813dca700 1 --2- 172.21.15.2:0/1380673158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c105660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.882+0000 7fe813dca700 1 -- 172.21.15.2:0/1380673158 >> 172.21.15.2:0/1380673158 conn(0x7fe80c100820 msgr2=0x7fe80c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.882+0000 7fe813dca700 1 -- 172.21.15.2:0/1380673158 shutdown_connections 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.882+0000 7fe813dca700 1 -- 172.21.15.2:0/1380673158 wait complete. 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe813dca700 1 Processor -- start 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe813dca700 1 -- start start 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe813dca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe813dca700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c19c870 con 0x7fe80c105240 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe811b66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe811b66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c19c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49800/0 (socket says 172.21.15.2:49800) 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe811b66700 1 -- 172.21.15.2:0/1901410750 learned_addr learned my addr 172.21.15.2:0/1901410750 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe811b66700 1 -- 172.21.15.2:0/1901410750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc00b7f0 con 0x7fe80c105240 2024-03-16T10:35:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.883+0000 7fe811b66700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c19c330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00b340 tx=0x7fe7fc004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.884+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7fc00de20 con 0x7fe80c105240 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.884+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7fc010bb0 con 0x7fe80c105240 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.884+0000 7fe813dca700 1 -- 172.21.15.2:0/1901410750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c19ca70 con 0x7fe80c105240 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.884+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7fc021b30 con 0x7fe80c105240 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.884+0000 7fe813dca700 1 -- 172.21.15.2:0/1901410750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80c19ceb0 con 0x7fe80c105240 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.885+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45402+0+0 (secure 0 0 0) 0x7fe7fc010d20 con 0x7fe80c105240 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.885+0000 7fe802ffd700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.885+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 --> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7f803b120 con 0x7fe7f8038550 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.885+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe7fc04c7d0 con 0x7fe80c105240 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.885+0000 7fe811365700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 msgr2=0x7fe7f803aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3739300318 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:46.885+0000 7fe811365700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:47.086+0000 7fe811365700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 msgr2=0x7fe7f803aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3739300318 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:47.086+0000 7fe811365700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:35:53.416 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:47.487+0000 7fe811365700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 msgr2=0x7fe7f803aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3739300318 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:47.487+0000 7fe811365700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:48.287+0000 7fe811365700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 msgr2=0x7fe7f803aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3739300318 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:48.287+0000 7fe811365700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:49.888+0000 7fe811365700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 msgr2=0x7fe7f803aa10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3739300318 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:49.888+0000 7fe811365700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:51.599+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 10) v1 ==== 45169+0+0 (secure 0 0 0) 0x7fe7fc01ed60 con 0x7fe80c105240 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:51.599+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 msgr2=0x7fe7f803aa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:51.599+0000 7fe802ffd700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.600+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 11) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fe7fc021410 con 0x7fe80c105240 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.600+0000 7fe802ffd700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fe7f803bfc0 0x7fe7f803e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.600+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7f803b120 con 0x7fe7f803bfc0 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.601+0000 7fe811365700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fe7f803bfc0 0x7fe7f803e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.601+0000 7fe811365700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fe7f803bfc0 0x7fe7f803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe808003a10 tx=0x7fe808009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:53.417 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.602+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mgr.14164 v2:172.21.15.2:6800/486800969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fe7f803b120 con 0x7fe7f803bfc0 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.608+0000 7fe800ff9700 1 -- 172.21.15.2:0/1901410750 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe7f4002600 con 0x7fe7f803bfc0 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.608+0000 7fe802ffd700 1 -- 172.21.15.2:0/1901410750 <== mgr.14164 v2:172.21.15.2:6800/486800969 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fe7f4002600 con 0x7fe7f803bfc0 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.608+0000 7fe800ff9700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fe7f803bfc0 msgr2=0x7fe7f803e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.608+0000 7fe800ff9700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fe7f803bfc0 0x7fe7f803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe808003a10 tx=0x7fe808009270 comp rx=0 tx=0).stop 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.608+0000 7fe800ff9700 1 -- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 msgr2=0x7fe80c19c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.608+0000 7fe800ff9700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c19c330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00b340 tx=0x7fe7fc004070 comp rx=0 tx=0).stop 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.609+0000 7fe800ff9700 1 -- 172.21.15.2:0/1901410750 shutdown_connections 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.609+0000 7fe800ff9700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fe7f803bfc0 0x7fe7f803e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.609+0000 7fe800ff9700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe80c105240 0x7fe80c19c330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.609+0000 7fe800ff9700 1 --2- 172.21.15.2:0/1901410750 >> [v2:172.21.15.2:6800/3739300318,v1:172.21.15.2:6801/3739300318] conn(0x7fe7f8038550 0x7fe7f803aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.609+0000 7fe800ff9700 1 -- 172.21.15.2:0/1901410750 >> 172.21.15.2:0/1901410750 conn(0x7fe80c100820 msgr2=0x7fe80c18e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.609+0000 7fe800ff9700 1 -- 172.21.15.2:0/1901410750 shutdown_connections 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:52.609+0000 7fe800ff9700 1 -- 172.21.15.2:0/1901410750 wait complete. 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 9 is available 2024-03-16T10:35:53.418 INFO:teuthology.orchestra.run.smithi002.stdout:Generating a dashboard self-signed certificate... 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:51] ENGINE Bus STARTING 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:52] ENGINE Serving on https://172.21.15.2:7150 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:52] ENGINE Serving on http://172.21.15.2:8765 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: [16/Mar/2024:10:35:52] ENGINE Bus STARTED 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: mgrmap e11: smithi002.zevrtm(active, since 1.00554s) 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T10:35:53.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:53 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.863+0000 7ff6bb33a700 1 Processor -- start 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6bb33a700 1 -- start start 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6bb33a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b41055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6bb33a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4105bb0 con 0x7ff6b41051c0 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6b90d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b41055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6b90d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b41055e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44850/0 (socket says 172.21.15.2:44850) 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6b90d6700 1 -- 172.21.15.2:0/3842122860 learned_addr learned my addr 172.21.15.2:0/3842122860 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6b90d6700 1 -- 172.21.15.2:0/3842122860 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b41063c0 con 0x7ff6b41051c0 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.864+0000 7ff6b90d6700 1 --2- 172.21.15.2:0/3842122860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b41055e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00098a0 tx=0x7ff6b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfbe5731d3ce9d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:54.725 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6abfff700 1 -- 172.21.15.2:0/3842122860 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6b0004180 con 0x7ff6b41051c0 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6abfff700 1 -- 172.21.15.2:0/3842122860 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6b00042e0 con 0x7ff6b41051c0 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6abfff700 1 -- 172.21.15.2:0/3842122860 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6b0011b10 con 0x7ff6b41051c0 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6bb33a700 1 -- 172.21.15.2:0/3842122860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 msgr2=0x7ff6b41055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6bb33a700 1 --2- 172.21.15.2:0/3842122860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b41055e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00098a0 tx=0x7ff6b0009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6bb33a700 1 -- 172.21.15.2:0/3842122860 shutdown_connections 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6bb33a700 1 --2- 172.21.15.2:0/3842122860 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b41055e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.865+0000 7ff6bb33a700 1 -- 172.21.15.2:0/3842122860 >> 172.21.15.2:0/3842122860 conn(0x7ff6b4100780 msgr2=0x7ff6b4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.866+0000 7ff6bb33a700 1 -- 172.21.15.2:0/3842122860 shutdown_connections 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.866+0000 7ff6bb33a700 1 -- 172.21.15.2:0/3842122860 wait complete. 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.866+0000 7ff6bb33a700 1 Processor -- start 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.866+0000 7ff6bb33a700 1 -- start start 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.866+0000 7ff6bb33a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b4198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.866+0000 7ff6bb33a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b41994f0 con 0x7ff6b41051c0 2024-03-16T10:35:54.726 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6b90d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b4198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:54.727 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6b90d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b4198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44854/0 (socket says 172.21.15.2:44854) 2024-03-16T10:35:54.727 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6b90d6700 1 -- 172.21.15.2:0/1070219877 learned_addr learned my addr 172.21.15.2:0/1070219877 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:54.727 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6b90d6700 1 -- 172.21.15.2:0/1070219877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0009550 con 0x7ff6b41051c0 2024-03-16T10:35:54.727 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6b90d6700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b4198fb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009fb0 tx=0x7ff6b0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:54.729 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6aa7fc700 1 -- 172.21.15.2:0/1070219877 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6b00038b0 con 0x7ff6b41051c0 2024-03-16T10:35:54.729 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6aa7fc700 1 -- 172.21.15.2:0/1070219877 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6b0003a10 con 0x7ff6b41051c0 2024-03-16T10:35:54.729 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b41996f0 con 0x7ff6b41051c0 2024-03-16T10:35:54.729 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.867+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b4199b90 con 0x7ff6b41051c0 2024-03-16T10:35:54.729 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.868+0000 7ff6aa7fc700 1 -- 172.21.15.2:0/1070219877 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6b0019770 con 0x7ff6b41051c0 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.868+0000 7ff6aa7fc700 1 -- 172.21.15.2:0/1070219877 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 11) v1 ==== 45296+0+0 (secure 0 0 0) 0x7ff6b00198d0 con 0x7ff6b41051c0 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.869+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b41095d0 con 0x7ff6b41051c0 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.869+0000 7ff6aa7fc700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7ff6a0038430 0x7ff6a003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.869+0000 7ff6aa7fc700 1 -- 172.21.15.2:0/1070219877 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff6b004d460 con 0x7ff6b41051c0 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.869+0000 7ff6b88d5700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7ff6a0038430 0x7ff6a003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.869+0000 7ff6b88d5700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7ff6a0038430 0x7ff6a003a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4006fd0 tx=0x7ff6a4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:53.878+0000 7ff6aa7fc700 1 -- 172.21.15.2:0/1070219877 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff6b0022d70 con 0x7ff6b41051c0 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.099+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6b419c8a0 con 0x7ff6a0038430 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.176+0000 7ff6aa7fc700 1 -- 172.21.15.2:0/1070219877 <== mgr.14164 v2:172.21.15.2:6800/486800969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ff6b419c8a0 con 0x7ff6a0038430 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7ff6a0038430 msgr2=0x7ff6a003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7ff6a0038430 0x7ff6a003a8f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4006fd0 tx=0x7ff6a4006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 msgr2=0x7ff6b4198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b4198fb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009fb0 tx=0x7ff6b0011890 comp rx=0 tx=0).stop 2024-03-16T10:35:54.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 shutdown_connections 2024-03-16T10:35:54.731 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7ff6a0038430 0x7ff6a003a8f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:54.731 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 --2- 172.21.15.2:0/1070219877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b41051c0 0x7ff6b4198fb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:54.731 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 >> 172.21.15.2:0/1070219877 conn(0x7ff6b4100780 msgr2=0x7ff6b4101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:54.731 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 shutdown_connections 2024-03-16T10:35:54.731 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:54.178+0000 7ff6bb33a700 1 -- 172.21.15.2:0/1070219877 wait complete. 2024-03-16T10:35:54.731 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial admin user... 2024-03-16T10:35:55.111 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:54 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:55.112 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:54 smithi002 ceph-mon[29248]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:55.112 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:54 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:55.112 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:54 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:55.112 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:54 smithi002 ceph-mon[29248]: mgrmap e12: smithi002.zevrtm(active, since 3s) 2024-03-16T10:35:56.846 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:56 smithi002 ceph-mon[29248]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:35:56.846 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:56 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:35:56.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ykRJrPPYaetw1X/bID5uxeLk8qoXB2GgUAclloM8eGBa6i1i9I.DG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710585355, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-16T10:35:56.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.139+0000 7fdee2d23700 1 Processor -- start 2024-03-16T10:35:56.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.139+0000 7fdee2d23700 1 -- start start 2024-03-16T10:35:56.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdee2d23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc079160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:56.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdee2d23700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc079730 con 0x7fdedc07ad00 2024-03-16T10:35:56.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdee0abf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc079160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdee0abf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc079160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44868/0 (socket says 172.21.15.2:44868) 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdee0abf700 1 -- 172.21.15.2:0/106548606 learned_addr learned my addr 172.21.15.2:0/106548606 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdee0abf700 1 -- 172.21.15.2:0/106548606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc079870 con 0x7fdedc07ad00 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdee0abf700 1 --2- 172.21.15.2:0/106548606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc079160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fded00098a0 tx=0x7fded0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50da6bab24657e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdedb7fe700 1 -- 172.21.15.2:0/106548606 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded0004180 con 0x7fdedc07ad00 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdedb7fe700 1 -- 172.21.15.2:0/106548606 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fded00042e0 con 0x7fdedc07ad00 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.140+0000 7fdedb7fe700 1 -- 172.21.15.2:0/106548606 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded0011b10 con 0x7fdedc07ad00 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.141+0000 7fdee2d23700 1 -- 172.21.15.2:0/106548606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 msgr2=0x7fdedc079160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.141+0000 7fdee2d23700 1 --2- 172.21.15.2:0/106548606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc079160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fded00098a0 tx=0x7fded0009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.141+0000 7fdee2d23700 1 -- 172.21.15.2:0/106548606 shutdown_connections 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.141+0000 7fdee2d23700 1 --2- 172.21.15.2:0/106548606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc079160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.141+0000 7fdee2d23700 1 -- 172.21.15.2:0/106548606 >> 172.21.15.2:0/106548606 conn(0x7fdedc1018e0 msgr2=0x7fdedc103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.141+0000 7fdee2d23700 1 -- 172.21.15.2:0/106548606 shutdown_connections 2024-03-16T10:35:56.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.141+0000 7fdee2d23700 1 -- 172.21.15.2:0/106548606 wait complete. 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee2d23700 1 Processor -- start 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee2d23700 1 -- start start 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee2d23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee2d23700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc19c830 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee0abf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee0abf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44882/0 (socket says 172.21.15.2:44882) 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee0abf700 1 -- 172.21.15.2:0/2731887509 learned_addr learned my addr 172.21.15.2:0/2731887509 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee0abf700 1 -- 172.21.15.2:0/2731887509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0009550 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee0abf700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc19c2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fded0009fb0 tx=0x7fded0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fded9ffb700 1 -- 172.21.15.2:0/2731887509 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded00038b0 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc19ca30 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.142+0000 7fded9ffb700 1 -- 172.21.15.2:0/2731887509 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fded0003a10 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.143+0000 7fded9ffb700 1 -- 172.21.15.2:0/2731887509 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded0019770 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.143+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdedc19ced0 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.143+0000 7fded9ffb700 1 -- 172.21.15.2:0/2731887509 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45402+0+0 (secure 0 0 0) 0x7fded001ac10 con 0x7fdedc07ad00 2024-03-16T10:35:56.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.144+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdedc10c700 con 0x7fdedc07ad00 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.144+0000 7fded9ffb700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fdecc038500 0x7fdecc03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.144+0000 7fded9ffb700 1 -- 172.21.15.2:0/2731887509 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fded004c630 con 0x7fdedc07ad00 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.144+0000 7fdedbfff700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fdecc038500 0x7fdecc03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.144+0000 7fdedbfff700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fdecc038500 0x7fdecc03a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006fd0 tx=0x7fdec8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.148+0000 7fded9ffb700 1 -- 172.21.15.2:0/2731887509 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fded0022700 con 0x7fdedc07ad00 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.374+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdedc10c8c0 con 0x7fdecc038500 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.621+0000 7fded9ffb700 1 -- 172.21.15.2:0/2731887509 <== mgr.14164 v2:172.21.15.2:6800/486800969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fdedc10c8c0 con 0x7fdecc038500 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fdecc038500 msgr2=0x7fdecc03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fdecc038500 0x7fdecc03a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdec8006fd0 tx=0x7fdec8006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 msgr2=0x7fdedc19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc19c2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fded0009fb0 tx=0x7fded0011890 comp rx=0 tx=0).stop 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 shutdown_connections 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fdecc038500 0x7fdecc03a9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:56.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 --2- 172.21.15.2:0/2731887509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdedc07ad00 0x7fdedc19c2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 >> 172.21.15.2:0/2731887509 conn(0x7fdedc1018e0 msgr2=0x7fdedc102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 shutdown_connections 2024-03-16T10:35:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:55.623+0000 7fdee2d23700 1 -- 172.21.15.2:0/2731887509 wait complete. 2024-03-16T10:35:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:Fetching dashboard port number... 2024-03-16T10:35:57.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:57 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2588737114' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 8443 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.268+0000 7fd61c59c700 1 Processor -- start 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.268+0000 7fd61c59c700 1 -- start start 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.268+0000 7fd61c59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614105670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.268+0000 7fd61c59c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd614105c40 con 0x7fd614105250 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.268+0000 7fd61a338700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614105670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.268+0000 7fd61a338700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614105670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44886/0 (socket says 172.21.15.2:44886) 2024-03-16T10:35:58.054 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd61a338700 1 -- 172.21.15.2:0/3865263675 learned_addr learned my addr 172.21.15.2:0/3865263675 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd61a338700 1 -- 172.21.15.2:0/3865263675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614106450 con 0x7fd614105250 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd61a338700 1 --2- 172.21.15.2:0/3865263675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614105670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd604009e80 tx=0x7fd60400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e71eea13a0c107a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd619336700 1 -- 172.21.15.2:0/3865263675 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd60400e070 con 0x7fd614105250 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd619336700 1 -- 172.21.15.2:0/3865263675 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd60400b8c0 con 0x7fd614105250 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd619336700 1 -- 172.21.15.2:0/3865263675 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6040044d0 con 0x7fd614105250 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd61c59c700 1 -- 172.21.15.2:0/3865263675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 msgr2=0x7fd614105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.269+0000 7fd61c59c700 1 --2- 172.21.15.2:0/3865263675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614105670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd604009e80 tx=0x7fd60400b330 comp rx=0 tx=0).stop 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 -- 172.21.15.2:0/3865263675 shutdown_connections 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 --2- 172.21.15.2:0/3865263675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614105670 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 -- 172.21.15.2:0/3865263675 >> 172.21.15.2:0/3865263675 conn(0x7fd614100810 msgr2=0x7fd614102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 -- 172.21.15.2:0/3865263675 shutdown_connections 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 -- 172.21.15.2:0/3865263675 wait complete. 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 Processor -- start 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 -- start start 2024-03-16T10:35:58.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61c59c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6141994e0 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.270+0000 7fd61a338700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd61a338700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44892/0 (socket says 172.21.15.2:44892) 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd61a338700 1 -- 172.21.15.2:0/2588737114 learned_addr learned my addr 172.21.15.2:0/2588737114 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd61a338700 1 -- 172.21.15.2:0/2588737114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd604009b30 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd61a338700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614198fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd604000f80 tx=0x7fd60400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd6137fe700 1 -- 172.21.15.2:0/2588737114 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd60400e040 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd6137fe700 1 -- 172.21.15.2:0/2588737114 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd604003c30 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6141996e0 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd6137fe700 1 -- 172.21.15.2:0/2588737114 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd604012670 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.271+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd614199be0 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.272+0000 7fd6137fe700 1 -- 172.21.15.2:0/2588737114 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45402+0+0 (secure 0 0 0) 0x7fd604019040 con 0x7fd614105250 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.272+0000 7fd6137fe700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd5fc038590 0x7fd5fc03aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:58.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.272+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd614109680 con 0x7fd614105250 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.272+0000 7fd6137fe700 1 -- 172.21.15.2:0/2588737114 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd60404bf80 con 0x7fd614105250 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.272+0000 7fd619b37700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd5fc038590 0x7fd5fc03aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.273+0000 7fd619b37700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd5fc038590 0x7fd5fc03aa50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd608006fd0 tx=0x7fd608006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.278+0000 7fd6137fe700 1 -- 172.21.15.2:0/2588737114 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd60401f080 con 0x7fd614105250 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.479+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd61419c830 con 0x7fd614105250 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.480+0000 7fd6137fe700 1 -- 172.21.15.2:0/2588737114 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd60404a0b0 con 0x7fd614105250 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd5fc038590 msgr2=0x7fd5fc03aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd5fc038590 0x7fd5fc03aa50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd608006fd0 tx=0x7fd608006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 msgr2=0x7fd614198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614198fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd604000f80 tx=0x7fd60400bd10 comp rx=0 tx=0).stop 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 shutdown_connections 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd5fc038590 0x7fd5fc03aa50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 --2- 172.21.15.2:0/2588737114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd614105250 0x7fd614198fa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 >> 172.21.15.2:0/2588737114 conn(0x7fd614100810 msgr2=0x7fd614101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:58.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 shutdown_connections 2024-03-16T10:35:58.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:57.482+0000 7fd61c59c700 1 -- 172.21.15.2:0/2588737114 wait complete. 2024-03-16T10:35:58.061 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T10:35:58.061 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-03-16T10:35:58.068 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T10:35:58.068 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-03-16T10:35:58.068 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-03-16T10:35:58.068 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-16T10:35:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph Dashboard is now available at: 2024-03-16T10:35:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:35:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: URL: https://smithi002.front.sepia.ceph.com:8443/ 2024-03-16T10:35:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: User: admin 2024-03-16T10:35:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: Password: o17buz8wor 2024-03-16T10:35:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:35:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:Saving cluster configuration to /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config directory 2024-03-16T10:35:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling autotune for osd_memory_target 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.471+0000 7face8c65700 1 Processor -- start 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.471+0000 7face8c65700 1 -- start start 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.471+0000 7face8c65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face4106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.471+0000 7face8c65700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face4106d40 con 0x7face4106350 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.471+0000 7face259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face4106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.471+0000 7face259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face4106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44906/0 (socket says 172.21.15.2:44906) 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.471+0000 7face259c700 1 -- 172.21.15.2:0/1709358768 learned_addr learned my addr 172.21.15.2:0/1709358768 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.472+0000 7face259c700 1 -- 172.21.15.2:0/1709358768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4107550 con 0x7face4106350 2024-03-16T10:35:59.278 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.472+0000 7face259c700 1 --2- 172.21.15.2:0/1709358768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face4106770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7facd40098a0 tx=0x7facd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16322b6a96b864b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.472+0000 7face159a700 1 -- 172.21.15.2:0/1709358768 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd4004180 con 0x7face4106350 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.472+0000 7face159a700 1 -- 172.21.15.2:0/1709358768 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facd40042e0 con 0x7face4106350 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.472+0000 7face159a700 1 -- 172.21.15.2:0/1709358768 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd4011b10 con 0x7face4106350 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.472+0000 7face8c65700 1 -- 172.21.15.2:0/1709358768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 msgr2=0x7face4106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.472+0000 7face8c65700 1 --2- 172.21.15.2:0/1709358768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face4106770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7facd40098a0 tx=0x7facd4009bb0 comp rx=0 tx=0).stop 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.473+0000 7face8c65700 1 -- 172.21.15.2:0/1709358768 shutdown_connections 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.473+0000 7face8c65700 1 --2- 172.21.15.2:0/1709358768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face4106770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.473+0000 7face8c65700 1 -- 172.21.15.2:0/1709358768 >> 172.21.15.2:0/1709358768 conn(0x7face4101910 msgr2=0x7face4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.473+0000 7face8c65700 1 -- 172.21.15.2:0/1709358768 shutdown_connections 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.473+0000 7face8c65700 1 -- 172.21.15.2:0/1709358768 wait complete. 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.473+0000 7face8c65700 1 Processor -- start 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.473+0000 7face8c65700 1 -- start start 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face8c65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face8c65700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face419a620 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44918/0 (socket says 172.21.15.2:44918) 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face259c700 1 -- 172.21.15.2:0/528188032 learned_addr learned my addr 172.21.15.2:0/528188032 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face259c700 1 -- 172.21.15.2:0/528188032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd4009550 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face259c700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face419a0e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facd40117b0 tx=0x7facd4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd40038b0 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facd4003a10 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face419a820 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facd40246e0 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.474+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face419acc0 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.475+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45402+0+0 (secure 0 0 0) 0x7facd402d3f0 con 0x7face4106350 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.475+0000 7facd37fe700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7faccc038590 0x7faccc03aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:35:59.280 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.475+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7facd404d5d0 con 0x7face4106350 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.475+0000 7face1d9b700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7faccc038590 0x7faccc03aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.476+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face40623e0 con 0x7face4106350 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.476+0000 7face1d9b700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7faccc038590 0x7faccc03aa50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7facd8006fd0 tx=0x7facd8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.480+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7facd402d740 con 0x7face4106350 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.621+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7facd4021d60 con 0x7face4106350 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.680+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7face419dc50 con 0x7face4106350 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.681+0000 7facd37fe700 1 -- 172.21.15.2:0/528188032 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7facd4010e30 con 0x7face4106350 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.682+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7faccc038590 msgr2=0x7faccc03aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.682+0000 7face8c65700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7faccc038590 0x7faccc03aa50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7facd8006fd0 tx=0x7facd8006e40 comp rx=0 tx=0).stop 2024-03-16T10:35:59.281 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.682+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 msgr2=0x7face419a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:35:59.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.682+0000 7face8c65700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face419a0e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facd40117b0 tx=0x7facd4011890 comp rx=0 tx=0).stop 2024-03-16T10:35:59.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.683+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 shutdown_connections 2024-03-16T10:35:59.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.683+0000 7face8c65700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7faccc038590 0x7faccc03aa50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:59.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.683+0000 7face8c65700 1 --2- 172.21.15.2:0/528188032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7face4106350 0x7face419a0e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:35:59.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.683+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 >> 172.21.15.2:0/528188032 conn(0x7face4101910 msgr2=0x7face41024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:35:59.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.683+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 shutdown_connections 2024-03-16T10:35:59.282 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:58.683+0000 7face8c65700 1 -- 172.21.15.2:0/528188032 wait complete. 2024-03-16T10:35:59.689 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:35:59 smithi002 ceph-mon[29248]: mgrmap e13: smithi002.zevrtm(active, since 7s) 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62d632b700 1 Processor -- start 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62d632b700 1 -- start start 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62d632b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d0106770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62d632b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0106d40 con 0x7f62d0106350 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d0106770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d0106770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44934/0 (socket says 172.21.15.2:44934) 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62cffff700 1 -- 172.21.15.2:0/2361333242 learned_addr learned my addr 172.21.15.2:0/2361333242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.707+0000 7f62cffff700 1 -- 172.21.15.2:0/2361333242 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d0107550 con 0x7f62d0106350 2024-03-16T10:36:00.584 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62cffff700 1 --2- 172.21.15.2:0/2361333242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d0106770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098a0 tx=0x7f62c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad37f3ba33e760a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62ceffd700 1 -- 172.21.15.2:0/2361333242 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62c0004180 con 0x7f62d0106350 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62ceffd700 1 -- 172.21.15.2:0/2361333242 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62c00042e0 con 0x7f62d0106350 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62ceffd700 1 -- 172.21.15.2:0/2361333242 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62c0011b10 con 0x7f62d0106350 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62d632b700 1 -- 172.21.15.2:0/2361333242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 msgr2=0x7f62d0106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62d632b700 1 --2- 172.21.15.2:0/2361333242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d0106770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62c00098a0 tx=0x7f62c0009bb0 comp rx=0 tx=0).stop 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62d632b700 1 -- 172.21.15.2:0/2361333242 shutdown_connections 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62d632b700 1 --2- 172.21.15.2:0/2361333242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d0106770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.708+0000 7f62d632b700 1 -- 172.21.15.2:0/2361333242 >> 172.21.15.2:0/2361333242 conn(0x7f62d0101910 msgr2=0x7f62d0103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.709+0000 7f62d632b700 1 -- 172.21.15.2:0/2361333242 shutdown_connections 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.709+0000 7f62d632b700 1 -- 172.21.15.2:0/2361333242 wait complete. 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.709+0000 7f62d632b700 1 Processor -- start 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.709+0000 7f62d632b700 1 -- start start 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.709+0000 7f62d632b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:00.585 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.709+0000 7f62d632b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d019a640 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44944/0 (socket says 172.21.15.2:44944) 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cffff700 1 -- 172.21.15.2:0/1187081412 learned_addr learned my addr 172.21.15.2:0/1187081412 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cffff700 1 -- 172.21.15.2:0/1187081412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0009550 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cffff700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d019a100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62c00117b0 tx=0x7f62c0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cd7fa700 1 -- 172.21.15.2:0/1187081412 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62c00038b0 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cd7fa700 1 -- 172.21.15.2:0/1187081412 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62c0003a10 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62cd7fa700 1 -- 172.21.15.2:0/1187081412 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f62c00246e0 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d019a840 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.710+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d019ace0 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.711+0000 7f62cd7fa700 1 -- 172.21.15.2:0/1187081412 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f62c001a430 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.711+0000 7f62cd7fa700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f62b8038630 0x7f62b803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.711+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d010a780 con 0x7f62d0106350 2024-03-16T10:36:00.586 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.711+0000 7f62cd7fa700 1 -- 172.21.15.2:0/1187081412 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f62c004c700 con 0x7f62d0106350 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.711+0000 7f62cf7fe700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f62b8038630 0x7f62b803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.712+0000 7f62cf7fe700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f62b8038630 0x7f62b803aaf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62c4006fd0 tx=0x7f62c4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:35:59.716+0000 7f62cd7fa700 1 -- 172.21.15.2:0/1187081412 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62c001a780 con 0x7f62d0106350 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.041+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f62d019d990 con 0x7f62d0106350 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.043+0000 7f62cd7fa700 1 -- 172.21.15.2:0/1187081412 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f62c0018b40 con 0x7f62d0106350 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f62b8038630 msgr2=0x7f62b803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f62b8038630 0x7f62b803aaf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62c4006fd0 tx=0x7f62c4006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 msgr2=0x7f62d019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d019a100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62c00117b0 tx=0x7f62c0011890 comp rx=0 tx=0).stop 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 shutdown_connections 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f62b8038630 0x7f62b803aaf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 --2- 172.21.15.2:0/1187081412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62d0106350 0x7f62d019a100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 >> 172.21.15.2:0/1187081412 conn(0x7f62d0101910 msgr2=0x7f62d01024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 shutdown_connections 2024-03-16T10:36:00.587 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-03-16T10:36:00.046+0000 7f62d632b700 1 -- 172.21.15.2:0/1187081412 wait complete. 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout:Or, if you are only running a single cluster on this host: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: ceph telemetry on 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout:For more information see: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.588 INFO:teuthology.orchestra.run.smithi002.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-16T10:36:00.589 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:00.589 INFO:teuthology.orchestra.run.smithi002.stdout:Bootstrap complete. 2024-03-16T10:36:00.620 INFO:tasks.cephadm:Fetching config... 2024-03-16T10:36:00.621 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:36:00.621 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-16T10:36:00.636 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-16T10:36:00.636 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:36:00.636 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-16T10:36:00.691 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-16T10:36:00.692 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:36:00.692 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/keyring of=/dev/stdout 2024-03-16T10:36:00.759 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-16T10:36:00.759 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:36:00.759 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-16T10:36:00.813 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-16T10:36:00.813 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfD3ktJzskklXwp11K0/nBSlFpMprbzlonPgm5WWfzYfDqPcrcfwbqxhW4YxRsXG7jg0V0AccCz5fYSHJASt6KBhKzreozmga8NiFENVMiSIMseFpXBb0oTSFC8e/wttxVXMNYM45r2AwH+rezqyauBhvbRTLAJtjNKRaRjaopi4v3B2KXKuYqaSNiZNwiC4QBewcLWQJFoWIsNiPSiNmDueV8WiTXjOQT4KIW9lp0Ye+iUPeUzKdOzTkgim7kqFWyUXcDt9zfpvQaf6N4PRJ1U5qDQd3FxHhaUxXA8mnZ8m7HmJyRYAiiMQ6FJiXfRc5xnZjgTS3UA7Cs2HEn10N22x9oQI0ONrhldxfnCb1ChrGhM3sqaM54JBqXMMlj8iyb4w3yYfZ9xlXlib20d+XMrO7nSCySXtO0bweJNxv2MZfhqRuaMDks6W8HKezodmtsqB7WN1ROP6su+1lW7dEjUG/jkOJ2SgNNSyDVFxbQK97AjBbE0BxSjv8kdCGYQHM= ceph-b3aed5e6-e380-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T10:36:00.891 INFO:teuthology.orchestra.run.smithi002.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfD3ktJzskklXwp11K0/nBSlFpMprbzlonPgm5WWfzYfDqPcrcfwbqxhW4YxRsXG7jg0V0AccCz5fYSHJASt6KBhKzreozmga8NiFENVMiSIMseFpXBb0oTSFC8e/wttxVXMNYM45r2AwH+rezqyauBhvbRTLAJtjNKRaRjaopi4v3B2KXKuYqaSNiZNwiC4QBewcLWQJFoWIsNiPSiNmDueV8WiTXjOQT4KIW9lp0Ye+iUPeUzKdOzTkgim7kqFWyUXcDt9zfpvQaf6N4PRJ1U5qDQd3FxHhaUxXA8mnZ8m7HmJyRYAiiMQ6FJiXfRc5xnZjgTS3UA7Cs2HEn10N22x9oQI0ONrhldxfnCb1ChrGhM3sqaM54JBqXMMlj8iyb4w3yYfZ9xlXlib20d+XMrO7nSCySXtO0bweJNxv2MZfhqRuaMDks6W8HKezodmtsqB7WN1ROP6su+1lW7dEjUG/jkOJ2SgNNSyDVFxbQK97AjBbE0BxSjv8kdCGYQHM= ceph-b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:36:00.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfD3ktJzskklXwp11K0/nBSlFpMprbzlonPgm5WWfzYfDqPcrcfwbqxhW4YxRsXG7jg0V0AccCz5fYSHJASt6KBhKzreozmga8NiFENVMiSIMseFpXBb0oTSFC8e/wttxVXMNYM45r2AwH+rezqyauBhvbRTLAJtjNKRaRjaopi4v3B2KXKuYqaSNiZNwiC4QBewcLWQJFoWIsNiPSiNmDueV8WiTXjOQT4KIW9lp0Ye+iUPeUzKdOzTkgim7kqFWyUXcDt9zfpvQaf6N4PRJ1U5qDQd3FxHhaUxXA8mnZ8m7HmJyRYAiiMQ6FJiXfRc5xnZjgTS3UA7Cs2HEn10N22x9oQI0ONrhldxfnCb1ChrGhM3sqaM54JBqXMMlj8iyb4w3yYfZ9xlXlib20d+XMrO7nSCySXtO0bweJNxv2MZfhqRuaMDks6W8HKezodmtsqB7WN1ROP6su+1lW7dEjUG/jkOJ2SgNNSyDVFxbQK97AjBbE0BxSjv8kdCGYQHM= ceph-b3aed5e6-e380-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T10:36:00.951 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfD3ktJzskklXwp11K0/nBSlFpMprbzlonPgm5WWfzYfDqPcrcfwbqxhW4YxRsXG7jg0V0AccCz5fYSHJASt6KBhKzreozmga8NiFENVMiSIMseFpXBb0oTSFC8e/wttxVXMNYM45r2AwH+rezqyauBhvbRTLAJtjNKRaRjaopi4v3B2KXKuYqaSNiZNwiC4QBewcLWQJFoWIsNiPSiNmDueV8WiTXjOQT4KIW9lp0Ye+iUPeUzKdOzTkgim7kqFWyUXcDt9zfpvQaf6N4PRJ1U5qDQd3FxHhaUxXA8mnZ8m7HmJyRYAiiMQ6FJiXfRc5xnZjgTS3UA7Cs2HEn10N22x9oQI0ONrhldxfnCb1ChrGhM3sqaM54JBqXMMlj8iyb4w3yYfZ9xlXlib20d+XMrO7nSCySXtO0bweJNxv2MZfhqRuaMDks6W8HKezodmtsqB7WN1ROP6su+1lW7dEjUG/jkOJ2SgNNSyDVFxbQK97AjBbE0BxSjv8kdCGYQHM= ceph-b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:36:00.966 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-16T10:36:01.169 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:36:01.294 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:01 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1187081412' entity='client.admin' 2024-03-16T10:36:03.741 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:03.741 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:03.742 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:36:03.742 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:03.742 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:36:03.742 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-16T10:36:03.742 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:36:03.742 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:03 smithi002 ceph-mon[29248]: Deploying daemon ceph-exporter.smithi002 on smithi002 2024-03-16T10:36:03.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 -- 172.21.15.2:0/1937973468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 msgr2=0x7f3bac0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 --2- 172.21.15.2:0/1937973468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0ffa60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009910 tx=0x7f3ba0009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 -- 172.21.15.2:0/1937973468 shutdown_connections 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 --2- 172.21.15.2:0/1937973468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0ffa60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 -- 172.21.15.2:0/1937973468 >> 172.21.15.2:0/1937973468 conn(0x7f3bac0fabe0 msgr2=0x7f3bac0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 -- 172.21.15.2:0/1937973468 shutdown_connections 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 -- 172.21.15.2:0/1937973468 wait complete. 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 Processor -- start 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.833+0000 7f3bb2da1700 1 -- start start 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb2da1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb2da1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac069710 con 0x7f3bac0ff640 2024-03-16T10:36:03.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb0b3d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb0b3d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45364/0 (socket says 172.21.15.2:45364) 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb0b3d700 1 -- 172.21.15.2:0/247156920 learned_addr learned my addr 172.21.15.2:0/247156920 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb0b3d700 1 -- 172.21.15.2:0/247156920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba00095f0 con 0x7f3bac0ff640 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb0b3d700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0691d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006010 tx=0x7f3ba00050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3ba9ffb700 1 -- 172.21.15.2:0/247156920 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ba001d070 con 0x7f3bac0ff640 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3ba9ffb700 1 -- 172.21.15.2:0/247156920 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ba000be20 con 0x7f3bac0ff640 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.834+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac198910 con 0x7f3bac0ff640 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.835+0000 7f3ba9ffb700 1 -- 172.21.15.2:0/247156920 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ba0022de0 con 0x7f3bac0ff640 2024-03-16T10:36:03.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.835+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac198db0 con 0x7f3bac0ff640 2024-03-16T10:36:03.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.835+0000 7f3ba9ffb700 1 -- 172.21.15.2:0/247156920 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f3ba000f4b0 con 0x7f3bac0ff640 2024-03-16T10:36:03.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.836+0000 7f3ba9ffb700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b9c038580 0x7f3b9c03aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:03.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.836+0000 7f3ba9ffb700 1 -- 172.21.15.2:0/247156920 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ba004c680 con 0x7f3bac0ff640 2024-03-16T10:36:03.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.836+0000 7f3babfff700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b9c038580 0x7f3b9c03aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:03.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.836+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bac18e470 con 0x7f3bac0ff640 2024-03-16T10:36:03.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.836+0000 7f3babfff700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b9c038580 0x7f3b9c03aa40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b98006fd0 tx=0x7f3b98006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:03.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:03.841+0000 7f3ba9ffb700 1 -- 172.21.15.2:0/247156920 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3ba0027020 con 0x7f3bac0ff640 2024-03-16T10:36:04.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.102+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3bac0623e0 con 0x7f3bac0ff640 2024-03-16T10:36:04.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.105+0000 7f3ba9ffb700 1 -- 172.21.15.2:0/247156920 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3bac0623e0 con 0x7f3bac0ff640 2024-03-16T10:36:04.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b9c038580 msgr2=0x7f3b9c03aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b9c038580 0x7f3b9c03aa40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b98006fd0 tx=0x7f3b98006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 msgr2=0x7f3bac0691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0691d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006010 tx=0x7f3ba00050b0 comp rx=0 tx=0).stop 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 shutdown_connections 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b9c038580 0x7f3b9c03aa40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 --2- 172.21.15.2:0/247156920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bac0ff640 0x7f3bac0691d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 >> 172.21.15.2:0/247156920 conn(0x7f3bac0fabe0 msgr2=0x7f3bac0fb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 shutdown_connections 2024-03-16T10:36:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:04.107+0000 7f3bb2da1700 1 -- 172.21.15.2:0/247156920 wait complete. 2024-03-16T10:36:05.112 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-16T10:36:05.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-16T10:36:05.423 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:05 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/247156920' entity='client.admin' 2024-03-16T10:36:05.856 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:36:08.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.042+0000 7f581d931700 1 -- 172.21.15.2:0/3389865667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 msgr2=0x7f58180ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:08.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.042+0000 7f581d931700 1 --2- 172.21.15.2:0/3389865667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f58180ffa60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5808009910 tx=0x7f5808009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:08.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.042+0000 7f581d931700 1 -- 172.21.15.2:0/3389865667 shutdown_connections 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.042+0000 7f581d931700 1 --2- 172.21.15.2:0/3389865667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f58180ffa60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.042+0000 7f581d931700 1 -- 172.21.15.2:0/3389865667 >> 172.21.15.2:0/3389865667 conn(0x7f58180fac00 msgr2=0x7f58180fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.042+0000 7f581d931700 1 -- 172.21.15.2:0/3389865667 shutdown_connections 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.042+0000 7f581d931700 1 -- 172.21.15.2:0/3389865667 wait complete. 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.043+0000 7f581d931700 1 Processor -- start 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.043+0000 7f581d931700 1 -- start start 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.043+0000 7f581d931700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f5818069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.043+0000 7f581d931700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818069a70 con 0x7f58180ff640 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.043+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f5818069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.043+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f5818069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45372/0 (socket says 172.21.15.2:45372) 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.043+0000 7f5816ffd700 1 -- 172.21.15.2:0/2528310642 learned_addr learned my addr 172.21.15.2:0/2528310642 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.044+0000 7f5816ffd700 1 -- 172.21.15.2:0/2528310642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080095f0 con 0x7f58180ff640 2024-03-16T10:36:08.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.044+0000 7f5816ffd700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f5818069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5808006010 tx=0x7f5808009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:08.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.044+0000 7f581c92f700 1 -- 172.21.15.2:0/2528310642 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f580801d070 con 0x7f58180ff640 2024-03-16T10:36:08.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.044+0000 7f581c92f700 1 -- 172.21.15.2:0/2528310642 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5808005dc0 con 0x7f58180ff640 2024-03-16T10:36:08.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.044+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5818198910 con 0x7f58180ff640 2024-03-16T10:36:08.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.044+0000 7f581c92f700 1 -- 172.21.15.2:0/2528310642 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5808022e90 con 0x7f58180ff640 2024-03-16T10:36:08.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.044+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5818198d50 con 0x7f58180ff640 2024-03-16T10:36:08.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.045+0000 7f581c92f700 1 -- 172.21.15.2:0/2528310642 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f580800f4f0 con 0x7f58180ff640 2024-03-16T10:36:08.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.045+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581818e470 con 0x7f58180ff640 2024-03-16T10:36:08.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.045+0000 7f581c92f700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5800038530 0x7f580003a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:08.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.045+0000 7f581c92f700 1 -- 172.21.15.2:0/2528310642 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f580804cac0 con 0x7f58180ff640 2024-03-16T10:36:08.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.046+0000 7f58167fc700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5800038530 0x7f580003a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:08.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.046+0000 7f58167fc700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5800038530 0x7f580003a9f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f580c006fd0 tx=0x7f580c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:08.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.050+0000 7f581c92f700 1 -- 172.21.15.2:0/2528310642 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f580802a950 con 0x7f58180ff640 2024-03-16T10:36:08.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.257+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f58180611f0 con 0x7f5800038530 2024-03-16T10:36:08.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.260+0000 7f581c92f700 1 -- 172.21.15.2:0/2528310642 <== mgr.14164 v2:172.21.15.2:6800/486800969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f58180611f0 con 0x7f5800038530 2024-03-16T10:36:08.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.262+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5800038530 msgr2=0x7f580003a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.262+0000 7f581d931700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5800038530 0x7f580003a9f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f580c006fd0 tx=0x7f580c006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.262+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 msgr2=0x7f5818069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.262+0000 7f581d931700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f5818069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5808006010 tx=0x7f5808009f80 comp rx=0 tx=0).stop 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.263+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 shutdown_connections 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.263+0000 7f581d931700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5800038530 0x7f580003a9f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.263+0000 7f581d931700 1 --2- 172.21.15.2:0/2528310642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f58180ff640 0x7f5818069530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.263+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 >> 172.21.15.2:0/2528310642 conn(0x7f58180fac00 msgr2=0x7f58180fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.263+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 shutdown_connections 2024-03-16T10:36:08.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:08.263+0000 7f581d931700 1 -- 172.21.15.2:0/2528310642 wait complete. 2024-03-16T10:36:09.122 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi026 2024-03-16T10:36:09.123 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:36:09.123 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.conf 2024-03-16T10:36:09.142 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:36:09.142 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:36:09.199 INFO:tasks.cephadm:Adding host smithi026 to orchestrator... 2024-03-16T10:36:09.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch host add smithi026 2024-03-16T10:36:09.394 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:36:09.394 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:09.394 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:09.395 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:09.395 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:09.395 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:09.395 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:36:09.395 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-16T10:36:09.395 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:36:09.395 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:09 smithi002 ceph-mon[29248]: Deploying daemon crash.smithi002 on smithi002 2024-03-16T10:36:10.351 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:36:12.488 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:12 smithi002 ceph-mon[29248]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:12.488 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:12 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:12.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.596+0000 7f0620459700 1 -- 172.21.15.2:0/2120453951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 msgr2=0x7f0618102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:12.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.596+0000 7f0620459700 1 --2- 172.21.15.2:0/2120453951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618102220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0608009910 tx=0x7f0608009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:12.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.596+0000 7f0620459700 1 -- 172.21.15.2:0/2120453951 shutdown_connections 2024-03-16T10:36:12.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.596+0000 7f0620459700 1 --2- 172.21.15.2:0/2120453951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618102220 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:12.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.596+0000 7f0620459700 1 -- 172.21.15.2:0/2120453951 >> 172.21.15.2:0/2120453951 conn(0x7f06180fa420 msgr2=0x7f06180fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:12.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.596+0000 7f0620459700 1 -- 172.21.15.2:0/2120453951 shutdown_connections 2024-03-16T10:36:12.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.596+0000 7f0620459700 1 -- 172.21.15.2:0/2120453951 wait complete. 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.597+0000 7f0620459700 1 Processor -- start 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.597+0000 7f0620459700 1 -- start start 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.597+0000 7f0620459700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.597+0000 7f0620459700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0618196190 con 0x7f0618101e00 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.597+0000 7f061e1f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.597+0000 7f061e1f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37586/0 (socket says 172.21.15.2:37586) 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.597+0000 7f061e1f5700 1 -- 172.21.15.2:0/3389309523 learned_addr learned my addr 172.21.15.2:0/3389309523 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.598+0000 7f061e1f5700 1 -- 172.21.15.2:0/3389309523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06080095f0 con 0x7f0618101e00 2024-03-16T10:36:12.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.598+0000 7f061e1f5700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618195c50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0608006010 tx=0x7f0608009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:12.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.598+0000 7f060f7fe700 1 -- 172.21.15.2:0/3389309523 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f060801d070 con 0x7f0618101e00 2024-03-16T10:36:12.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.598+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0618196390 con 0x7f0618101e00 2024-03-16T10:36:12.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.598+0000 7f060f7fe700 1 -- 172.21.15.2:0/3389309523 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0608005dc0 con 0x7f0618101e00 2024-03-16T10:36:12.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.598+0000 7f060f7fe700 1 -- 172.21.15.2:0/3389309523 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0608022e90 con 0x7f0618101e00 2024-03-16T10:36:12.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.598+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0618196830 con 0x7f0618101e00 2024-03-16T10:36:12.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.599+0000 7f060f7fe700 1 -- 172.21.15.2:0/3389309523 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f060800f4f0 con 0x7f0618101e00 2024-03-16T10:36:12.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.599+0000 7f060f7fe700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0604038530 0x7f060403a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:12.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.599+0000 7f060f7fe700 1 -- 172.21.15.2:0/3389309523 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f060804c630 con 0x7f0618101e00 2024-03-16T10:36:12.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.599+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061818f630 con 0x7f0618101e00 2024-03-16T10:36:12.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.599+0000 7f061d9f4700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0604038530 0x7f060403a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:12.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.600+0000 7f061d9f4700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0604038530 0x7f060403a9f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0614006fd0 tx=0x7f0614006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:12.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.604+0000 7f060f7fe700 1 -- 172.21.15.2:0/3389309523 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0608027020 con 0x7f0618101e00 2024-03-16T10:36:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:12.817+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}) v1 -- 0x7f061802cf20 con 0x7f0604038530 2024-03-16T10:36:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:13 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:13 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:13 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:13 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:13 smithi002 ceph-mon[29248]: Deploying daemon node-exporter.smithi002 on smithi002 2024-03-16T10:36:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:14 smithi002 ceph-mon[29248]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:36:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:14 smithi002 ceph-mon[29248]: Deploying cephadm binary to smithi026 2024-03-16T10:36:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:14 smithi002 ceph-mon[29248]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:15.538 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi026' with addr '172.21.15.26' 2024-03-16T10:36:15.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.535+0000 7f060f7fe700 1 -- 172.21.15.2:0/3389309523 <== mgr.14164 v2:172.21.15.2:6800/486800969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f061802cf20 con 0x7f0604038530 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0604038530 msgr2=0x7f060403a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0604038530 0x7f060403a9f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0614006fd0 tx=0x7f0614006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 msgr2=0x7f0618195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618195c50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0608006010 tx=0x7f0608009f80 comp rx=0 tx=0).stop 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 shutdown_connections 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0604038530 0x7f060403a9f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 --2- 172.21.15.2:0/3389309523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0618101e00 0x7f0618195c50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 >> 172.21.15.2:0/3389309523 conn(0x7f06180fa420 msgr2=0x7f06180fae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 shutdown_connections 2024-03-16T10:36:15.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:15.538+0000 7f0620459700 1 -- 172.21.15.2:0/3389309523 wait complete. 2024-03-16T10:36:16.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T10:36:16.669 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:36:16.792 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:16 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:16.792 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:16 smithi002 ceph-mon[29248]: Added host smithi026 2024-03-16T10:36:16.792 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:16 smithi002 ceph-mon[29248]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:17.951 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:17 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:17.951 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:17 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:17.951 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:17 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:17.951 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:17 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:17.951 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:17 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:17.951 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:17 smithi002 ceph-mon[29248]: Deploying daemon alertmanager.smithi002 on smithi002 2024-03-16T10:36:17.951 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:17 smithi002 ceph-mon[29248]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 -- 172.21.15.2:0/199462080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 msgr2=0x7fa3900fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 --2- 172.21.15.2:0/199462080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa3900fee10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa380009910 tx=0x7fa380009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 -- 172.21.15.2:0/199462080 shutdown_connections 2024-03-16T10:36:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 --2- 172.21.15.2:0/199462080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa3900fee10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 -- 172.21.15.2:0/199462080 >> 172.21.15.2:0/199462080 conn(0x7fa3900fa330 msgr2=0x7fa3900fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 -- 172.21.15.2:0/199462080 shutdown_connections 2024-03-16T10:36:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 -- 172.21.15.2:0/199462080 wait complete. 2024-03-16T10:36:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 Processor -- start 2024-03-16T10:36:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.166+0000 7fa396a0b700 1 -- start start 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa396a0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa390195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa396a0b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa390196150 con 0x7fa3900fe9f0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa390195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa390195c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37604/0 (socket says 172.21.15.2:37604) 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38ffff700 1 -- 172.21.15.2:0/1530899979 learned_addr learned my addr 172.21.15.2:0/1530899979 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38ffff700 1 -- 172.21.15.2:0/1530899979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800095f0 con 0x7fa3900fe9f0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38ffff700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa390195c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa380006010 tx=0x7fa380009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38d7fa700 1 -- 172.21.15.2:0/1530899979 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa38001d070 con 0x7fa3900fe9f0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa390196350 con 0x7fa3900fe9f0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38d7fa700 1 -- 172.21.15.2:0/1530899979 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa380005dc0 con 0x7fa3900fe9f0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa38d7fa700 1 -- 172.21.15.2:0/1530899979 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa380022e90 con 0x7fa3900fe9f0 2024-03-16T10:36:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.167+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3901967f0 con 0x7fa3900fe9f0 2024-03-16T10:36:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.168+0000 7fa38d7fa700 1 -- 172.21.15.2:0/1530899979 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7fa38000f4f0 con 0x7fa3900fe9f0 2024-03-16T10:36:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.168+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39018f570 con 0x7fa3900fe9f0 2024-03-16T10:36:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.168+0000 7fa38d7fa700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fa378038530 0x7fa37803a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.169+0000 7fa38d7fa700 1 -- 172.21.15.2:0/1530899979 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa38004c6c0 con 0x7fa3900fe9f0 2024-03-16T10:36:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.169+0000 7fa38f7fe700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fa378038530 0x7fa37803a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.169+0000 7fa38f7fe700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fa378038530 0x7fa37803a9f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa384006fd0 tx=0x7fa384006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:18.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.173+0000 7fa38d7fa700 1 -- 172.21.15.2:0/1530899979 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa38002a950 con 0x7fa3900fe9f0 2024-03-16T10:36:18.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.382+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3900ff520 con 0x7fa378038530 2024-03-16T10:36:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.383+0000 7fa38d7fa700 1 -- 172.21.15.2:0/1530899979 <== mgr.14164 v2:172.21.15.2:6800/486800969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7fa3900ff520 con 0x7fa378038530 2024-03-16T10:36:18.384 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:36:18.384 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}] 2024-03-16T10:36:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.385+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fa378038530 msgr2=0x7fa37803a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.385+0000 7fa396a0b700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fa378038530 0x7fa37803a9f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa384006fd0 tx=0x7fa384006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.385+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 msgr2=0x7fa390195c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.385+0000 7fa396a0b700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa390195c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa380006010 tx=0x7fa380009f80 comp rx=0 tx=0).stop 2024-03-16T10:36:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.386+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 shutdown_connections 2024-03-16T10:36:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.386+0000 7fa396a0b700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fa378038530 0x7fa37803a9f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.386+0000 7fa396a0b700 1 --2- 172.21.15.2:0/1530899979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3900fe9f0 0x7fa390195c10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.386+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 >> 172.21.15.2:0/1530899979 conn(0x7fa3900fa330 msgr2=0x7fa3900fae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.386+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 shutdown_connections 2024-03-16T10:36:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:18.386+0000 7fa396a0b700 1 -- 172.21.15.2:0/1530899979 wait complete. 2024-03-16T10:36:19.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:18 smithi002 ceph-mon[29248]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:36:19.371 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-16T10:36:19.371 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-16T10:36:19.579 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:36:19.952 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:19 smithi002 ceph-mon[29248]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:22.481 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:22 smithi002 ceph-mon[29248]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:22.481 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:22 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/1340459626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 msgr2=0x7fd6ec1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 --2- 172.21.15.2:0/1340459626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec1035b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009910 tx=0x7fd6dc009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/1340459626 shutdown_connections 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 --2- 172.21.15.2:0/1340459626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec1035b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/1340459626 >> 172.21.15.2:0/1340459626 conn(0x7fd6ec0fabd0 msgr2=0x7fd6ec0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/1340459626 shutdown_connections 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/1340459626 wait complete. 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.760+0000 7fd6f1ebd700 1 Processor -- start 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6f1ebd700 1 -- start start 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6f1ebd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6f1ebd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1960b0 con 0x7fd6ec1011c0 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:22.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36212/0 (socket says 172.21.15.2:36212) 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6eb7fe700 1 -- 172.21.15.2:0/746955465 learned_addr learned my addr 172.21.15.2:0/746955465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6eb7fe700 1 -- 172.21.15.2:0/746955465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0095f0 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6eb7fe700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec195b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006010 tx=0x7fd6dc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6e8ff9700 1 -- 172.21.15.2:0/746955465 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6dc01d070 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6e8ff9700 1 -- 172.21.15.2:0/746955465 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6dc00be20 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec1962b0 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6e8ff9700 1 -- 172.21.15.2:0/746955465 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd6dc00f460 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.761+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec196750 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.762+0000 7fd6e8ff9700 1 -- 172.21.15.2:0/746955465 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7fd6dc00f720 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.762+0000 7fd6e8ff9700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd6d40385a0 0x7fd6d403aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.762+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec18f560 con 0x7fd6ec1011c0 2024-03-16T10:36:22.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.763+0000 7fd6e8ff9700 1 -- 172.21.15.2:0/746955465 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd6dc04c820 con 0x7fd6ec1011c0 2024-03-16T10:36:22.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.763+0000 7fd6eaffd700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd6d40385a0 0x7fd6d403aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:22.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.763+0000 7fd6eaffd700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd6d40385a0 0x7fd6d403aa60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:22.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.767+0000 7fd6e8ff9700 1 -- 172.21.15.2:0/746955465 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd6dc020dd0 con 0x7fd6ec1011c0 2024-03-16T10:36:22.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:22.975+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd6ec02d140 con 0x7fd6ec1011c0 2024-03-16T10:36:23.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.224+0000 7fd6e8ff9700 1 -- 172.21.15.2:0/746955465 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd6dc020370 con 0x7fd6ec1011c0 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd6d40385a0 msgr2=0x7fd6d403aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd6d40385a0 0x7fd6d403aa60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 msgr2=0x7fd6ec195b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec195b70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006010 tx=0x7fd6dc0050b0 comp rx=0 tx=0).stop 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 shutdown_connections 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fd6d40385a0 0x7fd6d403aa60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 --2- 172.21.15.2:0/746955465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6ec1011c0 0x7fd6ec195b70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 >> 172.21.15.2:0/746955465 conn(0x7fd6ec0fabd0 msgr2=0x7fd6ec0fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 shutdown_connections 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:36:23.227+0000 7fd6f1ebd700 1 -- 172.21.15.2:0/746955465 wait complete. 2024-03-16T10:36:23.228 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-03-16T10:36:23.329 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:23 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/746955465' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T10:36:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:24 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/746955465' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T10:36:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:24 smithi002 ceph-mon[29248]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T10:36:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:24 smithi002 ceph-mon[29248]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:24.881 INFO:tasks.cephadm:Adding mon.smithi002 on smithi002 2024-03-16T10:36:24.881 INFO:tasks.cephadm:Adding mon.smithi026 on smithi026 2024-03-16T10:36:24.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch apply mon '2;smithi002:172.21.15.2=smithi002;smithi026:172.21.15.26=smithi026' 2024-03-16T10:36:25.090 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:25.136 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:26.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:25 smithi002 ceph-mon[29248]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.634+0000 7f65d90b2700 1 -- 172.21.15.26:0/101342521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 msgr2=0x7f65d4100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.634+0000 7f65d90b2700 1 --2- 172.21.15.26:0/101342521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4100b60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009920 tx=0x7f65c8009c30 comp rx=0 tx=0).stop 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.634+0000 7f65d90b2700 1 -- 172.21.15.26:0/101342521 shutdown_connections 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.634+0000 7f65d90b2700 1 --2- 172.21.15.26:0/101342521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4100b60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.634+0000 7f65d90b2700 1 -- 172.21.15.26:0/101342521 >> 172.21.15.26:0/101342521 conn(0x7f65d40fbd00 msgr2=0x7f65d40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.634+0000 7f65d90b2700 1 -- 172.21.15.26:0/101342521 shutdown_connections 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.634+0000 7f65d90b2700 1 -- 172.21.15.26:0/101342521 wait complete. 2024-03-16T10:36:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.635+0000 7f65d90b2700 1 Processor -- start 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.636+0000 7f65d90b2700 1 -- start start 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.636+0000 7f65d90b2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.636+0000 7f65d90b2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4196180 con 0x7f65d4100740 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.636+0000 7f65d2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.636+0000 7f65d2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:42836/0 (socket says 172.21.15.26:42836) 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.636+0000 7f65d2d9d700 1 -- 172.21.15.26:0/3171095905 learned_addr learned my addr 172.21.15.26:0/3171095905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.636+0000 7f65d2d9d700 1 -- 172.21.15.26:0/3171095905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8009580 con 0x7f65d4100740 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.637+0000 7f65d2d9d700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4195c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f65c8004eb0 tx=0x7f65c8004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.637+0000 7f65bbfff700 1 -- 172.21.15.26:0/3171095905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c801d070 con 0x7f65d4100740 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.637+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4196380 con 0x7f65d4100740 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.637+0000 7f65bbfff700 1 -- 172.21.15.26:0/3171095905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65c800bc10 con 0x7f65d4100740 2024-03-16T10:36:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.637+0000 7f65bbfff700 1 -- 172.21.15.26:0/3171095905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c80039d0 con 0x7f65d4100740 2024-03-16T10:36:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.637+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d4196820 con 0x7f65d4100740 2024-03-16T10:36:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.638+0000 7f65bbfff700 1 -- 172.21.15.26:0/3171095905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f65c8003c90 con 0x7f65d4100740 2024-03-16T10:36:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.638+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d418f570 con 0x7f65d4100740 2024-03-16T10:36:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.638+0000 7f65bbfff700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f65bc0385e0 0x7f65bc03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.638+0000 7f65bbfff700 1 -- 172.21.15.26:0/3171095905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f65c804c080 con 0x7f65d4100740 2024-03-16T10:36:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.638+0000 7f65d259c700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f65bc0385e0 0x7f65bc03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.639+0000 7f65d259c700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f65bc0385e0 0x7f65bc03aaa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006fd0 tx=0x7f65c4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.643+0000 7f65bbfff700 1 -- 172.21.15.26:0/3171095905 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f65c8029770 con 0x7f65d4100740 2024-03-16T10:36:26.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.860+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 --> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi002:172.21.15.2=smithi002;smithi026:172.21.15.26=smithi026", "target": ["mon-mgr", ""]}) v1 -- 0x7f65d40612a0 con 0x7f65bc0385e0 2024-03-16T10:36:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.866+0000 7f65bbfff700 1 -- 172.21.15.26:0/3171095905 <== mgr.14164 v2:172.21.15.2:6800/486800969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f65d40612a0 con 0x7f65bc0385e0 2024-03-16T10:36:26.867 INFO:teuthology.orchestra.run.smithi026.stdout:Scheduled mon update... 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f65bc0385e0 msgr2=0x7f65bc03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f65bc0385e0 0x7f65bc03aaa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006fd0 tx=0x7f65c4006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 msgr2=0x7f65d4195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4195c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f65c8004eb0 tx=0x7f65c8004f90 comp rx=0 tx=0).stop 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 shutdown_connections 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f65bc0385e0 0x7f65bc03aaa0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 --2- 172.21.15.26:0/3171095905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65d4100740 0x7f65d4195c40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 >> 172.21.15.26:0/3171095905 conn(0x7f65d40fbd00 msgr2=0x7f65d40fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 shutdown_connections 2024-03-16T10:36:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:26.868+0000 7f65d90b2700 1 -- 172.21.15.26:0/3171095905 wait complete. 2024-03-16T10:36:27.518 DEBUG:teuthology.orchestra.run.smithi026:mon.smithi026> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi026.service 2024-03-16T10:36:27.521 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:27.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:27.752 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi002:172.21.15.2=smithi002;smithi026:172.21.15.26=smithi026", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: Saving service mon spec with placement smithi002:172.21.15.2=smithi002;smithi026:172.21.15.26=smithi026;count:2 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-16T10:36:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-16T10:36:28.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-16T10:36:28.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:28.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: Deploying daemon grafana.smithi002 on smithi002 2024-03-16T10:36:28.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:27 smithi002 ceph-mon[29248]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.095+0000 7f0c10fd2700 1 -- 172.21.15.26:0/3449292564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 msgr2=0x7f0c0c0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.095+0000 7f0c10fd2700 1 --2- 172.21.15.26:0/3449292564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0ffa50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b0a0 tx=0x7f0c0400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.095+0000 7f0c10fd2700 1 -- 172.21.15.26:0/3449292564 shutdown_connections 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.095+0000 7f0c10fd2700 1 --2- 172.21.15.26:0/3449292564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0ffa50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.095+0000 7f0c10fd2700 1 -- 172.21.15.26:0/3449292564 >> 172.21.15.26:0/3449292564 conn(0x7f0c0c0fabd0 msgr2=0x7f0c0c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.095+0000 7f0c10fd2700 1 -- 172.21.15.26:0/3449292564 shutdown_connections 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.095+0000 7f0c10fd2700 1 -- 172.21.15.26:0/3449292564 wait complete. 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c10fd2700 1 Processor -- start 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c10fd2700 1 -- start start 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c10fd2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c10fd2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0fcf30 con 0x7f0c0c0ff630 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c0a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c0a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0fc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:42858/0 (socket says 172.21.15.26:42858) 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c0a59c700 1 -- 172.21.15.26:0/1835011301 learned_addr learned my addr 172.21.15.26:0/1835011301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.096+0000 7f0c0a59c700 1 -- 172.21.15.26:0/1835011301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04009d00 con 0x7f0c0c0ff630 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.097+0000 7f0c0a59c700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0fc9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c040062a0 tx=0x7f0c0400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.097+0000 7f0c037fe700 1 -- 172.21.15.26:0/1835011301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c0400e040 con 0x7f0c0c0ff630 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.097+0000 7f0c037fe700 1 -- 172.21.15.26:0/1835011301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c04003d70 con 0x7f0c0c0ff630 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.097+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c0fb1c0 con 0x7f0c0c0ff630 2024-03-16T10:36:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.097+0000 7f0c037fe700 1 -- 172.21.15.26:0/1835011301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c0401c890 con 0x7f0c0c0ff630 2024-03-16T10:36:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.097+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0fb660 con 0x7f0c0c0ff630 2024-03-16T10:36:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.098+0000 7f0c037fe700 1 -- 172.21.15.26:0/1835011301 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f0c0401cb50 con 0x7f0c0c0ff630 2024-03-16T10:36:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.098+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c0fc070 con 0x7f0c0c0ff630 2024-03-16T10:36:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.098+0000 7f0c037fe700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0bf8038590 0x7f0bf803aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.098+0000 7f0c037fe700 1 -- 172.21.15.26:0/1835011301 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0c0404c720 con 0x7f0c0c0ff630 2024-03-16T10:36:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.098+0000 7f0c09d9b700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0bf8038590 0x7f0bf803aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.099+0000 7f0c09d9b700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0bf8038590 0x7f0bf803aa50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4006fd0 tx=0x7f0bf4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:29.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.103+0000 7f0c037fe700 1 -- 172.21.15.26:0/1835011301 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0c04024b50 con 0x7f0c0c0ff630 2024-03-16T10:36:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.390+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0c0c105d90 con 0x7f0c0c0ff630 2024-03-16T10:36:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.390+0000 7f0c037fe700 1 -- 172.21.15.26:0/1835011301 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f0c04024b50 con 0x7f0c0c0ff630 2024-03-16T10:36:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:29.391 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0bf8038590 msgr2=0x7f0bf803aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0bf8038590 0x7f0bf803aa50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4006fd0 tx=0x7f0bf4006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 msgr2=0x7f0c0c0fc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0fc9f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c040062a0 tx=0x7f0c0400bd40 comp rx=0 tx=0).stop 2024-03-16T10:36:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 shutdown_connections 2024-03-16T10:36:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f0bf8038590 0x7f0bf803aa50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 --2- 172.21.15.26:0/1835011301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c0c0ff630 0x7f0c0c0fc9f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 >> 172.21.15.26:0/1835011301 conn(0x7f0c0c0fabd0 msgr2=0x7f0c0c068f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 shutdown_connections 2024-03-16T10:36:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:29.392+0000 7f0c10fd2700 1 -- 172.21.15.26:0/1835011301 wait complete. 2024-03-16T10:36:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:29.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:29 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/1835011301' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:30.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:30 smithi002 ceph-mon[29248]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:31.020 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:31.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:31.225 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:32.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:32 smithi002 ceph-mon[29248]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:32.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:32 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:32.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.798+0000 7f43850b6700 1 -- 172.21.15.26:0/1384427595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 msgr2=0x7f4380103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.798+0000 7f43850b6700 1 --2- 172.21.15.26:0/1384427595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380103650 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4368009910 tx=0x7f4368009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.798+0000 7f43850b6700 1 -- 172.21.15.26:0/1384427595 shutdown_connections 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.798+0000 7f43850b6700 1 --2- 172.21.15.26:0/1384427595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380103650 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.798+0000 7f43850b6700 1 -- 172.21.15.26:0/1384427595 >> 172.21.15.26:0/1384427595 conn(0x7f43800fabd0 msgr2=0x7f43800fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.798+0000 7f43850b6700 1 -- 172.21.15.26:0/1384427595 shutdown_connections 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.798+0000 7f43850b6700 1 -- 172.21.15.26:0/1384427595 wait complete. 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.799+0000 7f43850b6700 1 Processor -- start 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.799+0000 7f43850b6700 1 -- start start 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.799+0000 7f43850b6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.799+0000 7f43850b6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4380100fd0 con 0x7f4380101260 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f437ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f437ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:42864/0 (socket says 172.21.15.26:42864) 2024-03-16T10:36:32.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f437ed9d700 1 -- 172.21.15.26:0/3621586694 learned_addr learned my addr 172.21.15.26:0/3621586694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:32.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f437ed9d700 1 -- 172.21.15.26:0/3621586694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680095f0 con 0x7f4380101260 2024-03-16T10:36:32.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f437ed9d700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380100a90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4368009ed0 tx=0x7f4368004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:32.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f4377fff700 1 -- 172.21.15.26:0/3621586694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f436801d070 con 0x7f4380101260 2024-03-16T10:36:32.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43800ff210 con 0x7f4380101260 2024-03-16T10:36:32.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f4377fff700 1 -- 172.21.15.26:0/3621586694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f436800be20 con 0x7f4380101260 2024-03-16T10:36:32.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f4377fff700 1 -- 172.21.15.26:0/3621586694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f436800f460 con 0x7f4380101260 2024-03-16T10:36:32.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.800+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43800ff6b0 con 0x7f4380101260 2024-03-16T10:36:32.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.801+0000 7f4377fff700 1 -- 172.21.15.26:0/3621586694 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f436800b960 con 0x7f4380101260 2024-03-16T10:36:32.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.801+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4380106e90 con 0x7f4380101260 2024-03-16T10:36:32.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.801+0000 7f4377fff700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f436c0385a0 0x7f436c03aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:32.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.802+0000 7f4377fff700 1 -- 172.21.15.26:0/3621586694 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f436804c710 con 0x7f4380101260 2024-03-16T10:36:32.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.802+0000 7f437e59c700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f436c0385a0 0x7f436c03aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:32.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.802+0000 7f437e59c700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f436c0385a0 0x7f436c03aa60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4370006fd0 tx=0x7f4370006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:32.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:32.810+0000 7f4377fff700 1 -- 172.21.15.26:0/3621586694 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f436802a360 con 0x7f4380101260 2024-03-16T10:36:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.091+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f43800623e0 con 0x7f4380101260 2024-03-16T10:36:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.091+0000 7f4377fff700 1 -- 172.21.15.26:0/3621586694 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f4368027030 con 0x7f4380101260 2024-03-16T10:36:33.093 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:33.093 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f436c0385a0 msgr2=0x7f436c03aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f436c0385a0 0x7f436c03aa60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4370006fd0 tx=0x7f4370006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 msgr2=0x7f4380100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380100a90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4368009ed0 tx=0x7f4368004690 comp rx=0 tx=0).stop 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 shutdown_connections 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f436c0385a0 0x7f436c03aa60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 --2- 172.21.15.26:0/3621586694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101260 0x7f4380100a90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 >> 172.21.15.26:0/3621586694 conn(0x7f43800fabd0 msgr2=0x7f43800fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 shutdown_connections 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:33.093+0000 7f43850b6700 1 -- 172.21.15.26:0/3621586694 wait complete. 2024-03-16T10:36:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:33.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:33 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/3621586694' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:34.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:34 smithi002 ceph-mon[29248]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:34.598 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:34.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:34.806 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:36.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:35 smithi002 ceph-mon[29248]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.190+0000 7f5928060700 1 -- 172.21.15.26:0/1557958090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 msgr2=0x7f5920100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.190+0000 7f5928060700 1 --2- 172.21.15.26:0/1557958090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920100b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f591c009910 tx=0x7f591c009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.190+0000 7f5928060700 1 -- 172.21.15.26:0/1557958090 shutdown_connections 2024-03-16T10:36:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.190+0000 7f5928060700 1 --2- 172.21.15.26:0/1557958090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920100b40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.190+0000 7f5928060700 1 -- 172.21.15.26:0/1557958090 >> 172.21.15.26:0/1557958090 conn(0x7f59200fbcc0 msgr2=0x7f59200fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.190+0000 7f5928060700 1 -- 172.21.15.26:0/1557958090 shutdown_connections 2024-03-16T10:36:36.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.190+0000 7f5928060700 1 -- 172.21.15.26:0/1557958090 wait complete. 2024-03-16T10:36:36.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.191+0000 7f5928060700 1 Processor -- start 2024-03-16T10:36:36.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.191+0000 7f5928060700 1 -- start start 2024-03-16T10:36:36.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.191+0000 7f5928060700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:36.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.191+0000 7f5928060700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920196110 con 0x7f5920100720 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.191+0000 7f5925dfc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.191+0000 7f5925dfc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:42880/0 (socket says 172.21.15.26:42880) 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.191+0000 7f5925dfc700 1 -- 172.21.15.26:0/486896273 learned_addr learned my addr 172.21.15.26:0/486896273 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.192+0000 7f5925dfc700 1 -- 172.21.15.26:0/486896273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0095f0 con 0x7f5920100720 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.192+0000 7f5925dfc700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920195bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f591c006010 tx=0x7f591c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.192+0000 7f5916ffd700 1 -- 172.21.15.26:0/486896273 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f591c01d070 con 0x7f5920100720 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.192+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5920196310 con 0x7f5920100720 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.192+0000 7f5916ffd700 1 -- 172.21.15.26:0/486896273 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f591c00be20 con 0x7f5920100720 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.192+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59201967b0 con 0x7f5920100720 2024-03-16T10:36:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.192+0000 7f5916ffd700 1 -- 172.21.15.26:0/486896273 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f591c022de0 con 0x7f5920100720 2024-03-16T10:36:36.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.193+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592018f550 con 0x7f5920100720 2024-03-16T10:36:36.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.193+0000 7f5916ffd700 1 -- 172.21.15.26:0/486896273 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f591c0226c0 con 0x7f5920100720 2024-03-16T10:36:36.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.193+0000 7f5916ffd700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f590c038580 0x7f590c03aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:36.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.193+0000 7f5916ffd700 1 -- 172.21.15.26:0/486896273 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f591c04c940 con 0x7f5920100720 2024-03-16T10:36:36.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.193+0000 7f59255fb700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f590c038580 0x7f590c03aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.194+0000 7f59255fb700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f590c038580 0x7f590c03aa40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5910006fd0 tx=0x7f5910006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:36.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.199+0000 7f5916ffd700 1 -- 172.21.15.26:0/486896273 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f591c02ba00 con 0x7f5920100720 2024-03-16T10:36:36.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.481+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f59200623e0 con 0x7f5920100720 2024-03-16T10:36:36.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.481+0000 7f5916ffd700 1 -- 172.21.15.26:0/486896273 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f59200623e0 con 0x7f5920100720 2024-03-16T10:36:36.483 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:36.483 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f590c038580 msgr2=0x7f590c03aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f590c038580 0x7f590c03aa40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5910006fd0 tx=0x7f5910006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 msgr2=0x7f5920195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920195bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f591c006010 tx=0x7f591c0050b0 comp rx=0 tx=0).stop 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 shutdown_connections 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f590c038580 0x7f590c03aa40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 --2- 172.21.15.26:0/486896273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5920100720 0x7f5920195bd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 >> 172.21.15.26:0/486896273 conn(0x7f59200fbcc0 msgr2=0x7f59200fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 shutdown_connections 2024-03-16T10:36:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:36.484+0000 7f5928060700 1 -- 172.21.15.26:0/486896273 wait complete. 2024-03-16T10:36:36.486 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:37.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:37 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/486896273' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:38.045 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:38.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:38.254 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:38.298 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:38.490 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:38 smithi002 ceph-mon[29248]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 -- 172.21.15.26:0/3923246372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 msgr2=0x7f3ba00ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 --2- 172.21.15.26:0/3923246372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba00ffa50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b0a0 tx=0x7f3b9800b3b0 comp rx=0 tx=0).stop 2024-03-16T10:36:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 -- 172.21.15.26:0/3923246372 shutdown_connections 2024-03-16T10:36:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 --2- 172.21.15.26:0/3923246372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba00ffa50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 -- 172.21.15.26:0/3923246372 >> 172.21.15.26:0/3923246372 conn(0x7f3ba00fabd0 msgr2=0x7f3ba00fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 -- 172.21.15.26:0/3923246372 shutdown_connections 2024-03-16T10:36:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 -- 172.21.15.26:0/3923246372 wait complete. 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.706+0000 7f3ba52af700 1 Processor -- start 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3ba52af700 1 -- start start 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3ba52af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3ba52af700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0069a70 con 0x7f3ba00ff630 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3b9effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3b9effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba0069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:41718/0 (socket says 172.21.15.26:41718) 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3b9effd700 1 -- 172.21.15.26:0/2025364622 learned_addr learned my addr 172.21.15.26:0/2025364622 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3b9effd700 1 -- 172.21.15.26:0/2025364622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b98009d00 con 0x7f3ba00ff630 2024-03-16T10:36:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3b9effd700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba0069530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b980062a0 tx=0x7f3b9800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.707+0000 7f3b97fff700 1 -- 172.21.15.26:0/2025364622 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b9800e040 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.708+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba0198950 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.708+0000 7f3b97fff700 1 -- 172.21.15.26:0/2025364622 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b98003d70 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.708+0000 7f3b97fff700 1 -- 172.21.15.26:0/2025364622 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b9801c890 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.708+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba0198d90 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.708+0000 7f3b97fff700 1 -- 172.21.15.26:0/2025364622 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f3b9801cb50 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.708+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba018e460 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.709+0000 7f3b97fff700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b8c038590 0x7f3b8c03aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.709+0000 7f3b97fff700 1 -- 172.21.15.26:0/2025364622 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3b9804c6c0 con 0x7f3ba00ff630 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.709+0000 7f3b9e7fc700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b8c038590 0x7f3b8c03aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.709+0000 7f3b9e7fc700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b8c038590 0x7f3b8c03aa50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3b88006fd0 tx=0x7f3b88006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:39.718+0000 7f3b97fff700 1 -- 172.21.15.26:0/2025364622 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3b9801a590 con 0x7f3ba00ff630 2024-03-16T10:36:39.996 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:39 smithi002 ceph-mon[29248]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.000+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3ba00623e0 con 0x7f3ba00ff630 2024-03-16T10:36:40.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.000+0000 7f3b97fff700 1 -- 172.21.15.26:0/2025364622 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f3b9801a3b0 con 0x7f3ba00ff630 2024-03-16T10:36:40.001 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:40.001 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b8c038590 msgr2=0x7f3b8c03aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b8c038590 0x7f3b8c03aa50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3b88006fd0 tx=0x7f3b88006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:40.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 msgr2=0x7f3ba0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba0069530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b980062a0 tx=0x7f3b9800bd40 comp rx=0 tx=0).stop 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 shutdown_connections 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f3b8c038590 0x7f3b8c03aa50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 --2- 172.21.15.26:0/2025364622 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba00ff630 0x7f3ba0069530 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.002+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 >> 172.21.15.26:0/2025364622 conn(0x7f3ba00fabd0 msgr2=0x7f3ba00fb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.003+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 shutdown_connections 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:40.003+0000 7f3ba52af700 1 -- 172.21.15.26:0/2025364622 wait complete. 2024-03-16T10:36:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:40.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/2025364622' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:40.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:40.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:40.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:40.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:40.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:40.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:40.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:40.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:40 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:41.470 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:41.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:41.675 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:41.721 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:42 smithi002 ceph-mon[29248]: Deploying daemon prometheus.smithi002 on smithi002 2024-03-16T10:36:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:42 smithi002 ceph-mon[29248]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:42 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:43.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.218+0000 7f64d5a52700 1 -- 172.21.15.26:0/2854456370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 msgr2=0x7f64d0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:43.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.218+0000 7f64d5a52700 1 --2- 172.21.15.26:0/2854456370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0100b60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009920 tx=0x7f64b8009c30 comp rx=0 tx=0).stop 2024-03-16T10:36:43.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.218+0000 7f64d5a52700 1 -- 172.21.15.26:0/2854456370 shutdown_connections 2024-03-16T10:36:43.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.218+0000 7f64d5a52700 1 --2- 172.21.15.26:0/2854456370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0100b60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:43.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.218+0000 7f64d5a52700 1 -- 172.21.15.26:0/2854456370 >> 172.21.15.26:0/2854456370 conn(0x7f64d00fbd00 msgr2=0x7f64d00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.218+0000 7f64d5a52700 1 -- 172.21.15.26:0/2854456370 shutdown_connections 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.218+0000 7f64d5a52700 1 -- 172.21.15.26:0/2854456370 wait complete. 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.219+0000 7f64d5a52700 1 Processor -- start 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.219+0000 7f64d5a52700 1 -- start start 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.219+0000 7f64d5a52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.219+0000 7f64d5a52700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0074f60 con 0x7f64d0100740 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.219+0000 7f64ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.219+0000 7f64ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:41740/0 (socket says 172.21.15.26:41740) 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.219+0000 7f64ceffd700 1 -- 172.21.15.26:0/1499006160 learned_addr learned my addr 172.21.15.26:0/1499006160 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.220+0000 7f64ceffd700 1 -- 172.21.15.26:0/1499006160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009580 con 0x7f64d0100740 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.220+0000 7f64ceffd700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0074a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64b8004f60 tx=0x7f64b8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.220+0000 7f64d4a50700 1 -- 172.21.15.26:0/1499006160 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64b801d070 con 0x7f64d0100740 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.220+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00731a0 con 0x7f64d0100740 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.220+0000 7f64d4a50700 1 -- 172.21.15.26:0/1499006160 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64b8005da0 con 0x7f64d0100740 2024-03-16T10:36:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.220+0000 7f64d4a50700 1 -- 172.21.15.26:0/1499006160 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64b800f460 con 0x7f64d0100740 2024-03-16T10:36:43.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.220+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d0073640 con 0x7f64d0100740 2024-03-16T10:36:43.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.221+0000 7f64d4a50700 1 -- 172.21.15.26:0/1499006160 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f64b800b960 con 0x7f64d0100740 2024-03-16T10:36:43.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.221+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d0106ea0 con 0x7f64d0100740 2024-03-16T10:36:43.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.221+0000 7f64d4a50700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f64bc0385e0 0x7f64bc03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:43.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.221+0000 7f64d4a50700 1 -- 172.21.15.26:0/1499006160 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f64b804c530 con 0x7f64d0100740 2024-03-16T10:36:43.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.221+0000 7f64ce7fc700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f64bc0385e0 0x7f64bc03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:43.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.222+0000 7f64ce7fc700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f64bc0385e0 0x7f64bc03aaa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006fd0 tx=0x7f64c0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:43.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.230+0000 7f64d4a50700 1 -- 172.21.15.26:0/1499006160 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f64b802a360 con 0x7f64d0100740 2024-03-16T10:36:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.513+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f64d00623e0 con 0x7f64d0100740 2024-03-16T10:36:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.514+0000 7f64d4a50700 1 -- 172.21.15.26:0/1499006160 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f64b8027070 con 0x7f64d0100740 2024-03-16T10:36:43.515 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:43.515 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f64bc0385e0 msgr2=0x7f64bc03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f64bc0385e0 0x7f64bc03aaa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006fd0 tx=0x7f64c0006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 msgr2=0x7f64d0074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0074a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64b8004f60 tx=0x7f64b8005040 comp rx=0 tx=0).stop 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 shutdown_connections 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f64bc0385e0 0x7f64bc03aaa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 --2- 172.21.15.26:0/1499006160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0100740 0x7f64d0074a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 >> 172.21.15.26:0/1499006160 conn(0x7f64d00fbd00 msgr2=0x7f64d00fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 shutdown_connections 2024-03-16T10:36:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:43.516+0000 7f64d5a52700 1 -- 172.21.15.26:0/1499006160 wait complete. 2024-03-16T10:36:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:43.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:43 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/1499006160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:45.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:44 smithi002 ceph-mon[29248]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:45.121 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:45.121 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:46.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:45 smithi002 ceph-mon[29248]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.755+0000 7f02257e6700 1 -- 172.21.15.26:0/1634092216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 msgr2=0x7f0220103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.755+0000 7f02257e6700 1 --2- 172.21.15.26:0/1634092216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220103640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0218009910 tx=0x7f0218009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.755+0000 7f02257e6700 1 -- 172.21.15.26:0/1634092216 shutdown_connections 2024-03-16T10:36:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.755+0000 7f02257e6700 1 --2- 172.21.15.26:0/1634092216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220103640 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.755+0000 7f02257e6700 1 -- 172.21.15.26:0/1634092216 >> 172.21.15.26:0/1634092216 conn(0x7f02200fabe0 msgr2=0x7f02200fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.755+0000 7f02257e6700 1 -- 172.21.15.26:0/1634092216 shutdown_connections 2024-03-16T10:36:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.755+0000 7f02257e6700 1 -- 172.21.15.26:0/1634092216 wait complete. 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f02257e6700 1 Processor -- start 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f02257e6700 1 -- start start 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f02257e6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f02257e6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220100fc0 con 0x7f0220101250 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f021effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f021effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220100a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:38408/0 (socket says 172.21.15.26:38408) 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f021effd700 1 -- 172.21.15.26:0/4147249464 learned_addr learned my addr 172.21.15.26:0/4147249464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.756+0000 7f021effd700 1 -- 172.21.15.26:0/4147249464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02180095f0 con 0x7f0220101250 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.757+0000 7f021effd700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220100a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0218009ed0 tx=0x7f0218004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.757+0000 7f0217fff700 1 -- 172.21.15.26:0/4147249464 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f021801d070 con 0x7f0220101250 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.757+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200ff200 con 0x7f0220101250 2024-03-16T10:36:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.757+0000 7f0217fff700 1 -- 172.21.15.26:0/4147249464 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f021800be20 con 0x7f0220101250 2024-03-16T10:36:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.757+0000 7f0217fff700 1 -- 172.21.15.26:0/4147249464 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f021800f460 con 0x7f0220101250 2024-03-16T10:36:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.757+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02200ff6a0 con 0x7f0220101250 2024-03-16T10:36:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.758+0000 7f0217fff700 1 -- 172.21.15.26:0/4147249464 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f021800b960 con 0x7f0220101250 2024-03-16T10:36:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.758+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220106ea0 con 0x7f0220101250 2024-03-16T10:36:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.758+0000 7f0217fff700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f020c0385a0 0x7f020c03aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.758+0000 7f0217fff700 1 -- 172.21.15.26:0/4147249464 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f021804c9e0 con 0x7f0220101250 2024-03-16T10:36:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.758+0000 7f021e7fc700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f020c0385a0 0x7f020c03aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.759+0000 7f021e7fc700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f020c0385a0 0x7f020c03aa60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0208006fd0 tx=0x7f0208006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:46.763+0000 7f0217fff700 1 -- 172.21.15.26:0/4147249464 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f021802ba00 con 0x7f0220101250 2024-03-16T10:36:47.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.049+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f02200623e0 con 0x7f0220101250 2024-03-16T10:36:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.050+0000 7f0217fff700 1 -- 172.21.15.26:0/4147249464 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f021802b3b0 con 0x7f0220101250 2024-03-16T10:36:47.051 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:47.051 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f020c0385a0 msgr2=0x7f020c03aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f020c0385a0 0x7f020c03aa60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0208006fd0 tx=0x7f0208006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 msgr2=0x7f0220100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220100a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0218009ed0 tx=0x7f0218004690 comp rx=0 tx=0).stop 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 shutdown_connections 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f020c0385a0 0x7f020c03aa60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 --2- 172.21.15.26:0/4147249464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220101250 0x7f0220100a80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 >> 172.21.15.26:0/4147249464 conn(0x7f02200fabe0 msgr2=0x7f02200fb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 shutdown_connections 2024-03-16T10:36:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:47.052+0000 7f02257e6700 1 -- 172.21.15.26:0/4147249464 wait complete. 2024-03-16T10:36:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:47.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:47 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/4147249464' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:48.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:48 smithi002 ceph-mon[29248]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:48.579 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:48.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:48.827 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:49.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:49 smithi002 ceph-mon[29248]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.270+0000 7f6d53b08700 1 -- 172.21.15.26:0/915165424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 msgr2=0x7f6d4c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.270+0000 7f6d53b08700 1 --2- 172.21.15.26:0/915165424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c1013e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009910 tx=0x7f6d48009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.270+0000 7f6d53b08700 1 -- 172.21.15.26:0/915165424 shutdown_connections 2024-03-16T10:36:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.270+0000 7f6d53b08700 1 --2- 172.21.15.26:0/915165424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c1013e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.270+0000 7f6d53b08700 1 -- 172.21.15.26:0/915165424 >> 172.21.15.26:0/915165424 conn(0x7f6d4c0fabd0 msgr2=0x7f6d4c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.270+0000 7f6d53b08700 1 -- 172.21.15.26:0/915165424 shutdown_connections 2024-03-16T10:36:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.270+0000 7f6d53b08700 1 -- 172.21.15.26:0/915165424 wait complete. 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d53b08700 1 Processor -- start 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d53b08700 1 -- start start 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d53b08700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d53b08700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c19c700 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d518a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d518a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:38414/0 (socket says 172.21.15.26:38414) 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d518a4700 1 -- 172.21.15.26:0/660713876 learned_addr learned my addr 172.21.15.26:0/660713876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.271+0000 7f6d518a4700 1 -- 172.21.15.26:0/660713876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d480095f0 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.272+0000 7f6d518a4700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c19c1c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009ed0 tx=0x7f6d48004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.272+0000 7f6d42ffd700 1 -- 172.21.15.26:0/660713876 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d4801d070 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.272+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c19c900 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.272+0000 7f6d42ffd700 1 -- 172.21.15.26:0/660713876 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d4800be20 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.272+0000 7f6d42ffd700 1 -- 172.21.15.26:0/660713876 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d4800f460 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.272+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c19cda0 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.273+0000 7f6d42ffd700 1 -- 172.21.15.26:0/660713876 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45444+0+0 (secure 0 0 0) 0x7f6d48022800 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.273+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c1962a0 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.273+0000 7f6d42ffd700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f6d380385a0 0x7f6d3803aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.273+0000 7f6d42ffd700 1 -- 172.21.15.26:0/660713876 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d4804ca30 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.273+0000 7f6d510a3700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f6d380385a0 0x7f6d3803aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.274+0000 7f6d510a3700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f6d380385a0 0x7f6d3803aa60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c006fd0 tx=0x7f6d3c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:50.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.278+0000 7f6d42ffd700 1 -- 172.21.15.26:0/660713876 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6d4802bba0 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.561+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d4c0623e0 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.561+0000 7f6d42ffd700 1 -- 172.21.15.26:0/660713876 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f6d4802bba0 con 0x7f6d4c0fefb0 2024-03-16T10:36:50.563 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:50.563 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f6d380385a0 msgr2=0x7f6d3803aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f6d380385a0 0x7f6d3803aa60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c006fd0 tx=0x7f6d3c006e40 comp rx=0 tx=0).stop 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 msgr2=0x7f6d4c19c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c19c1c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009ed0 tx=0x7f6d48004690 comp rx=0 tx=0).stop 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 shutdown_connections 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f6d380385a0 0x7f6d3803aa60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 --2- 172.21.15.26:0/660713876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4c0fefb0 0x7f6d4c19c1c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 >> 172.21.15.26:0/660713876 conn(0x7f6d4c0fabd0 msgr2=0x7f6d4c0fb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.563+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 shutdown_connections 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:50.564+0000 7f6d53b08700 1 -- 172.21.15.26:0/660713876 wait complete. 2024-03-16T10:36:50.565 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:50.964 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:50 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/660713876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:52.168 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:52.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:52.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:52 smithi002 ceph-mon[29248]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:36:52.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:52.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:52.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' 2024-03-16T10:36:52.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:52 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 -- 172.21.15.26:0/1740730399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 msgr2=0x7f5c18100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 --2- 172.21.15.26:0/1740730399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18100b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800b0a0 tx=0x7f5c0800b3b0 comp rx=0 tx=0).stop 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 -- 172.21.15.26:0/1740730399 shutdown_connections 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 --2- 172.21.15.26:0/1740730399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18100b70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 -- 172.21.15.26:0/1740730399 >> 172.21.15.26:0/1740730399 conn(0x7f5c180fbcd0 msgr2=0x7f5c180fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 -- 172.21.15.26:0/1740730399 shutdown_connections 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 -- 172.21.15.26:0/1740730399 wait complete. 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.957+0000 7f5c1d542700 1 Processor -- start 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.958+0000 7f5c1d542700 1 -- start start 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.958+0000 7f5c1d542700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:53.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.958+0000 7f5c1d542700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c18196100 con 0x7f5c18100750 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.958+0000 7f5c16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.958+0000 7f5c16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:38440/0 (socket says 172.21.15.26:38440) 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.958+0000 7f5c16ffd700 1 -- 172.21.15.26:0/982370301 learned_addr learned my addr 172.21.15.26:0/982370301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.958+0000 7f5c16ffd700 1 -- 172.21.15.26:0/982370301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08009d00 con 0x7f5c18100750 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.959+0000 7f5c16ffd700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18195bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800bc90 tx=0x7f5c0800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.959+0000 7f5c0ffff700 1 -- 172.21.15.26:0/982370301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c0800e040 con 0x7f5c18100750 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.959+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c18196300 con 0x7f5c18100750 2024-03-16T10:36:53.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.959+0000 7f5c0ffff700 1 -- 172.21.15.26:0/982370301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c08003d50 con 0x7f5c18100750 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.959+0000 7f5c0ffff700 1 -- 172.21.15.26:0/982370301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c0801c8f0 con 0x7f5c18100750 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.959+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c181967a0 con 0x7f5c18100750 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.960+0000 7f5c0ffff700 1 -- 172.21.15.26:0/982370301 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 14) v1 ==== 45458+0+0 (secure 0 0 0) 0x7f5c0801cbb0 con 0x7f5c18100750 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.960+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1818f580 con 0x7f5c18100750 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.960+0000 7f5c0ffff700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 0x7f5c0403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.960+0000 7f5c0ffff700 1 -- 172.21.15.26:0/982370301 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c0804c780 con 0x7f5c18100750 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.960+0000 7f5c167fc700 1 -- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 msgr2=0x7f5c0403aaf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/486800969 2024-03-16T10:36:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.960+0000 7f5c167fc700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 0x7f5c0403aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:36:53.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:53.969+0000 7f5c0ffff700 1 -- 172.21.15.26:0/982370301 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5c0801a590 con 0x7f5c18100750 2024-03-16T10:36:54.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.161+0000 7f5c167fc700 1 -- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 msgr2=0x7f5c0403aaf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/486800969 2024-03-16T10:36:54.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.161+0000 7f5c167fc700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 0x7f5c0403aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:36:54.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.254+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5c180623e0 con 0x7f5c18100750 2024-03-16T10:36:54.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.255+0000 7f5c0ffff700 1 -- 172.21.15.26:0/982370301 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f5c0801a3b0 con 0x7f5c18100750 2024-03-16T10:36:54.256 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:54.256 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:54.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 msgr2=0x7f5c0403aaf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:36:54.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 0x7f5c0403aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:54.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 msgr2=0x7f5c18195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18195bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800bc90 tx=0x7f5c0800bd70 comp rx=0 tx=0).stop 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 shutdown_connections 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7f5c04038630 0x7f5c0403aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 --2- 172.21.15.26:0/982370301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c18100750 0x7f5c18195bc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 >> 172.21.15.26:0/982370301 conn(0x7f5c180fbcd0 msgr2=0x7f5c180fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 shutdown_connections 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:54.257+0000 7f5c1d542700 1 -- 172.21.15.26:0/982370301 wait complete. 2024-03-16T10:36:54.259 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:54.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:54 smithi002 ceph-mon[29248]: from='mgr.14164 172.21.15.2:0/779446619' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-16T10:36:54.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:54 smithi002 ceph-mon[29248]: mgrmap e14: smithi002.zevrtm(active, since 61s) 2024-03-16T10:36:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:55 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/982370301' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:55.838 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:55.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:56.044 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.682+0000 7fc5da2a3700 1 -- 172.21.15.26:0/3387864875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 msgr2=0x7fc5d4100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.682+0000 7fc5da2a3700 1 --2- 172.21.15.26:0/3387864875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4100b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009910 tx=0x7fc5c4009c20 comp rx=0 tx=0).stop 2024-03-16T10:36:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.682+0000 7fc5da2a3700 1 -- 172.21.15.26:0/3387864875 shutdown_connections 2024-03-16T10:36:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.682+0000 7fc5da2a3700 1 --2- 172.21.15.26:0/3387864875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4100b40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.682+0000 7fc5da2a3700 1 -- 172.21.15.26:0/3387864875 >> 172.21.15.26:0/3387864875 conn(0x7fc5d40fbcc0 msgr2=0x7fc5d40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.682+0000 7fc5da2a3700 1 -- 172.21.15.26:0/3387864875 shutdown_connections 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.682+0000 7fc5da2a3700 1 -- 172.21.15.26:0/3387864875 wait complete. 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.683+0000 7fc5da2a3700 1 Processor -- start 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.683+0000 7fc5da2a3700 1 -- start start 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.683+0000 7fc5da2a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.683+0000 7fc5da2a3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4074f60 con 0x7fc5d4100720 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.683+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.683+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:35514/0 (socket says 172.21.15.26:35514) 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.683+0000 7fc5d3fff700 1 -- 172.21.15.26:0/685121143 learned_addr learned my addr 172.21.15.26:0/685121143 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.684+0000 7fc5d3fff700 1 -- 172.21.15.26:0/685121143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40095f0 con 0x7fc5d4100720 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.684+0000 7fc5d3fff700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4074a20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:36:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.684+0000 7fc5d17fa700 1 -- 172.21.15.26:0/685121143 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5c401d070 con 0x7fc5d4100720 2024-03-16T10:36:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.684+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d40731a0 con 0x7fc5d4100720 2024-03-16T10:36:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.684+0000 7fc5d17fa700 1 -- 172.21.15.26:0/685121143 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5c4005dc0 con 0x7fc5d4100720 2024-03-16T10:36:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.684+0000 7fc5d17fa700 1 -- 172.21.15.26:0/685121143 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc5c4022e90 con 0x7fc5d4100720 2024-03-16T10:36:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.684+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d4073640 con 0x7fc5d4100720 2024-03-16T10:36:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.685+0000 7fc5d17fa700 1 -- 172.21.15.26:0/685121143 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 14) v1 ==== 45458+0+0 (secure 0 0 0) 0x7fc5c400b960 con 0x7fc5d4100720 2024-03-16T10:36:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.685+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d4106e80 con 0x7fc5d4100720 2024-03-16T10:36:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.685+0000 7fc5d17fa700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 0x7fc5bc03aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:36:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.685+0000 7fc5d17fa700 1 -- 172.21.15.26:0/685121143 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc5c404c5d0 con 0x7fc5d4100720 2024-03-16T10:36:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.685+0000 7fc5d37fe700 1 -- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 msgr2=0x7fc5bc03aa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/486800969 2024-03-16T10:36:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.685+0000 7fc5d37fe700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 0x7fc5bc03aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:36:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.693+0000 7fc5d17fa700 1 -- 172.21.15.26:0/685121143 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc5c402a540 con 0x7fc5d4100720 2024-03-16T10:36:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.886+0000 7fc5d37fe700 1 -- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 msgr2=0x7fc5bc03aa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/486800969 2024-03-16T10:36:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.886+0000 7fc5d37fe700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 0x7fc5bc03aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:36:57.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.978+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc5d40623e0 con 0x7fc5d4100720 2024-03-16T10:36:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.979+0000 7fc5d17fa700 1 -- 172.21.15.26:0/685121143 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fc5c4027020 con 0x7fc5d4100720 2024-03-16T10:36:57.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:36:57.980 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 msgr2=0x7fc5bc03aa40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 0x7fc5bc03aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 msgr2=0x7fc5d4074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4074a20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006010 tx=0x7fc5c4009f80 comp rx=0 tx=0).stop 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 shutdown_connections 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:6800/486800969,v1:172.21.15.2:6801/486800969] conn(0x7fc5bc038580 0x7fc5bc03aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 --2- 172.21.15.26:0/685121143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d4100720 0x7fc5d4074a20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 >> 172.21.15.26:0/685121143 conn(0x7fc5d40fbcc0 msgr2=0x7fc5d40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:36:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 shutdown_connections 2024-03-16T10:36:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:36:57.981+0000 7fc5da2a3700 1 -- 172.21.15.26:0/685121143 wait complete. 2024-03-16T10:36:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:36:58.101 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:36:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/685121143' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:36:59.541 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:36:59.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:36:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:36:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:37:00.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: Active manager daemon smithi002.zevrtm restarted 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: Activating manager daemon smithi002.zevrtm 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: osdmap e5: 0 total, 0 up, 0 in 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: mgrmap e15: smithi002.zevrtm(active, starting, since 0.00309876s) 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:37:00.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.179+0000 7f0745126700 1 -- 172.21.15.26:0/1450470465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 msgr2=0x7f07400ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.179+0000 7f0745126700 1 --2- 172.21.15.26:0/1450470465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f07400ffa60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0734009910 tx=0x7f0734009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.179+0000 7f0745126700 1 -- 172.21.15.26:0/1450470465 shutdown_connections 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.179+0000 7f0745126700 1 --2- 172.21.15.26:0/1450470465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f07400ffa60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.179+0000 7f0745126700 1 -- 172.21.15.26:0/1450470465 >> 172.21.15.26:0/1450470465 conn(0x7f07400fabe0 msgr2=0x7f07400fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.179+0000 7f0745126700 1 -- 172.21.15.26:0/1450470465 shutdown_connections 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.179+0000 7f0745126700 1 -- 172.21.15.26:0/1450470465 wait complete. 2024-03-16T10:37:01.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f0745126700 1 Processor -- start 2024-03-16T10:37:01.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f0745126700 1 -- start start 2024-03-16T10:37:01.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f0745126700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f0740195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:01.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f0745126700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0740196100 con 0x7f07400ff640 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f073ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f0740195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f073ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f0740195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:35524/0 (socket says 172.21.15.26:35524) 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f073ed9d700 1 -- 172.21.15.26:0/3188264712 learned_addr learned my addr 172.21.15.26:0/3188264712 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.180+0000 7f073ed9d700 1 -- 172.21.15.26:0/3188264712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340095f0 con 0x7f07400ff640 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.181+0000 7f073ed9d700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f0740195bc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0734006010 tx=0x7f0734009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.181+0000 7f0727fff700 1 -- 172.21.15.26:0/3188264712 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f073401d070 con 0x7f07400ff640 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.181+0000 7f0727fff700 1 -- 172.21.15.26:0/3188264712 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0734005dc0 con 0x7f07400ff640 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.181+0000 7f0727fff700 1 -- 172.21.15.26:0/3188264712 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0734022e90 con 0x7f07400ff640 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.181+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0740196300 con 0x7f07400ff640 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.181+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07401967a0 con 0x7f07400ff640 2024-03-16T10:37:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.182+0000 7f0727fff700 1 -- 172.21.15.26:0/3188264712 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 45310+0+0 (secure 0 0 0) 0x7f073400f4f0 con 0x7f07400ff640 2024-03-16T10:37:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.182+0000 7f0727fff700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f07280383b0 0x7f072803a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.182+0000 7f0727fff700 1 -- 172.21.15.26:0/3188264712 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f073404d490 con 0x7f07400ff640 2024-03-16T10:37:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.182+0000 7f073e59c700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f07280383b0 0x7f072803a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.182+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074018f6e0 con 0x7f07400ff640 2024-03-16T10:37:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.182+0000 7f073e59c700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f07280383b0 0x7f072803a870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0730006fd0 tx=0x7f0730006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:01.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.191+0000 7f0727fff700 1 -- 172.21.15.26:0/3188264712 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0734027070 con 0x7f07400ff640 2024-03-16T10:37:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.487+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0740105da0 con 0x7f07400ff640 2024-03-16T10:37:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.488+0000 7f0727fff700 1 -- 172.21.15.26:0/3188264712 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f073402a950 con 0x7f07400ff640 2024-03-16T10:37:01.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:01.489 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f07280383b0 msgr2=0x7f072803a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f07280383b0 0x7f072803a870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0730006fd0 tx=0x7f0730006e40 comp rx=0 tx=0).stop 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 msgr2=0x7f0740195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f0740195bc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0734006010 tx=0x7f0734009f80 comp rx=0 tx=0).stop 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 shutdown_connections 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f07280383b0 0x7f072803a870 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 --2- 172.21.15.26:0/3188264712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07400ff640 0x7f0740195bc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 >> 172.21.15.26:0/3188264712 conn(0x7f07400fabe0 msgr2=0x7f07400683b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 shutdown_connections 2024-03-16T10:37:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:01.490+0000 7f0745126700 1 -- 172.21.15.26:0/3188264712 wait complete. 2024-03-16T10:37:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: [16/Mar/2024:10:37:00] ENGINE Bus STARTING 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: [16/Mar/2024:10:37:00] ENGINE Serving on http://172.21.15.2:8765 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: [16/Mar/2024:10:37:00] ENGINE Serving on https://172.21.15.2:7150 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: [16/Mar/2024:10:37:00] ENGINE Bus STARTED 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:01.545 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:01 smithi002 ceph-mon[29248]: mgrmap e16: smithi002.zevrtm(active, since 1.00547s) 2024-03-16T10:37:02.559 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:02.559 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:02 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/3188264712' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:03.070 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:37:03.070 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:37:03.492 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:03 smithi002 ceph-mon[29248]: mgrmap e17: smithi002.zevrtm(active, since 2s) 2024-03-16T10:37:03.492 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:37:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-16T10:37:05.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:05.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:05.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:05.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:05.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:37:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 -- 172.21.15.26:0/3119678320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 msgr2=0x7f8d381035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 --2- 172.21.15.26:0/3119678320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d381035c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009910 tx=0x7f8d28009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 -- 172.21.15.26:0/3119678320 shutdown_connections 2024-03-16T10:37:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 --2- 172.21.15.26:0/3119678320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d381035c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 -- 172.21.15.26:0/3119678320 >> 172.21.15.26:0/3119678320 conn(0x7f8d380fabe0 msgr2=0x7f8d380fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 -- 172.21.15.26:0/3119678320 shutdown_connections 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 -- 172.21.15.26:0/3119678320 wait complete. 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.403+0000 7f8d3d56e700 1 Processor -- start 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.404+0000 7f8d3d56e700 1 -- start start 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.404+0000 7f8d3d56e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d38100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.404+0000 7f8d3d56e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38100f40 con 0x7f8d381011d0 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.404+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d38100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.404+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d38100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:35544/0 (socket says 172.21.15.26:35544) 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.404+0000 7f8d36ffd700 1 -- 172.21.15.26:0/1802217066 learned_addr learned my addr 172.21.15.26:0/1802217066 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.404+0000 7f8d36ffd700 1 -- 172.21.15.26:0/1802217066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d280095f0 con 0x7f8d381011d0 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.405+0000 7f8d36ffd700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d38100a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009ed0 tx=0x7f8d28004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.405+0000 7f8d1ffff700 1 -- 172.21.15.26:0/1802217066 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d2801d070 con 0x7f8d381011d0 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.405+0000 7f8d1ffff700 1 -- 172.21.15.26:0/1802217066 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d2800be20 con 0x7f8d381011d0 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.405+0000 7f8d1ffff700 1 -- 172.21.15.26:0/1802217066 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d2800f460 con 0x7f8d381011d0 2024-03-16T10:37:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.405+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d380ff180 con 0x7f8d381011d0 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.405+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d380ff620 con 0x7f8d381011d0 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.406+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d38106ea0 con 0x7f8d381011d0 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.406+0000 7f8d1ffff700 1 -- 172.21.15.26:0/1802217066 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 45500+0+0 (secure 0 0 0) 0x7f8d2800f650 con 0x7f8d381011d0 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.406+0000 7f8d1ffff700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d20038690 0x7f8d2003ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.406+0000 7f8d1ffff700 1 -- 172.21.15.26:0/1802217066 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f8d2804cb20 con 0x7f8d381011d0 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.406+0000 7f8d367fc700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d20038690 0x7f8d2003ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.407+0000 7f8d367fc700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d20038690 0x7f8d2003ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006fd0 tx=0x7f8d2c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.412+0000 7f8d1ffff700 1 -- 172.21.15.26:0/1802217066 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d2802ba00 con 0x7f8d381011d0 2024-03-16T10:37:06.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.706+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8d380623e0 con 0x7f8d381011d0 2024-03-16T10:37:06.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.706+0000 7f8d1ffff700 1 -- 172.21.15.26:0/1802217066 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f8d2802b3b0 con 0x7f8d381011d0 2024-03-16T10:37:06.707 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:06.707 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d20038690 msgr2=0x7f8d2003ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d20038690 0x7f8d2003ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c006fd0 tx=0x7f8d2c006e40 comp rx=0 tx=0).stop 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 msgr2=0x7f8d38100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d38100a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009ed0 tx=0x7f8d28004690 comp rx=0 tx=0).stop 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 shutdown_connections 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d20038690 0x7f8d2003ab50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 --2- 172.21.15.26:0/1802217066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d381011d0 0x7f8d38100a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 >> 172.21.15.26:0/1802217066 conn(0x7f8d380fabe0 msgr2=0x7f8d380fb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 shutdown_connections 2024-03-16T10:37:06.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:06.709+0000 7f8d3d56e700 1 -- 172.21.15.26:0/1802217066 wait complete. 2024-03-16T10:37:06.711 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/1802217066' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:07.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:07 smithi002 ceph-mon[29248]: Deploying daemon ceph-exporter.smithi026 on smithi026 2024-03-16T10:37:08.406 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:37:08.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:37:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.368+0000 7fc213052700 1 -- 172.21.15.26:0/949126274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 msgr2=0x7fc20c0ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.368+0000 7fc213052700 1 --2- 172.21.15.26:0/949126274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c0ff370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc20000aa20 tx=0x7fc20000ad30 comp rx=0 tx=0).stop 2024-03-16T10:37:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.368+0000 7fc213052700 1 -- 172.21.15.26:0/949126274 shutdown_connections 2024-03-16T10:37:10.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.368+0000 7fc213052700 1 --2- 172.21.15.26:0/949126274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c0ff370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.368+0000 7fc213052700 1 -- 172.21.15.26:0/949126274 >> 172.21.15.26:0/949126274 conn(0x7fc20c0fa820 msgr2=0x7fc20c0fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.368+0000 7fc213052700 1 -- 172.21.15.26:0/949126274 shutdown_connections 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.368+0000 7fc213052700 1 -- 172.21.15.26:0/949126274 wait complete. 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.369+0000 7fc213052700 1 Processor -- start 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.369+0000 7fc213052700 1 -- start start 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.369+0000 7fc213052700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c1960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.369+0000 7fc213052700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c196610 con 0x7fc20c0fef90 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.369+0000 7fc210dee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c1960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.369+0000 7fc210dee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c1960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:40768/0 (socket says 172.21.15.26:40768) 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.369+0000 7fc210dee700 1 -- 172.21.15.26:0/3493125263 learned_addr learned my addr 172.21.15.26:0/3493125263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.370+0000 7fc210dee700 1 -- 172.21.15.26:0/3493125263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20000a700 con 0x7fc20c0fef90 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.370+0000 7fc210dee700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c1960d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc20000c6d0 tx=0x7fc200004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.370+0000 7fc209ffb700 1 -- 172.21.15.26:0/3493125263 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc20001d070 con 0x7fc20c0fef90 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.370+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c196810 con 0x7fc20c0fef90 2024-03-16T10:37:10.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.370+0000 7fc209ffb700 1 -- 172.21.15.26:0/3493125263 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc200005420 con 0x7fc20c0fef90 2024-03-16T10:37:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.370+0000 7fc209ffb700 1 -- 172.21.15.26:0/3493125263 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc20000e600 con 0x7fc20c0fef90 2024-03-16T10:37:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.370+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c196c30 con 0x7fc20c0fef90 2024-03-16T10:37:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.371+0000 7fc209ffb700 1 -- 172.21.15.26:0/3493125263 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 45500+0+0 (secure 0 0 0) 0x7fc200016470 con 0x7fc20c0fef90 2024-03-16T10:37:10.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.371+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c100470 con 0x7fc20c0fef90 2024-03-16T10:37:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.371+0000 7fc209ffb700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc1fc038690 0x7fc1fc03ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.371+0000 7fc209ffb700 1 -- 172.21.15.26:0/3493125263 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc200012070 con 0x7fc20c0fef90 2024-03-16T10:37:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.372+0000 7fc20bfff700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc1fc038690 0x7fc1fc03ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:10.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.372+0000 7fc20bfff700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc1fc038690 0x7fc1fc03ab50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006fd0 tx=0x7fc1f8006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.377+0000 7fc209ffb700 1 -- 172.21.15.26:0/3493125263 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc20002b440 con 0x7fc20c0fef90 2024-03-16T10:37:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.666+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc20c197520 con 0x7fc20c0fef90 2024-03-16T10:37:10.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.677+0000 7fc209ffb700 1 -- 172.21.15.26:0/3493125263 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fc20c197520 con 0x7fc20c0fef90 2024-03-16T10:37:10.678 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:10.678 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:37:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc1fc038690 msgr2=0x7fc1fc03ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc1fc038690 0x7fc1fc03ab50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006fd0 tx=0x7fc1f8006e40 comp rx=0 tx=0).stop 2024-03-16T10:37:10.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 msgr2=0x7fc20c1960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c1960d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc20000c6d0 tx=0x7fc200004cd0 comp rx=0 tx=0).stop 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 shutdown_connections 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc1fc038690 0x7fc1fc03ab50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 --2- 172.21.15.26:0/3493125263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c0fef90 0x7fc20c1960d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 >> 172.21.15.26:0/3493125263 conn(0x7fc20c0fa820 msgr2=0x7fc20c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 shutdown_connections 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:10.679+0000 7fc213052700 1 -- 172.21.15.26:0/3493125263 wait complete. 2024-03-16T10:37:10.681 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:37:11.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:10 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/3493125263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:12.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:37:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-16T10:37:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:11 smithi002 ceph-mon[29248]: Deploying daemon crash.smithi026 on smithi026 2024-03-16T10:37:12.395 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:37:12.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:37:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.361+0000 7fa57b59e700 1 -- 172.21.15.26:0/2728060315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 msgr2=0x7fa57c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.361+0000 7fa57b59e700 1 --2- 172.21.15.26:0/2728060315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c100080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa574009910 tx=0x7fa574009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.361+0000 7fa57b59e700 1 -- 172.21.15.26:0/2728060315 shutdown_connections 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.361+0000 7fa57b59e700 1 --2- 172.21.15.26:0/2728060315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c100080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.361+0000 7fa57b59e700 1 -- 172.21.15.26:0/2728060315 >> 172.21.15.26:0/2728060315 conn(0x7fa57c0fa650 msgr2=0x7fa57c0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.361+0000 7fa57b59e700 1 -- 172.21.15.26:0/2728060315 shutdown_connections 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.361+0000 7fa57b59e700 1 -- 172.21.15.26:0/2728060315 wait complete. 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.362+0000 7fa57b59e700 1 Processor -- start 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.362+0000 7fa57b59e700 1 -- start start 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.362+0000 7fa57b59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c0fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.362+0000 7fa57b59e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0fc9c0 con 0x7fa57c0ffca0 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.362+0000 7fa57a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c0fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.362+0000 7fa57a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c0fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:40796/0 (socket says 172.21.15.26:40796) 2024-03-16T10:37:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.362+0000 7fa57a59c700 1 -- 172.21.15.26:0/1657531374 learned_addr learned my addr 172.21.15.26:0/1657531374 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.363+0000 7fa57a59c700 1 -- 172.21.15.26:0/1657531374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740095f0 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.363+0000 7fa57a59c700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c0fab00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa574005490 tx=0x7fa574005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.363+0000 7fa56b7fe700 1 -- 172.21.15.26:0/1657531374 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa57401d070 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.363+0000 7fa56b7fe700 1 -- 172.21.15.26:0/1657531374 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa574005700 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.363+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c0fb040 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.363+0000 7fa56b7fe700 1 -- 172.21.15.26:0/1657531374 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa574003950 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.363+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c0fb460 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.364+0000 7fa56b7fe700 1 -- 172.21.15.26:0/1657531374 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 45500+0+0 (secure 0 0 0) 0x7fa574003c10 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.364+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57c0fec10 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.364+0000 7fa56b7fe700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa564038690 0x7fa56403ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.364+0000 7fa56b7fe700 1 -- 172.21.15.26:0/1657531374 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa57404c2f0 con 0x7fa57c0ffca0 2024-03-16T10:37:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.364+0000 7fa579d9b700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa564038690 0x7fa56403ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:14.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.365+0000 7fa579d9b700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa564038690 0x7fa56403ab50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.370+0000 7fa56b7fe700 1 -- 172.21.15.26:0/1657531374 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa574029950 con 0x7fa57c0ffca0 2024-03-16T10:37:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.654+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa57c100c60 con 0x7fa57c0ffca0 2024-03-16T10:37:14.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.655+0000 7fa56b7fe700 1 -- 172.21.15.26:0/1657531374 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fa574026020 con 0x7fa57c0ffca0 2024-03-16T10:37:14.656 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:14.656 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:37:14.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa564038690 msgr2=0x7fa56403ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa564038690 0x7fa56403ab50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570006e40 comp rx=0 tx=0).stop 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 msgr2=0x7fa57c0fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c0fab00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa574005490 tx=0x7fa574005570 comp rx=0 tx=0).stop 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 shutdown_connections 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa564038690 0x7fa56403ab50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 --2- 172.21.15.26:0/1657531374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa57c0ffca0 0x7fa57c0fab00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 >> 172.21.15.26:0/1657531374 conn(0x7fa57c0fa650 msgr2=0x7fa57c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 shutdown_connections 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:14.657+0000 7fa57b59e700 1 -- 172.21.15.26:0/1657531374 wait complete. 2024-03-16T10:37:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:37:15.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:14 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/1657531374' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:16.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:15 smithi002 ceph-mon[29248]: Deploying daemon node-exporter.smithi026 on smithi026 2024-03-16T10:37:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:37:16.304 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:37:16.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:37:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.380+0000 7f08e6753700 1 -- 172.21.15.26:0/2616430806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 msgr2=0x7f08e0106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.380+0000 7f08e6753700 1 --2- 172.21.15.26:0/2616430806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e0106aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009910 tx=0x7f08d0009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.380+0000 7f08e6753700 1 -- 172.21.15.26:0/2616430806 shutdown_connections 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.380+0000 7f08e6753700 1 --2- 172.21.15.26:0/2616430806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e0106aa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.380+0000 7f08e6753700 1 -- 172.21.15.26:0/2616430806 >> 172.21.15.26:0/2616430806 conn(0x7f08e00fc1d0 msgr2=0x7f08e00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.380+0000 7f08e6753700 1 -- 172.21.15.26:0/2616430806 shutdown_connections 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.380+0000 7f08e6753700 1 -- 172.21.15.26:0/2616430806 wait complete. 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.381+0000 7f08e6753700 1 Processor -- start 2024-03-16T10:37:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.381+0000 7f08e6753700 1 -- start start 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.381+0000 7f08e6753700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.381+0000 7f08e6753700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e0074fc0 con 0x7f08e01066c0 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e00730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:37136/0 (socket says 172.21.15.26:37136) 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dffff700 1 -- 172.21.15.26:0/3511154116 learned_addr learned my addr 172.21.15.26:0/3511154116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dffff700 1 -- 172.21.15.26:0/3511154116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d00095f0 con 0x7f08e01066c0 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dffff700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e00730b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08d0005490 tx=0x7f08d0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dd7fa700 1 -- 172.21.15.26:0/3511154116 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d001d070 con 0x7f08e01066c0 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dd7fa700 1 -- 172.21.15.26:0/3511154116 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08d0005700 con 0x7f08e01066c0 2024-03-16T10:37:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e00735f0 con 0x7f08e01066c0 2024-03-16T10:37:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08dd7fa700 1 -- 172.21.15.26:0/3511154116 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d00038e0 con 0x7f08e01066c0 2024-03-16T10:37:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.382+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e0073a70 con 0x7f08e01066c0 2024-03-16T10:37:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.383+0000 7f08dd7fa700 1 -- 172.21.15.26:0/3511154116 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 45500+0+0 (secure 0 0 0) 0x7f08d0003ba0 con 0x7f08e01066c0 2024-03-16T10:37:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.383+0000 7f08dd7fa700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08cc038690 0x7f08cc03ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.383+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e0101e00 con 0x7f08e01066c0 2024-03-16T10:37:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.384+0000 7f08dd7fa700 1 -- 172.21.15.26:0/3511154116 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f08d004c3c0 con 0x7f08e01066c0 2024-03-16T10:37:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.384+0000 7f08df7fe700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08cc038690 0x7f08cc03ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.384+0000 7f08df7fe700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08cc038690 0x7f08cc03ab50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006fd0 tx=0x7f08c8006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:19.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.389+0000 7f08dd7fa700 1 -- 172.21.15.26:0/3511154116 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08d0026020 con 0x7f08e01066c0 2024-03-16T10:37:19.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.677+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f08e00623e0 con 0x7f08e01066c0 2024-03-16T10:37:19.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.677+0000 7f08dd7fa700 1 -- 172.21.15.26:0/3511154116 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f08d0029360 con 0x7f08e01066c0 2024-03-16T10:37:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:19.679 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:37:19.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.679+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08cc038690 msgr2=0x7f08cc03ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.679+0000 7f08e6753700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08cc038690 0x7f08cc03ab50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006fd0 tx=0x7f08c8006e40 comp rx=0 tx=0).stop 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.679+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 msgr2=0x7f08e00730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.679+0000 7f08e6753700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e00730b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08d0005490 tx=0x7f08d0005570 comp rx=0 tx=0).stop 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.680+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 shutdown_connections 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.680+0000 7f08e6753700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08cc038690 0x7f08cc03ab50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.680+0000 7f08e6753700 1 --2- 172.21.15.26:0/3511154116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08e01066c0 0x7f08e00730b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.680+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 >> 172.21.15.26:0/3511154116 conn(0x7f08e00fc1d0 msgr2=0x7f08e01082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.680+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 shutdown_connections 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:19.680+0000 7f08e6753700 1 -- 172.21.15.26:0/3511154116 wait complete. 2024-03-16T10:37:19.681 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: Deploying daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:37:20.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:20 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/3511154116' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:21.312 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:37:21.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:37:21.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:21 smithi002 ceph-mon[29248]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:21.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:21 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.603+0000 7fa19f6b8700 1 -- 172.21.15.26:0/3308894580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 msgr2=0x7fa198101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:23.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.603+0000 7fa19f6b8700 1 --2- 172.21.15.26:0/3308894580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa198101c70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa194009910 tx=0x7fa194009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.603+0000 7fa19f6b8700 1 -- 172.21.15.26:0/3308894580 shutdown_connections 2024-03-16T10:37:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.603+0000 7fa19f6b8700 1 --2- 172.21.15.26:0/3308894580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa198101c70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.603+0000 7fa19f6b8700 1 -- 172.21.15.26:0/3308894580 >> 172.21.15.26:0/3308894580 conn(0x7fa1980fb0e0 msgr2=0x7fa1980fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.603+0000 7fa19f6b8700 1 -- 172.21.15.26:0/3308894580 shutdown_connections 2024-03-16T10:37:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.603+0000 7fa19f6b8700 1 -- 172.21.15.26:0/3308894580 wait complete. 2024-03-16T10:37:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.604+0000 7fa19f6b8700 1 Processor -- start 2024-03-16T10:37:23.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.604+0000 7fa19f6b8700 1 -- start start 2024-03-16T10:37:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.604+0000 7fa19f6b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa1980ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.604+0000 7fa19f6b8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981013b0 con 0x7fa198101890 2024-03-16T10:37:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.604+0000 7fa19d454700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa1980ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa19d454700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa1980ff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:37166/0 (socket says 172.21.15.26:37166) 2024-03-16T10:37:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa19d454700 1 -- 172.21.15.26:0/1544230805 learned_addr learned my addr 172.21.15.26:0/1544230805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:23.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa19d454700 1 -- 172.21.15.26:0/1544230805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1940095f0 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa19d454700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa1980ff4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa194005490 tx=0x7fa194005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa18e7fc700 1 -- 172.21.15.26:0/1544230805 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa19401d070 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1980ffa30 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa18e7fc700 1 -- 172.21.15.26:0/1544230805 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa194005700 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa18e7fc700 1 -- 172.21.15.26:0/1544230805 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa194003950 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.605+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1980ffe50 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.606+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198069060 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.607+0000 7fa18e7fc700 1 -- 172.21.15.26:0/1544230805 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 45500+0+0 (secure 0 0 0) 0x7fa194003c10 con 0x7fa198101890 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.607+0000 7fa18e7fc700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa1840386e0 0x7fa18403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.607+0000 7fa18e7fc700 1 -- 172.21.15.26:0/1544230805 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa19404c3b0 con 0x7fa198101890 2024-03-16T10:37:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.607+0000 7fa19cc53700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa1840386e0 0x7fa18403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.608+0000 7fa19cc53700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa1840386e0 0x7fa18403aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa188006fd0 tx=0x7fa188006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.612+0000 7fa18e7fc700 1 -- 172.21.15.26:0/1544230805 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa19402a590 con 0x7fa198101890 2024-03-16T10:37:23.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.902+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa19810ab20 con 0x7fa198101890 2024-03-16T10:37:23.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.902+0000 7fa18e7fc700 1 -- 172.21.15.26:0/1544230805 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fa194026070 con 0x7fa198101890 2024-03-16T10:37:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:23.904 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.904+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa1840386e0 msgr2=0x7fa18403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.904+0000 7fa19f6b8700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa1840386e0 0x7fa18403aba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa188006fd0 tx=0x7fa188006e40 comp rx=0 tx=0).stop 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.904+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 msgr2=0x7fa1980ff4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.904+0000 7fa19f6b8700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa1980ff4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa194005490 tx=0x7fa194005570 comp rx=0 tx=0).stop 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.904+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 shutdown_connections 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.904+0000 7fa19f6b8700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa1840386e0 0x7fa18403aba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.904+0000 7fa19f6b8700 1 --2- 172.21.15.26:0/1544230805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa198101890 0x7fa1980ff4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.905+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 >> 172.21.15.26:0/1544230805 conn(0x7fa1980fb0e0 msgr2=0x7fa198104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.905+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 shutdown_connections 2024-03-16T10:37:23.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:23.905+0000 7fa19f6b8700 1 -- 172.21.15.26:0/1544230805 wait complete. 2024-03-16T10:37:23.907 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:37:24.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:37:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:23 smithi002 ceph-mon[29248]: Deploying daemon mon.smithi026 on smithi026 2024-03-16T10:37:25.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:24 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/1544230805' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:26.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:25 smithi002 ceph-mon[29248]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:26.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:25 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:26.183 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:37:26.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:37:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:37:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.136+0000 7f3864aad700 1 -- 172.21.15.26:0/3285090678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 msgr2=0x7f3860105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.136+0000 7f3864aad700 1 --2- 172.21.15.26:0/3285090678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860105ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3850009910 tx=0x7f3850009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.136+0000 7f3864aad700 1 -- 172.21.15.26:0/3285090678 shutdown_connections 2024-03-16T10:37:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.136+0000 7f3864aad700 1 --2- 172.21.15.26:0/3285090678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860105ad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.136+0000 7f3864aad700 1 -- 172.21.15.26:0/3285090678 >> 172.21.15.26:0/3285090678 conn(0x7f3860076390 msgr2=0x7f38600fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.136+0000 7f3864aad700 1 -- 172.21.15.26:0/3285090678 shutdown_connections 2024-03-16T10:37:28.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.136+0000 7f3864aad700 1 -- 172.21.15.26:0/3285090678 wait complete. 2024-03-16T10:37:28.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.137+0000 7f3864aad700 1 Processor -- start 2024-03-16T10:37:28.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.137+0000 7f3864aad700 1 -- start start 2024-03-16T10:37:28.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.137+0000 7f3864aad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:28.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.137+0000 7f3864aad700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860069390 con 0x7f38601056f0 2024-03-16T10:37:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.137+0000 7f385e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f385e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:41638/0 (socket says 172.21.15.26:41638) 2024-03-16T10:37:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f385e59c700 1 -- 172.21.15.26:0/4082050524 learned_addr learned my addr 172.21.15.26:0/4082050524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f385e59c700 1 -- 172.21.15.26:0/4082050524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500095f0 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f385e59c700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860196eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3850005490 tx=0x7f3850005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f38577fe700 1 -- 172.21.15.26:0/4082050524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f385001d070 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f38577fe700 1 -- 172.21.15.26:0/4082050524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3850005700 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860197450 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f38577fe700 1 -- 172.21.15.26:0/4082050524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3850003950 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.138+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38601978d0 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.139+0000 7f38577fe700 1 -- 172.21.15.26:0/4082050524 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 45500+0+0 (secure 0 0 0) 0x7f3850003c10 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.139+0000 7f38577fe700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f384c038690 0x7f384c03ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.139+0000 7f38577fe700 1 -- 172.21.15.26:0/4082050524 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f385004c370 con 0x7f38601056f0 2024-03-16T10:37:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.139+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860100e30 con 0x7f38601056f0 2024-03-16T10:37:28.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.140+0000 7f385dd9b700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f384c038690 0x7f384c03ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:28.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.140+0000 7f385dd9b700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f384c038690 0x7f384c03ab50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3848006fd0 tx=0x7f3848006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:28.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.145+0000 7f38577fe700 1 -- 172.21.15.26:0/4082050524 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3850026020 con 0x7f38601056f0 2024-03-16T10:37:28.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:27 smithi002 ceph-mon[29248]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.433+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3860068a50 con 0x7f38601056f0 2024-03-16T10:37:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.434+0000 7f38577fe700 1 -- 172.21.15.26:0/4082050524 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f3850029360 con 0x7f38601056f0 2024-03-16T10:37:28.435 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:28.435 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":1,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:34:38.356713Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-16T10:37:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f384c038690 msgr2=0x7f384c03ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f384c038690 0x7f384c03ab50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3848006fd0 tx=0x7f3848006e40 comp rx=0 tx=0).stop 2024-03-16T10:37:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 msgr2=0x7f3860196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860196eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3850005490 tx=0x7f3850005570 comp rx=0 tx=0).stop 2024-03-16T10:37:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 shutdown_connections 2024-03-16T10:37:28.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f384c038690 0x7f384c03ab50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 --2- 172.21.15.26:0/4082050524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38601056f0 0x7f3860196eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 >> 172.21.15.26:0/4082050524 conn(0x7f3860076390 msgr2=0x7f38600fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 shutdown_connections 2024-03-16T10:37:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:28.436+0000 7f3864aad700 1 -- 172.21.15.26:0/4082050524 wait complete. 2024-03-16T10:37:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 1 2024-03-16T10:37:30.184 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-16T10:37:30.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T10:37:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: mon.smithi002 calling monitor election 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/1182345832' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:37:33.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: mon.smithi026 calling monitor election 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: mon.smithi002 is new leader, mons smithi002,smithi026 in quorum (ranks 0,1) 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: monmap e2: 2 mons at {smithi002=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],smithi026=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: fsmap 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: osdmap e5: 0 total, 0 up, 0 in 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: mgrmap e17: smithi002.zevrtm(active, since 33s) 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: Standby manager daemon smithi026.omjkpo started 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/1182345832' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: overall HEALTH_OK 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/1182345832' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:37:33.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:33.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/1182345832' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:37:33.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:33.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:33.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:33.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:33.934 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.883+0000 7fb3efe74700 1 -- 172.21.15.26:0/3999017847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 msgr2=0x7fb3d4007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.883+0000 7fb3efe74700 1 --2- 172.21.15.26:0/3999017847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3d4007c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80ff640 tx=0x7fb3e400feb0 comp rx=0 tx=0).stop 2024-03-16T10:37:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.883+0000 7fb3efe74700 1 -- 172.21.15.26:0/3999017847 shutdown_connections 2024-03-16T10:37:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.883+0000 7fb3efe74700 1 --2- 172.21.15.26:0/3999017847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3d4007c60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.883+0000 7fb3efe74700 1 -- 172.21.15.26:0/3999017847 >> 172.21.15.26:0/3999017847 conn(0x7fb3e80fb260 msgr2=0x7fb3e80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.883+0000 7fb3efe74700 1 -- 172.21.15.26:0/3999017847 shutdown_connections 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.883+0000 7fb3efe74700 1 -- 172.21.15.26:0/3999017847 wait complete. 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.884+0000 7fb3efe74700 1 Processor -- start 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.884+0000 7fb3efe74700 1 -- start start 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.884+0000 7fb3efe74700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3e8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.884+0000 7fb3efe74700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 0x7fb3e819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.884+0000 7fb3efe74700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8198ef0 con 0x7fb3d40057d0 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.884+0000 7fb3efe74700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8199060 con 0x7fb3e81988d0 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3ed40f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 0x7fb3e819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3edc10700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3e8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3edc10700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3e8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.26:41684/0 (socket says 172.21.15.26:41684) 2024-03-16T10:37:34.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3edc10700 1 -- 172.21.15.26:0/235899150 learned_addr learned my addr 172.21.15.26:0/235899150 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3ed40f700 1 -- 172.21.15.26:0/235899150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 msgr2=0x7fb3e819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3ed40f700 1 -- 172.21.15.26:0/235899150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 msgr2=0x7fb3e819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3ed40f700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 0x7fb3e819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3ed40f700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 0x7fb3e819cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3edc10700 1 -- 172.21.15.26:0/235899150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 msgr2=0x7fb3e819cc80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3edc10700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 0x7fb3e819cc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3edc10700 1 -- 172.21.15.26:0/235899150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e400f900 con 0x7fb3d40057d0 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3edc10700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3e8198390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4001eb0 tx=0x7fb3e40046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3deffd700 1 -- 172.21.15.26:0/235899150 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e4013b30 con 0x7fb3d40057d0 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3deffd700 1 -- 172.21.15.26:0/235899150 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3e40044f0 con 0x7fb3d40057d0 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e819d280 con 0x7fb3d40057d0 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3deffd700 1 -- 172.21.15.26:0/235899150 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3e4011470 con 0x7fb3d40057d0 2024-03-16T10:37:34.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.885+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e819d7d0 con 0x7fb3d40057d0 2024-03-16T10:37:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.886+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e818dec0 con 0x7fb3d40057d0 2024-03-16T10:37:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.887+0000 7fb3deffd700 1 -- 172.21.15.26:0/235899150 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 90466+0+0 (secure 0 0 0) 0x7fb3e4013c90 con 0x7fb3d40057d0 2024-03-16T10:37:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.887+0000 7fb3deffd700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb3cc06c9a0 0x7fb3cc06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.887+0000 7fb3deffd700 1 -- 172.21.15.26:0/235899150 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb3e4000e40 con 0x7fb3d40057d0 2024-03-16T10:37:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.887+0000 7fb3ed40f700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb3cc06c9a0 0x7fb3cc06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.888+0000 7fb3ed40f700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb3cc06c9a0 0x7fb3cc06ee60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009850 tx=0x7fb3d8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:34.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:34.892+0000 7fb3deffd700 1 -- 172.21.15.26:0/235899150 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb3e4058fe0 con 0x7fb3d40057d0 2024-03-16T10:37:34.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:34 smithi026 ceph-mon[37041]: mgrmap e18: smithi002.zevrtm(active, since 33s), standbys: smithi026.omjkpo 2024-03-16T10:37:34.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:34 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:37:34.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:34 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:37:34.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:34 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:35.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:34 smithi002 ceph-mon[29248]: mgrmap e18: smithi002.zevrtm(active, since 33s), standbys: smithi026.omjkpo 2024-03-16T10:37:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:34 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:37:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:34 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:37:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:34 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:37:35.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.186+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb3e8104ef0 con 0x7fb3d40057d0 2024-03-16T10:37:35.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.186+0000 7fb3deffd700 1 -- 172.21.15.26:0/235899150 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1014 (secure 0 0 0) 0x7fb3e8104ef0 con 0x7fb3d40057d0 2024-03-16T10:37:35.187 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-03-16T10:37:35.187 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","modified":"2024-03-16T10:37:28.545499Z","created":"2024-03-16T10:34:38.356713Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi002","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi026","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T10:37:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb3cc06c9a0 msgr2=0x7fb3cc06ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb3cc06c9a0 0x7fb3cc06ee60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009850 tx=0x7fb3d8008040 comp rx=0 tx=0).stop 2024-03-16T10:37:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 msgr2=0x7fb3e8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3e8198390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4001eb0 tx=0x7fb3e40046f0 comp rx=0 tx=0).stop 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 shutdown_connections 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb3cc06c9a0 0x7fb3cc06ee60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3d40057d0 0x7fb3e8198390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 --2- 172.21.15.26:0/235899150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3e81988d0 0x7fb3e819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 >> 172.21.15.26:0/235899150 conn(0x7fb3e80fb260 msgr2=0x7fb3e80fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 shutdown_connections 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:37:35.188+0000 7fb3efe74700 1 -- 172.21.15.26:0/235899150 wait complete. 2024-03-16T10:37:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:dumped monmap epoch 2 2024-03-16T10:37:35.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:35 smithi026 ceph-mon[37041]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:35.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:35 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:35.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:35 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/235899150' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:36.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:35 smithi002 ceph-mon[29248]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:36.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:35 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:35 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/235899150' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T10:37:36.128 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-16T10:37:36.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-16T10:37:36.355 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:37:36.737 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.737 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.737 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:36.737 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:37:36.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:36.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:37:36.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:37:36.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:37.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:37 smithi026 ceph-mon[37041]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:37.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:37 smithi026 ceph-mon[37041]: Reconfiguring mon.smithi002 (unknown last config time)... 2024-03-16T10:37:37.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:37 smithi026 ceph-mon[37041]: Reconfiguring daemon mon.smithi002 on smithi002 2024-03-16T10:37:37.995 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:37 smithi002 ceph-mon[29248]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:37.995 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:37 smithi002 ceph-mon[29248]: Reconfiguring mon.smithi002 (unknown last config time)... 2024-03-16T10:37:37.996 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:37 smithi002 ceph-mon[29248]: Reconfiguring daemon mon.smithi002 on smithi002 2024-03-16T10:37:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.552+0000 7f93fb99a700 1 -- 172.21.15.2:0/695143769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f41066c0 msgr2=0x7f93f4106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.552+0000 7f93fb99a700 1 --2- 172.21.15.2:0/695143769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f41066c0 0x7f93f4106aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009910 tx=0x7f93e4009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.552+0000 7f93fb99a700 1 -- 172.21.15.2:0/695143769 shutdown_connections 2024-03-16T10:37:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.552+0000 7f93fb99a700 1 --2- 172.21.15.2:0/695143769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f41066c0 0x7f93f4106aa0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009910 tx=0x7f93e4009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.552+0000 7f93fb99a700 1 -- 172.21.15.2:0/695143769 >> 172.21.15.2:0/695143769 conn(0x7f93f40fc1d0 msgr2=0x7f93f40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.552+0000 7f93fb99a700 1 -- 172.21.15.2:0/695143769 shutdown_connections 2024-03-16T10:37:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.552+0000 7f93fb99a700 1 -- 172.21.15.2:0/695143769 wait complete. 2024-03-16T10:37:38.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.553+0000 7f93fb99a700 1 Processor -- start 2024-03-16T10:37:38.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.553+0000 7f93fb99a700 1 -- start start 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93fb99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93f4196330 0x7f93f4196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93fb99a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f4196c50 0x7f93f419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93fb99a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419b1f0 con 0x7f93f4196c50 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93fb99a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f419b360 con 0x7f93f4196330 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f8f35700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f4196c50 0x7f93f419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f8f35700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f4196c50 0x7f93f419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50772/0 (socket says 172.21.15.2:50772) 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f8f35700 1 -- 172.21.15.2:0/2128288432 learned_addr learned my addr 172.21.15.2:0/2128288432 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f9736700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93f4196330 0x7f93f4196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f8f35700 1 -- 172.21.15.2:0/2128288432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93f4196330 msgr2=0x7f93f4196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f8f35700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93f4196330 0x7f93f4196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f8f35700 1 -- 172.21.15.2:0/2128288432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40095f0 con 0x7f93f4196c50 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f8f35700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f4196c50 0x7f93f419ac20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b700 tx=0x7f93e800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93f27fc700 1 -- 172.21.15.2:0/2128288432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e8011800 con 0x7f93f4196c50 2024-03-16T10:37:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.554+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f419b640 con 0x7f93f4196c50 2024-03-16T10:37:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.555+0000 7f93f27fc700 1 -- 172.21.15.2:0/2128288432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93e8011e20 con 0x7f93f4196c50 2024-03-16T10:37:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.555+0000 7f93f27fc700 1 -- 172.21.15.2:0/2128288432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93e800f2e0 con 0x7f93f4196c50 2024-03-16T10:37:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.555+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f41a0ec0 con 0x7f93f4196c50 2024-03-16T10:37:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.556+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4104970 con 0x7f93f4196c50 2024-03-16T10:37:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.556+0000 7f93f27fc700 1 -- 172.21.15.2:0/2128288432 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 90466+0+0 (secure 0 0 0) 0x7f93e8010380 con 0x7f93f4196c50 2024-03-16T10:37:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.557+0000 7f93f27fc700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f93dc06c950 0x7f93dc06ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.557+0000 7f93f27fc700 1 -- 172.21.15.2:0/2128288432 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f93e808a860 con 0x7f93f4196c50 2024-03-16T10:37:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.557+0000 7f93f9736700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f93dc06c950 0x7f93dc06ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.557+0000 7f93f9736700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f93dc06c950 0x7f93dc06ee10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93e4006010 tx=0x7f93e400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.562+0000 7f93f27fc700 1 -- 172.21.15.2:0/2128288432 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f93e8055eb0 con 0x7f93f4196c50 2024-03-16T10:37:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.765+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f93f4101e00 con 0x7f93f4196c50 2024-03-16T10:37:38.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.765+0000 7f93f27fc700 1 -- 172.21.15.2:0/2128288432 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+221 (secure 0 0 0) 0x7f93e80594d0 con 0x7f93f4196c50 2024-03-16T10:37:38.767 INFO:teuthology.orchestra.run.smithi002.stdout:# minimal ceph.conf for b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:37:38.767 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-03-16T10:37:38.767 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:37:38.767 INFO:teuthology.orchestra.run.smithi002.stdout: mon_host = [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] 2024-03-16T10:37:38.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f93dc06c950 msgr2=0x7f93dc06ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:38.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f93dc06c950 0x7f93dc06ee10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93e4006010 tx=0x7f93e400b540 comp rx=0 tx=0).stop 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f4196c50 msgr2=0x7f93f419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f4196c50 0x7f93f419ac20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b700 tx=0x7f93e800bac0 comp rx=0 tx=0).stop 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 shutdown_connections 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f93dc06c950 0x7f93dc06ee10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93f4196330 0x7f93f4196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 --2- 172.21.15.2:0/2128288432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93f4196c50 0x7f93f419ac20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 >> 172.21.15.2:0/2128288432 conn(0x7f93f40fc1d0 msgr2=0x7f93f40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.767+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 shutdown_connections 2024-03-16T10:37:38.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:38.768+0000 7f93fb99a700 1 -- 172.21.15.2:0/2128288432 wait complete. 2024-03-16T10:37:39.574 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-16T10:37:39.574 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:37:39.574 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T10:37:39.604 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:37:39.604 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:37:39.670 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:37:39.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T10:37:39.705 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:37:39.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2128288432' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: Reconfiguring mgr.smithi002.zevrtm (unknown last config time)... 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:39.763 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:39 smithi002 ceph-mon[29248]: Reconfiguring daemon mgr.smithi002.zevrtm on smithi002 2024-03-16T10:37:39.775 INFO:tasks.cephadm:Deploying OSDs... 2024-03-16T10:37:39.776 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:37:39.776 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T10:37:39.791 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T10:37:39.791 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-16 10:36:51.370596657 +0000 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-16 10:31:30.325464616 +0000 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-16 10:31:30.325464616 +0000 2024-03-16T10:37:39.845 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-16 10:31:30.325464616 +0000 2024-03-16T10:37:39.846 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T10:37:39.913 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-16T10:37:39.913 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-16T10:37:39.913 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.00010361 s, 4.9 MB/s 2024-03-16T10:37:39.914 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T10:37:39.970 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-03-16T10:37:39.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2128288432' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: Reconfiguring mgr.smithi002.zevrtm (unknown last config time)... 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:39 smithi026 ceph-mon[37041]: Reconfiguring daemon mgr.smithi002.zevrtm on smithi002 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-16 10:36:51.370596657 +0000 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-16 10:31:30.655457312 +0000 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-16 10:31:30.655457312 +0000 2024-03-16T10:37:40.026 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-16 10:31:30.655457312 +0000 2024-03-16T10:37:40.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T10:37:40.092 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-16T10:37:40.092 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-16T10:37:40.092 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 9.9749e-05 s, 5.1 MB/s 2024-03-16T10:37:40.093 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T10:37:40.149 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-03-16T10:37:40.202 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T10:37:40.202 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:40.203 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-16T10:37:40.203 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:40.203 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:40.203 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-16 10:36:51.370596657 +0000 2024-03-16T10:37:40.203 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-16 10:31:30.979450141 +0000 2024-03-16T10:37:40.203 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-16 10:31:30.979450141 +0000 2024-03-16T10:37:40.203 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-16 10:31:30.979450141 +0000 2024-03-16T10:37:40.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T10:37:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-16T10:37:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-16T10:37:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 9.8326e-05 s, 5.2 MB/s 2024-03-16T10:37:40.269 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T10:37:40.324 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-03-16T10:37:40.379 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T10:37:40.379 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:40.379 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-16T10:37:40.379 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:40.379 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:40.380 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-03-16 10:36:51.371596635 +0000 2024-03-16T10:37:40.380 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-16 10:31:31.302442991 +0000 2024-03-16T10:37:40.380 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-16 10:31:31.302442991 +0000 2024-03-16T10:37:40.380 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-03-16 10:31:31.302442991 +0000 2024-03-16T10:37:40.380 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T10:37:40.444 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-03-16T10:37:40.445 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-03-16T10:37:40.445 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 9.5933e-05 s, 5.3 MB/s 2024-03-16T10:37:40.446 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T10:37:40.501 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:37:40.501 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T10:37:40.519 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T10:37:40.519 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-03-16T10:37:40.577 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-16 10:37:26.631230228 +0000 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-16 10:31:30.332704205 +0000 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-16 10:31:30.332704205 +0000 2024-03-16T10:37:40.578 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-16 10:31:30.332704205 +0000 2024-03-16T10:37:40.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T10:37:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-16T10:37:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-16T10:37:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000127 s, 4.0 MB/s 2024-03-16T10:37:40.649 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T10:37:40.708 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-03-16T10:37:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T10:37:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:40.766 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-16T10:37:40.766 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:40.766 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:40.766 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-16 10:37:26.631230228 +0000 2024-03-16T10:37:40.766 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-16 10:31:30.684698800 +0000 2024-03-16T10:37:40.766 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-16 10:31:30.684698800 +0000 2024-03-16T10:37:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-16 10:31:30.684698800 +0000 2024-03-16T10:37:40.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T10:37:40.836 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-16T10:37:40.836 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-16T10:37:40.836 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000124136 s, 4.1 MB/s 2024-03-16T10:37:40.837 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T10:37:40.896 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-03-16T10:37:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T10:37:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:40.954 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-16T10:37:40.954 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:40.954 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:40.954 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-16 10:37:26.632230213 +0000 2024-03-16T10:37:40.955 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-16 10:31:31.036693394 +0000 2024-03-16T10:37:40.955 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-16 10:31:31.036693394 +0000 2024-03-16T10:37:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-16 10:31:31.036693394 +0000 2024-03-16T10:37:40.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T10:37:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-16T10:37:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-16T10:37:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.00012057 s, 4.2 MB/s 2024-03-16T10:37:41.025 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T10:37:41.084 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-03-16 10:37:26.632230213 +0000 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-16 10:31:31.371688250 +0000 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-16 10:31:31.371688250 +0000 2024-03-16T10:37:41.143 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-03-16 10:31:31.371688250 +0000 2024-03-16T10:37:41.143 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T10:37:41.212 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-03-16T10:37:41.212 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-03-16T10:37:41.212 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000124112 s, 4.1 MB/s 2024-03-16T10:37:41.213 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T10:37:41.271 INFO:tasks.cephadm:Deploying osd.0 on smithi002 with /dev/vg_nvme/lv_4... 2024-03-16T10:37:41.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T10:37:42.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:41 smithi026 ceph-mon[37041]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:41 smithi026 ceph-mon[37041]: Reconfiguring ceph-exporter.smithi002 (monmap changed)... 2024-03-16T10:37:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:37:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:41 smithi026 ceph-mon[37041]: Reconfiguring daemon ceph-exporter.smithi002 on smithi002 2024-03-16T10:37:42.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:41 smithi002 ceph-mon[29248]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:42.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:42.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:42.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:41 smithi002 ceph-mon[29248]: Reconfiguring ceph-exporter.smithi002 (monmap changed)... 2024-03-16T10:37:42.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:37:42.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:42.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:41 smithi002 ceph-mon[29248]: Reconfiguring daemon ceph-exporter.smithi002 on smithi002 2024-03-16T10:37:42.346 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:37:43.574 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:43 smithi002 ceph-mon[29248]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:43.575 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:43 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:43.575 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:43 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:43.575 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:43 smithi002 ceph-mon[29248]: Reconfiguring crash.smithi002 (monmap changed)... 2024-03-16T10:37:43.575 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:43 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:37:43.575 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:43 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:43.575 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:43 smithi002 ceph-mon[29248]: Reconfiguring daemon crash.smithi002 on smithi002 2024-03-16T10:37:43.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:43 smithi026 ceph-mon[37041]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:43 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:43 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:43 smithi026 ceph-mon[37041]: Reconfiguring crash.smithi002 (monmap changed)... 2024-03-16T10:37:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:43 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:37:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:43 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:43 smithi026 ceph-mon[37041]: Reconfiguring daemon crash.smithi002 on smithi002 2024-03-16T10:37:44.817 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T10:37:44.817 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T10:37:44.817 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-03-16T10:37:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-03-16T10:37:44.818 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00855673 s, 1.2 GB/s 2024-03-16T10:37:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-03-16T10:37:45.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi002:vg_nvme/lv_4 2024-03-16T10:37:46.014 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:45 smithi002 ceph-mon[29248]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:46.014 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:46.014 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:46.014 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:45 smithi002 ceph-mon[29248]: Reconfiguring alertmanager.smithi002 (dependencies changed)... 2024-03-16T10:37:46.014 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:45 smithi002 ceph-mon[29248]: Reconfiguring daemon alertmanager.smithi002 on smithi002 2024-03-16T10:37:46.014 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:37:46.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:45 smithi026 ceph-mon[37041]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:45 smithi026 ceph-mon[37041]: Reconfiguring alertmanager.smithi002 (dependencies changed)... 2024-03-16T10:37:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:45 smithi026 ceph-mon[37041]: Reconfiguring daemon alertmanager.smithi002 on smithi002 2024-03-16T10:37:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:37:47.804 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:37:48.190 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:47 smithi002 ceph-mon[29248]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:48.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:47 smithi026 ceph-mon[37041]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:50.229 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:49 smithi002 ceph-mon[29248]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:50.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:49 smithi026 ceph-mon[37041]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 -- 172.21.15.2:0/1444701866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760100380 msgr2=0x7f27601007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 --2- 172.21.15.2:0/1444701866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760100380 0x7f27601007e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2754009910 tx=0x7f2754009c20 comp rx=0 tx=0).stop 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 -- 172.21.15.2:0/1444701866 shutdown_connections 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 --2- 172.21.15.2:0/1444701866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2760100380 0x7f27601007e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 --2- 172.21.15.2:0/1444701866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27600ff180 0x7f27600ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 -- 172.21.15.2:0/1444701866 >> 172.21.15.2:0/1444701866 conn(0x7f27600fa720 msgr2=0x7f27600fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 -- 172.21.15.2:0/1444701866 shutdown_connections 2024-03-16T10:37:50.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.334+0000 7f2767346700 1 -- 172.21.15.2:0/1444701866 wait complete. 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f2767346700 1 Processor -- start 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f2767346700 1 -- start start 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f2767346700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ff180 0x7f2760103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f2767346700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2760100380 0x7f2760101760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f2767346700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760103670 con 0x7f27600ff180 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f2767346700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760101ca0 con 0x7f2760100380 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27650e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ff180 0x7f2760103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27650e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ff180 0x7f2760103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58708/0 (socket says 172.21.15.2:58708) 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27650e2700 1 -- 172.21.15.2:0/1337962158 learned_addr learned my addr 172.21.15.2:0/1337962158 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27648e1700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2760100380 0x7f2760101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27650e2700 1 -- 172.21.15.2:0/1337962158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2760100380 msgr2=0x7f2760101760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27650e2700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2760100380 0x7f2760101760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27650e2700 1 -- 172.21.15.2:0/1337962158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27540095f0 con 0x7f27600ff180 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.335+0000 7f27650e2700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ff180 0x7f2760103050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f275000b9c0 tx=0x7f275000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.336+0000 7f275e7fc700 1 -- 172.21.15.2:0/1337962158 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f275000c980 con 0x7f27600ff180 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.336+0000 7f275e7fc700 1 -- 172.21.15.2:0/1337962158 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2750014440 con 0x7f27600ff180 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.336+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2760101f80 con 0x7f27600ff180 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.336+0000 7f275e7fc700 1 -- 172.21.15.2:0/1337962158 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2750012570 con 0x7f27600ff180 2024-03-16T10:37:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.336+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2760102550 con 0x7f27600ff180 2024-03-16T10:37:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.337+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276004ea80 con 0x7f27600ff180 2024-03-16T10:37:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.337+0000 7f275e7fc700 1 -- 172.21.15.2:0/1337962158 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 90466+0+0 (secure 0 0 0) 0x7f275000cae0 con 0x7f27600ff180 2024-03-16T10:37:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.338+0000 7f275e7fc700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f274806c800 0x7f274806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:37:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.338+0000 7f275e7fc700 1 -- 172.21.15.2:0/1337962158 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f2750006e40 con 0x7f27600ff180 2024-03-16T10:37:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.338+0000 7f27648e1700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f274806c800 0x7f274806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:37:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.338+0000 7f27648e1700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f274806c800 0x7f274806ecc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f275400b5c0 tx=0x7f2754005160 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:37:50.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.342+0000 7f275e7fc700 1 -- 172.21.15.2:0/1337962158 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f27500594e0 con 0x7f27600ff180 2024-03-16T10:37:50.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:37:50.557+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f276019e740 con 0x7f274806c800 2024-03-16T10:37:51.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:37:51.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:37:51.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:37:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:37:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:37:52.039 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:51 smithi002 ceph-mon[29248]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:52.039 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:51 smithi002 ceph-mon[29248]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:37:52.039 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:52.039 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:52.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:51 smithi026 ceph-mon[37041]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:52.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:51 smithi026 ceph-mon[37041]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:37:52.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:52.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:53.135 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:52 smithi002 ceph-mon[29248]: Reconfiguring grafana.smithi002 (dependencies changed)... 2024-03-16T10:37:53.135 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:52 smithi002 ceph-mon[29248]: Reconfiguring daemon grafana.smithi002 on smithi002 2024-03-16T10:37:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:52 smithi026 ceph-mon[37041]: Reconfiguring grafana.smithi002 (dependencies changed)... 2024-03-16T10:37:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:52 smithi026 ceph-mon[37041]: Reconfiguring daemon grafana.smithi002 on smithi002 2024-03-16T10:37:54.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:53 smithi002 ceph-mon[29248]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:54.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:53 smithi026 ceph-mon[37041]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:56.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:55 smithi002 ceph-mon[29248]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:56.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:55 smithi026 ceph-mon[37041]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:57.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:57 smithi002 ceph-mon[29248]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:57.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:57.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:57.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:57 smithi002 ceph-mon[29248]: Reconfiguring prometheus.smithi002 (dependencies changed)... 2024-03-16T10:37:57.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:57 smithi002 ceph-mon[29248]: Reconfiguring daemon prometheus.smithi002 on smithi002 2024-03-16T10:37:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:57 smithi026 ceph-mon[37041]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:37:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:37:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:57 smithi026 ceph-mon[37041]: Reconfiguring prometheus.smithi002 (dependencies changed)... 2024-03-16T10:37:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:57 smithi026 ceph-mon[37041]: Reconfiguring daemon prometheus.smithi002 on smithi002 2024-03-16T10:37:58.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3994341081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4befd156-b58c-44e4-bc1e-f16112b3df27"}]: dispatch 2024-03-16T10:37:58.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3994341081' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4befd156-b58c-44e4-bc1e-f16112b3df27"}]': finished 2024-03-16T10:37:58.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:58 smithi002 ceph-mon[29248]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T10:37:58.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:37:58.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3035417188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:37:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:58 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3994341081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4befd156-b58c-44e4-bc1e-f16112b3df27"}]: dispatch 2024-03-16T10:37:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:58 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3994341081' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4befd156-b58c-44e4-bc1e-f16112b3df27"}]': finished 2024-03-16T10:37:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:58 smithi026 ceph-mon[37041]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T10:37:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:37:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:58 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3035417188' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:38:00.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:37:59 smithi002 ceph-mon[29248]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:00.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:37:59 smithi026 ceph-mon[37041]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:01.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:01.133 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:00 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:02.039 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:01 smithi002 ceph-mon[29248]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:02.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:01 smithi026 ceph-mon[37041]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:04.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:03 smithi002 ceph-mon[29248]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:04.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:03 smithi026 ceph-mon[37041]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:06.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:05 smithi002 ceph-mon[29248]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:06.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:05 smithi026 ceph-mon[37041]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:07.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:07 smithi002 ceph-mon[29248]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:07.989 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:07.989 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:07.989 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:38:07.989 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:08.105 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:07 smithi026 ceph-mon[37041]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:08.105 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:07 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:08.105 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:07 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:08.105 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:07 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:38:08.105 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:07 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:08.883 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:08 smithi002 ceph-mon[29248]: Reconfiguring ceph-exporter.smithi026 (monmap changed)... 2024-03-16T10:38:08.884 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:08 smithi002 ceph-mon[29248]: Reconfiguring daemon ceph-exporter.smithi026 on smithi026 2024-03-16T10:38:08.884 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:08 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:08.884 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:08 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:08.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:08 smithi026 ceph-mon[37041]: Reconfiguring ceph-exporter.smithi026 (monmap changed)... 2024-03-16T10:38:08.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:08 smithi026 ceph-mon[37041]: Reconfiguring daemon ceph-exporter.smithi026 on smithi026 2024-03-16T10:38:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:08 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:08 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:09.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: Reconfiguring crash.smithi026 (monmap changed)... 2024-03-16T10:38:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:38:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: Reconfiguring daemon crash.smithi026 on smithi026 2024-03-16T10:38:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T10:38:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:09 smithi026 ceph-mon[37041]: Deploying daemon osd.0 on smithi002 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: Reconfiguring crash.smithi026 (monmap changed)... 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: Reconfiguring daemon crash.smithi026 on smithi026 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:10.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:09 smithi002 ceph-mon[29248]: Deploying daemon osd.0 on smithi002 2024-03-16T10:38:11.604 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:11.605 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:11.605 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:11.605 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: Reconfiguring mgr.smithi026.omjkpo (monmap changed)... 2024-03-16T10:38:11.605 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:38:11.605 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:38:11.605 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:11.605 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:11 smithi002 ceph-mon[29248]: Reconfiguring daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:38:11.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:11.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:11.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:11.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: Reconfiguring mgr.smithi026.omjkpo (monmap changed)... 2024-03-16T10:38:11.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:38:11.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:38:11.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:11.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:11 smithi026 ceph-mon[37041]: Reconfiguring daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:38:12.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:12 smithi026 ceph-mon[37041]: Reconfiguring mon.smithi026 (monmap changed)... 2024-03-16T10:38:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:38:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:38:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:12 smithi026 ceph-mon[37041]: Reconfiguring daemon mon.smithi026 on smithi026 2024-03-16T10:38:13.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:13.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:13.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:12 smithi002 ceph-mon[29248]: Reconfiguring mon.smithi026 (monmap changed)... 2024-03-16T10:38:13.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:38:13.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:38:13.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:13.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:12 smithi002 ceph-mon[29248]: Reconfiguring daemon mon.smithi026 on smithi026 2024-03-16T10:38:14.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:14.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:38:14.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:38:14.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi002.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T10:38:14.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi002.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi002.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi002.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi002.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi002.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T10:38:14.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.275 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:14 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:14.363 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:14.363 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.363 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.363 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:38:14.363 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:38:14.363 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi002.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi002.front.sepia.ceph.com:9093"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi002.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi002.front.sepia.ceph.com:3000"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi002.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi002.front.sepia.ceph.com:9095"}]: dispatch 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:14.364 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:14 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:15.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:15.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:15.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:15.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:16.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:16 smithi026 ceph-mon[37041]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:16.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:16.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:16 smithi002 ceph-mon[29248]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:16.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:18.290 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:18 smithi002 ceph-mon[29248]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:18.290 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:18 smithi002 ceph-mon[29248]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T10:38:18.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.330+0000 7f275e7fc700 1 -- 172.21.15.2:0/1337962158 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f276019e740 con 0x7f274806c800 2024-03-16T10:38:18.331 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 0 on host 'smithi002' 2024-03-16T10:38:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f274806c800 msgr2=0x7f274806ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f274806c800 0x7f274806ecc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f275400b5c0 tx=0x7f2754005160 comp rx=0 tx=0).stop 2024-03-16T10:38:18.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ff180 msgr2=0x7f2760103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ff180 0x7f2760103050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f275000b9c0 tx=0x7f275000bd80 comp rx=0 tx=0).stop 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 shutdown_connections 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f274806c800 0x7f274806ecc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ff180 0x7f2760103050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 --2- 172.21.15.2:0/1337962158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2760100380 0x7f2760101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 >> 172.21.15.2:0/1337962158 conn(0x7f27600fa720 msgr2=0x7f27600fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 shutdown_connections 2024-03-16T10:38:18.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:18.332+0000 7f2767346700 1 -- 172.21.15.2:0/1337962158 wait complete. 2024-03-16T10:38:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:18 smithi026 ceph-mon[37041]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:18 smithi026 ceph-mon[37041]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T10:38:19.014 DEBUG:teuthology.orchestra.run.smithi002:osd.0> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.0.service 2024-03-16T10:38:19.016 INFO:tasks.cephadm:Deploying osd.1 on smithi002 with /dev/vg_nvme/lv_3... 2024-03-16T10:38:19.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T10:38:19.280 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:19 smithi002 ceph-mon[29248]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T10:38:19.280 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:19 smithi002 ceph-mon[29248]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T10:38:19.280 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:19.280 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:19 smithi002 ceph-mon[29248]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:38:19.280 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:19.280 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:19.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:19 smithi026 ceph-mon[37041]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T10:38:19.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:19 smithi026 ceph-mon[37041]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T10:38:19.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:19.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:19 smithi026 ceph-mon[37041]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:38:19.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:19.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:19.685 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:38:20.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:20 smithi002 ceph-mon[29248]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:20 smithi002 ceph-mon[29248]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-16T10:38:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:20 smithi002 ceph-mon[29248]: osdmap e8: 1 total, 0 up, 1 in 2024-03-16T10:38:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:20.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:20 smithi026 ceph-mon[37041]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:20.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:20 smithi026 ceph-mon[37041]: from='osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-16T10:38:20.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:20 smithi026 ceph-mon[37041]: osdmap e8: 1 total, 0 up, 1 in 2024-03-16T10:38:20.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:20 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:20.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:20 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:21 smithi026 ceph-mon[37041]: purged_snaps scrub starts 2024-03-16T10:38:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:21 smithi026 ceph-mon[37041]: purged_snaps scrub ok 2024-03-16T10:38:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:21 smithi026 ceph-mon[37041]: osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] boot 2024-03-16T10:38:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:21 smithi026 ceph-mon[37041]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T10:38:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:21 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:21 smithi002 ceph-mon[29248]: purged_snaps scrub starts 2024-03-16T10:38:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:21 smithi002 ceph-mon[29248]: purged_snaps scrub ok 2024-03-16T10:38:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:21 smithi002 ceph-mon[29248]: osd.0 [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] boot 2024-03-16T10:38:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:21 smithi002 ceph-mon[29248]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T10:38:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:21 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:38:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T10:38:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T10:38:21.617 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-03-16T10:38:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-03-16T10:38:21.617 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00941751 s, 1.1 GB/s 2024-03-16T10:38:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-03-16T10:38:22.074 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:22 smithi002 ceph-mon[29248]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:22.074 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:22 smithi002 ceph-mon[29248]: osdmap e10: 1 total, 1 up, 1 in 2024-03-16T10:38:22.345 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi002:vg_nvme/lv_3 2024-03-16T10:38:22.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:22 smithi026 ceph-mon[37041]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T10:38:22.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:22 smithi026 ceph-mon[37041]: osdmap e10: 1 total, 1 up, 1 in 2024-03-16T10:38:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:38:23.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:23 smithi026 ceph-mon[37041]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:23.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:23.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:23.774 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:23 smithi002 ceph-mon[29248]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:23.774 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:23.774 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 -- 172.21.15.2:0/4062116269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 msgr2=0x7f13f8106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/4062116269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f8106290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009910 tx=0x7f13ec009c20 comp rx=0 tx=0).stop 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 -- 172.21.15.2:0/4062116269 shutdown_connections 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/4062116269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f8106290 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/4062116269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f8101570 0x7f13f8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 -- 172.21.15.2:0/4062116269 >> 172.21.15.2:0/4062116269 conn(0x7f13f80faf80 msgr2=0x7f13f80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 -- 172.21.15.2:0/4062116269 shutdown_connections 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.283+0000 7f13fe7d6700 1 -- 172.21.15.2:0/4062116269 wait complete. 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.284+0000 7f13fe7d6700 1 Processor -- start 2024-03-16T10:38:24.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.284+0000 7f13fe7d6700 1 -- start start 2024-03-16T10:38:24.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.284+0000 7f13fe7d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f8101570 0x7f13f8100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:38:24.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.284+0000 7f13fe7d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f80ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:38:24.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.284+0000 7f13fe7d6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80ffa60 con 0x7f13f8103ea0 2024-03-16T10:38:24.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.284+0000 7f13fe7d6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80ffba0 con 0x7f13f8101570 2024-03-16T10:38:24.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f80ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:38:24.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f80ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34380/0 (socket says 172.21.15.2:34380) 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f77fe700 1 -- 172.21.15.2:0/3549697606 learned_addr learned my addr 172.21.15.2:0/3549697606 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f7fff700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f8101570 0x7f13f8100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f77fe700 1 -- 172.21.15.2:0/3549697606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f8101570 msgr2=0x7f13f8100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f77fe700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f8101570 0x7f13f8100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f77fe700 1 -- 172.21.15.2:0/3549697606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0095f0 con 0x7f13f8103ea0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f7fff700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f8101570 0x7f13f8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f77fe700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f80ff520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0092e0 tx=0x7f13ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f57fa700 1 -- 172.21.15.2:0/3549697606 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13ec01d070 con 0x7f13f8103ea0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f80ffe20 con 0x7f13f8103ea0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f57fa700 1 -- 172.21.15.2:0/3549697606 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13ec00f460 con 0x7f13f8103ea0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13f57fa700 1 -- 172.21.15.2:0/3549697606 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13ec021640 con 0x7f13f8103ea0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.285+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f8100390 con 0x7f13f8103ea0 2024-03-16T10:38:24.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.286+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f804ea80 con 0x7f13f8103ea0 2024-03-16T10:38:24.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.287+0000 7f13f57fa700 1 -- 172.21.15.2:0/3549697606 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 90466+0+0 (secure 0 0 0) 0x7f13ec00f5d0 con 0x7f13f8103ea0 2024-03-16T10:38:24.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.287+0000 7f13f57fa700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f13e006c800 0x7f13e006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:38:24.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.287+0000 7f13f57fa700 1 -- 172.21.15.2:0/3549697606 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f13ec08bf40 con 0x7f13f8103ea0 2024-03-16T10:38:24.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.287+0000 7f13f7fff700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f13e006c800 0x7f13e006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:38:24.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.288+0000 7f13f7fff700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f13e006c800 0x7f13e006ecc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13e8007900 tx=0x7f13e8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:38:24.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.296+0000 7f13f57fa700 1 -- 172.21.15.2:0/3549697606 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f13ec05aee0 con 0x7f13f8103ea0 2024-03-16T10:38:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:24.513+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f13f819e640 con 0x7f13e006c800 2024-03-16T10:38:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:25 smithi026 ceph-mon[37041]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:25.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:25 smithi026 ceph-mon[37041]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:38:25.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:25 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:38:25.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:25 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:38:25.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:25 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:25.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:25 smithi002 ceph-mon[29248]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:25.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:25 smithi002 ceph-mon[29248]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:38:25.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:25 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:38:25.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:25 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:38:25.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:25 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:27.737 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:27 smithi002 ceph-mon[29248]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:27.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:27 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1256651295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95448bdc-721b-45c0-8c58-0b416e188aa9"}]: dispatch 2024-03-16T10:38:27.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:27 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1256651295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95448bdc-721b-45c0-8c58-0b416e188aa9"}]': finished 2024-03-16T10:38:27.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:27 smithi002 ceph-mon[29248]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T10:38:27.738 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:27 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:27.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:27 smithi026 ceph-mon[37041]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:27.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:27 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1256651295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95448bdc-721b-45c0-8c58-0b416e188aa9"}]: dispatch 2024-03-16T10:38:27.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:27 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1256651295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95448bdc-721b-45c0-8c58-0b416e188aa9"}]': finished 2024-03-16T10:38:27.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:27 smithi026 ceph-mon[37041]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T10:38:27.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:27 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:28.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:28 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2444149009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:38:28.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:28 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2444149009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:38:29.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:29 smithi002 ceph-mon[29248]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:29.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:29 smithi026 ceph-mon[37041]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:30.757 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:30 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:30.826 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:30 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:31.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:31 smithi002 ceph-mon[29248]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:31 smithi026 ceph-mon[37041]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:33.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:33.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:33.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:33.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:38:33.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:33.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:33.670 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:38:33.670 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:33.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:33.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:33.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:33.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:38:33.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:38:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:34.570 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:34 smithi002 ceph-mon[29248]: Detected new or changed devices on smithi002 2024-03-16T10:38:34.570 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:34 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi002 to 11845M 2024-03-16T10:38:34.570 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:34 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:34.793 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:34 smithi026 ceph-mon[37041]: Detected new or changed devices on smithi002 2024-03-16T10:38:34.794 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:34 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi002 to 11845M 2024-03-16T10:38:34.794 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:34 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:35.789 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:35 smithi002 ceph-mon[29248]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:35.789 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:35 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:38:35.789 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:35 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:35.789 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:35 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:35.789 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:35 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:35.833 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:35 smithi026 ceph-mon[37041]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:35.833 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:35 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:38:35.833 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:35 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:35.834 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:35 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:35.834 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:35 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:36.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:36 smithi002 ceph-mon[29248]: Deploying daemon osd.1 on smithi002 2024-03-16T10:38:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:36 smithi026 ceph-mon[37041]: Deploying daemon osd.1 on smithi002 2024-03-16T10:38:37.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:37 smithi002 ceph-mon[29248]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:37.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:37 smithi026 ceph-mon[37041]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:39.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:39 smithi002 ceph-mon[29248]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:39.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:39 smithi026 ceph-mon[37041]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:41.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:41 smithi002 ceph-mon[29248]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:41.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:41.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:41.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:41.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:38:41.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:41.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:41 smithi026 ceph-mon[37041]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:41.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:41.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:41.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:41.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:38:41.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:44.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:43 smithi002 ceph-mon[29248]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:44.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:43 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:44.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:43 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:44.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:43 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:44.091 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:43 smithi026 ceph-mon[37041]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:44.091 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:43 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:44.091 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:43 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:44.091 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:43 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:45.895 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:45 smithi002 ceph-mon[29248]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:45.895 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:45.895 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:45.895 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:45.979 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 1 on host 'smithi002' 2024-03-16T10:38:45.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.977+0000 7f13f57fa700 1 -- 172.21.15.2:0/3549697606 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f13f819e640 con 0x7f13e006c800 2024-03-16T10:38:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f13e006c800 msgr2=0x7f13e006ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f13e006c800 0x7f13e006ecc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13e8007900 tx=0x7f13e8008040 comp rx=0 tx=0).stop 2024-03-16T10:38:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 msgr2=0x7f13f80ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f80ff520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0092e0 tx=0x7f13ec00be20 comp rx=0 tx=0).stop 2024-03-16T10:38:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 shutdown_connections 2024-03-16T10:38:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f13e006c800 0x7f13e006ecc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:45.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f8101570 0x7f13f8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:45.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 --2- 172.21.15.2:0/3549697606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13f8103ea0 0x7f13f80ff520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:45.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 >> 172.21.15.2:0/3549697606 conn(0x7f13f80faf80 msgr2=0x7f13f80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:38:45.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 shutdown_connections 2024-03-16T10:38:45.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:45.980+0000 7f13fe7d6700 1 -- 172.21.15.2:0/3549697606 wait complete. 2024-03-16T10:38:45.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:45 smithi026 ceph-mon[37041]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:45.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:45.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:45.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:38:46.636 DEBUG:teuthology.orchestra.run.smithi002:osd.1> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.1.service 2024-03-16T10:38:46.639 INFO:tasks.cephadm:Deploying osd.2 on smithi002 with /dev/vg_nvme/lv_2... 2024-03-16T10:38:46.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T10:38:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:46 smithi026 ceph-mon[37041]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:38:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:46 smithi002 ceph-mon[29248]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:38:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:38:48.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:47 smithi002 ceph-mon[29248]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:48.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:47 smithi002 ceph-mon[29248]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T10:38:48.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:47 smithi002 ceph-mon[29248]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T10:38:48.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:47 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:48.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:47 smithi002 ceph-mon[29248]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:38:48.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:47 smithi026 ceph-mon[37041]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:48.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:47 smithi026 ceph-mon[37041]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T10:38:48.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:47 smithi026 ceph-mon[37041]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T10:38:48.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:47 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:48.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:47 smithi026 ceph-mon[37041]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:38:48.298 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:38:48 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[56344]: 2024-03-16T10:38:48.252+0000 7fe3344af700 -1 osd.1 0 waiting for initial osdmap 2024-03-16T10:38:48.299 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:38:48 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[56344]: 2024-03-16T10:38:48.256+0000 7fe32f2a2700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:38:49.507 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:49 smithi002 ceph-mon[29248]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:49.507 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:49 smithi002 ceph-mon[29248]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-16T10:38:49.507 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:49 smithi002 ceph-mon[29248]: osdmap e13: 2 total, 1 up, 2 in 2024-03-16T10:38:49.507 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:49 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:49.507 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:49 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:49.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:49 smithi026 ceph-mon[37041]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T10:38:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:49 smithi026 ceph-mon[37041]: from='osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-16T10:38:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:49 smithi026 ceph-mon[37041]: osdmap e13: 2 total, 1 up, 2 in 2024-03-16T10:38:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:49 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:49 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:49.881 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T10:38:49.881 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T10:38:49.881 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-03-16T10:38:49.881 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-03-16T10:38:49.881 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00957769 s, 1.1 GB/s 2024-03-16T10:38:49.881 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-03-16T10:38:50.540 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:50 smithi002 ceph-mon[29248]: purged_snaps scrub starts 2024-03-16T10:38:50.541 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:50 smithi002 ceph-mon[29248]: purged_snaps scrub ok 2024-03-16T10:38:50.541 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:50 smithi002 ceph-mon[29248]: osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] boot 2024-03-16T10:38:50.541 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:50 smithi002 ceph-mon[29248]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T10:38:50.541 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:50.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:50 smithi026 ceph-mon[37041]: purged_snaps scrub starts 2024-03-16T10:38:50.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:50 smithi026 ceph-mon[37041]: purged_snaps scrub ok 2024-03-16T10:38:50.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:50 smithi026 ceph-mon[37041]: osd.1 [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] boot 2024-03-16T10:38:50.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:50 smithi026 ceph-mon[37041]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T10:38:50.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:38:50.846 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi002:vg_nvme/lv_2 2024-03-16T10:38:51.547 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:51 smithi002 ceph-mon[29248]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:51.547 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:51 smithi002 ceph-mon[29248]: osdmap e15: 2 total, 2 up, 2 in 2024-03-16T10:38:51.575 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:38:51.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:51 smithi026 ceph-mon[37041]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:51 smithi026 ceph-mon[37041]: osdmap e15: 2 total, 2 up, 2 in 2024-03-16T10:38:52.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:52 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:52.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:52 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:52.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:52 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:52.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:52 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 -- 172.21.15.2:0/909323334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c0ffa00 msgr2=0x7f684c0ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 --2- 172.21.15.2:0/909323334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c0ffa00 0x7f684c0ffe20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6844009910 tx=0x7f6844009c20 comp rx=0 tx=0).stop 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 -- 172.21.15.2:0/909323334 shutdown_connections 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 --2- 172.21.15.2:0/909323334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c100c00 0x7f684c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 --2- 172.21.15.2:0/909323334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c0ffa00 0x7f684c0ffe20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 -- 172.21.15.2:0/909323334 >> 172.21.15.2:0/909323334 conn(0x7f684c0fafc0 msgr2=0x7f684c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 -- 172.21.15.2:0/909323334 shutdown_connections 2024-03-16T10:38:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 -- 172.21.15.2:0/909323334 wait complete. 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.033+0000 7f6853982700 1 Processor -- start 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6853982700 1 -- start start 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6853982700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c0ffa00 0x7f684c196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6853982700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c100c00 0x7f684c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6853982700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c197070 con 0x7f684c100c00 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6853982700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1971b0 con 0x7f684c0ffa00 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6850f1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c100c00 0x7f684c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6850f1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c100c00 0x7f684c196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50362/0 (socket says 172.21.15.2:50362) 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6850f1d700 1 -- 172.21.15.2:0/3220615265 learned_addr learned my addr 172.21.15.2:0/3220615265 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f685171e700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c0ffa00 0x7f684c196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:38:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6850f1d700 1 -- 172.21.15.2:0/3220615265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c0ffa00 msgr2=0x7f684c196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6850f1d700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c0ffa00 0x7f684c196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6850f1d700 1 -- 172.21.15.2:0/3220615265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440095f0 con 0x7f684c100c00 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6850f1d700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c100c00 0x7f684c196a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f684000d8d0 tx=0x7f684000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f684a7fc700 1 -- 172.21.15.2:0/3220615265 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68400098a0 con 0x7f684c100c00 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f684a7fc700 1 -- 172.21.15.2:0/3220615265 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6840010460 con 0x7f684c100c00 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.034+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c19bc10 con 0x7f684c100c00 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.035+0000 7f684a7fc700 1 -- 172.21.15.2:0/3220615265 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f684000f5f0 con 0x7f684c100c00 2024-03-16T10:38:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.035+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c19c1e0 con 0x7f684c100c00 2024-03-16T10:38:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.036+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c04ea80 con 0x7f684c100c00 2024-03-16T10:38:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.036+0000 7f684a7fc700 1 -- 172.21.15.2:0/3220615265 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 90466+0+0 (secure 0 0 0) 0x7f6840009a00 con 0x7f684c100c00 2024-03-16T10:38:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.036+0000 7f684a7fc700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f683406c800 0x7f683406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:38:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.036+0000 7f684a7fc700 1 -- 172.21.15.2:0/3220615265 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f684008ae40 con 0x7f684c100c00 2024-03-16T10:38:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.036+0000 7f685171e700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f683406c800 0x7f683406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:38:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.037+0000 7f685171e700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f683406c800 0x7f683406ecc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6844000c00 tx=0x7f6844005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:38:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.041+0000 7f684a7fc700 1 -- 172.21.15.2:0/3220615265 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6840059bb0 con 0x7f684c100c00 2024-03-16T10:38:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:38:53.257+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f684c069750 con 0x7f683406c800 2024-03-16T10:38:53.754 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:53 smithi002 ceph-mon[29248]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:53.755 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:53 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:38:53.755 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:53 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:38:53.755 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:53 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:53.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:53 smithi026 ceph-mon[37041]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:53.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:53 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:38:53.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:53 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:38:53.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:53 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:54.707 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:54 smithi002 ceph-mon[29248]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:38:54.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:54 smithi026 ceph-mon[37041]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:38:55.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:55 smithi002 ceph-mon[29248]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:55.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:55 smithi026 ceph-mon[37041]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:57.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:38:57.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:38:57.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi002 to 5922M 2024-03-16T10:38:57.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:57.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:57.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:38:57.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:57.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:57 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:58.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi002 to 5922M 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:58.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:57 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:38:59.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2004567021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce6a7a3f-db89-426c-bbb4-1339c5adac81"}]: dispatch 2024-03-16T10:38:59.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2004567021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce6a7a3f-db89-426c-bbb4-1339c5adac81"}]': finished 2024-03-16T10:38:59.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:58 smithi002 ceph-mon[29248]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T10:38:59.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:38:59.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3744079776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:38:59.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:59.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:38:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:59.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:58 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2004567021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce6a7a3f-db89-426c-bbb4-1339c5adac81"}]: dispatch 2024-03-16T10:38:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:58 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2004567021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce6a7a3f-db89-426c-bbb4-1339c5adac81"}]': finished 2024-03-16T10:38:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:58 smithi026 ceph-mon[37041]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T10:38:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:38:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:58 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3744079776' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:38:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:38:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:38:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:00.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:00 smithi002 ceph-mon[29248]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:00 smithi026 ceph-mon[37041]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:01.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:01 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:02 smithi002 ceph-mon[29248]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:02.458 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:02 smithi026 ceph-mon[37041]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:02.458 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:02.459 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:02.459 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:02.459 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:02.459 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:04.303 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:04 smithi002 ceph-mon[29248]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:04.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:04 smithi026 ceph-mon[37041]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:06.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:06 smithi002 ceph-mon[29248]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:06.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:06 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:39:06.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:06 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:06.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:06 smithi026 ceph-mon[37041]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:06 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:39:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:06 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:07.290 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:07 smithi002 ceph-mon[29248]: Deploying daemon osd.2 on smithi002 2024-03-16T10:39:07.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:07 smithi026 ceph-mon[37041]: Deploying daemon osd.2 on smithi002 2024-03-16T10:39:08.113 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:08 smithi002 ceph-mon[29248]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:08.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:08 smithi026 ceph-mon[37041]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:10.121 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:10 smithi002 ceph-mon[29248]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:10.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:10 smithi026 ceph-mon[37041]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:11.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:11 smithi026 ceph-mon[37041]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:11.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:11 smithi002 ceph-mon[29248]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:13.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.665+0000 7f684a7fc700 1 -- 172.21.15.2:0/3220615265 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f684c069750 con 0x7f683406c800 2024-03-16T10:39:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 2 on host 'smithi002' 2024-03-16T10:39:13.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f683406c800 msgr2=0x7f683406ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:13.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f683406c800 0x7f683406ecc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6844000c00 tx=0x7f6844005960 comp rx=0 tx=0).stop 2024-03-16T10:39:13.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c100c00 msgr2=0x7f684c196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c100c00 0x7f684c196a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f684000d8d0 tx=0x7f684000dbe0 comp rx=0 tx=0).stop 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 shutdown_connections 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f683406c800 0x7f683406ecc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f684c0ffa00 0x7f684c196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 --2- 172.21.15.2:0/3220615265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f684c100c00 0x7f684c196a50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.667+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 >> 172.21.15.2:0/3220615265 conn(0x7f684c0fafc0 msgr2=0x7f684c069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.668+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 shutdown_connections 2024-03-16T10:39:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:39:13.668+0000 7f6853982700 1 -- 172.21.15.2:0/3220615265 wait complete. 2024-03-16T10:39:13.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:13 smithi026 ceph-mon[37041]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:13.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:13 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:13.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:13 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:14.009 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:13 smithi002 ceph-mon[29248]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:14.009 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:13 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:14.009 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:13 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:14.641 DEBUG:teuthology.orchestra.run.smithi002:osd.2> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.2.service 2024-03-16T10:39:14.643 INFO:tasks.cephadm:Deploying osd.3 on smithi026 with /dev/vg_nvme/lv_4... 2024-03-16T10:39:14.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T10:39:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:39:14.948 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:14 smithi002 ceph-mon[29248]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:39:14.973 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:14 smithi026 ceph-mon[37041]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:39:15.879 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:15 smithi026 ceph-mon[37041]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:15.879 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:15 smithi026 ceph-mon[37041]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T10:39:15.879 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:15 smithi026 ceph-mon[37041]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T10:39:15.879 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:15.879 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:15 smithi026 ceph-mon[37041]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:39:15.879 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-mon[29248]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-mon[29248]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T10:39:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-mon[29248]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T10:39:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-mon[29248]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:39:16.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:16.049 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[63777]: 2024-03-16T10:39:15.740+0000 7f95edf4e700 -1 osd.2 0 waiting for initial osdmap 2024-03-16T10:39:16.049 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:15 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[63777]: 2024-03-16T10:39:15.745+0000 7f95e9542700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:39:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T10:39:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T10:39:16.677 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-03-16T10:39:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-03-16T10:39:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00830404 s, 1.3 GB/s 2024-03-16T10:39:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-03-16T10:39:17.031 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:16 smithi002 ceph-mon[29248]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-16T10:39:17.031 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:16 smithi002 ceph-mon[29248]: osdmap e18: 3 total, 2 up, 3 in 2024-03-16T10:39:17.031 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:17.031 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:17.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:16 smithi026 ceph-mon[37041]: from='osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-03-16T10:39:17.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:16 smithi026 ceph-mon[37041]: osdmap e18: 3 total, 2 up, 3 in 2024-03-16T10:39:17.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:17.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:17.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2024-03-16T10:39:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:39:17.830 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: purged_snaps scrub starts 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: purged_snaps scrub ok 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] boot 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:17.831 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:17 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:18.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: purged_snaps scrub starts 2024-03-16T10:39:18.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: purged_snaps scrub ok 2024-03-16T10:39:18.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-16T10:39:18.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:18.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: osd.2 [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] boot 2024-03-16T10:39:18.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T10:39:18.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:39:18.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:18.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:17 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 -- 172.21.15.26:0/3137811345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f84100be0 msgr2=0x7f4f84101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/3137811345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f84100be0 0x7f4f84101040 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009910 tx=0x7f4f74009c20 comp rx=0 tx=0).stop 2024-03-16T10:39:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 -- 172.21.15.26:0/3137811345 shutdown_connections 2024-03-16T10:39:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/3137811345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f84100be0 0x7f4f84101040 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/3137811345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f840ff9e0 0x7f4f840ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 -- 172.21.15.26:0/3137811345 >> 172.21.15.26:0/3137811345 conn(0x7f4f840faf80 msgr2=0x7f4f840fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:39:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 -- 172.21.15.26:0/3137811345 shutdown_connections 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.974+0000 7f4f8c72b700 1 -- 172.21.15.26:0/3137811345 wait complete. 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8c72b700 1 Processor -- start 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8c72b700 1 -- start start 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8c72b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f840ff9e0 0x7f4f84069b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8c72b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f84100be0 0x7f4f8419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8c72b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8419b7c0 con 0x7f4f84100be0 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8c72b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8419b900 con 0x7f4f840ff9e0 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8a4c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f840ff9e0 0x7f4f84069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8a4c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f840ff9e0 0x7f4f84069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39726/0 (socket says 172.21.15.26:39726) 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8a4c7700 1 -- 172.21.15.26:0/30702619 learned_addr learned my addr 172.21.15.26:0/30702619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f89cc6700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f84100be0 0x7f4f8419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8a4c7700 1 -- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f84100be0 msgr2=0x7f4f8419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8a4c7700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f84100be0 0x7f4f8419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.975+0000 7f4f8a4c7700 1 -- 172.21.15.26:0/30702619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f740095f0 con 0x7f4f840ff9e0 2024-03-16T10:39:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.976+0000 7f4f8a4c7700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f840ff9e0 0x7f4f84069b00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c8f0 tx=0x7f4f8000cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:39:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.976+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f800078e0 con 0x7f4f840ff9e0 2024-03-16T10:39:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.976+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f8000f450 con 0x7f4f840ff9e0 2024-03-16T10:39:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.976+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8419bbe0 con 0x7f4f840ff9e0 2024-03-16T10:39:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.976+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f80018610 con 0x7f4f840ff9e0 2024-03-16T10:39:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.976+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8419c130 con 0x7f4f840ff9e0 2024-03-16T10:39:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.977+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8418f560 con 0x7f4f840ff9e0 2024-03-16T10:39:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.977+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 90466+0+0 (secure 0 0 0) 0x7f4f8000f5c0 con 0x7f4f840ff9e0 2024-03-16T10:39:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.978+0000 7f4f7b7fe700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7006c800 0x7f4f7006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:39:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.978+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3221+0+0 (secure 0 0 0) 0x7f4f8008b0d0 con 0x7f4f840ff9e0 2024-03-16T10:39:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.978+0000 7f4f89cc6700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7006c800 0x7f4f7006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:39:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.978+0000 7f4f89cc6700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7006c800 0x7f4f7006ecc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f740051c0 tx=0x7f4f7400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:39:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:18.982+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4f80059b90 con 0x7f4f840ff9e0 2024-03-16T10:39:19.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:18 smithi002 ceph-mon[29248]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T10:39:19.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:18 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:39:19.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:19.193+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f840611f0 con 0x7f4f7006c800 2024-03-16T10:39:19.195 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:18 smithi026 ceph-mon[37041]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T10:39:19.195 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:18 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:39:19.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:19 smithi026 ceph-mon[37041]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T10:39:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:19 smithi026 ceph-mon[37041]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T10:39:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:39:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:39:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:39:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:19 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:19 smithi002 ceph-mon[29248]: pgmap v77: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T10:39:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:19 smithi002 ceph-mon[29248]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T10:39:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T10:39:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:39:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:39:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:19 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:20.549 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68237]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:39:20.549 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:20.549 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68237]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:20.550 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68237]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:20.891 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:20 smithi026 ceph-mon[37041]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:39:20.891 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:20 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:39:20.891 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:20 smithi026 ceph-mon[37041]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T10:39:20.891 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:20 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:20.919 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:20 smithi002 ceph-mon[29248]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:39:20.919 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T10:39:20.919 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:20 smithi002 ceph-mon[29248]: osdmap e22: 3 total, 3 up, 3 in 2024-03-16T10:39:20.919 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:20 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:20.919 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68242]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:39:20.919 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68242]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:20.919 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68242]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:20.919 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68242]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:21.189 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68245]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:39:21.189 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68245]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:21.189 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68245]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:21.189 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:20 smithi002 sudo[68245]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:21.445 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68346]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:39:21.446 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68346]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:21.446 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68346]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:21.446 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68346]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:21.721 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68377]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:39:21.721 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:21.721 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68377]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:21.722 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68377]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:21.980 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68440]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:39:21.980 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68440]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:21.980 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68440]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:21.980 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68440]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:22.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68462]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:39:22.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:22.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68462]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:22.256 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:21 smithi002 sudo[68462]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:22.268+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mon.1 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f4f800564d0 con 0x7f4f840ff9e0 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: osdmap e23: 3 total, 3 up, 3 in 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: Detected new or changed devices on smithi002 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi002 to 3948M 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.528 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:22.529 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:22.529 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.529 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:22.529 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:39:22.529 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41016]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T10:39:22.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 sudo[68499]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 sudo[68499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 sudo[68499]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 sudo[68499]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: osdmap e23: 3 total, 3 up, 3 in 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: Detected new or changed devices on smithi002 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi002 to 3948M 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:22.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:22.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:22.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:22 smithi002 ceph-mon[29248]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:39:22.792 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41016]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:22.793 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41016]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:22.793 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41016]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41019]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T10:39:23.810 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T10:39:23.810 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41019]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T10:39:23.810 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:22 smithi026 sudo[41019]: pam_unix(sudo:session): session closed for user root 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/741189728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d760311-5fda-425b-ac3c-5045ecb1cbea"}]: dispatch 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d760311-5fda-425b-ac3c-5045ecb1cbea"}]: dispatch 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: mgrmap e19: smithi002.zevrtm(active, since 2m), standbys: smithi026.omjkpo 2024-03-16T10:39:23.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d760311-5fda-425b-ac3c-5045ecb1cbea"}]': finished 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/741189728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d760311-5fda-425b-ac3c-5045ecb1cbea"}]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d760311-5fda-425b-ac3c-5045ecb1cbea"}]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: mgrmap e19: smithi002.zevrtm(active, since 2m), standbys: smithi026.omjkpo 2024-03-16T10:39:23.812 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T10:39:23.813 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d760311-5fda-425b-ac3c-5045ecb1cbea"}]': finished 2024-03-16T10:39:23.813 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T10:39:23.813 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:23.813 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:39:23.813 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:39:23.813 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T10:39:23.813 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:24 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/143562022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:39:24.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:24 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/143562022' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:39:25.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:25 smithi002 ceph-mon[29248]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:25.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:25 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:25.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:25 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:25 smithi026 ceph-mon[37041]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:25 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:25.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:25 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:26 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:26 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:26.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:26 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:26 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:26 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:26.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:26 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:27.930 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:27 smithi026 ceph-mon[37041]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:28.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:27 smithi002 ceph-mon[29248]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:29.840 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:29 smithi026 ceph-mon[37041]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:30.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:29 smithi002 ceph-mon[29248]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:30.914 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:30 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:39:30.915 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:30 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:30.915 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:30 smithi026 ceph-mon[37041]: Deploying daemon osd.3 on smithi026 2024-03-16T10:39:30.915 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:30 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:30 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:39:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:30 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:31.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:30 smithi002 ceph-mon[29248]: Deploying daemon osd.3 on smithi026 2024-03-16T10:39:31.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:30 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:31.782 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:31 smithi026 ceph-mon[37041]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:31.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:31 smithi002 ceph-mon[29248]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:33.993 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:33 smithi026 ceph-mon[37041]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:34.022 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:33 smithi002 ceph-mon[29248]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:36 smithi026 ceph-mon[37041]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:36 smithi002 ceph-mon[29248]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:38.137 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:38 smithi026 ceph-mon[37041]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:38.210 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 3 on host 'smithi026' 2024-03-16T10:39:38.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.209+0000 7f4f7b7fe700 1 -- 172.21.15.26:0/30702619 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4f840611f0 con 0x7f4f7006c800 2024-03-16T10:39:38.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.211+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7006c800 msgr2=0x7f4f7006ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.211+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7006c800 0x7f4f7006ecc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f740051c0 tx=0x7f4f7400b540 comp rx=0 tx=0).stop 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.211+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f840ff9e0 msgr2=0x7f4f84069b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.211+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f840ff9e0 0x7f4f84069b00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4f8000c8f0 tx=0x7f4f8000cc00 comp rx=0 tx=0).stop 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.212+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 shutdown_connections 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.212+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7006c800 0x7f4f7006ecc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.212+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f840ff9e0 0x7f4f84069b00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.212+0000 7f4f8c72b700 1 --2- 172.21.15.26:0/30702619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f84100be0 0x7f4f8419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.212+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 >> 172.21.15.26:0/30702619 conn(0x7f4f840faf80 msgr2=0x7f4f841041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.212+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 shutdown_connections 2024-03-16T10:39:38.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:38.212+0000 7f4f8c72b700 1 -- 172.21.15.26:0/30702619 wait complete. 2024-03-16T10:39:38.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:38 smithi002 ceph-mon[29248]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:39.422 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:39 smithi026 ceph-mon[37041]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:39.422 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:39 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:39.422 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:39 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:39.422 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:39 smithi026 ceph-mon[37041]: from='osd.3 [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:39:39.422 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:39 smithi026 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:39:39.441 DEBUG:teuthology.orchestra.run.smithi026:osd.3> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.3.service 2024-03-16T10:39:39.443 INFO:tasks.cephadm:Deploying osd.4 on smithi026 with /dev/vg_nvme/lv_3... 2024-03-16T10:39:39.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T10:39:39.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:39 smithi002 ceph-mon[29248]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:39.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:39 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:39.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:39 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:39.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:39 smithi002 ceph-mon[29248]: from='osd.3 [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:39:39.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:39 smithi002 ceph-mon[29248]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:39:39.691 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:39:40.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T10:39:40.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-mon[37041]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T10:39:40.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:40.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-mon[37041]: from='osd.3 [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:39:40.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:39:40.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:40.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:40.483 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[43138]: 2024-03-16T10:39:40.182+0000 7f3535178700 -1 osd.3 0 waiting for initial osdmap 2024-03-16T10:39:40.483 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:39:40 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[43138]: 2024-03-16T10:39:40.188+0000 7f353176e700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:39:40.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:40 smithi002 ceph-mon[29248]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T10:39:40.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:40 smithi002 ceph-mon[29248]: osdmap e25: 4 total, 3 up, 4 in 2024-03-16T10:39:40.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:40 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:40.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:40 smithi002 ceph-mon[29248]: from='osd.3 [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:39:40.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:40 smithi002 ceph-mon[29248]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:39:40.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:40 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:40.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:40 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:41.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:41 smithi026 ceph-mon[37041]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:41.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:41 smithi026 ceph-mon[37041]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-03-16T10:39:41.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:41 smithi026 ceph-mon[37041]: osdmap e26: 4 total, 3 up, 4 in 2024-03-16T10:39:41.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:41.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:41 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:41.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:41 smithi002 ceph-mon[29248]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:41.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:41 smithi002 ceph-mon[29248]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-03-16T10:39:41.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:41 smithi002 ceph-mon[29248]: osdmap e26: 4 total, 3 up, 4 in 2024-03-16T10:39:41.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:41.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:41 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:42.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:42 smithi026 ceph-mon[37041]: purged_snaps scrub starts 2024-03-16T10:39:42.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:42 smithi026 ceph-mon[37041]: purged_snaps scrub ok 2024-03-16T10:39:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:42 smithi026 ceph-mon[37041]: osd.3 [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] boot 2024-03-16T10:39:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:42 smithi026 ceph-mon[37041]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T10:39:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:42 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:42 smithi026 ceph-mon[37041]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T10:39:42.395 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T10:39:42.395 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T10:39:42.395 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-03-16T10:39:42.395 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-03-16T10:39:42.395 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0102351 s, 1.0 GB/s 2024-03-16T10:39:42.395 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-03-16T10:39:42.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:42 smithi002 ceph-mon[29248]: purged_snaps scrub starts 2024-03-16T10:39:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:42 smithi002 ceph-mon[29248]: purged_snaps scrub ok 2024-03-16T10:39:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:42 smithi002 ceph-mon[29248]: osd.3 [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] boot 2024-03-16T10:39:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:42 smithi002 ceph-mon[29248]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T10:39:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:42 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:39:42.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:42 smithi002 ceph-mon[29248]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T10:39:43.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:43 smithi026 ceph-mon[37041]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:43.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:43 smithi026 ceph-mon[37041]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T10:39:43.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:43 smithi002 ceph-mon[29248]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-16T10:39:43.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:43 smithi002 ceph-mon[29248]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T10:39:43.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2024-03-16T10:39:44.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:44 smithi026 ceph-mon[37041]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T10:39:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:39:44.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:44 smithi002 ceph-mon[29248]: osdmap e30: 4 total, 4 up, 4 in 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: osdmap e31: 4 total, 4 up, 4 in 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: Detected new or changed devices on smithi026 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi026 to 14917M 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:45.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:45.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:45.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:45.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:45.688 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:39:45.688 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: osdmap e31: 4 total, 4 up, 4 in 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: Detected new or changed devices on smithi026 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi026 to 14917M 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:39:45.689 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:39:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.223+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/2493069614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0101ce0 msgr2=0x7fa5a0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.223+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/2493069614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0101ce0 0x7fa5a0102140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa590009910 tx=0x7fa590009c20 comp rx=0 tx=0).stop 2024-03-16T10:39:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.223+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/2493069614 shutdown_connections 2024-03-16T10:39:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.223+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/2493069614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0101ce0 0x7fa5a0102140 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.223+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/2493069614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a0100ae0 0x7fa5a0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.223+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/2493069614 >> 172.21.15.26:0/2493069614 conn(0x7fa5a00fc0a0 msgr2=0x7fa5a00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:39:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.224+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/2493069614 shutdown_connections 2024-03-16T10:39:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.224+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/2493069614 wait complete. 2024-03-16T10:39:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.224+0000 7fa5a7c4b700 1 Processor -- start 2024-03-16T10:39:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.224+0000 7fa5a7c4b700 1 -- start start 2024-03-16T10:39:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a7c4b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0100ae0 0x7fa5a0074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:39:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a7c4b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a0101ce0 0x7fa5a00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a7c4b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a00736e0 con 0x7fa5a0101ce0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a7c4b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0073820 con 0x7fa5a0100ae0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a59e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0100ae0 0x7fa5a0074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a59e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0100ae0 0x7fa5a0074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37556/0 (socket says 172.21.15.26:37556) 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a59e7700 1 -- 172.21.15.26:0/3499235654 learned_addr learned my addr 172.21.15.26:0/3499235654 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a51e6700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a0101ce0 0x7fa5a00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a59e7700 1 -- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a0101ce0 msgr2=0x7fa5a00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a59e7700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a0101ce0 0x7fa5a00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a59e7700 1 -- 172.21.15.26:0/3499235654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5900095f0 con 0x7fa5a0100ae0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a59e7700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0100ae0 0x7fa5a0074ab0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00d900 tx=0x7fa59c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa596ffd700 1 -- 172.21.15.26:0/3499235654 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa59c0041f0 con 0x7fa5a0100ae0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa596ffd700 1 -- 172.21.15.26:0/3499235654 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa59c004350 con 0x7fa5a0100ae0 2024-03-16T10:39:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa596ffd700 1 -- 172.21.15.26:0/3499235654 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa59c003dc0 con 0x7fa5a0100ae0 2024-03-16T10:39:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a0073b00 con 0x7fa5a0100ae0 2024-03-16T10:39:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.225+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a019e560 con 0x7fa5a0100ae0 2024-03-16T10:39:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.226+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a0190660 con 0x7fa5a0100ae0 2024-03-16T10:39:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.227+0000 7fa596ffd700 1 -- 172.21.15.26:0/3499235654 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fa59c009730 con 0x7fa5a0100ae0 2024-03-16T10:39:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.227+0000 7fa596ffd700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa58c06c880 0x7fa58c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:39:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.227+0000 7fa596ffd700 1 -- 172.21.15.26:0/3499235654 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7fa59c08b7c0 con 0x7fa5a0100ae0 2024-03-16T10:39:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.228+0000 7fa5a51e6700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa58c06c880 0x7fa58c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:39:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.228+0000 7fa5a51e6700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa58c06c880 0x7fa58c06ed40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa5900051c0 tx=0x7fa590005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:39:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.232+0000 7fa596ffd700 1 -- 172.21.15.26:0/3499235654 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa59c059f00 con 0x7fa5a0100ae0 2024-03-16T10:39:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:39:46.443+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5a00611f0 con 0x7fa58c06c880 2024-03-16T10:39:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:47 smithi026 ceph-mon[37041]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-16T10:39:47.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:47 smithi026 ceph-mon[37041]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:39:47.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:47 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:39:47.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:47 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:39:47.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:47 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:47.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:47 smithi002 ceph-mon[29248]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-03-16T10:39:47.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:47 smithi002 ceph-mon[29248]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:39:47.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:47 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:39:47.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:47 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:39:47.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:47 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:49.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:49 smithi002 ceph-mon[29248]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:39:49.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:49 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/960286237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "250a3093-6818-4f4f-85d5-d1f2e4381afa"}]: dispatch 2024-03-16T10:39:49.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:49 smithi002 ceph-mon[29248]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "250a3093-6818-4f4f-85d5-d1f2e4381afa"}]: dispatch 2024-03-16T10:39:49.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:49 smithi002 ceph-mon[29248]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "250a3093-6818-4f4f-85d5-d1f2e4381afa"}]': finished 2024-03-16T10:39:49.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:49 smithi002 ceph-mon[29248]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T10:39:49.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:49 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:39:49.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:49 smithi026 ceph-mon[37041]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T10:39:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:49 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/960286237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "250a3093-6818-4f4f-85d5-d1f2e4381afa"}]: dispatch 2024-03-16T10:39:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:49 smithi026 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "250a3093-6818-4f4f-85d5-d1f2e4381afa"}]: dispatch 2024-03-16T10:39:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:49 smithi026 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "250a3093-6818-4f4f-85d5-d1f2e4381afa"}]': finished 2024-03-16T10:39:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:49 smithi026 ceph-mon[37041]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T10:39:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:49 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:39:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:50 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/2211661105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:39:50.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:50 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/2211661105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:39:51.542 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:51 smithi002 ceph-mon[29248]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-16T10:39:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:51 smithi026 ceph-mon[37041]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-16T10:39:53.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:53 smithi002 ceph-mon[29248]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:39:53.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:53 smithi026 ceph-mon[37041]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T10:39:55.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:55 smithi002 ceph-mon[29248]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:39:55.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:55 smithi026 ceph-mon[37041]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T10:39:56.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:56 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:39:56.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:56 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:56 smithi026 ceph-mon[37041]: Deploying daemon osd.4 on smithi026 2024-03-16T10:39:56.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:56 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:39:56.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:56 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:39:56.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:56 smithi002 ceph-mon[29248]: Deploying daemon osd.4 on smithi026 2024-03-16T10:39:57.459 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:57 smithi026 ceph-mon[37041]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:39:57.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:57 smithi002 ceph-mon[29248]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:39:59.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:39:59 smithi002 ceph-mon[29248]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:39:59.585 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:39:59 smithi026 ceph-mon[37041]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:00.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:00 smithi002 ceph-mon[29248]: overall HEALTH_OK 2024-03-16T10:40:00.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:00.573 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:00 smithi026 ceph-mon[37041]: overall HEALTH_OK 2024-03-16T10:40:00.573 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:00 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:01.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:01 smithi026 ceph-mon[37041]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:01.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:01 smithi002 ceph-mon[29248]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:02.725 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:02.725 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:02.725 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:02 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:02.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:02.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:02.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:02 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:03.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:03 smithi026 ceph-mon[37041]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:03.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:03 smithi002 ceph-mon[29248]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.207+0000 7fa596ffd700 1 -- 172.21.15.26:0/3499235654 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa5a00611f0 con 0x7fa58c06c880 2024-03-16T10:40:04.208 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 4 on host 'smithi026' 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa58c06c880 msgr2=0x7fa58c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa58c06c880 0x7fa58c06ed40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa5900051c0 tx=0x7fa590005a90 comp rx=0 tx=0).stop 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0100ae0 msgr2=0x7fa5a0074ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0100ae0 0x7fa5a0074ab0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00d900 tx=0x7fa59c00dcc0 comp rx=0 tx=0).stop 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 shutdown_connections 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa58c06c880 0x7fa58c06ed40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5a0100ae0 0x7fa5a0074ab0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 --2- 172.21.15.26:0/3499235654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a0101ce0 0x7fa5a00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.209+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 >> 172.21.15.26:0/3499235654 conn(0x7fa5a00fc0a0 msgr2=0x7fa5a01052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.210+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 shutdown_connections 2024-03-16T10:40:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:04.210+0000 7fa5a7c4b700 1 -- 172.21.15.26:0/3499235654 wait complete. 2024-03-16T10:40:05.045 DEBUG:teuthology.orchestra.run.smithi026:osd.4> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.4.service 2024-03-16T10:40:05.048 INFO:tasks.cephadm:Deploying osd.5 on smithi026 with /dev/vg_nvme/lv_2... 2024-03-16T10:40:05.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T10:40:05.291 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:05 smithi026 ceph-mon[37041]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:05.291 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:05.292 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:05 smithi002 ceph-mon[29248]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:05.600 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:40:05 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[49782]: 2024-03-16T10:40:05.289+0000 7f798cc95640 -1 osd.4 0 log_to_monitors true 2024-03-16T10:40:05.933 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:40:06.289 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:06 smithi026 ceph-mon[37041]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:40:06.289 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:06 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:06.289 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:06 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:06.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:06 smithi002 ceph-mon[29248]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:40:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:06 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:06 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:07.471 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:07 smithi026 ceph-mon[37041]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:07.472 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:07 smithi026 ceph-mon[37041]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T10:40:07.472 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:07 smithi026 ceph-mon[37041]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T10:40:07.472 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:07 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:07.472 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:07 smithi026 ceph-mon[37041]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:40:07.472 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:40:07 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[49782]: 2024-03-16T10:40:07.180+0000 7f7981af8700 -1 osd.4 0 waiting for initial osdmap 2024-03-16T10:40:07.472 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:40:07 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[49782]: 2024-03-16T10:40:07.189+0000 7f797c0ea700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:40:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:07 smithi002 ceph-mon[29248]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:07 smithi002 ceph-mon[29248]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T10:40:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:07 smithi002 ceph-mon[29248]: osdmap e33: 5 total, 4 up, 5 in 2024-03-16T10:40:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:07 smithi002 ceph-mon[29248]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:40:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T10:40:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T10:40:07.841 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2024-03-16T10:40:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2024-03-16T10:40:07.841 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010822 s, 969 MB/s 2024-03-16T10:40:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2024-03-16T10:40:08.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:08 smithi026 ceph-mon[37041]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-03-16T10:40:08.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:08 smithi026 ceph-mon[37041]: osdmap e34: 5 total, 4 up, 5 in 2024-03-16T10:40:08.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:08 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:08.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:08 smithi002 ceph-mon[29248]: from='osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-03-16T10:40:08.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:08 smithi002 ceph-mon[29248]: osdmap e34: 5 total, 4 up, 5 in 2024-03-16T10:40:08.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:08 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:09.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2024-03-16T10:40:09.475 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:09 smithi026 ceph-mon[37041]: purged_snaps scrub starts 2024-03-16T10:40:09.475 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:09 smithi026 ceph-mon[37041]: purged_snaps scrub ok 2024-03-16T10:40:09.475 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:09 smithi026 ceph-mon[37041]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:09.475 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:09 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:09.475 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:09 smithi026 ceph-mon[37041]: osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] boot 2024-03-16T10:40:09.475 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:09 smithi026 ceph-mon[37041]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T10:40:09.475 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:09 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:09 smithi002 ceph-mon[29248]: purged_snaps scrub starts 2024-03-16T10:40:09.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:09 smithi002 ceph-mon[29248]: purged_snaps scrub ok 2024-03-16T10:40:09.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:09 smithi002 ceph-mon[29248]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-16T10:40:09.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:09 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:09.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:09 smithi002 ceph-mon[29248]: osd.4 [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] boot 2024-03-16T10:40:09.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:09 smithi002 ceph-mon[29248]: osdmap e35: 5 total, 5 up, 5 in 2024-03-16T10:40:09.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:09 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:40:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:40:10.440 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:10 smithi026 ceph-mon[37041]: osdmap e36: 5 total, 5 up, 5 in 2024-03-16T10:40:10.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:10 smithi002 ceph-mon[29248]: osdmap e36: 5 total, 5 up, 5 in 2024-03-16T10:40:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:11 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:11.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:11.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:11.544 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:11.544 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:11.544 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:11.544 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:11.544 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:11.544 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:11.544 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:11 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.078+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/1954327394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 msgr2=0x7f0d64106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.078+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/1954327394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64106310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009910 tx=0x7f0d50009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/1954327394 shutdown_connections 2024-03-16T10:40:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/1954327394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64106310 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/1954327394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d641015f0 0x7f0d641039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/1954327394 >> 172.21.15.26:0/1954327394 conn(0x7f0d640fafc0 msgr2=0x7f0d640fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/1954327394 shutdown_connections 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/1954327394 wait complete. 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 Processor -- start 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.079+0000 7f0d6b1e5700 1 -- start start 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d6b1e5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d641015f0 0x7f0d64196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d6b1e5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d6b1e5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d641970b0 con 0x7f0d641015f0 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d6b1e5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d641971f0 con 0x7f0d64103f20 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d63fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d63fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37224/0 (socket says 172.21.15.26:37224) 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d63fff700 1 -- 172.21.15.26:0/4141878324 learned_addr learned my addr 172.21.15.26:0/4141878324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d68f81700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d641015f0 0x7f0d64196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d63fff700 1 -- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d641015f0 msgr2=0x7f0d64196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d63fff700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d641015f0 0x7f0d64196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d63fff700 1 -- 172.21.15.26:0/4141878324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500095f0 con 0x7f0d64103f20 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d68f81700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d641015f0 0x7f0d64196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d63fff700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64196a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006010 tx=0x7f0d5000be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d61ffb700 1 -- 172.21.15.26:0/4141878324 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d5001d070 con 0x7f0d64103f20 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6419bbf0 con 0x7f0d64103f20 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d61ffb700 1 -- 172.21.15.26:0/4141878324 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d50005dc0 con 0x7f0d64103f20 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d61ffb700 1 -- 172.21.15.26:0/4141878324 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d50021640 con 0x7f0d64103f20 2024-03-16T10:40:12.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.080+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6419c0e0 con 0x7f0d64103f20 2024-03-16T10:40:12.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.081+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6404ea80 con 0x7f0d64103f20 2024-03-16T10:40:12.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.082+0000 7f0d61ffb700 1 -- 172.21.15.26:0/4141878324 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f0d5000f880 con 0x7f0d64103f20 2024-03-16T10:40:12.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.082+0000 7f0d61ffb700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f0d5406c890 0x7f0d5406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:12.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.082+0000 7f0d61ffb700 1 -- 172.21.15.26:0/4141878324 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f0d5008c9e0 con 0x7f0d64103f20 2024-03-16T10:40:12.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.083+0000 7f0d68f81700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f0d5406c890 0x7f0d5406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:12.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.083+0000 7f0d68f81700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f0d5406c890 0x7f0d5406ed50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d58005950 tx=0x7f0d580058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:12.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.087+0000 7f0d61ffb700 1 -- 172.21.15.26:0/4141878324 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0d5005af70 con 0x7f0d64103f20 2024-03-16T10:40:12.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:12.303+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d64105eb0 con 0x7f0d5406c890 2024-03-16T10:40:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:12 smithi002 ceph-mon[29248]: Detected new or changed devices on smithi026 2024-03-16T10:40:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:12 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi026 to 7458M 2024-03-16T10:40:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:12.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:12 smithi026 ceph-mon[37041]: Detected new or changed devices on smithi026 2024-03-16T10:40:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:12 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi026 to 7458M 2024-03-16T10:40:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:12.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:13 smithi002 ceph-mon[29248]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:13 smithi002 ceph-mon[29248]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:40:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:13 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:40:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:13 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:40:13.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:13 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:13.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:13 smithi026 ceph-mon[37041]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:13.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:13 smithi026 ceph-mon[37041]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:40:13.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:13 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T10:40:13.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:13 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T10:40:13.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:13 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:15 smithi002 ceph-mon[29248]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:15 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/2487532314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b16f392a-1d2f-47b5-b356-cab94bc834cf"}]: dispatch 2024-03-16T10:40:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:15 smithi002 ceph-mon[29248]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b16f392a-1d2f-47b5-b356-cab94bc834cf"}]: dispatch 2024-03-16T10:40:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:15 smithi002 ceph-mon[29248]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b16f392a-1d2f-47b5-b356-cab94bc834cf"}]': finished 2024-03-16T10:40:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:15 smithi002 ceph-mon[29248]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T10:40:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:15.606 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:15 smithi026 ceph-mon[37041]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:15.606 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:15 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/2487532314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b16f392a-1d2f-47b5-b356-cab94bc834cf"}]: dispatch 2024-03-16T10:40:15.606 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:15 smithi026 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b16f392a-1d2f-47b5-b356-cab94bc834cf"}]: dispatch 2024-03-16T10:40:15.607 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:15 smithi026 ceph-mon[37041]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b16f392a-1d2f-47b5-b356-cab94bc834cf"}]': finished 2024-03-16T10:40:15.607 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:15 smithi026 ceph-mon[37041]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T10:40:15.607 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:15.607 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:16.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:16 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/3914127031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:40:16.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:16 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/3914127031' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T10:40:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:17 smithi026 ceph-mon[37041]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:17.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:17 smithi002 ceph-mon[29248]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:19.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:19 smithi026 ceph-mon[37041]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:19.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:19 smithi002 ceph-mon[29248]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:21.507 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:21 smithi026 ceph-mon[37041]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:21.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:21 smithi002 ceph-mon[29248]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:22.477 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:40:22.478 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:22 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:22.478 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:22 smithi026 ceph-mon[37041]: Deploying daemon osd.5 on smithi026 2024-03-16T10:40:22.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:40:22.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:22 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:22.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:22 smithi002 ceph-mon[29248]: Deploying daemon osd.5 on smithi026 2024-03-16T10:40:23.598 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:23 smithi026 ceph-mon[37041]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:23.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:23 smithi002 ceph-mon[29248]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:25.584 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:25 smithi026 ceph-mon[37041]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:25.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:25 smithi002 ceph-mon[29248]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:27.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:27 smithi026 ceph-mon[37041]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:27.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:27 smithi002 ceph-mon[29248]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:28.923 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:28 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:28.923 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:28 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:28.924 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:28 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:28 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:29.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:28 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:29.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:28 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:29.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:29 smithi002 ceph-mon[29248]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:29.893 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:29 smithi026 ceph-mon[37041]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:30.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:30 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:30 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:31.674 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 5 on host 'smithi026' 2024-03-16T10:40:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.673+0000 7f0d61ffb700 1 -- 172.21.15.26:0/4141878324 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0d64105eb0 con 0x7f0d5406c890 2024-03-16T10:40:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.675+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f0d5406c890 msgr2=0x7f0d5406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.675+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f0d5406c890 0x7f0d5406ed50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d58005950 tx=0x7f0d580058e0 comp rx=0 tx=0).stop 2024-03-16T10:40:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 msgr2=0x7f0d64196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64196a90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d50006010 tx=0x7f0d5000be20 comp rx=0 tx=0).stop 2024-03-16T10:40:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 shutdown_connections 2024-03-16T10:40:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f0d5406c890 0x7f0d5406ed50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d641015f0 0x7f0d64196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 --2- 172.21.15.26:0/4141878324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d64103f20 0x7f0d64196a90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 >> 172.21.15.26:0/4141878324 conn(0x7f0d640fafc0 msgr2=0x7f0d640fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 shutdown_connections 2024-03-16T10:40:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:31.676+0000 7f0d6b1e5700 1 -- 172.21.15.26:0/4141878324 wait complete. 2024-03-16T10:40:31.938 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:31 smithi026 ceph-mon[37041]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:31.939 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:31 smithi026 ceph-mon[37041]: from='osd.5 [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:40:31.939 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:31 smithi026 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:40:32.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:31 smithi002 ceph-mon[29248]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:32.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:31 smithi002 ceph-mon[29248]: from='osd.5 [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:40:32.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:31 smithi002 ceph-mon[29248]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:40:32.347 DEBUG:teuthology.orchestra.run.smithi026:osd.5> sudo journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.5.service 2024-03-16T10:40:32.349 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-16T10:40:32.349 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-16T10:40:32.575 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:32.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='osd.5 [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:32.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: osdmap e38: 6 total, 5 up, 6 in 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='osd.5 [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:33.482 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:40:33 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[56653]: 2024-03-16T10:40:33.274+0000 7fe99d585700 -1 osd.5 0 waiting for initial osdmap 2024-03-16T10:40:33.482 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:40:33 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[56653]: 2024-03-16T10:40:33.283+0000 7fe996b75700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:40:33.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:33 smithi026 ceph-mon[37041]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:33.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:33 smithi026 ceph-mon[37041]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-03-16T10:40:33.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:33 smithi026 ceph-mon[37041]: osdmap e39: 6 total, 5 up, 6 in 2024-03-16T10:40:33.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:33.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:33.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:33 smithi002 ceph-mon[29248]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:33.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:33 smithi002 ceph-mon[29248]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-03-16T10:40:33.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:33 smithi002 ceph-mon[29248]: osdmap e39: 6 total, 5 up, 6 in 2024-03-16T10:40:33.901 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:33.902 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:34.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 -- 172.21.15.2:0/3974849196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b0100be0 msgr2=0x7f62b0101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:34.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/3974849196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b0100be0 0x7f62b0101040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6298009910 tx=0x7f6298009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:34.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 -- 172.21.15.2:0/3974849196 shutdown_connections 2024-03-16T10:40:34.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/3974849196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b0100be0 0x7f62b0101040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:34.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/3974849196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62b00ff9e0 0x7f62b00ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:34.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 -- 172.21.15.2:0/3974849196 >> 172.21.15.2:0/3974849196 conn(0x7f62b00faf80 msgr2=0x7f62b00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:34.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 -- 172.21.15.2:0/3974849196 shutdown_connections 2024-03-16T10:40:34.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.434+0000 7f62b4f4a700 1 -- 172.21.15.2:0/3974849196 wait complete. 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62b4f4a700 1 Processor -- start 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62b4f4a700 1 -- start start 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62b4f4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b00ff9e0 0x7f62b0103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62b4f4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62b0100be0 0x7f62b0102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62b4f4a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0102850 con 0x7f62b00ff9e0 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62b4f4a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0102990 con 0x7f62b0100be0 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62ae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b00ff9e0 0x7f62b0103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62ae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b00ff9e0 0x7f62b0103c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55788/0 (socket says 172.21.15.2:55788) 2024-03-16T10:40:34.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.435+0000 7f62ae59c700 1 -- 172.21.15.2:0/4003329483 learned_addr learned my addr 172.21.15.2:0/4003329483 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62add9b700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62b0100be0 0x7f62b0102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62ae59c700 1 -- 172.21.15.2:0/4003329483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62b0100be0 msgr2=0x7f62b0102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62ae59c700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62b0100be0 0x7f62b0102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62ae59c700 1 -- 172.21.15.2:0/4003329483 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980095f0 con 0x7f62b00ff9e0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62ae59c700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b00ff9e0 0x7f62b0103c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c8f0 tx=0x7f62a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62a77fe700 1 -- 172.21.15.2:0/4003329483 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a0007a70 con 0x7f62b00ff9e0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b0102c70 con 0x7f62b00ff9e0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62a77fe700 1 -- 172.21.15.2:0/4003329483 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62a0007bd0 con 0x7f62b00ff9e0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62a77fe700 1 -- 172.21.15.2:0/4003329483 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a0018630 con 0x7f62b00ff9e0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.436+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b019e550 con 0x7f62b00ff9e0 2024-03-16T10:40:34.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.437+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b004ea80 con 0x7f62b00ff9e0 2024-03-16T10:40:34.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.437+0000 7f62a77fe700 1 -- 172.21.15.2:0/4003329483 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f62a000f450 con 0x7f62b00ff9e0 2024-03-16T10:40:34.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.438+0000 7f62a77fe700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f629c06c880 0x7f629c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:34.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.438+0000 7f62a77fe700 1 -- 172.21.15.2:0/4003329483 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f62a008b980 con 0x7f62b00ff9e0 2024-03-16T10:40:34.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.438+0000 7f62add9b700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f629c06c880 0x7f629c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:34.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.438+0000 7f62add9b700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f629c06c880 0x7f629c06ed40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6298006010 tx=0x7f629800b580 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:34.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.443+0000 7f62a77fe700 1 -- 172.21.15.2:0/4003329483 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62a0059e10 con 0x7f62b00ff9e0 2024-03-16T10:40:34.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.647+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f62b019e6e0 con 0x7f62b00ff9e0 2024-03-16T10:40:34.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.647+0000 7f62a77fe700 1 -- 172.21.15.2:0/4003329483 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f62a00599a0 con 0x7f62b00ff9e0 2024-03-16T10:40:34.648 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:40:34.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f629c06c880 msgr2=0x7f629c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f629c06c880 0x7f629c06ed40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6298006010 tx=0x7f629800b580 comp rx=0 tx=0).stop 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b00ff9e0 msgr2=0x7f62b0103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b00ff9e0 0x7f62b0103c20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c8f0 tx=0x7f62a000ccb0 comp rx=0 tx=0).stop 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 shutdown_connections 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f629c06c880 0x7f629c06ed40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62b00ff9e0 0x7f62b0103c20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 --2- 172.21.15.2:0/4003329483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62b0100be0 0x7f62b0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.649+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 >> 172.21.15.2:0/4003329483 conn(0x7f62b00faf80 msgr2=0x7f62b0068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.650+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 shutdown_connections 2024-03-16T10:40:34.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:34.650+0000 7f62b4f4a700 1 -- 172.21.15.2:0/4003329483 wait complete. 2024-03-16T10:40:35.392 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1710585634,"num_in_osds":6,"osd_in_since":1710585614,"num_remapped_pgs":0} 2024-03-16T10:40:35.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T10:40:35.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:35 smithi002 ceph-mon[29248]: purged_snaps scrub starts 2024-03-16T10:40:35.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:35 smithi002 ceph-mon[29248]: purged_snaps scrub ok 2024-03-16T10:40:35.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:35 smithi002 ceph-mon[29248]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:35.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:35 smithi002 ceph-mon[29248]: osd.5 [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] boot 2024-03-16T10:40:35.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:35 smithi002 ceph-mon[29248]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T10:40:35.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:35 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:35.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:35 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/4003329483' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T10:40:35.590 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:35 smithi026 ceph-mon[37041]: purged_snaps scrub starts 2024-03-16T10:40:35.590 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:35 smithi026 ceph-mon[37041]: purged_snaps scrub ok 2024-03-16T10:40:35.590 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:35 smithi026 ceph-mon[37041]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-16T10:40:35.590 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:35 smithi026 ceph-mon[37041]: osd.5 [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] boot 2024-03-16T10:40:35.590 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:35 smithi026 ceph-mon[37041]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T10:40:35.590 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:35 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:40:35.591 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:35 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/4003329483' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T10:40:35.610 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:36 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T10:40:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:40:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:36.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:36 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1431771542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b41038a0 msgr2=0x7fd4b41078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1431771542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b41038a0 0x7fd4b41078f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009910 tx=0x7fd4a4009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1431771542 shutdown_connections 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1431771542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b41038a0 0x7fd4b41078f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1431771542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b4102ef0 0x7fd4b41032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1431771542 >> 172.21.15.2:0/1431771542 conn(0x7fd4b40fe780 msgr2=0x7fd4b4100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1431771542 shutdown_connections 2024-03-16T10:40:37.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.264+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1431771542 wait complete. 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4bc3e3700 1 Processor -- start 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4bc3e3700 1 -- start start 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4bc3e3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b4102ef0 0x7fd4b419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4bc3e3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b41038a0 0x7fd4b419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4bc3e3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4198ab0 con 0x7fd4b4102ef0 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4bc3e3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4198c20 con 0x7fd4b41038a0 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4ba17f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b4102ef0 0x7fd4b419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4ba17f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b4102ef0 0x7fd4b419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55802/0 (socket says 172.21.15.2:55802) 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4ba17f700 1 -- 172.21.15.2:0/1651323918 learned_addr learned my addr 172.21.15.2:0/1651323918 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.265+0000 7fd4b997e700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b41038a0 0x7fd4b419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4ba17f700 1 -- 172.21.15.2:0/1651323918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b41038a0 msgr2=0x7fd4b419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4ba17f700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b41038a0 0x7fd4b419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4ba17f700 1 -- 172.21.15.2:0/1651323918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a40095f0 con 0x7fd4b4102ef0 2024-03-16T10:40:37.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4ba17f700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b4102ef0 0x7fd4b419ea00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000d8d0 tx=0x7fd4b000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:37.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4ab7fe700 1 -- 172.21.15.2:0/1651323918 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4b00098a0 con 0x7fd4b4102ef0 2024-03-16T10:40:37.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4ab7fe700 1 -- 172.21.15.2:0/1651323918 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4b0010460 con 0x7fd4b4102ef0 2024-03-16T10:40:37.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b4198f00 con 0x7fd4b4102ef0 2024-03-16T10:40:37.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4ab7fe700 1 -- 172.21.15.2:0/1651323918 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4b000f5f0 con 0x7fd4b4102ef0 2024-03-16T10:40:37.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.266+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b4199420 con 0x7fd4b4102ef0 2024-03-16T10:40:37.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.267+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b410b2f0 con 0x7fd4b4102ef0 2024-03-16T10:40:37.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.267+0000 7fd4ab7fe700 1 -- 172.21.15.2:0/1651323918 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fd4b0009a00 con 0x7fd4b4102ef0 2024-03-16T10:40:37.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.268+0000 7fd4ab7fe700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd4a006c880 0x7fd4a006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:37.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.268+0000 7fd4ab7fe700 1 -- 172.21.15.2:0/1651323918 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd4b008b810 con 0x7fd4b4102ef0 2024-03-16T10:40:37.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.268+0000 7fd4b997e700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd4a006c880 0x7fd4a006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:37.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.268+0000 7fd4b997e700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd4a006c880 0x7fd4a006ed40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd4b419a2b0 tx=0x7fd4a401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:37.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.272+0000 7fd4ab7fe700 1 -- 172.21.15.2:0/1651323918 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd4b0056160 con 0x7fd4b4102ef0 2024-03-16T10:40:37.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.477+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4b404eab0 con 0x7fd4b4102ef0 2024-03-16T10:40:37.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.478+0000 7fd4ab7fe700 1 -- 172.21.15.2:0/1651323918 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11226 (secure 0 0 0) 0x7fd4b0059780 con 0x7fd4b4102ef0 2024-03-16T10:40:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:40:37.479 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","created":"2024-03-16T10:34:41.302807+0000","modified":"2024-03-16T10:40:35.271642+0000","last_up_change":"2024-03-16T10:40:34.269636+0000","last_in_change":"2024-03-16T10:40:14.935894+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:39:18.453991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4befd156-b58c-44e4-bc1e-f16112b3df27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6803","nonce":702078877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6805","nonce":702078877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6809","nonce":702078877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6807","nonce":702078877}]},"public_addr":"172.21.15.2:6803/702078877","cluster_addr":"172.21.15.2:6805/702078877","heartbeat_back_addr":"172.21.15.2:6809/702078877","heartbeat_front_addr":"172.21.15.2:6807/702078877","state":["exists","up"]},{"osd":1,"uuid":"95448bdc-721b-45c0-8c58-0b416e188aa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6811","nonce":1466336624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6813","nonce":1466336624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6817","nonce":1466336624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6815","nonce":1466336624}]},"public_addr":"172.21.15.2:6811/1466336624","cluster_addr":"172.21.15.2:6813/1466336624","heartbeat_back_addr":"172.21.15.2:6817/1466336624","heartbeat_front_addr":"172.21.15.2:6815/1466336624","state":["exists","up"]},{"osd":2,"uuid":"ce6a7a3f-db89-426c-bbb4-1339c5adac81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6819","nonce":869328850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6821","nonce":869328850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6825","nonce":869328850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6823","nonce":869328850}]},"public_addr":"172.21.15.2:6819/869328850","cluster_addr":"172.21.15.2:6821/869328850","heartbeat_back_addr":"172.21.15.2:6825/869328850","heartbeat_front_addr":"172.21.15.2:6823/869328850","state":["exists","up"]},{"osd":3,"uuid":"8d760311-5fda-425b-ac3c-5045ecb1cbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6801","nonce":4202679460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6803","nonce":4202679460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6807","nonce":4202679460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6805","nonce":4202679460}]},"public_addr":"172.21.15.26:6801/4202679460","cluster_addr":"172.21.15.26:6803/4202679460","heartbeat_back_addr":"172.21.15.26:6807/4202679460","heartbeat_front_addr":"172.21.15.26:6805/4202679460","state":["exists","up"]},{"osd":4,"uuid":"250a3093-6818-4f4f-85d5-d1f2e4381afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6809","nonce":412631433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6811","nonce":412631433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6815","nonce":412631433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6813","nonce":412631433}]},"public_addr":"172.21.15.26:6809/412631433","cluster_addr":"172.21.15.26:6811/412631433","heartbeat_back_addr":"172.21.15.26:6815/412631433","heartbeat_front_addr":"172.21.15.26:6813/412631433","state":["exists","up"]},{"osd":5,"uuid":"b16f392a-1d2f-47b5-b356-cab94bc834cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6817","nonce":3448963017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6819","nonce":3448963017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6823","nonce":3448963017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6821","nonce":3448963017}]},"public_addr":"172.21.15.26:6817/3448963017","cluster_addr":"172.21.15.26:6819/3448963017","heartbeat_back_addr":"172.21.15.26:6823/3448963017","heartbeat_front_addr":"172.21.15.26:6821/3448963017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:18.975873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:47.618973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:15.584615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:39.921213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:06.286712+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:32.567112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/1971862470":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6801/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/2348229359":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6801/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1144469616":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3462160730":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/1789736475":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/1498322259":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/2565089320":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3421178356":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6801/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/2322904669":"2024-03-17T10:35:17.551709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:40:37.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.479+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd4a006c880 msgr2=0x7fd4a006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.479+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd4a006c880 0x7fd4a006ed40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd4b419a2b0 tx=0x7fd4a401a040 comp rx=0 tx=0).stop 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b4102ef0 msgr2=0x7fd4b419ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b4102ef0 0x7fd4b419ea00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000d8d0 tx=0x7fd4b000dbe0 comp rx=0 tx=0).stop 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 shutdown_connections 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd4a006c880 0x7fd4a006ed40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4b4102ef0 0x7fd4b419ea00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 --2- 172.21.15.2:0/1651323918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4b41038a0 0x7fd4b419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 >> 172.21.15.2:0/1651323918 conn(0x7fd4b40fe780 msgr2=0x7fd4b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 shutdown_connections 2024-03-16T10:40:37.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:37.480+0000 7fd4bc3e3700 1 -- 172.21.15.2:0/1651323918 wait complete. 2024-03-16T10:40:37.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:37 smithi002 ceph-mon[29248]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:37.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:37 smithi002 ceph-mon[29248]: Detected new or changed devices on smithi026 2024-03-16T10:40:37.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:37 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi026 to 4972M 2024-03-16T10:40:37.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:37 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:37.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:37 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:37.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:37 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:37.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:37 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:37.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:37 smithi026 ceph-mon[37041]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:37.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:37 smithi026 ceph-mon[37041]: Detected new or changed devices on smithi026 2024-03-16T10:40:37.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:37 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi026 to 4972M 2024-03-16T10:40:37.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:37 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:40:37.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:37 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:40:37.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:37 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:40:37.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:37 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:40:38.217 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:39:18.453991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:40:38.218 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-16T10:40:38.433 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:38.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:38 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1651323918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:40:38.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:38 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1651323918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:40:39.530 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:39 smithi002 ceph-mon[29248]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:39.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:39 smithi026 ceph-mon[37041]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:40.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 -- 172.21.15.2:0/2074971582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 msgr2=0x7faaec106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:40.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/2074971582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec106e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faadc009910 tx=0x7faadc009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:40.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 -- 172.21.15.2:0/2074971582 shutdown_connections 2024-03-16T10:40:40.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/2074971582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec100a70 0x7faaec100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:40.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/2074971582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec106e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:40.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 -- 172.21.15.2:0/2074971582 >> 172.21.15.2:0/2074971582 conn(0x7faaec0fc5a0 msgr2=0x7faaec0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:40.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 -- 172.21.15.2:0/2074971582 shutdown_connections 2024-03-16T10:40:40.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.080+0000 7faaf0e1f700 1 -- 172.21.15.2:0/2074971582 wait complete. 2024-03-16T10:40:40.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faaf0e1f700 1 Processor -- start 2024-03-16T10:40:40.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faaf0e1f700 1 -- start start 2024-03-16T10:40:40.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faaf0e1f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec100a70 0x7faaec19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:40.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faaf0e1f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:40.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faaf0e1f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec19d330 con 0x7faaec106a70 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faaf0e1f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec196910 con 0x7faaec100a70 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faae9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faae9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55816/0 (socket says 172.21.15.2:55816) 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faae9d9b700 1 -- 172.21.15.2:0/4213792226 learned_addr learned my addr 172.21.15.2:0/4213792226 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faaea59c700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec100a70 0x7faaec19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faae9d9b700 1 -- 172.21.15.2:0/4213792226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec100a70 msgr2=0x7faaec19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faae9d9b700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec100a70 0x7faaec19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.081+0000 7faae9d9b700 1 -- 172.21.15.2:0/4213792226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0095f0 con 0x7faaec106a70 2024-03-16T10:40:40.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.082+0000 7faae9d9b700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec19cd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faae000b4b0 tx=0x7faae000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:40.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.082+0000 7faadb7fe700 1 -- 172.21.15.2:0/4213792226 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae0011800 con 0x7faaec106a70 2024-03-16T10:40:40.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.082+0000 7faadb7fe700 1 -- 172.21.15.2:0/4213792226 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faae0011e20 con 0x7faaec106a70 2024-03-16T10:40:40.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.082+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec196bf0 con 0x7faaec106a70 2024-03-16T10:40:40.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.082+0000 7faadb7fe700 1 -- 172.21.15.2:0/4213792226 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae000f2e0 con 0x7faaec106a70 2024-03-16T10:40:40.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.082+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec197140 con 0x7faaec106a70 2024-03-16T10:40:40.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.083+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec109070 con 0x7faaec106a70 2024-03-16T10:40:40.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.083+0000 7faadb7fe700 1 -- 172.21.15.2:0/4213792226 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7faae0010380 con 0x7faaec106a70 2024-03-16T10:40:40.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.084+0000 7faadb7fe700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faad406c880 0x7faad406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:40.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.084+0000 7faadb7fe700 1 -- 172.21.15.2:0/4213792226 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7faae008b510 con 0x7faaec106a70 2024-03-16T10:40:40.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.084+0000 7faaea59c700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faad406c880 0x7faad406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:40.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.084+0000 7faaea59c700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faad406c880 0x7faad406ed40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faadc00b5c0 tx=0x7faadc01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:40.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.088+0000 7faadb7fe700 1 -- 172.21.15.2:0/4213792226 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faae00598f0 con 0x7faaec106a70 2024-03-16T10:40:40.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.287+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faaec197dd0 con 0x7faaec106a70 2024-03-16T10:40:40.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.288+0000 7faadb7fe700 1 -- 172.21.15.2:0/4213792226 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7faae0059480 con 0x7faaec106a70 2024-03-16T10:40:40.289 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faad406c880 msgr2=0x7faad406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faad406c880 0x7faad406ed40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faadc00b5c0 tx=0x7faadc01a040 comp rx=0 tx=0).stop 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 msgr2=0x7faaec19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec19cd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faae000b4b0 tx=0x7faae000b870 comp rx=0 tx=0).stop 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 shutdown_connections 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faad406c880 0x7faad406ed40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaec100a70 0x7faaec19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.290+0000 7faaf0e1f700 1 --2- 172.21.15.2:0/4213792226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec106a70 0x7faaec19cd10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.291+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 >> 172.21.15.2:0/4213792226 conn(0x7faaec0fc5a0 msgr2=0x7faaec0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.291+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 shutdown_connections 2024-03-16T10:40:40.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:40.291+0000 7faaf0e1f700 1 -- 172.21.15.2:0/4213792226 wait complete. 2024-03-16T10:40:41.072 INFO:tasks.cephadm:Setting up client nodes... 2024-03-16T10:40:41.072 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-16T10:40:41.288 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:41 smithi002 ceph-mon[29248]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:41 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/4213792226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:40:41.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:41 smithi026 ceph-mon[37041]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:41.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:41 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/4213792226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:40:42.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 -- 172.21.15.2:0/3767840689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 msgr2=0x7ffa54101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:42.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/3767840689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54101050 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009910 tx=0x7ffa40009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:42.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 -- 172.21.15.2:0/3767840689 shutdown_connections 2024-03-16T10:40:42.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/3767840689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54101050 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:42.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/3767840689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa540ff9f0 0x7ffa540ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:42.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 -- 172.21.15.2:0/3767840689 >> 172.21.15.2:0/3767840689 conn(0x7ffa540fafb0 msgr2=0x7ffa540fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:42.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 -- 172.21.15.2:0/3767840689 shutdown_connections 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.901+0000 7ffa5ac07700 1 -- 172.21.15.2:0/3767840689 wait complete. 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa5ac07700 1 Processor -- start 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa5ac07700 1 -- start start 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa5ac07700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa540ff9f0 0x7ffa54196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa5ac07700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa5ac07700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541970e0 con 0x7ffa54100bf0 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa5ac07700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54197220 con 0x7ffa540ff9f0 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa53fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:42.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa53fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33728/0 (socket says 172.21.15.2:33728) 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa53fff700 1 -- 172.21.15.2:0/1030239062 learned_addr learned my addr 172.21.15.2:0/1030239062 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa589a3700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa540ff9f0 0x7ffa54196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa53fff700 1 -- 172.21.15.2:0/1030239062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa540ff9f0 msgr2=0x7ffa54196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa53fff700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa540ff9f0 0x7ffa54196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.902+0000 7ffa53fff700 1 -- 172.21.15.2:0/1030239062 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa54100bf0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.903+0000 7ffa53fff700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54196ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000bd70 tx=0x7ffa4000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.903+0000 7ffa51ffb700 1 -- 172.21.15.2:0/1030239062 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa4001d070 con 0x7ffa54100bf0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.903+0000 7ffa51ffb700 1 -- 172.21.15.2:0/1030239062 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa4000f460 con 0x7ffa54100bf0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.903+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5419bc20 con 0x7ffa54100bf0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.903+0000 7ffa51ffb700 1 -- 172.21.15.2:0/1030239062 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa40017630 con 0x7ffa54100bf0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.903+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5419c190 con 0x7ffa54100bf0 2024-03-16T10:40:42.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.904+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5404ea80 con 0x7ffa54100bf0 2024-03-16T10:40:42.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.904+0000 7ffa51ffb700 1 -- 172.21.15.2:0/1030239062 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7ffa4000f5d0 con 0x7ffa54100bf0 2024-03-16T10:40:42.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.905+0000 7ffa51ffb700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ffa4406c880 0x7ffa4406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:42.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.905+0000 7ffa51ffb700 1 -- 172.21.15.2:0/1030239062 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ffa4008cfe0 con 0x7ffa54100bf0 2024-03-16T10:40:42.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.905+0000 7ffa589a3700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ffa4406c880 0x7ffa4406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:42.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.905+0000 7ffa589a3700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ffa4406c880 0x7ffa4406ed40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa54100a50 tx=0x7ffa48005cf0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:42.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:42.909+0000 7ffa51ffb700 1 -- 172.21.15.2:0/1030239062 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffa4005b340 con 0x7ffa54100bf0 2024-03-16T10:40:43.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.193+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ffa5419cab0 con 0x7ffa54100bf0 2024-03-16T10:40:43.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.197+0000 7ffa51ffb700 1 -- 172.21.15.2:0/1030239062 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7ffa4005aed0 con 0x7ffa54100bf0 2024-03-16T10:40:43.198 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-03-16T10:40:43.198 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQArd/VlaKecCxAAWnzf4w2WeM86YDtIf9B9AQ== 2024-03-16T10:40:43.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ffa4406c880 msgr2=0x7ffa4406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:43.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ffa4406c880 0x7ffa4406ed40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa54100a50 tx=0x7ffa48005cf0 comp rx=0 tx=0).stop 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 msgr2=0x7ffa54196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54196ac0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000bd70 tx=0x7ffa4000be50 comp rx=0 tx=0).stop 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 shutdown_connections 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ffa4406c880 0x7ffa4406ed40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa540ff9f0 0x7ffa54196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 --2- 172.21.15.2:0/1030239062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa54100bf0 0x7ffa54196ac0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 >> 172.21.15.2:0/1030239062 conn(0x7ffa540fafb0 msgr2=0x7ffa54069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 shutdown_connections 2024-03-16T10:40:43.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:43.199+0000 7ffa5ac07700 1 -- 172.21.15.2:0/1030239062 wait complete. 2024-03-16T10:40:43.502 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:43 smithi002 ceph-mon[29248]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:43.502 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:43 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1030239062' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:40:43.502 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:43 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1030239062' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T10:40:43.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:43 smithi026 ceph-mon[37041]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:43 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1030239062' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:40:43.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:43 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1030239062' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T10:40:43.816 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:40:43.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-16T10:40:43.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T10:40:43.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-16T10:40:44.079 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi026/config 2024-03-16T10:40:45.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:45 smithi026 ceph-mon[37041]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:45.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:45.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:45 smithi002 ceph-mon[29248]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:45.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:40:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 -- 172.21.15.26:0/1519132339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 msgr2=0x7f3638102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 --2- 172.21.15.26:0/1519132339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f3638102500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3634009910 tx=0x7f3634009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 -- 172.21.15.26:0/1519132339 shutdown_connections 2024-03-16T10:40:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 --2- 172.21.15.26:0/1519132339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36380feb60 0x7f36380fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 --2- 172.21.15.26:0/1519132339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f3638102500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 -- 172.21.15.26:0/1519132339 >> 172.21.15.26:0/1519132339 conn(0x7f36380fa700 msgr2=0x7f36380fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 -- 172.21.15.26:0/1519132339 shutdown_connections 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.739+0000 7f363f551700 1 -- 172.21.15.26:0/1519132339 wait complete. 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363f551700 1 Processor -- start 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363f551700 1 -- start start 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363f551700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36380feb60 0x7f3638101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363f551700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f36380ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363f551700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3638100370 con 0x7f36380feb60 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363f551700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36381004b0 con 0x7f36381020e0 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363caec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f36380ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363caec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f36380ffe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41866/0 (socket says 172.21.15.26:41866) 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363caec700 1 -- 172.21.15.26:0/2529203569 learned_addr learned my addr 172.21.15.26:0/2529203569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-03-16T10:40:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.740+0000 7f363d2ed700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36380feb60 0x7f3638101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f363caec700 1 -- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36380feb60 msgr2=0x7f3638101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f363caec700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36380feb60 0x7f3638101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f363caec700 1 -- 172.21.15.26:0/2529203569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340095f0 con 0x7f36381020e0 2024-03-16T10:40:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f363caec700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f36380ffe30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f362800b480 tx=0x7f362800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f362e7fc700 1 -- 172.21.15.26:0/2529203569 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3628011800 con 0x7f36381020e0 2024-03-16T10:40:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f362e7fc700 1 -- 172.21.15.26:0/2529203569 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3628011e20 con 0x7f36381020e0 2024-03-16T10:40:45.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3638100790 con 0x7f36381020e0 2024-03-16T10:40:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f362e7fc700 1 -- 172.21.15.26:0/2529203569 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3628010ea0 con 0x7f36381020e0 2024-03-16T10:40:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.741+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363819e6c0 con 0x7f36381020e0 2024-03-16T10:40:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.742+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3638190720 con 0x7f36381020e0 2024-03-16T10:40:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.743+0000 7f362e7fc700 1 -- 172.21.15.26:0/2529203569 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f3628010380 con 0x7f36381020e0 2024-03-16T10:40:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.743+0000 7f362e7fc700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f362406c880 0x7f362406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:45.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.743+0000 7f362e7fc700 1 -- 172.21.15.26:0/2529203569 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f362808b420 con 0x7f36381020e0 2024-03-16T10:40:45.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.744+0000 7f363d2ed700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f362406c880 0x7f362406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:45.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.744+0000 7f363d2ed700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f362406c880 0x7f362406ed40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f3634005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:45.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:45.748+0000 7f362e7fc700 1 -- 172.21.15.26:0/2529203569 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f36280598b0 con 0x7f36381020e0 2024-03-16T10:40:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.030+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f363802cf60 con 0x7f36381020e0 2024-03-16T10:40:46.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.035+0000 7f362e7fc700 1 -- 172.21.15.26:0/2529203569 <== mon.1 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f3628059440 con 0x7f36381020e0 2024-03-16T10:40:46.036 INFO:teuthology.orchestra.run.smithi026.stdout:[client.1] 2024-03-16T10:40:46.036 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAud/Vl8W3wARAAiofX5n7bxQEQeNFWfe6StQ== 2024-03-16T10:40:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f362406c880 msgr2=0x7f362406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f362406c880 0x7f362406ed40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3634000c00 tx=0x7f3634005960 comp rx=0 tx=0).stop 2024-03-16T10:40:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 msgr2=0x7f36380ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f36380ffe30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f362800b480 tx=0x7f362800b790 comp rx=0 tx=0).stop 2024-03-16T10:40:46.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 shutdown_connections 2024-03-16T10:40:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f362406c880 0x7f362406ed40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f36380feb60 0x7f3638101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 --2- 172.21.15.26:0/2529203569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36381020e0 0x7f36380ffe30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 >> 172.21.15.26:0/2529203569 conn(0x7f36380fa700 msgr2=0x7f36381054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 shutdown_connections 2024-03-16T10:40:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:40:46.037+0000 7f363f551700 1 -- 172.21.15.26:0/2529203569 wait complete. 2024-03-16T10:40:46.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:46 smithi002 ceph-mon[29248]: from='client.? 172.21.15.26:0/2529203569' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:40:46.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:46 smithi002 ceph-mon[29248]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:40:46.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:46 smithi002 ceph-mon[29248]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T10:40:46.685 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:46 smithi026 ceph-mon[37041]: from='client.? 172.21.15.26:0/2529203569' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:40:46.686 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:46 smithi026 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T10:40:46.686 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:46 smithi026 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T10:40:46.706 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:40:46.706 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-16T10:40:46.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-16T10:40:46.749 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T10:40:46.749 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-16T10:40:46.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-16T10:40:46.959 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:48.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:48 smithi002 ceph-mon[29248]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:48 smithi026 ceph-mon[37041]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 -- 172.21.15.2:0/788008502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0100a70 msgr2=0x7f3ab0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 --2- 172.21.15.2:0/788008502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0100a70 0x7f3ab0100ef0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009910 tx=0x7f3aa4009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 -- 172.21.15.2:0/788008502 shutdown_connections 2024-03-16T10:40:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 --2- 172.21.15.2:0/788008502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0100a70 0x7f3ab0100ef0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 --2- 172.21.15.2:0/788008502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ab0106a70 0x7f3ab0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 -- 172.21.15.2:0/788008502 >> 172.21.15.2:0/788008502 conn(0x7f3ab00fc5a0 msgr2=0x7f3ab00fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 -- 172.21.15.2:0/788008502 shutdown_connections 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.641+0000 7f3ab7441700 1 -- 172.21.15.2:0/788008502 wait complete. 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab7441700 1 Processor -- start 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab7441700 1 -- start start 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab7441700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ab0100a70 0x7f3ab00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab7441700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0106a70 0x7f3ab00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab7441700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0073b30 con 0x7f3ab0106a70 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab7441700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0073c70 con 0x7f3ab0100a70 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab49dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0106a70 0x7f3ab00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab49dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0106a70 0x7f3ab00735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33746/0 (socket says 172.21.15.2:33746) 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab49dc700 1 -- 172.21.15.2:0/3151777945 learned_addr learned my addr 172.21.15.2:0/3151777945 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.642+0000 7f3ab51dd700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ab0100a70 0x7f3ab00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3ab49dc700 1 -- 172.21.15.2:0/3151777945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ab0100a70 msgr2=0x7f3ab00730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3ab49dc700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ab0100a70 0x7f3ab00730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3ab49dc700 1 -- 172.21.15.2:0/3151777945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40095f0 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3ab51dd700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ab0100a70 0x7f3ab00730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3ab49dc700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0106a70 0x7f3ab00735f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400bd70 tx=0x7f3aa400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3aae7fc700 1 -- 172.21.15.2:0/3151777945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa401d070 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3aae7fc700 1 -- 172.21.15.2:0/3151777945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3aa400f460 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab0073ef0 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3aae7fc700 1 -- 172.21.15.2:0/3151777945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa4017630 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.643+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab01a2ef0 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.644+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab0109090 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.644+0000 7f3aae7fc700 1 -- 172.21.15.2:0/3151777945 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f3aa400f5d0 con 0x7f3ab0106a70 2024-03-16T10:40:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.645+0000 7f3aae7fc700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a9806c880 0x7f3a9806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.645+0000 7f3aae7fc700 1 -- 172.21.15.2:0/3151777945 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3aa408ce90 con 0x7f3ab0106a70 2024-03-16T10:40:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.645+0000 7f3ab51dd700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a9806c880 0x7f3a9806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.645+0000 7f3ab51dd700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a9806c880 0x7f3a9806ed40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0007eb0 tx=0x7f3aa00073c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.650+0000 7f3aae7fc700 1 -- 172.21.15.2:0/3151777945 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3aa40577e0 con 0x7f3ab0106a70 2024-03-16T10:40:48.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.917+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3ab0108a70 con 0x7f3ab0106a70 2024-03-16T10:40:48.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.921+0000 7f3aae7fc700 1 -- 172.21.15.2:0/3151777945 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173455 (secure 0 0 0) 0x7f3aa405ae00 con 0x7f3ab0106a70 2024-03-16T10:40:48.924 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:40:48.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a9806c880 msgr2=0x7f3a9806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:48.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a9806c880 0x7f3a9806ed40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0007eb0 tx=0x7f3aa00073c0 comp rx=0 tx=0).stop 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0106a70 msgr2=0x7f3ab00735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0106a70 0x7f3ab00735f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400bd70 tx=0x7f3aa400be50 comp rx=0 tx=0).stop 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 shutdown_connections 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a9806c880 0x7f3a9806ed40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ab0100a70 0x7f3ab00730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 --2- 172.21.15.2:0/3151777945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ab0106a70 0x7f3ab00735f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.923+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 >> 172.21.15.2:0/3151777945 conn(0x7f3ab00fc5a0 msgr2=0x7f3ab00fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.924+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 shutdown_connections 2024-03-16T10:40:48.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:48.924+0000 7f3ab7441700 1 -- 172.21.15.2:0/3151777945 wait complete. 2024-03-16T10:40:49.216 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:49 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3151777945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T10:40:49.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:49 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3151777945' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T10:40:49.538 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi002.zevrtm","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3195213211},{"type":"v1","addr":"172.21.15.2:6801","nonce":3195213211}]},"active_addr":"172.21.15.2:6801/3195213211","active_change":"2024-03-16T10:37:00.118191+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14236,"name":"smithi026.omjkpo","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.2:8443/","prometheus":"http://172.21.15.2:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":4201560868}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3514263595}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3582566838}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":401513378}]}]} 2024-03-16T10:40:49.541 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-16T10:40:49.541 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-16T10:40:49.541 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T10:40:49.752 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:50.186 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:50 smithi002 ceph-mon[29248]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:50 smithi026 ceph-mon[37041]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:51.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.302+0000 7f93709fd700 1 -- 172.21.15.2:0/4292552400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c0ff9a0 msgr2=0x7f936c0ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.302+0000 7f93709fd700 1 --2- 172.21.15.2:0/4292552400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c0ff9a0 0x7f936c0ffe20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9360009910 tx=0x7f9360009c20 comp rx=0 tx=0).stop 2024-03-16T10:40:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 -- 172.21.15.2:0/4292552400 shutdown_connections 2024-03-16T10:40:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 --2- 172.21.15.2:0/4292552400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c0ff9a0 0x7f936c0ffe20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 --2- 172.21.15.2:0/4292552400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f936c1059a0 0x7f936c105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 -- 172.21.15.2:0/4292552400 >> 172.21.15.2:0/4292552400 conn(0x7f936c0fb4b0 msgr2=0x7f936c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 -- 172.21.15.2:0/4292552400 shutdown_connections 2024-03-16T10:40:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 -- 172.21.15.2:0/4292552400 wait complete. 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 Processor -- start 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.303+0000 7f93709fd700 1 -- start start 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f93709fd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f936c0ff9a0 0x7f936c0fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f93709fd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c1059a0 0x7f936c0fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f93709fd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0fc760 con 0x7f936c1059a0 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f93709fd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0fc8a0 con 0x7f936c0ff9a0 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f9369d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c1059a0 0x7f936c0fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f9369d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c1059a0 0x7f936c0fc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33764/0 (socket says 172.21.15.2:33764) 2024-03-16T10:40:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f9369d9b700 1 -- 172.21.15.2:0/2044030540 learned_addr learned my addr 172.21.15.2:0/2044030540 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f936a59c700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f936c0ff9a0 0x7f936c0fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f9369d9b700 1 -- 172.21.15.2:0/2044030540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f936c0ff9a0 msgr2=0x7f936c0fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f9369d9b700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f936c0ff9a0 0x7f936c0fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f9369d9b700 1 -- 172.21.15.2:0/2044030540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600095f0 con 0x7f936c1059a0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f9369d9b700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c1059a0 0x7f936c0fc220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f936000bd70 tx=0x7f936000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f935b7fe700 1 -- 172.21.15.2:0/2044030540 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f936001d070 con 0x7f936c1059a0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f935b7fe700 1 -- 172.21.15.2:0/2044030540 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93600056e0 con 0x7f936c1059a0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c0fcb20 con 0x7f936c1059a0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f935b7fe700 1 -- 172.21.15.2:0/2044030540 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f936000fe40 con 0x7f936c1059a0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.304+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c1a2e90 con 0x7f936c1059a0 2024-03-16T10:40:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.305+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c107fc0 con 0x7f936c1059a0 2024-03-16T10:40:51.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.306+0000 7f935b7fe700 1 -- 172.21.15.2:0/2044030540 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f936000f680 con 0x7f936c1059a0 2024-03-16T10:40:51.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.306+0000 7f935b7fe700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f935406c820 0x7f935406ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:51.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.306+0000 7f935b7fe700 1 -- 172.21.15.2:0/2044030540 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f936008cd30 con 0x7f936c1059a0 2024-03-16T10:40:51.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.306+0000 7f936a59c700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f935406c820 0x7f935406ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:51.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.307+0000 7f936a59c700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f935406c820 0x7f935406ece0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f935c005950 tx=0x7f935c00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:51.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.311+0000 7f935b7fe700 1 -- 172.21.15.2:0/2044030540 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9360057680 con 0x7f936c1059a0 2024-03-16T10:40:51.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.513+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f936c1010e0 con 0x7f936c1059a0 2024-03-16T10:40:51.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.514+0000 7f935b7fe700 1 -- 172.21.15.2:0/2044030540 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11226 (secure 0 0 0) 0x7f936005aca0 con 0x7f936c1059a0 2024-03-16T10:40:51.515 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:40:51.515 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","created":"2024-03-16T10:34:41.302807+0000","modified":"2024-03-16T10:40:35.271642+0000","last_up_change":"2024-03-16T10:40:34.269636+0000","last_in_change":"2024-03-16T10:40:14.935894+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:39:18.453991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4befd156-b58c-44e4-bc1e-f16112b3df27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6803","nonce":702078877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6805","nonce":702078877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6809","nonce":702078877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6807","nonce":702078877}]},"public_addr":"172.21.15.2:6803/702078877","cluster_addr":"172.21.15.2:6805/702078877","heartbeat_back_addr":"172.21.15.2:6809/702078877","heartbeat_front_addr":"172.21.15.2:6807/702078877","state":["exists","up"]},{"osd":1,"uuid":"95448bdc-721b-45c0-8c58-0b416e188aa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6811","nonce":1466336624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6813","nonce":1466336624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6817","nonce":1466336624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6815","nonce":1466336624}]},"public_addr":"172.21.15.2:6811/1466336624","cluster_addr":"172.21.15.2:6813/1466336624","heartbeat_back_addr":"172.21.15.2:6817/1466336624","heartbeat_front_addr":"172.21.15.2:6815/1466336624","state":["exists","up"]},{"osd":2,"uuid":"ce6a7a3f-db89-426c-bbb4-1339c5adac81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6819","nonce":869328850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6821","nonce":869328850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6825","nonce":869328850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6823","nonce":869328850}]},"public_addr":"172.21.15.2:6819/869328850","cluster_addr":"172.21.15.2:6821/869328850","heartbeat_back_addr":"172.21.15.2:6825/869328850","heartbeat_front_addr":"172.21.15.2:6823/869328850","state":["exists","up"]},{"osd":3,"uuid":"8d760311-5fda-425b-ac3c-5045ecb1cbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6801","nonce":4202679460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6803","nonce":4202679460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6807","nonce":4202679460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6805","nonce":4202679460}]},"public_addr":"172.21.15.26:6801/4202679460","cluster_addr":"172.21.15.26:6803/4202679460","heartbeat_back_addr":"172.21.15.26:6807/4202679460","heartbeat_front_addr":"172.21.15.26:6805/4202679460","state":["exists","up"]},{"osd":4,"uuid":"250a3093-6818-4f4f-85d5-d1f2e4381afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6809","nonce":412631433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6811","nonce":412631433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6815","nonce":412631433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6813","nonce":412631433}]},"public_addr":"172.21.15.26:6809/412631433","cluster_addr":"172.21.15.26:6811/412631433","heartbeat_back_addr":"172.21.15.26:6815/412631433","heartbeat_front_addr":"172.21.15.26:6813/412631433","state":["exists","up"]},{"osd":5,"uuid":"b16f392a-1d2f-47b5-b356-cab94bc834cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6817","nonce":3448963017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6819","nonce":3448963017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6823","nonce":3448963017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6821","nonce":3448963017}]},"public_addr":"172.21.15.26:6817/3448963017","cluster_addr":"172.21.15.26:6819/3448963017","heartbeat_back_addr":"172.21.15.26:6823/3448963017","heartbeat_front_addr":"172.21.15.26:6821/3448963017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:18.975873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:47.618973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:15.584615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:39.921213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:06.286712+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:32.567112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/1971862470":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6801/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/2348229359":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6801/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1144469616":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3462160730":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/1789736475":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/1498322259":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/2565089320":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3421178356":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6801/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/2322904669":"2024-03-17T10:35:17.551709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:40:51.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f935406c820 msgr2=0x7f935406ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:51.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f935406c820 0x7f935406ece0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f935c005950 tx=0x7f935c00b410 comp rx=0 tx=0).stop 2024-03-16T10:40:51.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c1059a0 msgr2=0x7f936c0fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:51.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c1059a0 0x7f936c0fc220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f936000bd70 tx=0x7f936000bda0 comp rx=0 tx=0).stop 2024-03-16T10:40:51.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 shutdown_connections 2024-03-16T10:40:51.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f935406c820 0x7f935406ece0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:51.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f936c0ff9a0 0x7f936c0fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:51.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 --2- 172.21.15.2:0/2044030540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c1059a0 0x7f936c0fc220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:51.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.516+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 >> 172.21.15.2:0/2044030540 conn(0x7f936c0fb4b0 msgr2=0x7f936c109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:51.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.517+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 shutdown_connections 2024-03-16T10:40:51.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:51.517+0000 7f93709fd700 1 -- 172.21.15.2:0/2044030540 wait complete. 2024-03-16T10:40:52.121 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:52 smithi002 ceph-mon[29248]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:52.121 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:52 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2044030540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:40:52.144 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-16T10:40:52.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T10:40:52.353 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:52.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:52 smithi026 ceph-mon[37041]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:52.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:52 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2044030540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.059+0000 7f21bb8b8700 1 -- 172.21.15.2:0/563461047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 msgr2=0x7f21b4105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.059+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/563461047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b4105d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21a400b0a0 tx=0x7f21a400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.059+0000 7f21bb8b8700 1 -- 172.21.15.2:0/563461047 shutdown_connections 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.059+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/563461047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21b40ff9a0 0x7f21b40ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.059+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/563461047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b4105d80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.059+0000 7f21bb8b8700 1 -- 172.21.15.2:0/563461047 >> 172.21.15.2:0/563461047 conn(0x7f21b40fb4b0 msgr2=0x7f21b40fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 -- 172.21.15.2:0/563461047 shutdown_connections 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 -- 172.21.15.2:0/563461047 wait complete. 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 Processor -- start 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 -- start start 2024-03-16T10:40:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21b40ff9a0 0x7f21b40fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b40fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b40fc860 con 0x7f21b41059a0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21bb8b8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b40fc9d0 con 0x7f21b40ff9a0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21b8e53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b40fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21b8e53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b40fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34864/0 (socket says 172.21.15.2:34864) 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.060+0000 7f21b8e53700 1 -- 172.21.15.2:0/2544250909 learned_addr learned my addr 172.21.15.2:0/2544250909 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b9654700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21b40ff9a0 0x7f21b40fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b8e53700 1 -- 172.21.15.2:0/2544250909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21b40ff9a0 msgr2=0x7f21b40fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b8e53700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21b40ff9a0 0x7f21b40fbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b8e53700 1 -- 172.21.15.2:0/2544250909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4009d00 con 0x7f21b41059a0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b8e53700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b40fc320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21a800b4b0 tx=0x7f21a800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b27fc700 1 -- 172.21.15.2:0/2544250909 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21a8011800 con 0x7f21b41059a0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b27fc700 1 -- 172.21.15.2:0/2544250909 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21a8011e20 con 0x7f21b41059a0 2024-03-16T10:40:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b41a2d00 con 0x7f21b41059a0 2024-03-16T10:40:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21b27fc700 1 -- 172.21.15.2:0/2544250909 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21a800f2e0 con 0x7f21b41059a0 2024-03-16T10:40:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.061+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b41a30b0 con 0x7f21b41059a0 2024-03-16T10:40:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.062+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b4108fb0 con 0x7f21b41059a0 2024-03-16T10:40:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.063+0000 7f21b27fc700 1 -- 172.21.15.2:0/2544250909 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f21a8010380 con 0x7f21b41059a0 2024-03-16T10:40:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.063+0000 7f21b27fc700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f219c06c880 0x7f219c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.063+0000 7f21b27fc700 1 -- 172.21.15.2:0/2544250909 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f21a808b510 con 0x7f21b41059a0 2024-03-16T10:40:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.063+0000 7f21b9654700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f219c06c880 0x7f219c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.063+0000 7f21b9654700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f219c06c880 0x7f219c06ed40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21a400b830 tx=0x7f21a4006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:54.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.068+0000 7f21b27fc700 1 -- 172.21.15.2:0/2544250909 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f21a8055e60 con 0x7f21b41059a0 2024-03-16T10:40:54.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.270+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21b40fd2c0 con 0x7f21b41059a0 2024-03-16T10:40:54.272 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:54 smithi002 ceph-mon[29248]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:54.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.271+0000 7f21b27fc700 1 -- 172.21.15.2:0/2544250909 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11226 (secure 0 0 0) 0x7f21a8059480 con 0x7f21b41059a0 2024-03-16T10:40:54.272 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:40:54.272 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","created":"2024-03-16T10:34:41.302807+0000","modified":"2024-03-16T10:40:35.271642+0000","last_up_change":"2024-03-16T10:40:34.269636+0000","last_in_change":"2024-03-16T10:40:14.935894+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:39:18.453991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4befd156-b58c-44e4-bc1e-f16112b3df27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6803","nonce":702078877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6805","nonce":702078877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6809","nonce":702078877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6807","nonce":702078877}]},"public_addr":"172.21.15.2:6803/702078877","cluster_addr":"172.21.15.2:6805/702078877","heartbeat_back_addr":"172.21.15.2:6809/702078877","heartbeat_front_addr":"172.21.15.2:6807/702078877","state":["exists","up"]},{"osd":1,"uuid":"95448bdc-721b-45c0-8c58-0b416e188aa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6811","nonce":1466336624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6813","nonce":1466336624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6817","nonce":1466336624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6815","nonce":1466336624}]},"public_addr":"172.21.15.2:6811/1466336624","cluster_addr":"172.21.15.2:6813/1466336624","heartbeat_back_addr":"172.21.15.2:6817/1466336624","heartbeat_front_addr":"172.21.15.2:6815/1466336624","state":["exists","up"]},{"osd":2,"uuid":"ce6a7a3f-db89-426c-bbb4-1339c5adac81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6819","nonce":869328850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6821","nonce":869328850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6825","nonce":869328850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6823","nonce":869328850}]},"public_addr":"172.21.15.2:6819/869328850","cluster_addr":"172.21.15.2:6821/869328850","heartbeat_back_addr":"172.21.15.2:6825/869328850","heartbeat_front_addr":"172.21.15.2:6823/869328850","state":["exists","up"]},{"osd":3,"uuid":"8d760311-5fda-425b-ac3c-5045ecb1cbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6801","nonce":4202679460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6803","nonce":4202679460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6807","nonce":4202679460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6805","nonce":4202679460}]},"public_addr":"172.21.15.26:6801/4202679460","cluster_addr":"172.21.15.26:6803/4202679460","heartbeat_back_addr":"172.21.15.26:6807/4202679460","heartbeat_front_addr":"172.21.15.26:6805/4202679460","state":["exists","up"]},{"osd":4,"uuid":"250a3093-6818-4f4f-85d5-d1f2e4381afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6809","nonce":412631433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6811","nonce":412631433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6815","nonce":412631433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6813","nonce":412631433}]},"public_addr":"172.21.15.26:6809/412631433","cluster_addr":"172.21.15.26:6811/412631433","heartbeat_back_addr":"172.21.15.26:6815/412631433","heartbeat_front_addr":"172.21.15.26:6813/412631433","state":["exists","up"]},{"osd":5,"uuid":"b16f392a-1d2f-47b5-b356-cab94bc834cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6817","nonce":3448963017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6819","nonce":3448963017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6823","nonce":3448963017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6821","nonce":3448963017}]},"public_addr":"172.21.15.26:6817/3448963017","cluster_addr":"172.21.15.26:6819/3448963017","heartbeat_back_addr":"172.21.15.26:6823/3448963017","heartbeat_front_addr":"172.21.15.26:6821/3448963017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:18.975873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:47.618973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:15.584615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:39.921213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:06.286712+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:32.567112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/1971862470":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6801/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/2348229359":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6801/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1144469616":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3462160730":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/1789736475":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/1498322259":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/2565089320":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3421178356":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6801/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/2322904669":"2024-03-17T10:35:17.551709+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:40:54.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f219c06c880 msgr2=0x7f219c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f219c06c880 0x7f219c06ed40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21a400b830 tx=0x7f21a4006000 comp rx=0 tx=0).stop 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 msgr2=0x7f21b40fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b40fc320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21a800b4b0 tx=0x7f21a800b870 comp rx=0 tx=0).stop 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 shutdown_connections 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f219c06c880 0x7f219c06ed40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21b40ff9a0 0x7f21b40fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 --2- 172.21.15.2:0/2544250909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21b41059a0 0x7f21b40fc320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 >> 172.21.15.2:0/2544250909 conn(0x7f21b40fb4b0 msgr2=0x7f21b4069830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 shutdown_connections 2024-03-16T10:40:54.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:54.274+0000 7f21bb8b8700 1 -- 172.21.15.2:0/2544250909 wait complete. 2024-03-16T10:40:54.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:54 smithi026 ceph-mon[37041]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:55.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-16T10:40:55.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-16T10:40:55.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-16T10:40:55.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-16T10:40:55.088 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-16T10:40:55.088 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-16T10:40:55.354 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:55 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2544250909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:40:55.491 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:55 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2544250909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:40:56.361 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:56 smithi026 ceph-mon[37041]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:56.494 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:56 smithi002 ceph-mon[29248]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:40:57.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 -- 172.21.15.2:0/3311780191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94101c10 msgr2=0x7fda94101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:57.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 --2- 172.21.15.2:0/3311780191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94101c10 0x7fda94101ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00b0a0 tx=0x7fda8c00b3b0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 -- 172.21.15.2:0/3311780191 shutdown_connections 2024-03-16T10:40:57.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 --2- 172.21.15.2:0/3311780191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda94102530 0x7fda9410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 --2- 172.21.15.2:0/3311780191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94101c10 0x7fda94101ff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 -- 172.21.15.2:0/3311780191 >> 172.21.15.2:0/3311780191 conn(0x7fda94076640 msgr2=0x7fda940fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:57.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 -- 172.21.15.2:0/3311780191 shutdown_connections 2024-03-16T10:40:57.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.915+0000 7fda99ff5700 1 -- 172.21.15.2:0/3311780191 wait complete. 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda99ff5700 1 Processor -- start 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda99ff5700 1 -- start start 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda99ff5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda94101c10 0x7fda94196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda99ff5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94102530 0x7fda94196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda99ff5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda94197610 con 0x7fda94102530 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda99ff5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9419b360 con 0x7fda94101c10 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda92ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94102530 0x7fda94196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda92ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94102530 0x7fda94196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34892/0 (socket says 172.21.15.2:34892) 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda92ffd700 1 -- 172.21.15.2:0/4279650830 learned_addr learned my addr 172.21.15.2:0/4279650830 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.916+0000 7fda937fe700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda94101c10 0x7fda94196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda92ffd700 1 -- 172.21.15.2:0/4279650830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda94101c10 msgr2=0x7fda94196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda92ffd700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda94101c10 0x7fda94196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda92ffd700 1 -- 172.21.15.2:0/4279650830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c009d00 con 0x7fda94102530 2024-03-16T10:40:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda92ffd700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94102530 0x7fda94196f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00d6d0 tx=0x7fda7c00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda7c00f860 con 0x7fda94102530 2024-03-16T10:40:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda7c00fe80 con 0x7fda94102530 2024-03-16T10:40:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda7c010ea0 con 0x7fda94102530 2024-03-16T10:40:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9419b640 con 0x7fda94102530 2024-03-16T10:40:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.917+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9419bb90 con 0x7fda94102530 2024-03-16T10:40:57.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.918+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda9410df10 con 0x7fda94102530 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.919+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fda7c010460 con 0x7fda94102530 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda90ff9700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fda8006c880 0x7fda8006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fda7c08bab0 con 0x7fda94102530 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda937fe700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fda8006c880 0x7fda8006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda90ff9700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] conn(0x7fda80072560 0x7fda80074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 --> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda80075030 con 0x7fda80072560 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda7c08bf10 con 0x7fda94102530 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda93fff700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] conn(0x7fda80072560 0x7fda80074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:40:57.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda937fe700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fda8006c880 0x7fda8006ed40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda8c000f80 tx=0x7fda8c0060c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:57.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.920+0000 7fda93fff700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] conn(0x7fda80072560 0x7fda80074980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:40:57.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.921+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== osd.1 v2:172.21.15.2:6810/1466336624 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fda80075030 con 0x7fda80072560 2024-03-16T10:40:57.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.938+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 --> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda9419c100 con 0x7fda80072560 2024-03-16T10:40:57.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.938+0000 7fda90ff9700 1 -- 172.21.15.2:0/4279650830 <== osd.1 v2:172.21.15.2:6810/1466336624 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda9419c100 con 0x7fda80072560 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.938+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] conn(0x7fda80072560 msgr2=0x7fda80074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.938+0000 7fda99ff5700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] conn(0x7fda80072560 0x7fda80074980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fda8006c880 msgr2=0x7fda8006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fda8006c880 0x7fda8006ed40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda8c000f80 tx=0x7fda8c0060c0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94102530 msgr2=0x7fda94196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94102530 0x7fda94196f80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00d6d0 tx=0x7fda7c00d9e0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 shutdown_connections 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6810/1466336624,v1:172.21.15.2:6811/1466336624] conn(0x7fda80072560 0x7fda80074980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fda8006c880 0x7fda8006ed40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda94101c10 0x7fda94196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 --2- 172.21.15.2:0/4279650830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda94102530 0x7fda94196f80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:40:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 >> 172.21.15.2:0/4279650830 conn(0x7fda94076640 msgr2=0x7fda94102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:40:57.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 shutdown_connections 2024-03-16T10:40:57.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:40:57.939+0000 7fda99ff5700 1 -- 172.21.15.2:0/4279650830 wait complete. 2024-03-16T10:40:58.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:40:58 smithi026 ceph-mon[37041]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:58.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:40:58 smithi002 ceph-mon[29248]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:40:58.836 INFO:teuthology.orchestra.run.smithi002.stdout:60129542172 2024-03-16T10:40:58.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-16T10:41:00.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:00 smithi002 ceph-mon[29248]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:00 smithi026 ceph-mon[37041]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:01.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 -- 172.21.15.2:0/2462222439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 msgr2=0x7f2568100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:01.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 --2- 172.21.15.2:0/2462222439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f2568100990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2564009910 tx=0x7f2564009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:01.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 -- 172.21.15.2:0/2462222439 shutdown_connections 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 --2- 172.21.15.2:0/2462222439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2568100f60 0x7f25681045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 --2- 172.21.15.2:0/2462222439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f2568100990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 -- 172.21.15.2:0/2462222439 >> 172.21.15.2:0/2462222439 conn(0x7f2568074cc0 msgr2=0x7f25680750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 -- 172.21.15.2:0/2462222439 shutdown_connections 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.159+0000 7f25705c8700 1 -- 172.21.15.2:0/2462222439 wait complete. 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f25705c8700 1 Processor -- start 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f25705c8700 1 -- start start 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f25705c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f256819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f25705c8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2568100f60 0x7f256819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f25705c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256819d2b0 con 0x7f25681005b0 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f25705c8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568196820 con 0x7f2568100f60 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f256e364700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f256819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f256e364700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f256819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34914/0 (socket says 172.21.15.2:34914) 2024-03-16T10:41:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.160+0000 7f256e364700 1 -- 172.21.15.2:0/2296990559 learned_addr learned my addr 172.21.15.2:0/2296990559 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f256e364700 1 -- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2568100f60 msgr2=0x7f256819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f256e364700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2568100f60 0x7f256819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f256e364700 1 -- 172.21.15.2:0/2296990559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640095f0 con 0x7f25681005b0 2024-03-16T10:41:01.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f256e364700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f256819c6e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2564006010 tx=0x7f256400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256401d070 con 0x7f25681005b0 2024-03-16T10:41:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f256400f460 con 0x7f25681005b0 2024-03-16T10:41:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2568196aa0 con 0x7f25681005b0 2024-03-16T10:41:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2564017630 con 0x7f25681005b0 2024-03-16T10:41:01.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.161+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2568196f90 con 0x7f25681005b0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.162+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f256810cec0 con 0x7f25681005b0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.162+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f256400f5d0 con 0x7f25681005b0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f255b7fe700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f255406c890 0x7f255406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f256408cdf0 con 0x7f25681005b0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f256db63700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f255406c890 0x7f255406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f255b7fe700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] conn(0x7f2554072570 0x7f2554074990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 --> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2554075040 con 0x7f2554072570 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25640924d0 con 0x7f25681005b0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f256eb65700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] conn(0x7f2554072570 0x7f2554074990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.163+0000 7f256db63700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f255406c890 0x7f255406ed50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2568197c40 tx=0x7f255c005c80 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.164+0000 7f256eb65700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] conn(0x7f2554072570 0x7f2554074990 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:01.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.164+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== osd.5 v2:172.21.15.26:6816/3448963017 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2554075040 con 0x7f2554072570 2024-03-16T10:41:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.180+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 --> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2568197580 con 0x7f2554072570 2024-03-16T10:41:01.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f255b7fe700 1 -- 172.21.15.2:0/2296990559 <== osd.5 v2:172.21.15.26:6816/3448963017 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2568197580 con 0x7f2554072570 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] conn(0x7f2554072570 msgr2=0x7f2554074990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] conn(0x7f2554072570 0x7f2554074990 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f255406c890 msgr2=0x7f255406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f255406c890 0x7f255406ed50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2568197c40 tx=0x7f255c005c80 comp rx=0 tx=0).stop 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 msgr2=0x7f256819c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f256819c6e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2564006010 tx=0x7f256400be20 comp rx=0 tx=0).stop 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 shutdown_connections 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:6816/3448963017,v1:172.21.15.26:6817/3448963017] conn(0x7f2554072570 0x7f2554074990 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f255406c890 0x7f255406ed50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25681005b0 0x7f256819c6e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 --2- 172.21.15.2:0/2296990559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2568100f60 0x7f256819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:01.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 >> 172.21.15.2:0/2296990559 conn(0x7f2568074cc0 msgr2=0x7f2568103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 shutdown_connections 2024-03-16T10:41:01.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:01.181+0000 7f25705c8700 1 -- 172.21.15.2:0/2296990559 wait complete. 2024-03-16T10:41:01.366 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:01.382 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:01 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:02 smithi002 ceph-mon[29248]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:02.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:02 smithi026 ceph-mon[37041]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:03.793 INFO:teuthology.orchestra.run.smithi002.stdout:171798691848 2024-03-16T10:41:03.793 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-16T10:41:04.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 -- 172.21.15.2:0/3553560976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6000ffa10 msgr2=0x7ff6000ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 --2- 172.21.15.2:0/3553560976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6000ffa10 0x7ff6000ffe90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009910 tx=0x7ff5f4009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:04.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 -- 172.21.15.2:0/3553560976 shutdown_connections 2024-03-16T10:41:04.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 --2- 172.21.15.2:0/3553560976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6000ffa10 0x7ff6000ffe90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 --2- 172.21.15.2:0/3553560976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff600105a10 0x7ff600105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 -- 172.21.15.2:0/3553560976 >> 172.21.15.2:0/3553560976 conn(0x7ff6000fb540 msgr2=0x7ff6000fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 -- 172.21.15.2:0/3553560976 shutdown_connections 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.260+0000 7ff60728d700 1 -- 172.21.15.2:0/3553560976 wait complete. 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff60728d700 1 Processor -- start 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff60728d700 1 -- start start 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff60728d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6000ffa10 0x7ff60019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff60728d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600105a10 0x7ff60019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff60728d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60019c280 con 0x7ff600105a10 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff60728d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001957f0 con 0x7ff6000ffa10 2024-03-16T10:41:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff604828700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600105a10 0x7ff60019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff604828700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600105a10 0x7ff60019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53416/0 (socket says 172.21.15.2:53416) 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff604828700 1 -- 172.21.15.2:0/1977702488 learned_addr learned my addr 172.21.15.2:0/1977702488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.261+0000 7ff605029700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6000ffa10 0x7ff60019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff604828700 1 -- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6000ffa10 msgr2=0x7ff60019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff604828700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6000ffa10 0x7ff60019b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff604828700 1 -- 172.21.15.2:0/1977702488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40095f0 con 0x7ff600105a10 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff605029700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6000ffa10 0x7ff60019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff604828700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600105a10 0x7ff60019bbf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006010 tx=0x7ff5f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f401d070 con 0x7ff600105a10 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5f400f460 con 0x7ff600105a10 2024-03-16T10:41:04.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600195a70 con 0x7ff600105a10 2024-03-16T10:41:04.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5f4017630 con 0x7ff600105a10 2024-03-16T10:41:04.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.262+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff600195f60 con 0x7ff600105a10 2024-03-16T10:41:04.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.263+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff60010cf90 con 0x7ff600105a10 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.263+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7ff5f400f5d0 con 0x7ff600105a10 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff5fe7fc700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff5e806c880 0x7ff5e806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff5f408d1b0 con 0x7ff600105a10 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff605029700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff5e806c880 0x7ff5e806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff5fe7fc700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] conn(0x7ff5e8072560 0x7ff5e8074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 --> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5e8075030 con 0x7ff5e8072560 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5f408d610 con 0x7ff600105a10 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff605029700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff5e806c880 0x7ff5e806ed40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009fa0 tx=0x7ff5f0009570 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.264+0000 7ff60582a700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] conn(0x7ff5e8072560 0x7ff5e8074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.265+0000 7ff60582a700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] conn(0x7ff5e8072560 0x7ff5e8074980 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:04.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.266+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== osd.3 v2:172.21.15.26:6800/4202679460 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff5e8075030 con 0x7ff5e8072560 2024-03-16T10:41:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.281+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 --> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff600196480 con 0x7ff5e8072560 2024-03-16T10:41:04.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff5fe7fc700 1 -- 172.21.15.2:0/1977702488 <== osd.3 v2:172.21.15.26:6800/4202679460 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff600196480 con 0x7ff5e8072560 2024-03-16T10:41:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] conn(0x7ff5e8072560 msgr2=0x7ff5e8074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] conn(0x7ff5e8072560 0x7ff5e8074980 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff5e806c880 msgr2=0x7ff5e806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff5e806c880 0x7ff5e806ed40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009fa0 tx=0x7ff5f0009570 comp rx=0 tx=0).stop 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600105a10 msgr2=0x7ff60019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600105a10 0x7ff60019bbf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006010 tx=0x7ff5f400be20 comp rx=0 tx=0).stop 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 shutdown_connections 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:6800/4202679460,v1:172.21.15.26:6801/4202679460] conn(0x7ff5e8072560 0x7ff5e8074980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff5e806c880 0x7ff5e806ed40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6000ffa10 0x7ff60019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 --2- 172.21.15.2:0/1977702488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff600105a10 0x7ff60019bbf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 >> 172.21.15.2:0/1977702488 conn(0x7ff6000fb540 msgr2=0x7ff6000fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 shutdown_connections 2024-03-16T10:41:04.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.282+0000 7ff60728d700 1 -- 172.21.15.2:0/1977702488 wait complete. 2024-03-16T10:41:04.294 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:04 smithi002 ceph-mon[29248]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:04.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:04 smithi026 ceph-mon[37041]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:04.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.487+0000 7fe7baac2700 1 -- 172.21.15.2:0/3700898635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41059a0 msgr2=0x7fe7b4105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.487+0000 7fe7baac2700 1 --2- 172.21.15.2:0/3700898635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41059a0 0x7fe7b4105d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009910 tx=0x7fe7a8009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:04.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 -- 172.21.15.2:0/3700898635 shutdown_connections 2024-03-16T10:41:04.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 --2- 172.21.15.2:0/3700898635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7b40ff9a0 0x7fe7b40ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 --2- 172.21.15.2:0/3700898635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41059a0 0x7fe7b4105d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 -- 172.21.15.2:0/3700898635 >> 172.21.15.2:0/3700898635 conn(0x7fe7b40fb4b0 msgr2=0x7fe7b40fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 -- 172.21.15.2:0/3700898635 shutdown_connections 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 -- 172.21.15.2:0/3700898635 wait complete. 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 Processor -- start 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 -- start start 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b40ff9a0 0x7fe7b419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7b41059a0 0x7fe7b419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b419d290 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7baac2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b4196800 con 0x7fe7b41059a0 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7b885e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b40ff9a0 0x7fe7b419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:04.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7b885e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b40ff9a0 0x7fe7b419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53432/0 (socket says 172.21.15.2:53432) 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.488+0000 7fe7b885e700 1 -- 172.21.15.2:0/999634520 learned_addr learned my addr 172.21.15.2:0/999634520 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7b885e700 1 -- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7b41059a0 msgr2=0x7fe7b419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7b885e700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7b41059a0 0x7fe7b419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7b885e700 1 -- 172.21.15.2:0/999634520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80095f0 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7b885e700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b40ff9a0 0x7fe7b419c6c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800bd70 tx=0x7fe7a800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a801d070 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b4196a80 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7a800f460 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a8017630 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.489+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b4196f70 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.490+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7b410cea0 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.490+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fe7a800f5d0 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b1ffb700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe7a406c880 0x7fe7a406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe7a808d2b0 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b3fff700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe7a406c880 0x7fe7a406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b1ffb700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] conn(0x7fe7a4072560 0x7fe7a4074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 --> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7a4075030 con 0x7fe7a4072560 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7a808d710 con 0x7fe7b40ff9a0 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b905f700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] conn(0x7fe7a4072560 0x7fe7a4074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.491+0000 7fe7b3fff700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe7a406c880 0x7fe7a406ed40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4197c20 tx=0x7fe7a0005c80 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.492+0000 7fe7b905f700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] conn(0x7fe7a4072560 0x7fe7a4074980 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:04.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.492+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== osd.4 v2:172.21.15.26:6808/412631433 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe7a4075030 con 0x7fe7a4072560 2024-03-16T10:41:04.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.508+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 --> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7b4197490 con 0x7fe7a4072560 2024-03-16T10:41:04.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7b1ffb700 1 -- 172.21.15.2:0/999634520 <== osd.4 v2:172.21.15.26:6808/412631433 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe7b4197490 con 0x7fe7a4072560 2024-03-16T10:41:04.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] conn(0x7fe7a4072560 msgr2=0x7fe7a4074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] conn(0x7fe7a4072560 0x7fe7a4074980 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe7a406c880 msgr2=0x7fe7a406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe7a406c880 0x7fe7a406ed40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4197c20 tx=0x7fe7a0005c80 comp rx=0 tx=0).stop 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b40ff9a0 msgr2=0x7fe7b419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b40ff9a0 0x7fe7b419c6c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800bd70 tx=0x7fe7a800be50 comp rx=0 tx=0).stop 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 shutdown_connections 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:6808/412631433,v1:172.21.15.26:6809/412631433] conn(0x7fe7a4072560 0x7fe7a4074980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe7a406c880 0x7fe7a406ed40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b40ff9a0 0x7fe7b419c6c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 --2- 172.21.15.2:0/999634520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7b41059a0 0x7fe7b419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 >> 172.21.15.2:0/999634520 conn(0x7fe7b40fb4b0 msgr2=0x7fe7b4104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:04.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 shutdown_connections 2024-03-16T10:41:04.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:04.509+0000 7fe7baac2700 1 -- 172.21.15.2:0/999634520 wait complete. 2024-03-16T10:41:05.406 INFO:teuthology.orchestra.run.smithi002.stdout:115964117011 2024-03-16T10:41:05.407 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.464+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/2156019995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 msgr2=0x7fa7c8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.464+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/2156019995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c8106e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009910 tx=0x7fa7b8009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/2156019995 shutdown_connections 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/2156019995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7c8100a70 0x7fa7c8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/2156019995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c8106e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/2156019995 >> 172.21.15.2:0/2156019995 conn(0x7fa7c80fc5a0 msgr2=0x7fa7c80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/2156019995 shutdown_connections 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/2156019995 wait complete. 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 Processor -- start 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 -- start start 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.465+0000 7fa7cfc7d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7c8100a70 0x7fa7c819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cfc7d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:05.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cfc7d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c819d310 con 0x7fa7c8106a70 2024-03-16T10:41:05.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cfc7d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c81968f0 con 0x7fa7c8100a70 2024-03-16T10:41:05.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cd218700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:05.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cd218700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53410/0 (socket says 172.21.15.2:53410) 2024-03-16T10:41:05.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cd218700 1 -- 172.21.15.2:0/3584687916 learned_addr learned my addr 172.21.15.2:0/3584687916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:05.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cda19700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7c8100a70 0x7fa7c819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:05.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cd218700 1 -- 172.21.15.2:0/3584687916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7c8100a70 msgr2=0x7fa7c819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cd218700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7c8100a70 0x7fa7c819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cd218700 1 -- 172.21.15.2:0/3584687916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b80095f0 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.466+0000 7fa7cd218700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c819ccf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400b4b0 tx=0x7fa7c400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.467+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7c4011800 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.467+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c8196bd0 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.467+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7c4011e20 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.467+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7c400f2e0 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.467+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c8197120 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.467+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7c810dff0 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.468+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fa7c4010380 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7beffd700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa7b406c880 0x7fa7b406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa7c408ba50 con 0x7fa7c8106a70 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7cda19700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa7b406c880 0x7fa7b406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7beffd700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] conn(0x7fa7b4072560 0x7fa7b4074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:05.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 --> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7b4075030 con 0x7fa7b4072560 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7c408beb0 con 0x7fa7c8106a70 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7ce21a700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] conn(0x7fa7b4072560 0x7fa7b4074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7cda19700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa7b406c880 0x7fa7b406ed40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8005f50 tx=0x7fa7b801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.469+0000 7fa7ce21a700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] conn(0x7fa7b4072560 0x7fa7b4074980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.470+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== osd.2 v2:172.21.15.2:6818/869328850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa7b4075030 con 0x7fa7b4072560 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.485+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 --> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7c81975e0 con 0x7fa7b4072560 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7beffd700 1 -- 172.21.15.2:0/3584687916 <== osd.2 v2:172.21.15.2:6818/869328850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa7c81975e0 con 0x7fa7b4072560 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] conn(0x7fa7b4072560 msgr2=0x7fa7b4074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] conn(0x7fa7b4072560 0x7fa7b4074980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa7b406c880 msgr2=0x7fa7b406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa7b406c880 0x7fa7b406ed40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8005f50 tx=0x7fa7b801a040 comp rx=0 tx=0).stop 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 msgr2=0x7fa7c819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c819ccf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400b4b0 tx=0x7fa7c400b870 comp rx=0 tx=0).stop 2024-03-16T10:41:05.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 shutdown_connections 2024-03-16T10:41:05.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6818/869328850,v1:172.21.15.2:6819/869328850] conn(0x7fa7b4072560 0x7fa7b4074980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa7b406c880 0x7fa7b406ed40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7c8100a70 0x7fa7c819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 --2- 172.21.15.2:0/3584687916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7c8106a70 0x7fa7c819ccf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:05.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 >> 172.21.15.2:0/3584687916 conn(0x7fa7c80fc5a0 msgr2=0x7fa7c80fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:05.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 shutdown_connections 2024-03-16T10:41:05.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:03.486+0000 7fa7cfc7d700 1 -- 172.21.15.2:0/3584687916 wait complete. 2024-03-16T10:41:06.440 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:06 smithi026 ceph-mon[37041]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:06.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:06 smithi002 ceph-mon[29248]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:07.037 INFO:teuthology.orchestra.run.smithi002.stdout:150323855375 2024-03-16T10:41:07.038 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-16T10:41:07.038 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:07.425 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:08.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:08 smithi002 ceph-mon[29248]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 -- 172.21.15.2:0/989367458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 msgr2=0x7fe860073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 --2- 172.21.15.2:0/989367458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe860073560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe85800b0d0 tx=0x7fe85800b3e0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 -- 172.21.15.2:0/989367458 shutdown_connections 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 --2- 172.21.15.2:0/989367458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe860073b30 0x7fe86010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 --2- 172.21.15.2:0/989367458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe860073560 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 -- 172.21.15.2:0/989367458 >> 172.21.15.2:0/989367458 conn(0x7fe8600fc5a0 msgr2=0x7fe8600fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 -- 172.21.15.2:0/989367458 shutdown_connections 2024-03-16T10:41:08.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 -- 172.21.15.2:0/989367458 wait complete. 2024-03-16T10:41:08.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 Processor -- start 2024-03-16T10:41:08.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.351+0000 7fe8656fe700 1 -- start start 2024-03-16T10:41:08.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe8656fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe86019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:08.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe8656fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe860073b30 0x7fe86019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:08.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe8656fe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86019f380 con 0x7fe860073180 2024-03-16T10:41:08.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe8656fe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8601988f0 con 0x7fe860073b30 2024-03-16T10:41:08.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe860073b30 0x7fe86019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:08.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe86019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:08.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe86019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53454/0 (socket says 172.21.15.2:53454) 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe860073b30 0x7fe86019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56840/0 (socket says 172.21.15.2:56840) 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85effd700 1 -- 172.21.15.2:0/1298272067 learned_addr learned my addr 172.21.15.2:0/1298272067 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85effd700 1 -- 172.21.15.2:0/1298272067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe860073b30 msgr2=0x7fe86019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85effd700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe860073b30 0x7fe86019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85effd700 1 -- 172.21.15.2:0/1298272067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858009d00 con 0x7fe860073180 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.352+0000 7fe85effd700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe86019e7b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe858015040 tx=0x7fe858007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.353+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe85800e040 con 0x7fe860073180 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.353+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe858003c10 con 0x7fe860073180 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.353+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe860198b70 con 0x7fe860073180 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.353+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe85801d6d0 con 0x7fe860073180 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.353+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8601990c0 con 0x7fe860073180 2024-03-16T10:41:08.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.353+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8601100d0 con 0x7fe860073180 2024-03-16T10:41:08.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.354+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fe858004040 con 0x7fe860073180 2024-03-16T10:41:08.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.354+0000 7fe857fff700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe84c06c880 0x7fe84c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:08.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe85808d1a0 con 0x7fe860073180 2024-03-16T10:41:08.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe85e7fc700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe84c06c880 0x7fe84c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:08.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe857fff700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] conn(0x7fe84c072560 0x7fe84c074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:08.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 --> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe84c075030 con 0x7fe84c072560 2024-03-16T10:41:08.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe85808f870 con 0x7fe860073180 2024-03-16T10:41:08.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe85f7fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] conn(0x7fe84c072560 0x7fe84c074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:08.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe85e7fc700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe84c06c880 0x7fe84c06ed40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8600fd830 tx=0x7fe84800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:08.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.355+0000 7fe85f7fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] conn(0x7fe84c072560 0x7fe84c074980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:08.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.356+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== osd.0 v2:172.21.15.2:6802/702078877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe84c075030 con 0x7fe84c072560 2024-03-16T10:41:08.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.371+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 --> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe860199520 con 0x7fe84c072560 2024-03-16T10:41:08.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe857fff700 1 -- 172.21.15.2:0/1298272067 <== osd.0 v2:172.21.15.2:6802/702078877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe860199520 con 0x7fe84c072560 2024-03-16T10:41:08.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] conn(0x7fe84c072560 msgr2=0x7fe84c074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] conn(0x7fe84c072560 0x7fe84c074980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe84c06c880 msgr2=0x7fe84c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe84c06c880 0x7fe84c06ed40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8600fd830 tx=0x7fe84800b410 comp rx=0 tx=0).stop 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 msgr2=0x7fe86019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe86019e7b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe858015040 tx=0x7fe858007ab0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 shutdown_connections 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6802/702078877,v1:172.21.15.2:6803/702078877] conn(0x7fe84c072560 0x7fe84c074980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe84c06c880 0x7fe84c06ed40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe860073180 0x7fe86019e7b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 --2- 172.21.15.2:0/1298272067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe860073b30 0x7fe86019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 >> 172.21.15.2:0/1298272067 conn(0x7fe8600fc5a0 msgr2=0x7fe860107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 shutdown_connections 2024-03-16T10:41:08.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:08.372+0000 7fe8656fe700 1 -- 172.21.15.2:0/1298272067 wait complete. 2024-03-16T10:41:08.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:08 smithi026 ceph-mon[37041]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:09.092 INFO:teuthology.orchestra.run.smithi002.stdout:38654705701 2024-03-16T10:41:09.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-16T10:41:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:09.821 INFO:teuthology.orchestra.run.smithi002.stdout:81604378649 2024-03-16T10:41:09.821 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-16T10:41:10.462 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:10 smithi002 ceph-mon[29248]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:10.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:10 smithi026 ceph-mon[37041]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:12 smithi002 ceph-mon[29248]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:12 smithi026 ceph-mon[37041]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 -- 172.21.15.2:0/964793230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 msgr2=0x7f1068105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 --2- 172.21.15.2:0/964793230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f1068105d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f106000b0a0 tx=0x7f106000b3b0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 -- 172.21.15.2:0/964793230 shutdown_connections 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 --2- 172.21.15.2:0/964793230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680ff9a0 0x7f10680ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 --2- 172.21.15.2:0/964793230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f1068105d80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 -- 172.21.15.2:0/964793230 >> 172.21.15.2:0/964793230 conn(0x7f10680fb4b0 msgr2=0x7f10680fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 -- 172.21.15.2:0/964793230 shutdown_connections 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.400+0000 7f106ca44700 1 -- 172.21.15.2:0/964793230 wait complete. 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f106ca44700 1 Processor -- start 2024-03-16T10:41:13.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f106ca44700 1 -- start start 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f106ca44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680ff9a0 0x7f10680fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f106ca44700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f10680fc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f106ca44700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680fc7c0 con 0x7f10681059a0 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f106ca44700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680fc900 con 0x7f10680ff9a0 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f1065d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f10680fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f1065d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f10680fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53574/0 (socket says 172.21.15.2:53574) 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f1065d9b700 1 -- 172.21.15.2:0/153882164 learned_addr learned my addr 172.21.15.2:0/153882164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f106659c700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680ff9a0 0x7f10680fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f1065d9b700 1 -- 172.21.15.2:0/153882164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680ff9a0 msgr2=0x7f10680fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f1065d9b700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680ff9a0 0x7f10680fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.401+0000 7f1065d9b700 1 -- 172.21.15.2:0/153882164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1060009d00 con 0x7f10681059a0 2024-03-16T10:41:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.402+0000 7f1065d9b700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f10680fc280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f105000c8f0 tx=0x7f105000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:13.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.402+0000 7f105f7fe700 1 -- 172.21.15.2:0/153882164 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1050004a00 con 0x7f10681059a0 2024-03-16T10:41:13.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.402+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10681a2bc0 con 0x7f10681059a0 2024-03-16T10:41:13.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.402+0000 7f105f7fe700 1 -- 172.21.15.2:0/153882164 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10500054e0 con 0x7f10681059a0 2024-03-16T10:41:13.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.402+0000 7f105f7fe700 1 -- 172.21.15.2:0/153882164 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1050007cf0 con 0x7f10681059a0 2024-03-16T10:41:13.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.402+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10681a2f20 con 0x7f10681059a0 2024-03-16T10:41:13.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.403+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1068108fb0 con 0x7f10681059a0 2024-03-16T10:41:13.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.403+0000 7f105f7fe700 1 -- 172.21.15.2:0/153882164 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f1050005020 con 0x7f10681059a0 2024-03-16T10:41:13.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.403+0000 7f105f7fe700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f105406c880 0x7f105406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:13.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.404+0000 7f105f7fe700 1 -- 172.21.15.2:0/153882164 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1050013070 con 0x7f10681059a0 2024-03-16T10:41:13.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.404+0000 7f106659c700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f105406c880 0x7f105406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:13.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.404+0000 7f106659c700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f105406c880 0x7f105406ed40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1060000f80 tx=0x7f10600060c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:13.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.408+0000 7f105f7fe700 1 -- 172.21.15.2:0/153882164 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1050056090 con 0x7f10681059a0 2024-03-16T10:41:13.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.612+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f10680fd220 con 0x7f10681059a0 2024-03-16T10:41:13.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.612+0000 7f105f7fe700 1 -- 172.21.15.2:0/153882164 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10500596b0 con 0x7f10681059a0 2024-03-16T10:41:13.613 INFO:teuthology.orchestra.run.smithi002.stdout:60129542175 2024-03-16T10:41:13.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f105406c880 msgr2=0x7f105406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:13.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f105406c880 0x7f105406ed40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1060000f80 tx=0x7f10600060c0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 msgr2=0x7f10680fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f10680fc280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f105000c8f0 tx=0x7f105000cc00 comp rx=0 tx=0).stop 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 shutdown_connections 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f105406c880 0x7f105406ed40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10680ff9a0 0x7f10680fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 --2- 172.21.15.2:0/153882164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10681059a0 0x7f10680fc280 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.614+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 >> 172.21.15.2:0/153882164 conn(0x7f10680fb4b0 msgr2=0x7f1068104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.615+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 shutdown_connections 2024-03-16T10:41:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:13.615+0000 7f106ca44700 1 -- 172.21.15.2:0/153882164 wait complete. 2024-03-16T10:41:13.739 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:14.245 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:14 smithi002 ceph-mon[29248]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:14.245 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:14 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/153882164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T10:41:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.348+0000 7fe3c3970700 1 -- 172.21.15.2:0/2348210823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc100a80 msgr2=0x7fe3bc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.348+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2348210823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc100a80 0x7fe3bc100f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009910 tx=0x7fe3ac009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.348+0000 7fe3c3970700 1 -- 172.21.15.2:0/2348210823 shutdown_connections 2024-03-16T10:41:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.348+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2348210823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc100a80 0x7fe3bc100f00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.348+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2348210823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3bc106a80 0x7fe3bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.348+0000 7fe3c3970700 1 -- 172.21.15.2:0/2348210823 >> 172.21.15.2:0/2348210823 conn(0x7fe3bc0fc5b0 msgr2=0x7fe3bc0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.348+0000 7fe3c3970700 1 -- 172.21.15.2:0/2348210823 shutdown_connections 2024-03-16T10:41:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.349+0000 7fe3c3970700 1 -- 172.21.15.2:0/2348210823 wait complete. 2024-03-16T10:41:14.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.349+0000 7fe3c3970700 1 Processor -- start 2024-03-16T10:41:14.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.349+0000 7fe3c3970700 1 -- start start 2024-03-16T10:41:14.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c3970700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3bc100a80 0x7fe3bc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:14.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c3970700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc106a80 0x7fe3bc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:14.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c3970700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc073b30 con 0x7fe3bc106a80 2024-03-16T10:41:14.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c3970700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc073c70 con 0x7fe3bc100a80 2024-03-16T10:41:14.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c0f0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc106a80 0x7fe3bc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c0f0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc106a80 0x7fe3bc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53586/0 (socket says 172.21.15.2:53586) 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c0f0b700 1 -- 172.21.15.2:0/2073463767 learned_addr learned my addr 172.21.15.2:0/2073463767 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c170c700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3bc100a80 0x7fe3bc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c0f0b700 1 -- 172.21.15.2:0/2073463767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3bc100a80 msgr2=0x7fe3bc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c0f0b700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3bc100a80 0x7fe3bc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c0f0b700 1 -- 172.21.15.2:0/2073463767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0095f0 con 0x7fe3bc106a80 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.350+0000 7fe3c0f0b700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc106a80 0x7fe3bc0735f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.351+0000 7fe3b27fc700 1 -- 172.21.15.2:0/2073463767 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3ac01d070 con 0x7fe3bc106a80 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.351+0000 7fe3b27fc700 1 -- 172.21.15.2:0/2073463767 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3ac00f460 con 0x7fe3bc106a80 2024-03-16T10:41:14.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.351+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc073ef0 con 0x7fe3bc106a80 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.351+0000 7fe3b27fc700 1 -- 172.21.15.2:0/2073463767 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3ac0175f0 con 0x7fe3bc106a80 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.351+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc1a2f40 con 0x7fe3bc106a80 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.352+0000 7fe3b27fc700 1 -- 172.21.15.2:0/2073463767 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fe3ac0178b0 con 0x7fe3bc106a80 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.352+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc1090a0 con 0x7fe3bc106a80 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.352+0000 7fe3b27fc700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe3a806c880 0x7fe3a806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.353+0000 7fe3b27fc700 1 -- 172.21.15.2:0/2073463767 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe3ac08ce90 con 0x7fe3bc106a80 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.353+0000 7fe3c170c700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe3a806c880 0x7fe3a806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:14.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.353+0000 7fe3c170c700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe3a806c880 0x7fe3a806ed40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009a10 tx=0x7fe3b8009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:14.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.358+0000 7fe3b27fc700 1 -- 172.21.15.2:0/2073463767 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe3ac0577b0 con 0x7fe3bc106a80 2024-03-16T10:41:14.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:14 smithi026 ceph-mon[37041]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:14.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:14 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/153882164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T10:41:14.551 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:14.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.563+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe3bc1021c0 con 0x7fe3bc106a80 2024-03-16T10:41:14.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.564+0000 7fe3b27fc700 1 -- 172.21.15.2:0/2073463767 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe3ac005740 con 0x7fe3bc106a80 2024-03-16T10:41:14.565 INFO:teuthology.orchestra.run.smithi002.stdout:150323855376 2024-03-16T10:41:14.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe3a806c880 msgr2=0x7fe3a806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:14.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe3a806c880 0x7fe3a806ed40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009a10 tx=0x7fe3b8009190 comp rx=0 tx=0).stop 2024-03-16T10:41:14.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc106a80 msgr2=0x7fe3bc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:14.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc106a80 0x7fe3bc0735f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac00bd70 comp rx=0 tx=0).stop 2024-03-16T10:41:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 shutdown_connections 2024-03-16T10:41:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe3a806c880 0x7fe3a806ed40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3bc100a80 0x7fe3bc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 --2- 172.21.15.2:0/2073463767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3bc106a80 0x7fe3bc0735f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 >> 172.21.15.2:0/2073463767 conn(0x7fe3bc0fc5b0 msgr2=0x7fe3bc0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 shutdown_connections 2024-03-16T10:41:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:14.566+0000 7fe3c3970700 1 -- 172.21.15.2:0/2073463767 wait complete. 2024-03-16T10:41:15.255 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:15 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2073463767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T10:41:15.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:15 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2073463767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T10:41:15.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542175 for osd.1 2024-03-16T10:41:15.781 DEBUG:teuthology.parallel:result is None 2024-03-16T10:41:16.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:16 smithi002 ceph-mon[29248]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:16.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:16.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:16 smithi026 ceph-mon[37041]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:16.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:17.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 -- 172.21.15.2:0/1298491178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 msgr2=0x7f7b00100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:17.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1298491178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b00100f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009910 tx=0x7f7af0009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 -- 172.21.15.2:0/1298491178 shutdown_connections 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1298491178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b00100f10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1298491178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b00106a90 0x7f7b00106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 -- 172.21.15.2:0/1298491178 >> 172.21.15.2:0/1298491178 conn(0x7f7b000fc5a0 msgr2=0x7f7b000fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 -- 172.21.15.2:0/1298491178 shutdown_connections 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.107+0000 7f7b04b54700 1 -- 172.21.15.2:0/1298491178 wait complete. 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.108+0000 7f7b04b54700 1 Processor -- start 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.108+0000 7f7b04b54700 1 -- start start 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.108+0000 7f7b04b54700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b0019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.108+0000 7f7b04b54700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b00106a90 0x7f7b0019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:17.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.108+0000 7f7b04b54700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0019d2a0 con 0x7f7b00100a90 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.108+0000 7f7b04b54700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00196810 con 0x7f7b00106a90 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b0019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b0019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53616/0 (socket says 172.21.15.2:53616) 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afe59c700 1 -- 172.21.15.2:0/1402913010 learned_addr learned my addr 172.21.15.2:0/1402913010 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afdd9b700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b00106a90 0x7f7b0019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afe59c700 1 -- 172.21.15.2:0/1402913010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b00106a90 msgr2=0x7f7b0019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afe59c700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b00106a90 0x7f7b0019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afe59c700 1 -- 172.21.15.2:0/1402913010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af00095f0 con 0x7f7b00100a90 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.109+0000 7f7afe59c700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b0019c6d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800d900 tx=0x7f7ae800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.110+0000 7f7af77fe700 1 -- 172.21.15.2:0/1402913010 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae80041f0 con 0x7f7b00100a90 2024-03-16T10:41:17.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.110+0000 7f7af77fe700 1 -- 172.21.15.2:0/1402913010 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ae8004cf0 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.111+0000 7f7af77fe700 1 -- 172.21.15.2:0/1402913010 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae800b750 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.111+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b00196a90 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.111+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b00196fb0 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.113+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b001090b0 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.113+0000 7f7af77fe700 1 -- 172.21.15.2:0/1402913010 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f7ae8004350 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.113+0000 7f7af77fe700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f7aec06c8e0 0x7f7aec06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.113+0000 7f7af77fe700 1 -- 172.21.15.2:0/1402913010 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7ae808b540 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.113+0000 7f7afdd9b700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f7aec06c8e0 0x7f7aec06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.114+0000 7f7afdd9b700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f7aec06c8e0 0x7f7aec06eda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005b40 tx=0x7f7af001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.118+0000 7f7af77fe700 1 -- 172.21.15.2:0/1402913010 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7ae8059a40 con 0x7f7b00100a90 2024-03-16T10:41:17.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.327+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7b00197cd0 con 0x7f7b00100a90 2024-03-16T10:41:17.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.327+0000 7f7af77fe700 1 -- 172.21.15.2:0/1402913010 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ae80595d0 con 0x7f7b00100a90 2024-03-16T10:41:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:171798691850 2024-03-16T10:41:17.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f7aec06c8e0 msgr2=0x7f7aec06eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:17.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f7aec06c8e0 0x7f7aec06eda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005b40 tx=0x7f7af001a040 comp rx=0 tx=0).stop 2024-03-16T10:41:17.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 msgr2=0x7f7b0019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:17.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b0019c6d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800d900 tx=0x7f7ae800dcc0 comp rx=0 tx=0).stop 2024-03-16T10:41:17.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 shutdown_connections 2024-03-16T10:41:17.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f7aec06c8e0 0x7f7aec06eda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:17.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b00100a90 0x7f7b0019c6d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:17.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 --2- 172.21.15.2:0/1402913010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b00106a90 0x7f7b0019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:17.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 >> 172.21.15.2:0/1402913010 conn(0x7f7b000fc5a0 msgr2=0x7f7b000fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:17.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 shutdown_connections 2024-03-16T10:41:17.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:16.329+0000 7f7b04b54700 1 -- 172.21.15.2:0/1402913010 wait complete. 2024-03-16T10:41:17.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:17 smithi002 ceph-mon[29248]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:17.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:17 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1402913010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T10:41:17.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691848 got 171798691850 for osd.5 2024-03-16T10:41:17.667 DEBUG:teuthology.parallel:result is None 2024-03-16T10:41:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:17 smithi026 ceph-mon[37041]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:17.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:17 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1402913010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T10:41:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 -- 172.21.15.2:0/3902949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a01059a0 msgr2=0x7f03a0105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:18.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 --2- 172.21.15.2:0/3902949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a01059a0 0x7f03a0105d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f039000b0a0 tx=0x7f039000b3b0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 -- 172.21.15.2:0/3902949 shutdown_connections 2024-03-16T10:41:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 --2- 172.21.15.2:0/3902949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a00ff9a0 0x7f03a00ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 --2- 172.21.15.2:0/3902949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a01059a0 0x7f03a0105d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 -- 172.21.15.2:0/3902949 >> 172.21.15.2:0/3902949 conn(0x7f03a00fb4b0 msgr2=0x7f03a00fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 -- 172.21.15.2:0/3902949 shutdown_connections 2024-03-16T10:41:18.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.073+0000 7f03a4997700 1 -- 172.21.15.2:0/3902949 wait complete. 2024-03-16T10:41:18.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.074+0000 7f03a4997700 1 Processor -- start 2024-03-16T10:41:18.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.074+0000 7f03a4997700 1 -- start start 2024-03-16T10:41:18.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.074+0000 7f03a4997700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a00ff9a0 0x7f03a0197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.074+0000 7f03a4997700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a01059a0 0x7f03a0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.074+0000 7f03a4997700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0198300 con 0x7f03a00ff9a0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.074+0000 7f03a4997700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0198440 con 0x7f03a01059a0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a00ff9a0 0x7f03a0197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a00ff9a0 0x7f03a0197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53642/0 (socket says 172.21.15.2:53642) 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039e59c700 1 -- 172.21.15.2:0/3226780235 learned_addr learned my addr 172.21.15.2:0/3226780235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039dd9b700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a01059a0 0x7f03a0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039e59c700 1 -- 172.21.15.2:0/3226780235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a01059a0 msgr2=0x7f03a0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039e59c700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a01059a0 0x7f03a0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039e59c700 1 -- 172.21.15.2:0/3226780235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009d00 con 0x7f03a00ff9a0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039dd9b700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a01059a0 0x7f03a0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f039e59c700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a00ff9a0 0x7f03a0197880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0390000f80 tx=0x7f0390007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f03977fe700 1 -- 172.21.15.2:0/3226780235 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039000e040 con 0x7f03a00ff9a0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f03977fe700 1 -- 172.21.15.2:0/3226780235 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0390003c10 con 0x7f03a00ff9a0 2024-03-16T10:41:18.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a01986c0 con 0x7f03a00ff9a0 2024-03-16T10:41:18.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f03977fe700 1 -- 172.21.15.2:0/3226780235 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f039001d6b0 con 0x7f03a00ff9a0 2024-03-16T10:41:18.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.075+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a01a0c20 con 0x7f03a00ff9a0 2024-03-16T10:41:18.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.076+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a0107fa0 con 0x7f03a00ff9a0 2024-03-16T10:41:18.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.076+0000 7f03977fe700 1 -- 172.21.15.2:0/3226780235 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f0390004040 con 0x7f03a00ff9a0 2024-03-16T10:41:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.077+0000 7f03977fe700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f038c06c880 0x7f038c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.077+0000 7f039dd9b700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f038c06c880 0x7f038c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.077+0000 7f03977fe700 1 -- 172.21.15.2:0/3226780235 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f039008d020 con 0x7f03a00ff9a0 2024-03-16T10:41:18.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.077+0000 7f039dd9b700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f038c06c880 0x7f038c06ed40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03a0069770 tx=0x7f038800b5f0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.082+0000 7f03977fe700 1 -- 172.21.15.2:0/3226780235 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f03900579f0 con 0x7f03a00ff9a0 2024-03-16T10:41:18.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.284+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f03a0069550 con 0x7f03a00ff9a0 2024-03-16T10:41:18.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.285+0000 7f03977fe700 1 -- 172.21.15.2:0/3226780235 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f039005b010 con 0x7f03a00ff9a0 2024-03-16T10:41:18.286 INFO:teuthology.orchestra.run.smithi002.stdout:115964117014 2024-03-16T10:41:18.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f038c06c880 msgr2=0x7f038c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f038c06c880 0x7f038c06ed40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03a0069770 tx=0x7f038800b5f0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a00ff9a0 msgr2=0x7f03a0197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a00ff9a0 0x7f03a0197880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0390000f80 tx=0x7f0390007bd0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 shutdown_connections 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f038c06c880 0x7f038c06ed40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03a00ff9a0 0x7f03a0197880 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 --2- 172.21.15.2:0/3226780235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a01059a0 0x7f03a0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 >> 172.21.15.2:0/3226780235 conn(0x7f03a00fb4b0 msgr2=0x7f03a00fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 shutdown_connections 2024-03-16T10:41:18.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:18.287+0000 7f03a4997700 1 -- 172.21.15.2:0/3226780235 wait complete. 2024-03-16T10:41:19.224 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117014 for osd.3 2024-03-16T10:41:19.224 DEBUG:teuthology.parallel:result is None 2024-03-16T10:41:19.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:19 smithi002 ceph-mon[29248]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:19.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:19 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3226780235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T10:41:19.673 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855375 got 150323855376 for osd.4 2024-03-16T10:41:19.673 DEBUG:teuthology.parallel:result is None 2024-03-16T10:41:19.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:19 smithi026 ceph-mon[37041]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:19.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:19 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3226780235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 -- 172.21.15.2:0/3218192817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 msgr2=0x7f08f8102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/3218192817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f8102040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009910 tx=0x7f08f4009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 -- 172.21.15.2:0/3218192817 shutdown_connections 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/3218192817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08f8102580 0x7f08f810aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/3218192817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f8102040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 -- 172.21.15.2:0/3218192817 >> 172.21.15.2:0/3218192817 conn(0x7f08f80fb4b0 msgr2=0x7f08f80fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 -- 172.21.15.2:0/3218192817 shutdown_connections 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.671+0000 7f08ff3eb700 1 -- 172.21.15.2:0/3218192817 wait complete. 2024-03-16T10:41:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08ff3eb700 1 Processor -- start 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08ff3eb700 1 -- start start 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08ff3eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08ff3eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08f8102580 0x7f08f819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08ff3eb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f819f490 con 0x7f08f8101c60 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08ff3eb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f8198a70 con 0x7f08f8102580 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08fd187700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08fd187700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53674/0 (socket says 172.21.15.2:53674) 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08fd187700 1 -- 172.21.15.2:0/2283565527 learned_addr learned my addr 172.21.15.2:0/2283565527 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.672+0000 7f08fc986700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08f8102580 0x7f08f819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08fd187700 1 -- 172.21.15.2:0/2283565527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08f8102580 msgr2=0x7f08f819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08fd187700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08f8102580 0x7f08f819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08fd187700 1 -- 172.21.15.2:0/2283565527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40095f0 con 0x7f08f8101c60 2024-03-16T10:41:20.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08fd187700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f819e930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006010 tx=0x7f08f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:20.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08ee7fc700 1 -- 172.21.15.2:0/2283565527 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f401d070 con 0x7f08f8101c60 2024-03-16T10:41:20.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f8198cf0 con 0x7f08f8101c60 2024-03-16T10:41:20.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08ee7fc700 1 -- 172.21.15.2:0/2283565527 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08f400f460 con 0x7f08f8101c60 2024-03-16T10:41:20.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08ee7fc700 1 -- 172.21.15.2:0/2283565527 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f4017630 con 0x7f08f8101c60 2024-03-16T10:41:20.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.673+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f81991e0 con 0x7f08f8101c60 2024-03-16T10:41:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.674+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f81095d0 con 0x7f08f8101c60 2024-03-16T10:41:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.674+0000 7f08ee7fc700 1 -- 172.21.15.2:0/2283565527 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f08f400f5d0 con 0x7f08f8101c60 2024-03-16T10:41:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.675+0000 7f08ee7fc700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08e406c880 0x7f08e406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.675+0000 7f08ee7fc700 1 -- 172.21.15.2:0/2283565527 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f08f408ce10 con 0x7f08f8101c60 2024-03-16T10:41:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.675+0000 7f08fc986700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08e406c880 0x7f08e406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.675+0000 7f08fc986700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08e406c880 0x7f08e406ed40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08f8199c30 tx=0x7f08e8005f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.679+0000 7f08ee7fc700 1 -- 172.21.15.2:0/2283565527 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08f405b2b0 con 0x7f08f8101c60 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 -- 172.21.15.2:0/3984655016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018102550 msgr2=0x7fb01810aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 --2- 172.21.15.2:0/3984655016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018102550 0x7fb01810aa40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb014009910 tx=0x7fb014009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 -- 172.21.15.2:0/3984655016 shutdown_connections 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 --2- 172.21.15.2:0/3984655016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018102550 0x7fb01810aa40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 --2- 172.21.15.2:0/3984655016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb018101ba0 0x7fb018101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 -- 172.21.15.2:0/3984655016 >> 172.21.15.2:0/3984655016 conn(0x7fb0180fb480 msgr2=0x7fb0180fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 -- 172.21.15.2:0/3984655016 shutdown_connections 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.857+0000 7fb02010b700 1 -- 172.21.15.2:0/3984655016 wait complete. 2024-03-16T10:41:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb02010b700 1 Processor -- start 2024-03-16T10:41:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb02010b700 1 -- start start 2024-03-16T10:41:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb02010b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018101ba0 0x7fb01819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb02010b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb018102550 0x7fb01819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb02010b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01819d230 con 0x7fb018101ba0 2024-03-16T10:41:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb02010b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018196830 con 0x7fb018102550 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb01dea7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018101ba0 0x7fb01819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb01dea7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018101ba0 0x7fb01819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53688/0 (socket says 172.21.15.2:53688) 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb01dea7700 1 -- 172.21.15.2:0/1384444181 learned_addr learned my addr 172.21.15.2:0/1384444181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.858+0000 7fb01d6a6700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb018102550 0x7fb01819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb01dea7700 1 -- 172.21.15.2:0/1384444181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb018102550 msgr2=0x7fb01819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb01dea7700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb018102550 0x7fb01819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb01dea7700 1 -- 172.21.15.2:0/1384444181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140095f0 con 0x7fb018101ba0 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb01dea7700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018101ba0 0x7fb01819c6f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c930 tx=0x7fb00800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb00effd700 1 -- 172.21.15.2:0/1384444181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb008007ad0 con 0x7fb018101ba0 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb018196b10 con 0x7fb018101ba0 2024-03-16T10:41:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb00effd700 1 -- 172.21.15.2:0/1384444181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb00800ce80 con 0x7fb018101ba0 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.859+0000 7fb00effd700 1 -- 172.21.15.2:0/1384444181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb008018630 con 0x7fb018101ba0 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.860+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018197060 con 0x7fb018101ba0 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.861+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018105350 con 0x7fb018101ba0 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.861+0000 7fb00effd700 1 -- 172.21.15.2:0/1384444181 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fb008007c30 con 0x7fb018101ba0 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.861+0000 7fb00effd700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb00406c880 0x7fb00406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.861+0000 7fb00effd700 1 -- 172.21.15.2:0/1384444181 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb00808b890 con 0x7fb018101ba0 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.861+0000 7fb01d6a6700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb00406c880 0x7fb00406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.862+0000 7fb01d6a6700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb00406c880 0x7fb00406ed40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb018197c50 tx=0x7fb01401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:20.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.870+0000 7fb00effd700 1 -- 172.21.15.2:0/1384444181 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb008059d90 con 0x7fb018101ba0 2024-03-16T10:41:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.889+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f08f80698f0 con 0x7f08f8101c60 2024-03-16T10:41:20.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.889+0000 7f08ee7fc700 1 -- 172.21.15.2:0/2283565527 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f08f405ae40 con 0x7f08f8101c60 2024-03-16T10:41:20.890 INFO:teuthology.orchestra.run.smithi002.stdout:81604378652 2024-03-16T10:41:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08e406c880 msgr2=0x7f08e406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08e406c880 0x7f08e406ed40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08f8199c30 tx=0x7f08e8005f90 comp rx=0 tx=0).stop 2024-03-16T10:41:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 msgr2=0x7f08f819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f819e930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006010 tx=0x7f08f400be20 comp rx=0 tx=0).stop 2024-03-16T10:41:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 shutdown_connections 2024-03-16T10:41:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f08e406c880 0x7f08e406ed40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08f8101c60 0x7f08f819e930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 --2- 172.21.15.2:0/2283565527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08f8102580 0x7f08f819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 >> 172.21.15.2:0/2283565527 conn(0x7f08f80fb4b0 msgr2=0x7f08f810a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 shutdown_connections 2024-03-16T10:41:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:20.891+0000 7f08ff3eb700 1 -- 172.21.15.2:0/2283565527 wait complete. 2024-03-16T10:41:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.075+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb0180695f0 con 0x7fb018101ba0 2024-03-16T10:41:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.075+0000 7fb00effd700 1 -- 172.21.15.2:0/1384444181 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb008059920 con 0x7fb018101ba0 2024-03-16T10:41:21.076 INFO:teuthology.orchestra.run.smithi002.stdout:38654705703 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb00406c880 msgr2=0x7fb00406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb00406c880 0x7fb00406ed40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb018197c50 tx=0x7fb01401a040 comp rx=0 tx=0).stop 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018101ba0 msgr2=0x7fb01819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018101ba0 0x7fb01819c6f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c930 tx=0x7fb00800cc40 comp rx=0 tx=0).stop 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 shutdown_connections 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb00406c880 0x7fb00406ed40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb018101ba0 0x7fb01819c6f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 --2- 172.21.15.2:0/1384444181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb018102550 0x7fb01819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 >> 172.21.15.2:0/1384444181 conn(0x7fb0180fb480 msgr2=0x7fb018102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 shutdown_connections 2024-03-16T10:41:21.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:21.077+0000 7fb02010b700 1 -- 172.21.15.2:0/1384444181 wait complete. 2024-03-16T10:41:21.427 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:21 smithi002 ceph-mon[29248]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:21.427 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:21 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2283565527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T10:41:21.427 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:21 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1384444181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T10:41:21.555 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:21 smithi026 ceph-mon[37041]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:21.555 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:21 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2283565527' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T10:41:21.555 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:21 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1384444181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T10:41:21.791 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705701 got 38654705703 for osd.0 2024-03-16T10:41:21.791 DEBUG:teuthology.parallel:result is None 2024-03-16T10:41:22.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378652 for osd.2 2024-03-16T10:41:22.202 DEBUG:teuthology.parallel:result is None 2024-03-16T10:41:22.202 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-16T10:41:22.202 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T10:41:22.414 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:23.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:23 smithi002 ceph-mon[29248]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:23.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:23 smithi026 ceph-mon[37041]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 -- 172.21.15.2:0/2148750352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 msgr2=0x7f9674105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 --2- 172.21.15.2:0/2148750352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f9674105d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9664009910 tx=0x7f9664009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 -- 172.21.15.2:0/2148750352 shutdown_connections 2024-03-16T10:41:24.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 --2- 172.21.15.2:0/2148750352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96740ff9a0 0x7f96740ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 --2- 172.21.15.2:0/2148750352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f9674105d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 -- 172.21.15.2:0/2148750352 >> 172.21.15.2:0/2148750352 conn(0x7f96740fb4b0 msgr2=0x7f96740fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 -- 172.21.15.2:0/2148750352 shutdown_connections 2024-03-16T10:41:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 -- 172.21.15.2:0/2148750352 wait complete. 2024-03-16T10:41:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.041+0000 7f9679e88700 1 Processor -- start 2024-03-16T10:41:24.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9679e88700 1 -- start start 2024-03-16T10:41:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9679e88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96740ff9a0 0x7f96740fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9679e88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f96740fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9679e88700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740fc860 con 0x7f96741059a0 2024-03-16T10:41:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9679e88700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740fc9a0 con 0x7f96740ff9a0 2024-03-16T10:41:24.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9672ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f96740fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f96737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96740ff9a0 0x7f96740fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9672ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f96740fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40646/0 (socket says 172.21.15.2:40646) 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9672ffd700 1 -- 172.21.15.2:0/2564300609 learned_addr learned my addr 172.21.15.2:0/2564300609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9672ffd700 1 -- 172.21.15.2:0/2564300609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96740ff9a0 msgr2=0x7f96740fbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9672ffd700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96740ff9a0 0x7f96740fbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.042+0000 7f9672ffd700 1 -- 172.21.15.2:0/2564300609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640095f0 con 0x7f96741059a0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.043+0000 7f9672ffd700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f96740fc320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f966800b4b0 tx=0x7f966800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.043+0000 7f9670ff9700 1 -- 172.21.15.2:0/2564300609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9668011800 con 0x7f96741059a0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.043+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96741a2d40 con 0x7f96741059a0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.043+0000 7f9670ff9700 1 -- 172.21.15.2:0/2564300609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9668011e20 con 0x7f96741059a0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.043+0000 7f9670ff9700 1 -- 172.21.15.2:0/2564300609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f966800f2e0 con 0x7f96741059a0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.043+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96741a30a0 con 0x7f96741059a0 2024-03-16T10:41:24.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.044+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9674108fb0 con 0x7f96741059a0 2024-03-16T10:41:24.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.044+0000 7f9670ff9700 1 -- 172.21.15.2:0/2564300609 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f9668010380 con 0x7f96741059a0 2024-03-16T10:41:24.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.045+0000 7f9670ff9700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f965c06c880 0x7f965c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:24.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.045+0000 7f9670ff9700 1 -- 172.21.15.2:0/2564300609 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f966808b3c0 con 0x7f96741059a0 2024-03-16T10:41:24.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.045+0000 7f96737fe700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f965c06c880 0x7f965c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:24.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.045+0000 7f96737fe700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f965c06c880 0x7f965c06ed40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9664000c00 tx=0x7f966401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:24.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.049+0000 7f9670ff9700 1 -- 172.21.15.2:0/2564300609 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9668055e40 con 0x7f96741059a0 2024-03-16T10:41:24.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.246+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96740fd310 con 0x7f965c06c880 2024-03-16T10:41:24.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.247+0000 7f9670ff9700 1 -- 172.21.15.2:0/2564300609 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28202 (secure 0 0 0) 0x7f96740fd310 con 0x7f965c06c880 2024-03-16T10:41:24.249 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f965c06c880 msgr2=0x7f965c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f965c06c880 0x7f965c06ed40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9664000c00 tx=0x7f966401a040 comp rx=0 tx=0).stop 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 msgr2=0x7f96740fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f96740fc320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f966800b4b0 tx=0x7f966800b7c0 comp rx=0 tx=0).stop 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 shutdown_connections 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f965c06c880 0x7f965c06ed40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96740ff9a0 0x7f96740fbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:24.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 --2- 172.21.15.2:0/2564300609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96741059a0 0x7f96740fc320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:24.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 >> 172.21.15.2:0/2564300609 conn(0x7f96740fb4b0 msgr2=0x7f9674103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:24.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 shutdown_connections 2024-03-16T10:41:24.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:24.250+0000 7f9679e88700 1 -- 172.21.15.2:0/2564300609 wait complete. 2024-03-16T10:41:24.252 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-16T10:41:24.993 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-16T10:41:24.165028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":166060,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":562304852,"statfs":{"total":575970213888,"available":575800168448,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002315"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-16T10:40:34.273426+0000","last_change":"2024-03-16T10:40:08.189080+0000","last_active":"2024-03-16T10:40:34.273426+0000","last_peered":"2024-03-16T10:40:34.273426+0000","last_clean":"2024-03-16T10:40:34.273426+0000","last_became_active":"2024-03-16T10:40:08.188714+0000","last_became_peered":"2024-03-16T10:40:08.188714+0000","last_unstale":"2024-03-16T10:40:34.273426+0000","last_undegraded":"2024-03-16T10:40:34.273426+0000","last_fullsized":"2024-03-16T10:40:34.273426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:39:18.745361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:39:18.745361+0000","last_clean_scrub_stamp":"2024-03-16T10:39:18.745361+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:09:52.485270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":4,"up_from":35,"seq":150323855378,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.51600000000000001}]},{"osd":1,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.435},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41699999999999998}]},{"osd":2,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.499}]},{"osd":3,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47299999999999998}]},{"osd":2,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat Mar 16 10:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717760,"statfs":{"total":95995035648,"available":95966986240,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.33000000000000002}]},{"osd":1,"last update":"Sat Mar 16 10:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.34699999999999998}]},{"osd":3,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Mar 16 10:41:10 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":19.184000000000001,"5min":19.184000000000001,"15min":19.184000000000001},"last":0.752},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 16 10:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32000000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Sat Mar 16 10:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.38600000000000001}]},{"osd":3,"last update":"Sat Mar 16 10:40:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.0549999999999997,"5min":4.0549999999999997,"15min":4.0549999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.44500000000000001}]},{"osd":4,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32100000000000001},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.25800000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.24299999999999999}]},{"osd":2,"last update":"Sat Mar 16 10:40:18 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28599999999999998}]},{"osd":3,"last update":"Sat Mar 16 10:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.58199999999999996}]},{"osd":4,"last update":"Sat Mar 16 10:41:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:41:24.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T10:41:25.206 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:25.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:25 smithi002 ceph-mon[29248]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:25.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:25 smithi002 ceph-mon[29248]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:41:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:25 smithi026 ceph-mon[37041]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:25 smithi026 ceph-mon[37041]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:41:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.909+0000 7ff8ee65d700 1 -- 172.21.15.2:0/10733777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 msgr2=0x7ff8e8073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.909+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/10733777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e8073530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009910 tx=0x7ff8d8009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 -- 172.21.15.2:0/10733777 shutdown_connections 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/10733777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e8073a70 0x7ff8e8111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/10733777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e8073530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 -- 172.21.15.2:0/10733777 >> 172.21.15.2:0/10733777 conn(0x7ff8e80fc5a0 msgr2=0x7ff8e80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 -- 172.21.15.2:0/10733777 shutdown_connections 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 -- 172.21.15.2:0/10733777 wait complete. 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 Processor -- start 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.910+0000 7ff8ee65d700 1 -- start start 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8ee65d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8ee65d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e8073a70 0x7ff8e819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8ee65d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e819dc00 con 0x7ff8e8073150 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8ee65d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e81a18d0 con 0x7ff8e8073a70 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40662/0 (socket says 172.21.15.2:40662) 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e7fff700 1 -- 172.21.15.2:0/2637124402 learned_addr learned my addr 172.21.15.2:0/2637124402 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e77fe700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e8073a70 0x7ff8e819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e7fff700 1 -- 172.21.15.2:0/2637124402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e8073a70 msgr2=0x7ff8e819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e7fff700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e8073a70 0x7ff8e819d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e7fff700 1 -- 172.21.15.2:0/2637124402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80095f0 con 0x7ff8e8073150 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e7fff700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e819cfe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800bd70 tx=0x7ff8d800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e57fa700 1 -- 172.21.15.2:0/2637124402 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d801d070 con 0x7ff8e8073150 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e57fa700 1 -- 172.21.15.2:0/2637124402 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8d800f460 con 0x7ff8e8073150 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e81a1bb0 con 0x7ff8e8073150 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8e57fa700 1 -- 172.21.15.2:0/2637124402 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d8017630 con 0x7ff8e8073150 2024-03-16T10:41:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.911+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e81a2100 con 0x7ff8e8073150 2024-03-16T10:41:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.912+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e8074890 con 0x7ff8e8073150 2024-03-16T10:41:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.913+0000 7ff8e57fa700 1 -- 172.21.15.2:0/2637124402 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7ff8d800f5d0 con 0x7ff8e8073150 2024-03-16T10:41:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.913+0000 7ff8e57fa700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff8d006c880 0x7ff8d006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.913+0000 7ff8e77fe700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff8d006c880 0x7ff8d006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.913+0000 7ff8e57fa700 1 -- 172.21.15.2:0/2637124402 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff8d808cd90 con 0x7ff8e8073150 2024-03-16T10:41:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.914+0000 7ff8e77fe700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff8d006c880 0x7ff8d006ed40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005fd0 tx=0x7ff8dc00a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:26.918+0000 7ff8e57fa700 1 -- 172.21.15.2:0/2637124402 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff8d8057760 con 0x7ff8e8073150 2024-03-16T10:41:27.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.113+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8e81a2370 con 0x7ff8d006c880 2024-03-16T10:41:27.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.115+0000 7ff8e57fa700 1 -- 172.21.15.2:0/2637124402 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28118 (secure 0 0 0) 0x7ff8e81a2370 con 0x7ff8d006c880 2024-03-16T10:41:27.116 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff8d006c880 msgr2=0x7ff8d006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff8d006c880 0x7ff8d006ed40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005fd0 tx=0x7ff8dc00a560 comp rx=0 tx=0).stop 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 msgr2=0x7ff8e819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e819cfe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800bd70 tx=0x7ff8d800be50 comp rx=0 tx=0).stop 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 shutdown_connections 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff8d006c880 0x7ff8d006ed40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8e8073150 0x7ff8e819cfe0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 --2- 172.21.15.2:0/2637124402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e8073a70 0x7ff8e819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 >> 172.21.15.2:0/2637124402 conn(0x7ff8e80fc5a0 msgr2=0x7ff8e81007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 shutdown_connections 2024-03-16T10:41:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:27.117+0000 7ff8ee65d700 1 -- 172.21.15.2:0/2637124402 wait complete. 2024-03-16T10:41:27.119 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-03-16T10:41:27.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:27 smithi002 ceph-mon[29248]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:27.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:27 smithi002 ceph-mon[29248]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:41:27.713 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-03-16T10:41:26.165382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":166060,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":562304852,"statfs":{"total":575970213888,"available":575800168448,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":165927590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002355"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-16T10:40:34.273426+0000","last_change":"2024-03-16T10:40:08.189080+0000","last_active":"2024-03-16T10:40:34.273426+0000","last_peered":"2024-03-16T10:40:34.273426+0000","last_clean":"2024-03-16T10:40:34.273426+0000","last_became_active":"2024-03-16T10:40:08.188714+0000","last_became_peered":"2024-03-16T10:40:08.188714+0000","last_unstale":"2024-03-16T10:40:34.273426+0000","last_undegraded":"2024-03-16T10:40:34.273426+0000","last_fullsized":"2024-03-16T10:40:34.273426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:39:18.745361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:39:18.745361+0000","last_clean_scrub_stamp":"2024-03-16T10:39:18.745361+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:09:52.485270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45700000000000002}]},{"osd":1,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40500000000000003}]},{"osd":2,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.438},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.42699999999999999}]},{"osd":3,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.32700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.47299999999999998}]},{"osd":2,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat Mar 16 10:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717760,"statfs":{"total":95995035648,"available":95966986240,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.33000000000000002}]},{"osd":1,"last update":"Sat Mar 16 10:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.34699999999999998}]},{"osd":3,"last update":"Sat Mar 16 10:40:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Mar 16 10:41:10 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":19.184000000000001,"5min":19.184000000000001,"15min":19.184000000000001},"last":0.752},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 16 10:40:53 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.32000000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.192},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.24199999999999999}]},{"osd":2,"last update":"Sat Mar 16 10:41:23 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.375,"15min":0.38100000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.374,"15min":0.375},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.26200000000000001}]},{"osd":3,"last update":"Sat Mar 16 10:40:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.0549999999999997,"5min":4.0549999999999997,"15min":4.0549999999999997},"last":0.372},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.33900000000000002}]},{"osd":4,"last update":"Sat Mar 16 10:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.316},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 16 10:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.32600000000000001,"15min":0.32100000000000001},"min":{"1min":0.26900000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.25800000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Sat Mar 16 10:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.372,"15min":0.375},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.318},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.38400000000000001}]},{"osd":3,"last update":"Sat Mar 16 10:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.498}]},{"osd":4,"last update":"Sat Mar 16 10:41:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:41:27.714 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-16T10:41:27.714 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T10:41:27.715 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-16T10:41:27.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-16T10:41:27.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:27 smithi026 ceph-mon[37041]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:27.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:27 smithi026 ceph-mon[37041]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T10:41:27.926 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:29.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 -- 172.21.15.2:0/431935284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 msgr2=0x7f8f3c073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:29.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 --2- 172.21.15.2:0/431935284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c073530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009910 tx=0x7f8f2c009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:29.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 -- 172.21.15.2:0/431935284 shutdown_connections 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 --2- 172.21.15.2:0/431935284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3c073a70 0x7f8f3c111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 --2- 172.21.15.2:0/431935284 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c073530 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 -- 172.21.15.2:0/431935284 >> 172.21.15.2:0/431935284 conn(0x7f8f3c0fc5a0 msgr2=0x7f8f3c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 -- 172.21.15.2:0/431935284 shutdown_connections 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 -- 172.21.15.2:0/431935284 wait complete. 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.393+0000 7f8f4229f700 1 Processor -- start 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f4229f700 1 -- start start 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f4229f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f4229f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3c073a70 0x7f8f3c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f4229f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c19db80 con 0x7f8f3c073150 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f4229f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1a1850 con 0x7f8f3c073a70 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f3bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f3bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40686/0 (socket says 172.21.15.2:40686) 2024-03-16T10:41:29.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f3bfff700 1 -- 172.21.15.2:0/3556655057 learned_addr learned my addr 172.21.15.2:0/3556655057 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f33fff700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3c073a70 0x7f8f3c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f3bfff700 1 -- 172.21.15.2:0/3556655057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3c073a70 msgr2=0x7f8f3c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f3bfff700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3c073a70 0x7f8f3c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f3bfff700 1 -- 172.21.15.2:0/3556655057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c0095f0 con 0x7f8f3c073150 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f33fff700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3c073a70 0x7f8f3c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f3bfff700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c19cf60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bd70 tx=0x7f8f2c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f39ffb700 1 -- 172.21.15.2:0/3556655057 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2c01d070 con 0x7f8f3c073150 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f39ffb700 1 -- 172.21.15.2:0/3556655057 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f2c00f460 con 0x7f8f3c073150 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f39ffb700 1 -- 172.21.15.2:0/3556655057 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2c017630 con 0x7f8f3c073150 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.394+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c1a1b30 con 0x7f8f3c073150 2024-03-16T10:41:29.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.395+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c1a2030 con 0x7f8f3c073150 2024-03-16T10:41:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.395+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c074890 con 0x7f8f3c073150 2024-03-16T10:41:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.396+0000 7f8f39ffb700 1 -- 172.21.15.2:0/3556655057 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f8f2c00f5d0 con 0x7f8f3c073150 2024-03-16T10:41:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.396+0000 7f8f39ffb700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8f2806c880 0x7f8f2806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.396+0000 7f8f33fff700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8f2806c880 0x7f8f2806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.396+0000 7f8f39ffb700 1 -- 172.21.15.2:0/3556655057 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8f2c08ce40 con 0x7f8f3c073150 2024-03-16T10:41:29.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.397+0000 7f8f33fff700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8f2806c880 0x7f8f2806ed40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c19e4c0 tx=0x7f8f24005f50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:29.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.401+0000 7f8f39ffb700 1 -- 172.21.15.2:0/3556655057 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8f2c057790 con 0x7f8f3c073150 2024-03-16T10:41:29.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:29 smithi002 ceph-mon[29248]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:29.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.652+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8f3c04ea80 con 0x7f8f3c073150 2024-03-16T10:41:29.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.652+0000 7f8f39ffb700 1 -- 172.21.15.2:0/3556655057 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8f2c05adb0 con 0x7f8f3c073150 2024-03-16T10:41:29.653 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:41:29.653 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8f2806c880 msgr2=0x7f8f2806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8f2806c880 0x7f8f2806ed40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c19e4c0 tx=0x7f8f24005f50 comp rx=0 tx=0).stop 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 msgr2=0x7f8f3c19cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c19cf60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bd70 tx=0x7f8f2c00be50 comp rx=0 tx=0).stop 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 shutdown_connections 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8f2806c880 0x7f8f2806ed40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f3c073150 0x7f8f3c19cf60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 --2- 172.21.15.2:0/3556655057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3c073a70 0x7f8f3c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 >> 172.21.15.2:0/3556655057 conn(0x7f8f3c0fc5a0 msgr2=0x7f8f3c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 shutdown_connections 2024-03-16T10:41:29.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:29.655+0000 7f8f4229f700 1 -- 172.21.15.2:0/3556655057 wait complete. 2024-03-16T10:41:29.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:29 smithi026 ceph-mon[37041]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:30.250 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-16T10:41:30.250 INFO:tasks.cephadm:Setup complete, yielding 2024-03-16T10:41:30.251 INFO:teuthology.run_tasks:Running task print... 2024-03-16T10:41:30.260 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-03-16T10:41:30.260 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:41:30.268 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:41:30.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-16T10:41:30.473 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:30.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:30 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3556655057' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T10:41:30.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:30 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:30.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:30 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3556655057' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T10:41:30.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:30 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:31.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:31 smithi002 ceph-mon[29248]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:31.632 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:31 smithi026 ceph-mon[37041]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:32.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 -- 172.21.15.2:0/712954295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0101ce0 msgr2=0x7f87b0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 --2- 172.21.15.2:0/712954295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0101ce0 0x7f87b0102140 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009910 tx=0x7f87a0009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 -- 172.21.15.2:0/712954295 shutdown_connections 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 --2- 172.21.15.2:0/712954295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0101ce0 0x7f87b0102140 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 --2- 172.21.15.2:0/712954295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b0100ae0 0x7f87b0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 -- 172.21.15.2:0/712954295 >> 172.21.15.2:0/712954295 conn(0x7f87b00fc0a0 msgr2=0x7f87b00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 -- 172.21.15.2:0/712954295 shutdown_connections 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.091+0000 7f87b7d35700 1 -- 172.21.15.2:0/712954295 wait complete. 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b7d35700 1 Processor -- start 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b7d35700 1 -- start start 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b7d35700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0100ae0 0x7f87b01964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b7d35700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b0101ce0 0x7f87b0196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b7d35700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0197020 con 0x7f87b0100ae0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b7d35700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0197160 con 0x7f87b0101ce0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b5ad1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0100ae0 0x7f87b01964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b5ad1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0100ae0 0x7f87b01964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40704/0 (socket says 172.21.15.2:40704) 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b5ad1700 1 -- 172.21.15.2:0/440949816 learned_addr learned my addr 172.21.15.2:0/440949816 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b52d0700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b0101ce0 0x7f87b0196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b5ad1700 1 -- 172.21.15.2:0/440949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b0101ce0 msgr2=0x7f87b0196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b5ad1700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b0101ce0 0x7f87b0196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.092+0000 7f87b5ad1700 1 -- 172.21.15.2:0/440949816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00095f0 con 0x7f87b0100ae0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.093+0000 7f87b5ad1700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0100ae0 0x7f87b01964c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00e8f0 tx=0x7f87ac00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.093+0000 7f87a6ffd700 1 -- 172.21.15.2:0/440949816 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87ac009790 con 0x7f87b0100ae0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.093+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b019bbc0 con 0x7f87b0100ae0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.093+0000 7f87a6ffd700 1 -- 172.21.15.2:0/440949816 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87ac00c810 con 0x7f87b0100ae0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.093+0000 7f87a6ffd700 1 -- 172.21.15.2:0/440949816 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87ac003d20 con 0x7f87b0100ae0 2024-03-16T10:41:32.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.093+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b019c110 con 0x7f87b0100ae0 2024-03-16T10:41:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.094+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b0190660 con 0x7f87b0100ae0 2024-03-16T10:41:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.094+0000 7f87a6ffd700 1 -- 172.21.15.2:0/440949816 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f87ac0041f0 con 0x7f87b0100ae0 2024-03-16T10:41:32.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.095+0000 7f87a6ffd700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f879c06c880 0x7f879c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:32.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.095+0000 7f87a6ffd700 1 -- 172.21.15.2:0/440949816 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f87ac014070 con 0x7f87b0100ae0 2024-03-16T10:41:32.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.095+0000 7f87b52d0700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f879c06c880 0x7f879c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:32.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.095+0000 7f87b52d0700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f879c06c880 0x7f879c06ed40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b800 tx=0x7f87a0009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:32.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.099+0000 7f87a6ffd700 1 -- 172.21.15.2:0/440949816 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87ac05a020 con 0x7f87b0100ae0 2024-03-16T10:41:32.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.301+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f87b004ea80 con 0x7f87b0100ae0 2024-03-16T10:41:32.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.304+0000 7f87a6ffd700 1 -- 172.21.15.2:0/440949816 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f87ac054e60 con 0x7f87b0100ae0 2024-03-16T10:41:32.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f879c06c880 msgr2=0x7f879c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f879c06c880 0x7f879c06ed40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b800 tx=0x7f87a0009f70 comp rx=0 tx=0).stop 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0100ae0 msgr2=0x7f87b01964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0100ae0 0x7f87b01964c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00e8f0 tx=0x7f87ac00ec00 comp rx=0 tx=0).stop 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 shutdown_connections 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f879c06c880 0x7f879c06ed40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87b0100ae0 0x7f87b01964c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 --2- 172.21.15.2:0/440949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b0101ce0 0x7f87b0196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 >> 172.21.15.2:0/440949816 conn(0x7f87b00fc0a0 msgr2=0x7f87b0105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 shutdown_connections 2024-03-16T10:41:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:32.307+0000 7f87b7d35700 1 -- 172.21.15.2:0/440949816 wait complete. 2024-03-16T10:41:32.897 INFO:teuthology.run_tasks:Running task print... 2024-03-16T10:41:32.906 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-16T10:41:32.906 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:41:32.913 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:41:32.913 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-16T10:41:33.153 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:33.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:33 smithi002 ceph-mon[29248]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:33.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:33 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/440949816' entity='client.admin' 2024-03-16T10:41:33.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:41:33.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:33.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:41:33.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:33 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:33.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:33 smithi026 ceph-mon[37041]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:33 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/440949816' entity='client.admin' 2024-03-16T10:41:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:41:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:41:33.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:33 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 -- 172.21.15.2:0/1173729094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 msgr2=0x7f02bc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 --2- 172.21.15.2:0/1173729094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc100f00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009910 tx=0x7f02b4009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 -- 172.21.15.2:0/1173729094 shutdown_connections 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 --2- 172.21.15.2:0/1173729094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc101ce0 0x7f02bc102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 --2- 172.21.15.2:0/1173729094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc100f00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 -- 172.21.15.2:0/1173729094 >> 172.21.15.2:0/1173729094 conn(0x7f02bc0fc0a0 msgr2=0x7f02bc0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 -- 172.21.15.2:0/1173729094 shutdown_connections 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 -- 172.21.15.2:0/1173729094 wait complete. 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.722+0000 7f02c16c1700 1 Processor -- start 2024-03-16T10:41:34.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02c16c1700 1 -- start start 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02c16c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02c16c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc101ce0 0x7f02bc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02c16c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0736e0 con 0x7f02bc100ae0 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02c16c1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc073820 con 0x7f02bc101ce0 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02baffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02baffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38904/0 (socket says 172.21.15.2:38904) 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02baffd700 1 -- 172.21.15.2:0/43440745 learned_addr learned my addr 172.21.15.2:0/43440745 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02ba7fc700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc101ce0 0x7f02bc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02baffd700 1 -- 172.21.15.2:0/43440745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc101ce0 msgr2=0x7f02bc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02baffd700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc101ce0 0x7f02bc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02baffd700 1 -- 172.21.15.2:0/43440745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40095f0 con 0x7f02bc100ae0 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02baffd700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc074ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02b40059d0 tx=0x7f02b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02b3fff700 1 -- 172.21.15.2:0/43440745 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02b401d070 con 0x7f02bc100ae0 2024-03-16T10:41:34.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02b3fff700 1 -- 172.21.15.2:0/43440745 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02b4005dc0 con 0x7f02bc100ae0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc073aa0 con 0x7f02bc100ae0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.723+0000 7f02b3fff700 1 -- 172.21.15.2:0/43440745 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02b4021410 con 0x7f02bc100ae0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.724+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc074010 con 0x7f02bc100ae0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.724+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc1906e0 con 0x7f02bc100ae0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.725+0000 7f02b3fff700 1 -- 172.21.15.2:0/43440745 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f02b400f820 con 0x7f02bc100ae0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.725+0000 7f02b3fff700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f02a806c880 0x7f02a806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.725+0000 7f02b3fff700 1 -- 172.21.15.2:0/43440745 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f02b408ca20 con 0x7f02bc100ae0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.725+0000 7f02ba7fc700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f02a806c880 0x7f02a806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.726+0000 7f02ba7fc700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f02a806c880 0x7f02a806ed40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009a10 tx=0x7f02a4009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:34.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.730+0000 7f02b3fff700 1 -- 172.21.15.2:0/43440745 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f02b405ae00 con 0x7f02bc100ae0 2024-03-16T10:41:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.949+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02bc074010 con 0x7f02a806c880 2024-03-16T10:41:34.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.950+0000 7f02b3fff700 1 -- 172.21.15.2:0/43440745 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f02bc074010 con 0x7f02a806c880 2024-03-16T10:41:34.951 INFO:teuthology.orchestra.run.smithi002.stdout:Backend: cephadm 2024-03-16T10:41:34.951 INFO:teuthology.orchestra.run.smithi002.stdout:Available: Yes 2024-03-16T10:41:34.951 INFO:teuthology.orchestra.run.smithi002.stdout:Paused: No 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.952+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f02a806c880 msgr2=0x7f02a806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.952+0000 7f02c16c1700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f02a806c880 0x7f02a806ed40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009a10 tx=0x7f02a4009190 comp rx=0 tx=0).stop 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 msgr2=0x7f02bc074ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc074ab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02b40059d0 tx=0x7f02b400bd70 comp rx=0 tx=0).stop 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 shutdown_connections 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f02a806c880 0x7f02a806ed40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02bc100ae0 0x7f02bc074ab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:34.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 --2- 172.21.15.2:0/43440745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc101ce0 0x7f02bc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:34.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 >> 172.21.15.2:0/43440745 conn(0x7f02bc0fc0a0 msgr2=0x7f02bc105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:34.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 shutdown_connections 2024-03-16T10:41:34.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:34.953+0000 7f02c16c1700 1 -- 172.21.15.2:0/43440745 wait complete. 2024-03-16T10:41:35.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-16T10:41:35.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:35 smithi026 ceph-mon[37041]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:35.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:35 smithi026 ceph-mon[37041]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:35.740 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:35.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:35 smithi002 ceph-mon[29248]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:35.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:35 smithi002 ceph-mon[29248]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 -- 172.21.15.2:0/2375517471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24101ce0 msgr2=0x7feb24102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/2375517471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24101ce0 0x7feb24102140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feb10009910 tx=0x7feb10009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 -- 172.21.15.2:0/2375517471 shutdown_connections 2024-03-16T10:41:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/2375517471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24101ce0 0x7feb24102140 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/2375517471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24100ae0 0x7feb24100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 -- 172.21.15.2:0/2375517471 >> 172.21.15.2:0/2375517471 conn(0x7feb240fc0a0 msgr2=0x7feb240fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:37.362 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:37 smithi002 ceph-mon[29248]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 -- 172.21.15.2:0/2375517471 shutdown_connections 2024-03-16T10:41:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 -- 172.21.15.2:0/2375517471 wait complete. 2024-03-16T10:41:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 Processor -- start 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.360+0000 7feb2ab1e700 1 -- start start 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb2ab1e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24100ae0 0x7feb24198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb2ab1e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24101ce0 0x7feb24198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb2ab1e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241991d0 con 0x7feb24100ae0 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb2ab1e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb24199310 con 0x7feb24101ce0 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb288ba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24100ae0 0x7feb24198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb288ba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24100ae0 0x7feb24198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38916/0 (socket says 172.21.15.2:38916) 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb288ba700 1 -- 172.21.15.2:0/3743551390 learned_addr learned my addr 172.21.15.2:0/3743551390 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb23fff700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24101ce0 0x7feb24198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb288ba700 1 -- 172.21.15.2:0/3743551390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24101ce0 msgr2=0x7feb24198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb288ba700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24101ce0 0x7feb24198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb288ba700 1 -- 172.21.15.2:0/3743551390 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100095f0 con 0x7feb24100ae0 2024-03-16T10:41:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb288ba700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24100ae0 0x7feb24198700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c910 tx=0x7feb1800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb21ffb700 1 -- 172.21.15.2:0/3743551390 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb180041f0 con 0x7feb24100ae0 2024-03-16T10:41:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.361+0000 7feb21ffb700 1 -- 172.21.15.2:0/3743551390 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb1800ce60 con 0x7feb24100ae0 2024-03-16T10:41:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.362+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2419dd80 con 0x7feb24100ae0 2024-03-16T10:41:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.362+0000 7feb21ffb700 1 -- 172.21.15.2:0/3743551390 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb18003d90 con 0x7feb24100ae0 2024-03-16T10:41:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.362+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2419e2c0 con 0x7feb24100ae0 2024-03-16T10:41:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.363+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2404eaf0 con 0x7feb24100ae0 2024-03-16T10:41:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.363+0000 7feb21ffb700 1 -- 172.21.15.2:0/3743551390 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7feb18007900 con 0x7feb24100ae0 2024-03-16T10:41:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.364+0000 7feb21ffb700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb1406c880 0x7feb1406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.364+0000 7feb21ffb700 1 -- 172.21.15.2:0/3743551390 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7feb1808b570 con 0x7feb24100ae0 2024-03-16T10:41:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.364+0000 7feb23fff700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb1406c880 0x7feb1406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.364+0000 7feb23fff700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb1406c880 0x7feb1406ed40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb10009fd0 tx=0x7feb10005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:37.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.368+0000 7feb21ffb700 1 -- 172.21.15.2:0/3743551390 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7feb2404eaf0 con 0x7feb24100ae0 2024-03-16T10:41:37.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.580+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb2404eaf0 con 0x7feb1406c880 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.590+0000 7feb21ffb700 1 -- 172.21.15.2:0/3743551390 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3080 (secure 0 0 0) 0x7feb2404eaf0 con 0x7feb1406c880 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (3m) 2m ago 5m 26.2M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (5m) 2m ago 5m 7063k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (4m) 65s ago 4m 6966k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (5m) 2m ago 5m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (4m) 65s ago 4m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (3m) 2m ago 4m 80.0M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:41:37.591 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:9283,8765,8443 running (6m) 2m ago 6m 489M - 18.2.2-878-ga752c2f7 f692adb9a265 1b8a582e9d84 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (4m) 65s ago 4m 448M - 18.2.2-878-ga752c2f7 f692adb9a265 11a47ca222e7 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (6m) 2m ago 6m 43.5M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (4m) 65s ago 4m 40.2M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (5m) 2m ago 5m 16.4M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (4m) 65s ago 4m 15.7M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (3m) 2m ago 3m 39.2M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (2m) 2m ago 2m 40.5M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (2m) 2m ago 2m 44.1M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (2m) 65s ago 2m 41.7M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (96s) 65s ago 96s 39.4M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (70s) 65s ago 70s 13.9M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:41:37.592 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (3m) 2m ago 4m 40.5M - 2.43.0 a07b618ecd1d 02e5ed409aab 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.592+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb1406c880 msgr2=0x7feb1406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.592+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb1406c880 0x7feb1406ed40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb10009fd0 tx=0x7feb10005960 comp rx=0 tx=0).stop 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.592+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24100ae0 msgr2=0x7feb24198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.592+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24100ae0 0x7feb24198700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c910 tx=0x7feb1800cc20 comp rx=0 tx=0).stop 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.592+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 shutdown_connections 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.592+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb1406c880 0x7feb1406ed40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.592+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb24100ae0 0x7feb24198700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.593+0000 7feb2ab1e700 1 --2- 172.21.15.2:0/3743551390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24101ce0 0x7feb24198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.593+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 >> 172.21.15.2:0/3743551390 conn(0x7feb240fc0a0 msgr2=0x7feb24073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:37.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.593+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 shutdown_connections 2024-03-16T10:41:37.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:37.593+0000 7feb2ab1e700 1 -- 172.21.15.2:0/3743551390 wait complete. 2024-03-16T10:41:37.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:37 smithi026 ceph-mon[37041]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:38.344 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-16T10:41:38.554 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:38.676 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:38 smithi002 ceph-mon[29248]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:38.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:38 smithi026 ceph-mon[37041]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:39.643 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:39 smithi002 ceph-mon[29248]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:39.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:39 smithi026 ceph-mon[37041]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 -- 172.21.15.2:0/2665732130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 msgr2=0x7ff248101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/2665732130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009910 tx=0x7ff23c009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 -- 172.21.15.2:0/2665732130 shutdown_connections 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/2665732130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101040 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/2665732130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2480ff9e0 0x7ff2480ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 -- 172.21.15.2:0/2665732130 >> 172.21.15.2:0/2665732130 conn(0x7ff2480faf80 msgr2=0x7ff2480fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 -- 172.21.15.2:0/2665732130 shutdown_connections 2024-03-16T10:41:40.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.219+0000 7ff24c9b8700 1 -- 172.21.15.2:0/2665732130 wait complete. 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff24c9b8700 1 Processor -- start 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff24c9b8700 1 -- start start 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff24c9b8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2480ff9e0 0x7ff2481038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff24c9b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff24c9b8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248102500 con 0x7ff248100be0 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff24c9b8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff248102640 con 0x7ff2480ff9e0 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff245d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff245d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38944/0 (socket says 172.21.15.2:38944) 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff245d9b700 1 -- 172.21.15.2:0/946332206 learned_addr learned my addr 172.21.15.2:0/946332206 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff24659c700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2480ff9e0 0x7ff2481038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:40.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff245d9b700 1 -- 172.21.15.2:0/946332206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2480ff9e0 msgr2=0x7ff2481038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:40.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff245d9b700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2480ff9e0 0x7ff2481038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:40.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff245d9b700 1 -- 172.21.15.2:0/946332206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0095f0 con 0x7ff248100be0 2024-03-16T10:41:40.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.220+0000 7ff245d9b700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006010 tx=0x7ff23c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.221+0000 7ff2377fe700 1 -- 172.21.15.2:0/946332206 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff23c01d070 con 0x7ff248100be0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.221+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2481028c0 con 0x7ff248100be0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.221+0000 7ff2377fe700 1 -- 172.21.15.2:0/946332206 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff23c00f460 con 0x7ff248100be0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.221+0000 7ff2377fe700 1 -- 172.21.15.2:0/946332206 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff23c021620 con 0x7ff248100be0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.221+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff248102db0 con 0x7ff248100be0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.222+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24804ea80 con 0x7ff248100be0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.222+0000 7ff2377fe700 1 -- 172.21.15.2:0/946332206 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7ff23c00f5d0 con 0x7ff248100be0 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.222+0000 7ff2377fe700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff23006c880 0x7ff23006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:40.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.223+0000 7ff24659c700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff23006c880 0x7ff23006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:40.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.223+0000 7ff2377fe700 1 -- 172.21.15.2:0/946332206 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff23c08cb60 con 0x7ff248100be0 2024-03-16T10:41:40.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.223+0000 7ff24659c700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff23006c880 0x7ff23006ed40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff248100a40 tx=0x7ff238007400 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:40.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.228+0000 7ff2377fe700 1 -- 172.21.15.2:0/946332206 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff23c05af40 con 0x7ff248100be0 2024-03-16T10:41:40.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.441+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff248069790 con 0x7ff23006c880 2024-03-16T10:41:40.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.446+0000 7ff2377fe700 1 -- 172.21.15.2:0/946332206 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1280 (secure 0 0 0) 0x7ff248069790 con 0x7ff23006c880 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter 2/2 2m ago 6m * 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:crash 2/2 2m ago 6m * 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:mgr 2/2 2m ago 6m count:2 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:mon 2/2 2m ago 5m smithi002:172.21.15.2=smithi002;smithi026:172.21.15.26=smithi026;count:2 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:osd 6 2m ago - 2024-03-16T10:41:40.447 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff23006c880 msgr2=0x7ff23006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff23006c880 0x7ff23006ed40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff248100a40 tx=0x7ff238007400 comp rx=0 tx=0).stop 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 msgr2=0x7ff248101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006010 tx=0x7ff23c00bd70 comp rx=0 tx=0).stop 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 shutdown_connections 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff23006c880 0x7ff23006ed40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2480ff9e0 0x7ff2481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 --2- 172.21.15.2:0/946332206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff248100be0 0x7ff248101fc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 >> 172.21.15.2:0/946332206 conn(0x7ff2480faf80 msgr2=0x7ff248069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 shutdown_connections 2024-03-16T10:41:40.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:40.448+0000 7ff24c9b8700 1 -- 172.21.15.2:0/946332206 wait complete. 2024-03-16T10:41:41.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-16T10:41:41.231 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:41.586 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:41 smithi002 ceph-mon[29248]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:41.587 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:41 smithi002 ceph-mon[29248]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:41.722 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:41 smithi026 ceph-mon[37041]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:41.722 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:41 smithi026 ceph-mon[37041]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 -- 172.21.15.2:0/3367059051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c100bd0 msgr2=0x7f806c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 --2- 172.21.15.2:0/3367059051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c100bd0 0x7f806c101030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8068009910 tx=0x7f8068009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 -- 172.21.15.2:0/3367059051 shutdown_connections 2024-03-16T10:41:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 --2- 172.21.15.2:0/3367059051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c100bd0 0x7f806c101030 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 --2- 172.21.15.2:0/3367059051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c0ff9d0 0x7f806c0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 -- 172.21.15.2:0/3367059051 >> 172.21.15.2:0/3367059051 conn(0x7f806c0faf70 msgr2=0x7f806c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 -- 172.21.15.2:0/3367059051 shutdown_connections 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.863+0000 7f807369b700 1 -- 172.21.15.2:0/3367059051 wait complete. 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f807369b700 1 Processor -- start 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f807369b700 1 -- start start 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f807369b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c0ff9d0 0x7f806c103d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f807369b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c100bd0 0x7f806c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f807369b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c102910 con 0x7f806c0ff9d0 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f807369b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c102a50 con 0x7f806c100bd0 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f8071437700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c0ff9d0 0x7f806c103d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f8071437700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c0ff9d0 0x7f806c103d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34708/0 (socket says 172.21.15.2:34708) 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.864+0000 7f8071437700 1 -- 172.21.15.2:0/2708559866 learned_addr learned my addr 172.21.15.2:0/2708559866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f8070c36700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c100bd0 0x7f806c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f8071437700 1 -- 172.21.15.2:0/2708559866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c100bd0 msgr2=0x7f806c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f8071437700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c100bd0 0x7f806c1023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f8071437700 1 -- 172.21.15.2:0/2708559866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680095f0 con 0x7f806c0ff9d0 2024-03-16T10:41:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f8071437700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c0ff9d0 0x7f806c103d30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f805c00b480 tx=0x7f805c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f80627fc700 1 -- 172.21.15.2:0/2708559866 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f805c0118c0 con 0x7f806c0ff9d0 2024-03-16T10:41:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f80627fc700 1 -- 172.21.15.2:0/2708559866 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f805c010380 con 0x7f806c0ff9d0 2024-03-16T10:41:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806c102d30 con 0x7f806c0ff9d0 2024-03-16T10:41:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f80627fc700 1 -- 172.21.15.2:0/2708559866 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f805c00f4f0 con 0x7f806c0ff9d0 2024-03-16T10:41:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.865+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c19e600 con 0x7f806c0ff9d0 2024-03-16T10:41:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.866+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c04ea80 con 0x7f806c0ff9d0 2024-03-16T10:41:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.867+0000 7f80627fc700 1 -- 172.21.15.2:0/2708559866 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f805c0104f0 con 0x7f806c0ff9d0 2024-03-16T10:41:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.867+0000 7f80627fc700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f805806c880 0x7f805806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.867+0000 7f80627fc700 1 -- 172.21.15.2:0/2708559866 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f805c08b5c0 con 0x7f806c0ff9d0 2024-03-16T10:41:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.867+0000 7f8070c36700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f805806c880 0x7f805806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.867+0000 7f8070c36700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f805806c880 0x7f805806ed40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f806800b5c0 tx=0x7f8068005160 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:42.871+0000 7f80627fc700 1 -- 172.21.15.2:0/2708559866 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f805c0599a0 con 0x7f806c0ff9d0 2024-03-16T10:41:43.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.076+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f806c19e790 con 0x7f805806c880 2024-03-16T10:41:43.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.078+0000 7f80627fc700 1 -- 172.21.15.2:0/2708559866 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f806c19e790 con 0x7f805806c880 2024-03-16T10:41:43.079 INFO:teuthology.orchestra.run.smithi002.stdout:HOST ADDR LABELS STATUS 2024-03-16T10:41:43.079 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 172.21.15.2 2024-03-16T10:41:43.079 INFO:teuthology.orchestra.run.smithi002.stdout:smithi026 172.21.15.26 2024-03-16T10:41:43.079 INFO:teuthology.orchestra.run.smithi002.stdout:2 hosts in cluster 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f805806c880 msgr2=0x7f805806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f805806c880 0x7f805806ed40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f806800b5c0 tx=0x7f8068005160 comp rx=0 tx=0).stop 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c0ff9d0 msgr2=0x7f806c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c0ff9d0 0x7f806c103d30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f805c00b480 tx=0x7f805c00b840 comp rx=0 tx=0).stop 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 shutdown_connections 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f805806c880 0x7f805806ed40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f806c0ff9d0 0x7f806c103d30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 --2- 172.21.15.2:0/2708559866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f806c100bd0 0x7f806c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:43.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 >> 172.21.15.2:0/2708559866 conn(0x7f806c0faf70 msgr2=0x7f806c068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:43.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 shutdown_connections 2024-03-16T10:41:43.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:43.080+0000 7f807369b700 1 -- 172.21.15.2:0/2708559866 wait complete. 2024-03-16T10:41:43.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:43 smithi026 ceph-mon[37041]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:43.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:43 smithi026 ceph-mon[37041]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:43.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:43 smithi002 ceph-mon[29248]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:43.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:43 smithi002 ceph-mon[29248]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:43.831 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-16T10:41:44.047 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:45.591 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:45 smithi002 ceph-mon[29248]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:45.591 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:45.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 -- 172.21.15.2:0/4115418160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0101560 msgr2=0x7fe6b0103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 --2- 172.21.15.2:0/4115418160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0101560 0x7fe6b0103950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009910 tx=0x7fe6a0009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 -- 172.21.15.2:0/4115418160 shutdown_connections 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 --2- 172.21.15.2:0/4115418160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6b0103e90 0x7fe6b0106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 --2- 172.21.15.2:0/4115418160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0101560 0x7fe6b0103950 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 -- 172.21.15.2:0/4115418160 >> 172.21.15.2:0/4115418160 conn(0x7fe6b00faf70 msgr2=0x7fe6b00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 -- 172.21.15.2:0/4115418160 shutdown_connections 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.698+0000 7fe6b696b700 1 -- 172.21.15.2:0/4115418160 wait complete. 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6b696b700 1 Processor -- start 2024-03-16T10:41:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6b696b700 1 -- start start 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6b696b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6b0101560 0x7fe6b0196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6b696b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0103e90 0x7fe6b0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6b696b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0197160 con 0x7fe6b0103e90 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6b696b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b01972a0 con 0x7fe6b0101560 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6af7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0103e90 0x7fe6b0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6af7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0103e90 0x7fe6b0196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34728/0 (socket says 172.21.15.2:34728) 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6af7fe700 1 -- 172.21.15.2:0/3049450344 learned_addr learned my addr 172.21.15.2:0/3049450344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.699+0000 7fe6affff700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6b0101560 0x7fe6b0196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6af7fe700 1 -- 172.21.15.2:0/3049450344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6b0101560 msgr2=0x7fe6b0196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6af7fe700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6b0101560 0x7fe6b0196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6af7fe700 1 -- 172.21.15.2:0/3049450344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00095f0 con 0x7fe6b0103e90 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6affff700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6b0101560 0x7fe6b0196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6af7fe700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0103e90 0x7fe6b0196b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b4b0 tx=0x7fe6a400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6ad7fa700 1 -- 172.21.15.2:0/3049450344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6a4011800 con 0x7fe6b0103e90 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6ad7fa700 1 -- 172.21.15.2:0/3049450344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6a4011e20 con 0x7fe6b0103e90 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b019bcb0 con 0x7fe6b0103e90 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6ad7fa700 1 -- 172.21.15.2:0/3049450344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6a400f2e0 con 0x7fe6b0103e90 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.700+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b019c280 con 0x7fe6b0103e90 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.701+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b004ea80 con 0x7fe6b0103e90 2024-03-16T10:41:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.701+0000 7fe6ad7fa700 1 -- 172.21.15.2:0/3049450344 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fe6a4010380 con 0x7fe6b0103e90 2024-03-16T10:41:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.702+0000 7fe6ad7fa700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe69806c880 0x7fe69806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.702+0000 7fe6ad7fa700 1 -- 172.21.15.2:0/3049450344 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe6a408b4f0 con 0x7fe6b0103e90 2024-03-16T10:41:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.702+0000 7fe6affff700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe69806c880 0x7fe69806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.702+0000 7fe6affff700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe69806c880 0x7fe69806ed40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005b40 tx=0x7fe6a0005ab0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:45.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.706+0000 7fe6ad7fa700 1 -- 172.21.15.2:0/3049450344 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe6a40598d0 con 0x7fe6b0103e90 2024-03-16T10:41:45.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:45 smithi026 ceph-mon[37041]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:45.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:41:45.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.916+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6b019c4d0 con 0x7fe69806c880 2024-03-16T10:41:45.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.918+0000 7fe6ad7fa700 1 -- 172.21.15.2:0/3049450344 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fe6b019c4d0 con 0x7fe69806c880 2024-03-16T10:41:45.919 INFO:teuthology.orchestra.run.smithi002.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-16T10:41:45.919 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EV480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-16T10:41:45.919 INFO:teuthology.orchestra.run.smithi002.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 69s ago Has a FileSystem, LVM detected 2024-03-16T10:41:45.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe69806c880 msgr2=0x7fe69806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:45.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe69806c880 0x7fe69806ed40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0005b40 tx=0x7fe6a0005ab0 comp rx=0 tx=0).stop 2024-03-16T10:41:45.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0103e90 msgr2=0x7fe6b0196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:45.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0103e90 0x7fe6b0196b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400b4b0 tx=0x7fe6a400b870 comp rx=0 tx=0).stop 2024-03-16T10:41:45.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 shutdown_connections 2024-03-16T10:41:45.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fe69806c880 0x7fe69806ed40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:45.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6b0101560 0x7fe6b0196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:45.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 --2- 172.21.15.2:0/3049450344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6b0103e90 0x7fe6b0196b40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:45.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 >> 172.21.15.2:0/3049450344 conn(0x7fe6b00faf70 msgr2=0x7fe6b00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:45.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 shutdown_connections 2024-03-16T10:41:45.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:45.920+0000 7fe6b696b700 1 -- 172.21.15.2:0/3049450344 wait complete. 2024-03-16T10:41:46.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:46 smithi002 ceph-mon[29248]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:46.533 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:41:46.542 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:41:46.543 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-16T10:41:46.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:46 smithi026 ceph-mon[37041]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:47.543 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:47 smithi002 ceph-mon[29248]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:47.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:47 smithi026 ceph-mon[37041]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:48.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 -- 172.21.15.2:0/1816300108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c0ff9d0 msgr2=0x7f428c0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 --2- 172.21.15.2:0/1816300108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c0ff9d0 0x7f428c0ffdf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f427c009910 tx=0x7f427c009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 -- 172.21.15.2:0/1816300108 shutdown_connections 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 --2- 172.21.15.2:0/1816300108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c100bd0 0x7f428c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 --2- 172.21.15.2:0/1816300108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c0ff9d0 0x7f428c0ffdf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 -- 172.21.15.2:0/1816300108 >> 172.21.15.2:0/1816300108 conn(0x7f428c0faf70 msgr2=0x7f428c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 -- 172.21.15.2:0/1816300108 shutdown_connections 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 -- 172.21.15.2:0/1816300108 wait complete. 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.311+0000 7f42927ee700 1 Processor -- start 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f42927ee700 1 -- start start 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f42927ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c0ff9d0 0x7f428c196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f42927ee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c100bd0 0x7f428c196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f42927ee700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c1971b0 con 0x7f428c100bd0 2024-03-16T10:41:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f42927ee700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c1972f0 con 0x7f428c0ff9d0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4283fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c100bd0 0x7f428c196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4283fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c100bd0 0x7f428c196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34746/0 (socket says 172.21.15.2:34746) 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4283fff700 1 -- 172.21.15.2:0/2188235150 learned_addr learned my addr 172.21.15.2:0/2188235150 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f428bfff700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c0ff9d0 0x7f428c196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4283fff700 1 -- 172.21.15.2:0/2188235150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c0ff9d0 msgr2=0x7f428c196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4283fff700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c0ff9d0 0x7f428c196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4283fff700 1 -- 172.21.15.2:0/2188235150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0095f0 con 0x7f428c100bd0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4283fff700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c100bd0 0x7f428c196b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f427400b4b0 tx=0x7f427400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4289ffb700 1 -- 172.21.15.2:0/2188235150 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4274011800 con 0x7f428c100bd0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4289ffb700 1 -- 172.21.15.2:0/2188235150 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4274011e20 con 0x7f428c100bd0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428c19bd00 con 0x7f428c100bd0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.312+0000 7f4289ffb700 1 -- 172.21.15.2:0/2188235150 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f427400f2e0 con 0x7f428c100bd0 2024-03-16T10:41:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.313+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c19c2d0 con 0x7f428c100bd0 2024-03-16T10:41:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.313+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428c04ea80 con 0x7f428c100bd0 2024-03-16T10:41:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.314+0000 7f4289ffb700 1 -- 172.21.15.2:0/2188235150 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f4274010380 con 0x7f428c100bd0 2024-03-16T10:41:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.314+0000 7f4289ffb700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f427806c890 0x7f427806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.314+0000 7f4289ffb700 1 -- 172.21.15.2:0/2188235150 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f427408b4b0 con 0x7f428c100bd0 2024-03-16T10:41:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.314+0000 7f428bfff700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f427806c890 0x7f427806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.315+0000 7f428bfff700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f427806c890 0x7f427806ed50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f427c005090 tx=0x7f427c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.319+0000 7f4289ffb700 1 -- 172.21.15.2:0/2188235150 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4274059890 con 0x7f428c100bd0 2024-03-16T10:41:48.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:48.550+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f428c068b90 con 0x7f427806c890 2024-03-16T10:41:49.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:49 smithi026 ceph-mon[37041]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:49 smithi026 ceph-mon[37041]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:49.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:49 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-16T10:41:49.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:49 smithi002 ceph-mon[29248]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:49.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:49 smithi002 ceph-mon[29248]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:41:49.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:49 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-16T10:41:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.359+0000 7f4289ffb700 1 -- 172.21.15.2:0/2188235150 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f428c068b90 con 0x7f427806c890 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f427806c890 msgr2=0x7f427806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f427806c890 0x7f427806ed50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f427c005090 tx=0x7f427c00b540 comp rx=0 tx=0).stop 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c100bd0 msgr2=0x7f428c196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c100bd0 0x7f428c196b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f427400b4b0 tx=0x7f427400b7c0 comp rx=0 tx=0).stop 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 shutdown_connections 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f427806c890 0x7f427806ed50 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f427c005090 tx=0x7f427c00b540 comp rx=0 tx=0).stop 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f428c0ff9d0 0x7f428c196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 --2- 172.21.15.2:0/2188235150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f428c100bd0 0x7f428c196b90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:50.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 >> 172.21.15.2:0/2188235150 conn(0x7f428c0faf70 msgr2=0x7f428c068480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:50.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 shutdown_connections 2024-03-16T10:41:50.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:50.363+0000 7f42927ee700 1 -- 172.21.15.2:0/2188235150 wait complete. 2024-03-16T10:41:50.665 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-16T10:41:50.665 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:50 smithi002 ceph-mon[29248]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T10:41:50.666 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-16T10:41:50.666 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:50 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[29244]: 2024-03-16T10:41:50.341+0000 7ff8af255700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:41:50.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-16T10:41:50.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:50 smithi026 ceph-mon[37041]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T10:41:50.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-16T10:41:51.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T10:41:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:51.628 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: pgmap v175: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:51.628 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T10:41:51.628 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-16T10:41:51.628 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T10:41:51.628 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: osdmap e44: 6 total, 6 up, 6 in 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: fsmap cephfs:0 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: Saving service mds.cephfs spec with placement count:4 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:51.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:51 smithi002 ceph-mon[29248]: Deploying daemon mds.cephfs.smithi002.qvbseu on smithi002 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: pgmap v175: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: osdmap e43: 6 total, 6 up, 6 in 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: osdmap e44: 6 total, 6 up, 6 in 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: fsmap cephfs:0 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: Saving service mds.cephfs spec with placement count:4 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:41:51.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:51.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:41:51.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:51.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:41:51.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:41:51.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:51.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:51 smithi026 ceph-mon[37041]: Deploying daemon mds.cephfs.smithi002.qvbseu on smithi002 2024-03-16T10:41:52.631 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:52 smithi002 ceph-mon[29248]: osdmap e45: 6 total, 6 up, 6 in 2024-03-16T10:41:52.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:52 smithi026 ceph-mon[37041]: osdmap e45: 6 total, 6 up, 6 in 2024-03-16T10:41:53.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:53 smithi026 ceph-mon[37041]: pgmap v179: 65 pgs: 6 creating+peering, 4 active+clean, 55 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:53.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:53 smithi026 ceph-mon[37041]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-16T10:41:53.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:53 smithi026 ceph-mon[37041]: osdmap e46: 6 total, 6 up, 6 in 2024-03-16T10:41:53.733 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:53 smithi002 ceph-mon[29248]: pgmap v179: 65 pgs: 6 creating+peering, 4 active+clean, 55 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:53.733 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:53 smithi002 ceph-mon[29248]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-16T10:41:53.734 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:53 smithi002 ceph-mon[29248]: osdmap e46: 6 total, 6 up, 6 in 2024-03-16T10:41:53.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/177663031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 msgr2=0x7f9f68101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:53.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/177663031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101030 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009910 tx=0x7f9f5c009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/177663031 shutdown_connections 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/177663031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101030 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/177663031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680ff9d0 0x7f9f680ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/177663031 >> 172.21.15.2:0/177663031 conn(0x7f9f680faf70 msgr2=0x7f9f680fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/177663031 shutdown_connections 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.839+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/177663031 wait complete. 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f6c9cd700 1 Processor -- start 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f6c9cd700 1 -- start start 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f6c9cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680ff9d0 0x7f9f681038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:53.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f6c9cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f6c9cd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f681024f0 con 0x7f9f68100bd0 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f6c9cd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68102630 con 0x7f9f680ff9d0 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f65d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f65d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36804/0 (socket says 172.21.15.2:36804) 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.840+0000 7f9f65d9b700 1 -- 172.21.15.2:0/482857315 learned_addr learned my addr 172.21.15.2:0/482857315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f6659c700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680ff9d0 0x7f9f681038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f65d9b700 1 -- 172.21.15.2:0/482857315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680ff9d0 msgr2=0x7f9f681038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f65d9b700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680ff9d0 0x7f9f681038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f65d9b700 1 -- 172.21.15.2:0/482857315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0095f0 con 0x7f9f68100bd0 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f6659c700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680ff9d0 0x7f9f681038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:41:53.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f65d9b700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101fb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006010 tx=0x7f9f5c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:53.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f577fe700 1 -- 172.21.15.2:0/482857315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5c01d070 con 0x7f9f68100bd0 2024-03-16T10:41:53.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f681028b0 con 0x7f9f68100bd0 2024-03-16T10:41:53.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f68102d60 con 0x7f9f68100bd0 2024-03-16T10:41:53.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f577fe700 1 -- 172.21.15.2:0/482857315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f5c00f460 con 0x7f9f68100bd0 2024-03-16T10:41:53.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.841+0000 7f9f577fe700 1 -- 172.21.15.2:0/482857315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5c021620 con 0x7f9f68100bd0 2024-03-16T10:41:53.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.843+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f480052f0 con 0x7f9f68100bd0 2024-03-16T10:41:53.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.843+0000 7f9f577fe700 1 -- 172.21.15.2:0/482857315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f9f5c021780 con 0x7f9f68100bd0 2024-03-16T10:41:53.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.843+0000 7f9f577fe700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9f5006c880 0x7f9f5006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:53.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.844+0000 7f9f577fe700 1 -- 172.21.15.2:0/482857315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9f5c08cd30 con 0x7f9f68100bd0 2024-03-16T10:41:53.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.844+0000 7f9f6659c700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9f5006c880 0x7f9f5006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:53.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.844+0000 7f9f6659c700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9f5006c880 0x7f9f5006ed40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f58005fd0 tx=0x7f9f58005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:53.848+0000 7f9f577fe700 1 -- 172.21.15.2:0/482857315 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9f5c05af00 con 0x7f9f68100bd0 2024-03-16T10:41:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.108+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9f480061d0 con 0x7f9f68100bd0 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.108+0000 7f9f577fe700 1 -- 172.21.15.2:0/482857315 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1114 (secure 0 0 0) 0x7f9f5c026470 con 0x7f9f68100bd0 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:e2 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 2 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:41:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:41:50.343430+0000 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 0 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:in 2024-03-16T10:41:54.112 INFO:teuthology.orchestra.run.smithi002.stdout:up {} 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data disabled 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 0 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:41:54.113 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:41:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9f5006c880 msgr2=0x7f9f5006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9f5006c880 0x7f9f5006ed40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f58005fd0 tx=0x7f9f58005dc0 comp rx=0 tx=0).stop 2024-03-16T10:41:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 msgr2=0x7f9f68101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101fb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c006010 tx=0x7f9f5c00be20 comp rx=0 tx=0).stop 2024-03-16T10:41:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 shutdown_connections 2024-03-16T10:41:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9f5006c880 0x7f9f5006ed40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680ff9d0 0x7f9f681038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 --2- 172.21.15.2:0/482857315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f68100bd0 0x7f9f68101fb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 >> 172.21.15.2:0/482857315 conn(0x7f9f680faf70 msgr2=0x7f9f680fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 shutdown_connections 2024-03-16T10:41:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:54.111+0000 7f9f6c9cd700 1 -- 172.21.15.2:0/482857315 wait complete. 2024-03-16T10:41:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 2 2024-03-16T10:41:54.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:54 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/482857315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:41:54.790 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:54 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/482857315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:41:54.825 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:41:54.837 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:41:54.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-03-16T10:41:55.327 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:55.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:55 smithi002 ceph-mon[29248]: pgmap v181: 65 pgs: 6 creating+peering, 28 active+clean, 31 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:55.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:55 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:55.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:55 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:55.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:55 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:55.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:55 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:41:55.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:55 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:41:55.669 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:55 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:55.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:55 smithi026 ceph-mon[37041]: pgmap v181: 65 pgs: 6 creating+peering, 28 active+clean, 31 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:41:55.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:55 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:55.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:55 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:55.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:55 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:55.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:55 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:41:55.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:55 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:41:55.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:55 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: Deploying daemon mds.cephfs.smithi026.dzzagf on smithi026 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.2:6826/2835000958,v1:172.21.15.2:6827/2835000958] up:boot 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: daemon mds.cephfs.smithi002.qvbseu assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: Cluster is now healthy 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: fsmap cephfs:0 1 up:standby 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:creating} 2024-03-16T10:41:56.456 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:56 smithi002 ceph-mon[29248]: daemon mds.cephfs.smithi002.qvbseu is now active in filesystem cephfs as rank 0 2024-03-16T10:41:56.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: Deploying daemon mds.cephfs.smithi026.dzzagf on smithi026 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.2:6826/2835000958,v1:172.21.15.2:6827/2835000958] up:boot 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: daemon mds.cephfs.smithi002.qvbseu assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: Cluster is now healthy 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: fsmap cephfs:0 1 up:standby 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:creating} 2024-03-16T10:41:56.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:56 smithi026 ceph-mon[37041]: daemon mds.cephfs.smithi002.qvbseu is now active in filesystem cephfs as rank 0 2024-03-16T10:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 -- 172.21.15.2:0/1966454493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 msgr2=0x7f1d900ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 --2- 172.21.15.2:0/1966454493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d900ffe00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009910 tx=0x7f1d80009c20 comp rx=0 tx=0).stop 2024-03-16T10:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 -- 172.21.15.2:0/1966454493 shutdown_connections 2024-03-16T10:41:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 --2- 172.21.15.2:0/1966454493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90100be0 0x7f1d90101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 --2- 172.21.15.2:0/1966454493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d900ffe00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 -- 172.21.15.2:0/1966454493 >> 172.21.15.2:0/1966454493 conn(0x7f1d900faf80 msgr2=0x7f1d900fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 -- 172.21.15.2:0/1966454493 shutdown_connections 2024-03-16T10:41:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.991+0000 7f1d95f01700 1 -- 172.21.15.2:0/1966454493 wait complete. 2024-03-16T10:41:56.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d95f01700 1 Processor -- start 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d95f01700 1 -- start start 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d95f01700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d90069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d95f01700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90100be0 0x7f1d9019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d95f01700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9019b7d0 con 0x7f1d900ff9e0 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d95f01700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9019b910 con 0x7f1d90100be0 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d8f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d90069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d8f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d90069a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36832/0 (socket says 172.21.15.2:36832) 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d8f7fe700 1 -- 172.21.15.2:0/497150581 learned_addr learned my addr 172.21.15.2:0/497150581 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.992+0000 7f1d8effd700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90100be0 0x7f1d9019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:56.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d8f7fe700 1 -- 172.21.15.2:0/497150581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90100be0 msgr2=0x7f1d9019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d8f7fe700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90100be0 0x7f1d9019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d8f7fe700 1 -- 172.21.15.2:0/497150581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800095f0 con 0x7f1d900ff9e0 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d8f7fe700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d90069a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000bda0 tx=0x7f1d8000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d8cff9700 1 -- 172.21.15.2:0/497150581 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d8001d070 con 0x7f1d900ff9e0 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d8cff9700 1 -- 172.21.15.2:0/497150581 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d8000f460 con 0x7f1d900ff9e0 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9019bb60 con 0x7f1d900ff9e0 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d8cff9700 1 -- 172.21.15.2:0/497150581 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d80021620 con 0x7f1d900ff9e0 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.993+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9019c050 con 0x7f1d900ff9e0 2024-03-16T10:41:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.994+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9018f560 con 0x7f1d900ff9e0 2024-03-16T10:41:56.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.995+0000 7f1d8cff9700 1 -- 172.21.15.2:0/497150581 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f1d8000fa70 con 0x7f1d900ff9e0 2024-03-16T10:41:56.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.995+0000 7f1d8cff9700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f1d7806c880 0x7f1d7806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:41:56.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.995+0000 7f1d8cff9700 1 -- 172.21.15.2:0/497150581 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f1d8008cf10 con 0x7f1d900ff9e0 2024-03-16T10:41:56.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.995+0000 7f1d8effd700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f1d7806c880 0x7f1d7806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:41:56.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:56.995+0000 7f1d8effd700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f1d7806c880 0x7f1d7806ed40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005950 tx=0x7f1d8400a300 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:41:57.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.000+0000 7f1d8cff9700 1 -- 172.21.15.2:0/497150581 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1d8005afb0 con 0x7f1d900ff9e0 2024-03-16T10:41:57.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.246+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f1d9002cd00 con 0x7f1d900ff9e0 2024-03-16T10:41:57.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.369+0000 7f1d8cff9700 1 -- 172.21.15.2:0/497150581 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1d8005ab40 con 0x7f1d900ff9e0 2024-03-16T10:41:57.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f1d7806c880 msgr2=0x7f1d7806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:57.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f1d7806c880 0x7f1d7806ed40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005950 tx=0x7f1d8400a300 comp rx=0 tx=0).stop 2024-03-16T10:41:57.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 msgr2=0x7f1d90069a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:41:57.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d90069a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000bda0 tx=0x7f1d8000bdd0 comp rx=0 tx=0).stop 2024-03-16T10:41:57.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 shutdown_connections 2024-03-16T10:41:57.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f1d7806c880 0x7f1d7806ed40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:57.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d900ff9e0 0x7f1d90069a90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:57.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 --2- 172.21.15.2:0/497150581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90100be0 0x7f1d9019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:41:57.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.371+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 >> 172.21.15.2:0/497150581 conn(0x7f1d900faf80 msgr2=0x7f1d901041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:41:57.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.372+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 shutdown_connections 2024-03-16T10:41:57.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:41:57.372+0000 7f1d95f01700 1 -- 172.21.15.2:0/497150581 wait complete. 2024-03-16T10:41:57.670 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:57 smithi002 ceph-mon[29248]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-16T10:41:57.670 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:57 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.2:6826/2835000958,v1:172.21.15.2:6827/2835000958] up:active 2024-03-16T10:41:57.670 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:57 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2024-03-16T10:41:57.670 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:57 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/497150581' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-16T10:41:57.727 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:57 smithi026 ceph-mon[37041]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-16T10:41:57.727 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:57 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.2:6826/2835000958,v1:172.21.15.2:6827/2835000958] up:active 2024-03-16T10:41:57.728 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:57 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2024-03-16T10:41:57.728 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:57 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/497150581' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-16T10:41:58.017 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:41:58.026 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:41:58.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-03-16T10:41:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:41:58.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/497150581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-16T10:41:58.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2024-03-16T10:41:58.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:58.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:58.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:58.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:41:58.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:41:58.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/497150581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-16T10:41:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2024-03-16T10:41:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:41:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:41:58.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:41:58.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:41:59.402 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:59 smithi026 ceph-mon[37041]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-16T10:41:59.402 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:59 smithi026 ceph-mon[37041]: Deploying daemon mds.cephfs.smithi002.ijcmqs on smithi002 2024-03-16T10:41:59.402 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:59 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:boot 2024-03-16T10:41:59.402 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:59 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 1 up:standby 2024-03-16T10:41:59.402 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:59 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:41:59.402 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:41:59 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 1 up:standby 2024-03-16T10:41:59.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:59 smithi002 ceph-mon[29248]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-16T10:41:59.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:59 smithi002 ceph-mon[29248]: Deploying daemon mds.cephfs.smithi002.ijcmqs on smithi002 2024-03-16T10:41:59.629 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:59 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:boot 2024-03-16T10:41:59.630 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:59 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 1 up:standby 2024-03-16T10:41:59.630 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:59 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:41:59.630 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:41:59 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 1 up:standby 2024-03-16T10:42:00.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 -- 172.21.15.2:0/3865435312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 msgr2=0x7feb80101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:00.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 --2- 172.21.15.2:0/3865435312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 0x7feb80101030 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feb70009910 tx=0x7feb70009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:00.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 -- 172.21.15.2:0/3865435312 shutdown_connections 2024-03-16T10:42:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 --2- 172.21.15.2:0/3865435312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 0x7feb80101030 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 --2- 172.21.15.2:0/3865435312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800ff9d0 0x7feb800ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 -- 172.21.15.2:0/3865435312 >> 172.21.15.2:0/3865435312 conn(0x7feb800faf70 msgr2=0x7feb800fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 -- 172.21.15.2:0/3865435312 shutdown_connections 2024-03-16T10:42:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 -- 172.21.15.2:0/3865435312 wait complete. 2024-03-16T10:42:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.168+0000 7feb851d2700 1 Processor -- start 2024-03-16T10:42:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb851d2700 1 -- start start 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb851d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800ff9d0 0x7feb801964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb851d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 0x7feb80196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb851d2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb80197050 con 0x7feb80100bd0 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb851d2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb80197190 con 0x7feb800ff9d0 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb7e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 0x7feb80196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb7e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 0x7feb80196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36858/0 (socket says 172.21.15.2:36858) 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb7ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800ff9d0 0x7feb801964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb7e59c700 1 -- 172.21.15.2:0/77653536 learned_addr learned my addr 172.21.15.2:0/77653536 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb7ed9d700 1 -- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 msgr2=0x7feb80196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:00.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb7ed9d700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 0x7feb80196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.169+0000 7feb7ed9d700 1 -- 172.21.15.2:0/77653536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700095f0 con 0x7feb800ff9d0 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.170+0000 7feb7ed9d700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800ff9d0 0x7feb801964f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb7400e940 tx=0x7feb7400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.170+0000 7feb67fff700 1 -- 172.21.15.2:0/77653536 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb740041f0 con 0x7feb800ff9d0 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.170+0000 7feb67fff700 1 -- 172.21.15.2:0/77653536 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb74004350 con 0x7feb800ff9d0 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.170+0000 7feb67fff700 1 -- 172.21.15.2:0/77653536 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb74003c90 con 0x7feb800ff9d0 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.170+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8019bba0 con 0x7feb800ff9d0 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.170+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8019c0f0 con 0x7feb800ff9d0 2024-03-16T10:42:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.171+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8004ea80 con 0x7feb800ff9d0 2024-03-16T10:42:00.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.172+0000 7feb67fff700 1 -- 172.21.15.2:0/77653536 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7feb7400c7c0 con 0x7feb800ff9d0 2024-03-16T10:42:00.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.172+0000 7feb67fff700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb6806c880 0x7feb6806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:00.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.172+0000 7feb7e59c700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb6806c880 0x7feb6806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:00.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.172+0000 7feb67fff700 1 -- 172.21.15.2:0/77653536 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7feb74014070 con 0x7feb800ff9d0 2024-03-16T10:42:00.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.173+0000 7feb7e59c700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb6806c880 0x7feb6806ed40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb70009fd0 tx=0x7feb70005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:00.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.177+0000 7feb67fff700 1 -- 172.21.15.2:0/77653536 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7feb7405a050 con 0x7feb800ff9d0 2024-03-16T10:42:00.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:00.436+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7feb8019c7b0 con 0x7feb800ff9d0 2024-03-16T10:42:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:00 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:00.745 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:00 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:01.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.387+0000 7feb67fff700 1 -- 172.21.15.2:0/77653536 <== mon.1 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7feb74059be0 con 0x7feb800ff9d0 2024-03-16T10:42:01.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb6806c880 msgr2=0x7feb6806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb6806c880 0x7feb6806ed40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb70009fd0 tx=0x7feb70005940 comp rx=0 tx=0).stop 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800ff9d0 msgr2=0x7feb801964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800ff9d0 0x7feb801964f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb7400e940 tx=0x7feb7400ec50 comp rx=0 tx=0).stop 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 shutdown_connections 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7feb6806c880 0x7feb6806ed40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb800ff9d0 0x7feb801964f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 --2- 172.21.15.2:0/77653536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb80100bd0 0x7feb80196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.389+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 >> 172.21.15.2:0/77653536 conn(0x7feb800faf70 msgr2=0x7feb80068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.390+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 shutdown_connections 2024-03-16T10:42:01.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:01.390+0000 7feb851d2700 1 -- 172.21.15.2:0/77653536 wait complete. 2024-03-16T10:42:01.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:01 smithi026 ceph-mon[37041]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 4 op/s 2024-03-16T10:42:01.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:01 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/77653536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-16T10:42:01.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:01 smithi026 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-16T10:42:01.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:01 smithi002 ceph-mon[29248]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 4 op/s 2024-03-16T10:42:01.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:01 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/77653536' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-16T10:42:01.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:01 smithi002 ceph-mon[29248]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-16T10:42:02.157 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:42:02.167 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:42:02.167 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-03-16T10:42:02.678 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:02 smithi002 ceph-mon[29248]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-16T10:42:02.679 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:02 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 1 up:standby 2024-03-16T10:42:02.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:02 smithi026 ceph-mon[37041]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-16T10:42:02.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:02 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 1 up:standby 2024-03-16T10:42:02.732 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: Deploying daemon mds.cephfs.smithi026.ojzcsw on smithi026 2024-03-16T10:42:03.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] up:boot 2024-03-16T10:42:03.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2 up:standby 2024-03-16T10:42:03.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:03 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:03.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: Deploying daemon mds.cephfs.smithi026.ojzcsw on smithi026 2024-03-16T10:42:03.999 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] up:boot 2024-03-16T10:42:03.999 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2 up:standby 2024-03-16T10:42:03.999 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:03 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:42:04.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.331+0000 7fa1ad614700 1 -- 172.21.15.2:0/3526049012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8101570 msgr2=0x7fa1a8103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.331+0000 7fa1ad614700 1 --2- 172.21.15.2:0/3526049012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8101570 0x7fa1a8103960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa198009910 tx=0x7fa198009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 -- 172.21.15.2:0/3526049012 shutdown_connections 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 --2- 172.21.15.2:0/3526049012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a8103ea0 0x7fa1a8106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 --2- 172.21.15.2:0/3526049012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8101570 0x7fa1a8103960 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 -- 172.21.15.2:0/3526049012 >> 172.21.15.2:0/3526049012 conn(0x7fa1a80faf80 msgr2=0x7fa1a80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 -- 172.21.15.2:0/3526049012 shutdown_connections 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 -- 172.21.15.2:0/3526049012 wait complete. 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 Processor -- start 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.332+0000 7fa1ad614700 1 -- start start 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1ad614700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a8101570 0x7fa1a8100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:04.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1ad614700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103ea0 0x7fa1a80ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:04.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1ad614700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a80ffa60 con 0x7fa1a8103ea0 2024-03-16T10:42:04.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1ad614700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a80ffba0 con 0x7fa1a8101570 2024-03-16T10:42:04.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103ea0 0x7fa1a80ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:04.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103ea0 0x7fa1a80ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58768/0 (socket says 172.21.15.2:58768) 2024-03-16T10:42:04.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a67fc700 1 -- 172.21.15.2:0/1499058332 learned_addr learned my addr 172.21.15.2:0/1499058332 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:04.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a6ffd700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a8101570 0x7fa1a8100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:04.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a67fc700 1 -- 172.21.15.2:0/1499058332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a8101570 msgr2=0x7fa1a8100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:04.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a67fc700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a8101570 0x7fa1a8100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:04.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a67fc700 1 -- 172.21.15.2:0/1499058332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980095f0 con 0x7fa1a8103ea0 2024-03-16T10:42:04.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1a67fc700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103ea0 0x7fa1a80ff520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00d8d0 tx=0x7fa19c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:04.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa18ffff700 1 -- 172.21.15.2:0/1499058332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19c0098a0 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a80ffe80 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa18ffff700 1 -- 172.21.15.2:0/1499058332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa19c010460 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa18ffff700 1 -- 172.21.15.2:0/1499058332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19c00f5f0 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.333+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a819e500 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.334+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a804ea80 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.335+0000 7fa18ffff700 1 -- 172.21.15.2:0/1499058332 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fa19c009a00 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.335+0000 7fa18ffff700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa19006c880 0x7fa19006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.335+0000 7fa1a6ffd700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa19006c880 0x7fa19006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.335+0000 7fa18ffff700 1 -- 172.21.15.2:0/1499058332 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa19c08bb40 con 0x7fa1a8103ea0 2024-03-16T10:42:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.336+0000 7fa1a6ffd700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa19006c880 0x7fa19006ed40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa198005090 tx=0x7fa198005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:04.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.340+0000 7fa18ffff700 1 -- 172.21.15.2:0/1499058332 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa19c059be0 con 0x7fa1a8103ea0 2024-03-16T10:42:04.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.584+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1a819e690 con 0x7fa1a8103ea0 2024-03-16T10:42:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.736+0000 7fa18ffff700 1 -- 172.21.15.2:0/1499058332 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7fa19c059770 con 0x7fa1a8103ea0 2024-03-16T10:42:04.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa19006c880 msgr2=0x7fa19006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:04.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa19006c880 0x7fa19006ed40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa198005090 tx=0x7fa198005960 comp rx=0 tx=0).stop 2024-03-16T10:42:04.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103ea0 msgr2=0x7fa1a80ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:04.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103ea0 0x7fa1a80ff520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00d8d0 tx=0x7fa19c00dbe0 comp rx=0 tx=0).stop 2024-03-16T10:42:04.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 shutdown_connections 2024-03-16T10:42:04.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fa19006c880 0x7fa19006ed40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:04.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a8101570 0x7fa1a8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:04.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 --2- 172.21.15.2:0/1499058332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103ea0 0x7fa1a80ff520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:04.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 >> 172.21.15.2:0/1499058332 conn(0x7fa1a80faf80 msgr2=0x7fa1a80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:04.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 shutdown_connections 2024-03-16T10:42:04.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:04.738+0000 7fa1ad614700 1 -- 172.21.15.2:0/1499058332 wait complete. 2024-03-16T10:42:04.741 INFO:teuthology.orchestra.run.smithi002.stderr:inline data enabled 2024-03-16T10:42:04.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:04 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1499058332' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:42:05.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:04 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1499058332' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T10:42:05.364 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:42:05.374 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:42:05.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T10:42:05.606 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1499058332' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2 up:standby 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:05.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:05 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:06.229 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-03-16T10:42:06.229 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-16T10:42:06.229 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1499058332' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 2 up:standby 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:06.230 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:05 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 -- 172.21.15.2:0/3544825700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 msgr2=0x7f44441062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 --2- 172.21.15.2:0/3544825700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44441062b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4438009910 tx=0x7f4438009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 -- 172.21.15.2:0/3544825700 shutdown_connections 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 --2- 172.21.15.2:0/3544825700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44441062b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 --2- 172.21.15.2:0/3544825700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4444101590 0x7f4444103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 -- 172.21.15.2:0/3544825700 >> 172.21.15.2:0/3544825700 conn(0x7f44440fafc0 msgr2=0x7f44440fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 -- 172.21.15.2:0/3544825700 shutdown_connections 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 -- 172.21.15.2:0/3544825700 wait complete. 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 Processor -- start 2024-03-16T10:42:07.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.355+0000 7f444bcc0700 1 -- start start 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444bcc0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4444101590 0x7f4444100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444bcc0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44440ff540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444bcc0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440ffa80 con 0x7f4444103ec0 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444bcc0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440ffbc0 con 0x7f4444101590 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444925b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44440ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444925b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44440ff540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58794/0 (socket says 172.21.15.2:58794) 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444925b700 1 -- 172.21.15.2:0/1326459793 learned_addr learned my addr 172.21.15.2:0/1326459793 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f4449a5c700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4444101590 0x7f4444100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444925b700 1 -- 172.21.15.2:0/1326459793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4444101590 msgr2=0x7f4444100ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444925b700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4444101590 0x7f4444100ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444925b700 1 -- 172.21.15.2:0/1326459793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44380095f0 con 0x7f4444103ec0 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f444925b700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44440ff540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f443800b800 tx=0x7f4438005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.356+0000 7f4442ffd700 1 -- 172.21.15.2:0/1326459793 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f443801d070 con 0x7f4444103ec0 2024-03-16T10:42:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.357+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44440ffe40 con 0x7f4444103ec0 2024-03-16T10:42:07.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.357+0000 7f4442ffd700 1 -- 172.21.15.2:0/1326459793 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44380052a0 con 0x7f4444103ec0 2024-03-16T10:42:07.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.357+0000 7f4442ffd700 1 -- 172.21.15.2:0/1326459793 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4438017400 con 0x7f4444103ec0 2024-03-16T10:42:07.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.357+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44441003b0 con 0x7f4444103ec0 2024-03-16T10:42:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.358+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444404ea80 con 0x7f4444103ec0 2024-03-16T10:42:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.358+0000 7f4442ffd700 1 -- 172.21.15.2:0/1326459793 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f4438003780 con 0x7f4444103ec0 2024-03-16T10:42:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.359+0000 7f4442ffd700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f442c06c880 0x7f442c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.359+0000 7f4449a5c700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f442c06c880 0x7f442c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.359+0000 7f4442ffd700 1 -- 172.21.15.2:0/1326459793 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f443808cfb0 con 0x7f4444103ec0 2024-03-16T10:42:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.360+0000 7f4449a5c700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f442c06c880 0x7f442c06ed40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f443400f540 tx=0x7f443400b040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:07.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.365+0000 7f4442ffd700 1 -- 172.21.15.2:0/1326459793 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f443805aff0 con 0x7f4444103ec0 2024-03-16T10:42:07.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:boot 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:standby 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: Dropping low affinity active daemon mds.cephfs.smithi002.qvbseu in favor of higher affinity standby. 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: Replacing daemon mds.cephfs.smithi002.qvbseu as rank 0 with standby daemon mds.cephfs.smithi026.dzzagf 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 3 up:standby 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: osdmap e47: 6 total, 6 up, 6 in 2024-03-16T10:42:07.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:07 smithi026 ceph-mon[37041]: fsmap cephfs:1/1 {0=cephfs.smithi026.dzzagf=up:replay} 2 up:standby 2024-03-16T10:42:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-03-16T10:42:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:boot 2024-03-16T10:42:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:standby 2024-03-16T10:42:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: Dropping low affinity active daemon mds.cephfs.smithi002.qvbseu in favor of higher affinity standby. 2024-03-16T10:42:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: Replacing daemon mds.cephfs.smithi002.qvbseu as rank 0 with standby daemon mds.cephfs.smithi026.dzzagf 2024-03-16T10:42:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:42:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 3 up:standby 2024-03-16T10:42:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:42:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: osdmap e47: 6 total, 6 up, 6 in 2024-03-16T10:42:07.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:07 smithi002 ceph-mon[29248]: fsmap cephfs:1/1 {0=cephfs.smithi026.dzzagf=up:replay} 2 up:standby 2024-03-16T10:42:07.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.621+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f44441003b0 con 0x7f4444103ec0 2024-03-16T10:42:07.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.621+0000 7f4442ffd700 1 -- 172.21.15.2:0/1326459793 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1831 (secure 0 0 0) 0x7f44441003b0 con 0x7f4444103ec0 2024-03-16T10:42:07.622 INFO:teuthology.orchestra.run.smithi002.stdout:e14 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 14 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:07.212494+0000 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:42:07.623 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:42:07.624 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:reconnect seq 4 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:07.625 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:07.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.623+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f442c06c880 msgr2=0x7f442c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:07.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.623+0000 7f444bcc0700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f442c06c880 0x7f442c06ed40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f443400f540 tx=0x7f443400b040 comp rx=0 tx=0).stop 2024-03-16T10:42:07.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.623+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 msgr2=0x7f44440ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:07.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.623+0000 7f444bcc0700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44440ff540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f443800b800 tx=0x7f4438005f00 comp rx=0 tx=0).stop 2024-03-16T10:42:07.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.624+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 shutdown_connections 2024-03-16T10:42:07.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.624+0000 7f444bcc0700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f442c06c880 0x7f442c06ed40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:07.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.624+0000 7f444bcc0700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4444101590 0x7f4444100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:07.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.624+0000 7f444bcc0700 1 --2- 172.21.15.2:0/1326459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4444103ec0 0x7f44440ff540 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:07.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.624+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 >> 172.21.15.2:0/1326459793 conn(0x7f44440fafc0 msgr2=0x7f44440fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:07.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.624+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 shutdown_connections 2024-03-16T10:42:07.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:07.624+0000 7f444bcc0700 1 -- 172.21.15.2:0/1326459793 wait complete. 2024-03-16T10:42:07.627 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 14 2024-03-16T10:42:08.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:reconnect 2024-03-16T10:42:08.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] up:boot 2024-03-16T10:42:08.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] up:standby 2024-03-16T10:42:08.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: fsmap cephfs:1/1 {0=cephfs.smithi026.dzzagf=up:reconnect} 3 up:standby 2024-03-16T10:42:08.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:42:08.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1326459793' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:42:08.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:08.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:08 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:08.466 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-16T10:42:08.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:reconnect 2024-03-16T10:42:08.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] up:boot 2024-03-16T10:42:08.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] up:standby 2024-03-16T10:42:08.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: fsmap cephfs:1/1 {0=cephfs.smithi026.dzzagf=up:reconnect} 3 up:standby 2024-03-16T10:42:08.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:42:08.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1326459793' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:42:08.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:08.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:08 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:08.823 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:09.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:09 smithi026 ceph-mon[37041]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 307 B/s wr, 1 op/s 2024-03-16T10:42:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:09 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:rejoin 2024-03-16T10:42:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:09 smithi026 ceph-mon[37041]: fsmap cephfs:1/1 {0=cephfs.smithi026.dzzagf=up:rejoin} 3 up:standby 2024-03-16T10:42:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:09 smithi026 ceph-mon[37041]: daemon mds.cephfs.smithi026.dzzagf is now active in filesystem cephfs as rank 0 2024-03-16T10:42:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:09 smithi026 ceph-mon[37041]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:42:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:09 smithi002 ceph-mon[29248]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 307 B/s wr, 1 op/s 2024-03-16T10:42:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:09 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:rejoin 2024-03-16T10:42:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:09 smithi002 ceph-mon[29248]: fsmap cephfs:1/1 {0=cephfs.smithi026.dzzagf=up:rejoin} 3 up:standby 2024-03-16T10:42:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:09 smithi002 ceph-mon[29248]: daemon mds.cephfs.smithi026.dzzagf is now active in filesystem cephfs as rank 0 2024-03-16T10:42:09.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:09 smithi002 ceph-mon[29248]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:42:10.397 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:10 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:active 2024-03-16T10:42:10.397 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:10 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:42:10.397 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:10 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 -- 172.21.15.2:0/633765462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 msgr2=0x7ff1f00ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 --2- 172.21.15.2:0/633765462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f00ffe00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009910 tx=0x7ff1e0009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 -- 172.21.15.2:0/633765462 shutdown_connections 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 --2- 172.21.15.2:0/633765462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f0100be0 0x7ff1f0101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 --2- 172.21.15.2:0/633765462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f00ffe00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 -- 172.21.15.2:0/633765462 >> 172.21.15.2:0/633765462 conn(0x7ff1f00faf80 msgr2=0x7ff1f00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 -- 172.21.15.2:0/633765462 shutdown_connections 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.415+0000 7ff1f5525700 1 -- 172.21.15.2:0/633765462 wait complete. 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.416+0000 7ff1f5525700 1 Processor -- start 2024-03-16T10:42:10.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.416+0000 7ff1f5525700 1 -- start start 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.416+0000 7ff1f5525700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f01953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.416+0000 7ff1f5525700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f0100be0 0x7ff1f0195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.416+0000 7ff1f5525700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0195ea0 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.416+0000 7ff1f5525700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0195fe0 con 0x7ff1f0100be0 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.416+0000 7ff1eeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f01953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1eeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f01953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58808/0 (socket says 172.21.15.2:58808) 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1eeffd700 1 -- 172.21.15.2:0/3911334022 learned_addr learned my addr 172.21.15.2:0/3911334022 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1ee7fc700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f0100be0 0x7ff1f0195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1eeffd700 1 -- 172.21.15.2:0/3911334022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f0100be0 msgr2=0x7ff1f0195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1eeffd700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f0100be0 0x7ff1f0195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:10.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1eeffd700 1 -- 172.21.15.2:0/3911334022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00095f0 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1eeffd700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f01953d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00059d0 tx=0x7ff1e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1d7fff700 1 -- 172.21.15.2:0/3911334022 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1e001d070 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1d7fff700 1 -- 172.21.15.2:0/3911334022 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1e000f460 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f019a9f0 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1d7fff700 1 -- 172.21.15.2:0/3911334022 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1e0021640 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.417+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f019af30 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.418+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f018f560 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.419+0000 7ff1d7fff700 1 -- 172.21.15.2:0/3911334022 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7ff1e0003a60 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.419+0000 7ff1d7fff700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff1d806c880 0x7ff1d806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.419+0000 7ff1d7fff700 1 -- 172.21.15.2:0/3911334022 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff1e008d000 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.419+0000 7ff1ee7fc700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff1d806c880 0x7ff1d806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.419+0000 7ff1ee7fc700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff1d806c880 0x7ff1d806ed40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400ac50 tx=0x7ff1e400a380 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:10.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.424+0000 7ff1d7fff700 1 -- 172.21.15.2:0/3911334022 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff1e005b040 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:10 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] up:active 2024-03-16T10:42:10.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:10 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:42:10.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:10 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:10.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.672+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1f002cf60 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.673+0000 7ff1d7fff700 1 -- 172.21.15.2:0/3911334022 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4819 (secure 0 0 0) 0x7ff1e005abd0 con 0x7ff1f00ff9e0 2024-03-16T10:42:10.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff1d806c880 msgr2=0x7ff1d806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff1d806c880 0x7ff1d806ed40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400ac50 tx=0x7ff1e400a380 comp rx=0 tx=0).stop 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 msgr2=0x7ff1f01953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f01953d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00059d0 tx=0x7ff1e000bd70 comp rx=0 tx=0).stop 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 shutdown_connections 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff1d806c880 0x7ff1d806ed40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1f00ff9e0 0x7ff1f01953d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 --2- 172.21.15.2:0/3911334022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f0100be0 0x7ff1f0195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 >> 172.21.15.2:0/3911334022 conn(0x7ff1f00faf80 msgr2=0x7ff1f0104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 shutdown_connections 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:10.675+0000 7ff1f5525700 1 -- 172.21.15.2:0/3911334022 wait complete. 2024-03-16T10:42:10.676 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:42:10.686 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:42:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:11 smithi002 ceph-mon[29248]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-16T10:42:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:11 smithi002 ceph-mon[29248]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:standby 2024-03-16T10:42:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:11 smithi002 ceph-mon[29248]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:42:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:11 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3911334022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:42:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:11 smithi026 ceph-mon[37041]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-16T10:42:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:11 smithi026 ceph-mon[37041]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:standby 2024-03-16T10:42:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:11 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:42:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:11 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3911334022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:42:11.759 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-16T10:42:13.259 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:13 smithi002 ceph-mon[29248]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-16T10:42:13.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:13 smithi026 ceph-mon[37041]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-16T10:42:15.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:15 smithi002 ceph-mon[29248]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-16T10:42:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:15.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:15 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 -- 172.21.15.2:0/2612205841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 msgr2=0x7fd27c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 --2- 172.21.15.2:0/2612205841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c100f00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009910 tx=0x7fd26c009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 -- 172.21.15.2:0/2612205841 shutdown_connections 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 --2- 172.21.15.2:0/2612205841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27c101ce0 0x7fd27c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 --2- 172.21.15.2:0/2612205841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c100f00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 -- 172.21.15.2:0/2612205841 >> 172.21.15.2:0/2612205841 conn(0x7fd27c0fc0a0 msgr2=0x7fd27c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 -- 172.21.15.2:0/2612205841 shutdown_connections 2024-03-16T10:42:15.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 -- 172.21.15.2:0/2612205841 wait complete. 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.659+0000 7fd281f79700 1 Processor -- start 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd281f79700 1 -- start start 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd281f79700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd281f79700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27c101ce0 0x7fd27c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd281f79700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0736e0 con 0x7fd27c100ae0 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd281f79700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c073820 con 0x7fd27c101ce0 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd27b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:15.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd27b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46506/0 (socket says 172.21.15.2:46506) 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd27b7fe700 1 -- 172.21.15.2:0/1197039006 learned_addr learned my addr 172.21.15.2:0/1197039006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd27affd700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27c101ce0 0x7fd27c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd27b7fe700 1 -- 172.21.15.2:0/1197039006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27c101ce0 msgr2=0x7fd27c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd27b7fe700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27c101ce0 0x7fd27c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.660+0000 7fd27b7fe700 1 -- 172.21.15.2:0/1197039006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0095f0 con 0x7fd27c100ae0 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.661+0000 7fd27b7fe700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c074b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0059d0 tx=0x7fd26c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.661+0000 7fd278ff9700 1 -- 172.21.15.2:0/1197039006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26c01d070 con 0x7fd27c100ae0 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.661+0000 7fd278ff9700 1 -- 172.21.15.2:0/1197039006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd26c005dc0 con 0x7fd27c100ae0 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.661+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c073aa0 con 0x7fd27c100ae0 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.661+0000 7fd278ff9700 1 -- 172.21.15.2:0/1197039006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26c021410 con 0x7fd27c100ae0 2024-03-16T10:42:15.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.661+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c074010 con 0x7fd27c100ae0 2024-03-16T10:42:15.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.662+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c190660 con 0x7fd27c100ae0 2024-03-16T10:42:15.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.662+0000 7fd278ff9700 1 -- 172.21.15.2:0/1197039006 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fd26c003a60 con 0x7fd27c100ae0 2024-03-16T10:42:15.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.663+0000 7fd278ff9700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd26406c880 0x7fd26406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:15.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.663+0000 7fd278ff9700 1 -- 172.21.15.2:0/1197039006 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd26c08cd90 con 0x7fd27c100ae0 2024-03-16T10:42:15.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.663+0000 7fd27affd700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd26406c880 0x7fd26406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:15.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.663+0000 7fd27affd700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd26406c880 0x7fd26406ed40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd274009d10 tx=0x7fd274009040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:15.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.668+0000 7fd278ff9700 1 -- 172.21.15.2:0/1197039006 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd26c05add0 con 0x7fd27c100ae0 2024-03-16T10:42:15.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:15 smithi026 ceph-mon[37041]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-16T10:42:15.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:15.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:15 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:15.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.956+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fd27c074010 con 0x7fd27c100ae0 2024-03-16T10:42:15.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.957+0000 7fd278ff9700 1 -- 172.21.15.2:0/1197039006 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+97 (secure 0 0 0) 0x7fd27c074010 con 0x7fd27c100ae0 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd26406c880 msgr2=0x7fd26406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd26406c880 0x7fd26406ed40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd274009d10 tx=0x7fd274009040 comp rx=0 tx=0).stop 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 msgr2=0x7fd27c074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c074b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0059d0 tx=0x7fd26c00be20 comp rx=0 tx=0).stop 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 shutdown_connections 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fd26406c880 0x7fd26406ed40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c100ae0 0x7fd27c074b00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 --2- 172.21.15.2:0/1197039006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27c101ce0 0x7fd27c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 >> 172.21.15.2:0/1197039006 conn(0x7fd27c0fc0a0 msgr2=0x7fd27c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 shutdown_connections 2024-03-16T10:42:15.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:15.959+0000 7fd281f79700 1 -- 172.21.15.2:0/1197039006 wait complete. 2024-03-16T10:42:15.970 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:42:16.615 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-16T10:42:16.628 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:42:16.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:16.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:16.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:16.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:16.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:16.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:16.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:16 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1197039006' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-16T10:42:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- 172.21.15.2:0/1715359994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 msgr2=0x7f4ddc1003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 --2- 172.21.15.2:0/1715359994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc1003c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009880 tx=0x7f4dd802bce0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- 172.21.15.2:0/1715359994 shutdown_connections 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 --2- 172.21.15.2:0/1715359994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc101190 0x7f4ddc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 --2- 172.21.15.2:0/1715359994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc1003c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- 172.21.15.2:0/1715359994 >> 172.21.15.2:0/1715359994 conn(0x7f4ddc0fb530 msgr2=0x7f4ddc0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- 172.21.15.2:0/1715359994 shutdown_connections 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- 172.21.15.2:0/1715359994 wait complete. 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 Processor -- start 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- start start 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc101190 0x7f4ddc0726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc074580 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.752+0000 7f4de44af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc072bf0 con 0x7f4ddc101190 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de224b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de224b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc073fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46530/0 (socket says 172.21.15.2:46530) 2024-03-16T10:42:16.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de224b640 1 -- 172.21.15.2:0/1412191716 learned_addr learned my addr 172.21.15.2:0/1412191716 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de1a4a640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc101190 0x7f4ddc0726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de224b640 1 -- 172.21.15.2:0/1412191716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc101190 msgr2=0x7f4ddc0726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de224b640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc101190 0x7f4ddc0726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de224b640 1 -- 172.21.15.2:0/1412191716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8009560 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de224b640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc073fb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8000c00 tx=0x7f4dd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4dd37fe640 1 -- 172.21.15.2:0/1412191716 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd80037c0 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc072e70 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4dd37fe640 1 -- 172.21.15.2:0/1412191716 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dd8003de0 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4dd37fe640 1 -- 172.21.15.2:0/1412191716 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dd8034670 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.753+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc0733f0 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.754+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc04e030 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.755+0000 7f4dd37fe640 1 -- 172.21.15.2:0/1412191716 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f4dd8004dd0 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.755+0000 7f4dd37fe640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4dc806c7f0 0x7f4dc806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:16.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.755+0000 7f4dd37fe640 1 -- 172.21.15.2:0/1412191716 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4dd80ae210 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.756+0000 7f4de1a4a640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4dc806c7f0 0x7f4dc806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:16.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.756+0000 7f4de1a4a640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4dc806c7f0 0x7f4dc806ecb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc073d20 tx=0x7f4dcc00a2d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:16.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.760+0000 7f4dd37fe640 1 -- 172.21.15.2:0/1412191716 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4dd80787d0 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.948+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ddc18fb10 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.949+0000 7f4dd37fe640 1 -- 172.21.15.2:0/1412191716 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15027 (secure 0 0 0) 0x7f4dd807bdf0 con 0x7f4ddc0fffa0 2024-03-16T10:42:16.950 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:16.951 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":47,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","created":"2024-03-16T10:34:41.302807+0000","modified":"2024-03-16T10:42:06.208291+0000","last_up_change":"2024-03-16T10:40:34.269636+0000","last_in_change":"2024-03-16T10:40:14.935894+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:39:18.453991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T10:41:48.797183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T10:41:49.583833+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4befd156-b58c-44e4-bc1e-f16112b3df27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6803","nonce":702078877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6805","nonce":702078877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6809","nonce":702078877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":702078877},{"type":"v1","addr":"172.21.15.2:6807","nonce":702078877}]},"public_addr":"172.21.15.2:6803/702078877","cluster_addr":"172.21.15.2:6805/702078877","heartbeat_back_addr":"172.21.15.2:6809/702078877","heartbeat_front_addr":"172.21.15.2:6807/702078877","state":["exists","up"]},{"osd":1,"uuid":"95448bdc-721b-45c0-8c58-0b416e188aa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6811","nonce":1466336624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6813","nonce":1466336624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6817","nonce":1466336624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1466336624},{"type":"v1","addr":"172.21.15.2:6815","nonce":1466336624}]},"public_addr":"172.21.15.2:6811/1466336624","cluster_addr":"172.21.15.2:6813/1466336624","heartbeat_back_addr":"172.21.15.2:6817/1466336624","heartbeat_front_addr":"172.21.15.2:6815/1466336624","state":["exists","up"]},{"osd":2,"uuid":"ce6a7a3f-db89-426c-bbb4-1339c5adac81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6819","nonce":869328850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6821","nonce":869328850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6825","nonce":869328850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":869328850},{"type":"v1","addr":"172.21.15.2:6823","nonce":869328850}]},"public_addr":"172.21.15.2:6819/869328850","cluster_addr":"172.21.15.2:6821/869328850","heartbeat_back_addr":"172.21.15.2:6825/869328850","heartbeat_front_addr":"172.21.15.2:6823/869328850","state":["exists","up"]},{"osd":3,"uuid":"8d760311-5fda-425b-ac3c-5045ecb1cbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6801","nonce":4202679460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6803","nonce":4202679460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6807","nonce":4202679460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":4202679460},{"type":"v1","addr":"172.21.15.26:6805","nonce":4202679460}]},"public_addr":"172.21.15.26:6801/4202679460","cluster_addr":"172.21.15.26:6803/4202679460","heartbeat_back_addr":"172.21.15.26:6807/4202679460","heartbeat_front_addr":"172.21.15.26:6805/4202679460","state":["exists","up"]},{"osd":4,"uuid":"250a3093-6818-4f4f-85d5-d1f2e4381afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6809","nonce":412631433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6811","nonce":412631433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6815","nonce":412631433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":412631433},{"type":"v1","addr":"172.21.15.26:6813","nonce":412631433}]},"public_addr":"172.21.15.26:6809/412631433","cluster_addr":"172.21.15.26:6811/412631433","heartbeat_back_addr":"172.21.15.26:6815/412631433","heartbeat_front_addr":"172.21.15.26:6813/412631433","state":["exists","up"]},{"osd":5,"uuid":"b16f392a-1d2f-47b5-b356-cab94bc834cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6817","nonce":3448963017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6819","nonce":3448963017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6823","nonce":3448963017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3448963017},{"type":"v1","addr":"172.21.15.26:6821","nonce":3448963017}]},"public_addr":"172.21.15.26:6817/3448963017","cluster_addr":"172.21.15.26:6819/3448963017","heartbeat_back_addr":"172.21.15.26:6823/3448963017","heartbeat_front_addr":"172.21.15.26:6821/3448963017","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:18.975873+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:47.618973+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:15.584615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:39.921213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:06.286712+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:32.567112+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2322904669":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3421178356":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1498322259":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/3462160730":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/2565089320":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6800/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/1144469616":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6801/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6827/2835000958":"2024-03-17T10:42:06.208262+0000","172.21.15.2:6800/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/2348229359":"2024-03-17T10:35:51.598930+0000","172.21.15.2:0/1789736475":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1971862470":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6826/2835000958":"2024-03-17T10:42:06.208262+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4dc806c7f0 msgr2=0x7f4dc806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4dc806c7f0 0x7f4dc806ecb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc073d20 tx=0x7f4dcc00a2d0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 msgr2=0x7f4ddc073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc073fb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8000c00 tx=0x7f4dd8004270 comp rx=0 tx=0).stop 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 shutdown_connections 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4dc806c7f0 0x7f4dc806ecb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc101190 0x7f4ddc0726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 --2- 172.21.15.2:0/1412191716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ddc0fffa0 0x7f4ddc073fb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 >> 172.21.15.2:0/1412191716 conn(0x7f4ddc0fb530 msgr2=0x7f4ddc1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 shutdown_connections 2024-03-16T10:42:16.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:16.951+0000 7f4de44af640 1 -- 172.21.15.2:0/1412191716 wait complete. 2024-03-16T10:42:16.962 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:16.962 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:16.962 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:16.962 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:16.962 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:16.962 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:16.963 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:16 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1197039006' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-16T10:42:16.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:39:18.453991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T10:41:48.797183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T10:41:49.583833+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:42:16.964 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:42:17.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- 172.21.15.2:0/3972572701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 msgr2=0x7f6468102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 --2- 172.21.15.2:0/3972572701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468102e20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f64580098b0 tx=0x7f645802bcd0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- 172.21.15.2:0/3972572701 shutdown_connections 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 --2- 172.21.15.2:0/3972572701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6468103360 0x7f6468105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 --2- 172.21.15.2:0/3972572701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468102e20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- 172.21.15.2:0/3972572701 >> 172.21.15.2:0/3972572701 conn(0x7f64680fa440 msgr2=0x7f64680fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- 172.21.15.2:0/3972572701 shutdown_connections 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- 172.21.15.2:0/3972572701 wait complete. 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 Processor -- start 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- start start 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6468103360 0x7f64681960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64681966b0 con 0x7f6468100a30 2024-03-16T10:42:17.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.072+0000 7f646fbf0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6468196820 con 0x7f6468103360 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d98c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d98c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46556/0 (socket says 172.21.15.2:46556) 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d98c640 1 -- 172.21.15.2:0/222960109 learned_addr learned my addr 172.21.15.2:0/222960109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d18b640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6468103360 0x7f64681960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d98c640 1 -- 172.21.15.2:0/222960109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6468103360 msgr2=0x7f64681960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d98c640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6468103360 0x7f64681960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d98c640 1 -- 172.21.15.2:0/222960109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458009560 con 0x7f6468100a30 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646d98c640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468195ba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6458009430 tx=0x7f64580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f645effd640 1 -- 172.21.15.2:0/222960109 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64580039d0 con 0x7f6468100a30 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646819b1b0 con 0x7f6468100a30 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f645effd640 1 -- 172.21.15.2:0/222960109 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64580044e0 con 0x7f6468100a30 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f645effd640 1 -- 172.21.15.2:0/222960109 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6458034440 con 0x7f6468100a30 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.073+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646819b730 con 0x7f6468100a30 2024-03-16T10:42:17.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.074+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6468108bd0 con 0x7f6468100a30 2024-03-16T10:42:17.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.075+0000 7f645effd640 1 -- 172.21.15.2:0/222960109 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f6458004ff0 con 0x7f6468100a30 2024-03-16T10:42:17.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.075+0000 7f645effd640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f645406c800 0x7f645406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.075+0000 7f645effd640 1 -- 172.21.15.2:0/222960109 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f64580ae270 con 0x7f6468100a30 2024-03-16T10:42:17.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.075+0000 7f646d18b640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f645406c800 0x7f645406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.076+0000 7f646d18b640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f645406c800 0x7f645406ecc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6464005fd0 tx=0x7f6464005e20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:17.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.083+0000 7f645effd640 1 -- 172.21.15.2:0/222960109 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6458078820 con 0x7f6468100a30 2024-03-16T10:42:17.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.267+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f646819b950 con 0x7f6468100a30 2024-03-16T10:42:17.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.268+0000 7f645effd640 1 -- 172.21.15.2:0/222960109 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f646819b950 con 0x7f6468100a30 2024-03-16T10:42:17.269 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f645406c800 msgr2=0x7f645406ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f645406c800 0x7f645406ecc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6464005fd0 tx=0x7f6464005e20 comp rx=0 tx=0).stop 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 msgr2=0x7f6468195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468195ba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6458009430 tx=0x7f64580042f0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 shutdown_connections 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f645406c800 0x7f645406ecc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6468103360 0x7f64681960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 --2- 172.21.15.2:0/222960109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6468100a30 0x7f6468195ba0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 >> 172.21.15.2:0/222960109 conn(0x7f64680fa440 msgr2=0x7f64680fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:17.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 shutdown_connections 2024-03-16T10:42:17.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.270+0000 7f646fbf0640 1 -- 172.21.15.2:0/222960109 wait complete. 2024-03-16T10:42:17.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/4073291402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 msgr2=0x7ff7dc101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/4073291402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc101610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009880 tx=0x7ff7c402bce0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/4073291402 shutdown_connections 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/4073291402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc101610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/4073291402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7dc0fffa0 0x7ff7dc1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/4073291402 >> 172.21.15.2:0/4073291402 conn(0x7ff7dc0fb4f0 msgr2=0x7ff7dc0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/4073291402 shutdown_connections 2024-03-16T10:42:17.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/4073291402 wait complete. 2024-03-16T10:42:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 Processor -- start 2024-03-16T10:42:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.385+0000 7ff7e0ff0640 1 -- start start 2024-03-16T10:42:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7e0ff0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7dc0fffa0 0x7ff7dc195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7e0ff0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7e0ff0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc1965b0 con 0x7ff7dc101190 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7e0ff0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc196720 con 0x7ff7dc0fffa0 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7da59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7dc0fffa0 0x7ff7dc195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46570/0 (socket says 172.21.15.2:46570) 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d3fff640 1 -- 172.21.15.2:0/3908377098 learned_addr learned my addr 172.21.15.2:0/3908377098 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d3fff640 1 -- 172.21.15.2:0/3908377098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7dc0fffa0 msgr2=0x7ff7dc195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d3fff640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7dc0fffa0 0x7ff7dc195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d3fff640 1 -- 172.21.15.2:0/3908377098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4009560 con 0x7ff7dc101190 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7da59c640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7dc0fffa0 0x7ff7dc195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d3fff640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc195fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4036040 tx=0x7ff7c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d37fe640 1 -- 172.21.15.2:0/3908377098 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c40035f0 con 0x7ff7dc101190 2024-03-16T10:42:17.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc19b100 con 0x7ff7dc101190 2024-03-16T10:42:17.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d37fe640 1 -- 172.21.15.2:0/3908377098 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7c4003c10 con 0x7ff7dc101190 2024-03-16T10:42:17.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7d37fe640 1 -- 172.21.15.2:0/3908377098 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c4034440 con 0x7ff7dc101190 2024-03-16T10:42:17.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.386+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc19b680 con 0x7ff7dc101190 2024-03-16T10:42:17.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.388+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc18fb10 con 0x7ff7dc101190 2024-03-16T10:42:17.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.388+0000 7ff7d37fe640 1 -- 172.21.15.2:0/3908377098 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7ff7c4004ff0 con 0x7ff7dc101190 2024-03-16T10:42:17.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.388+0000 7ff7d37fe640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff7c806c7f0 0x7ff7c806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.388+0000 7ff7d37fe640 1 -- 172.21.15.2:0/3908377098 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff7c40ae080 con 0x7ff7dc101190 2024-03-16T10:42:17.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.388+0000 7ff7da59c640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff7c806c7f0 0x7ff7c806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.389+0000 7ff7da59c640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff7c806c7f0 0x7ff7c806ecb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc100ff0 tx=0x7ff7cc00b390 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:17.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.393+0000 7ff7d37fe640 1 -- 172.21.15.2:0/3908377098 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff7c40786e0 con 0x7ff7dc101190 2024-03-16T10:42:17.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.586+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff7dc108b80 con 0x7ff7dc101190 2024-03-16T10:42:17.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.586+0000 7ff7d37fe640 1 -- 172.21.15.2:0/3908377098 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff7c407bd00 con 0x7ff7dc101190 2024-03-16T10:42:17.587 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff7c806c7f0 msgr2=0x7ff7c806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff7c806c7f0 0x7ff7c806ecb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc100ff0 tx=0x7ff7cc00b390 comp rx=0 tx=0).stop 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 msgr2=0x7ff7dc195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc195fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4036040 tx=0x7ff7c4004040 comp rx=0 tx=0).stop 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 shutdown_connections 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7dc101190 0x7ff7dc195fe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7ff7c806c7f0 0x7ff7c806ecb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 --2- 172.21.15.2:0/3908377098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7dc0fffa0 0x7ff7dc195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 >> 172.21.15.2:0/3908377098 conn(0x7ff7dc0fb4f0 msgr2=0x7ff7dc1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:17.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 shutdown_connections 2024-03-16T10:42:17.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.588+0000 7ff7e0ff0640 1 -- 172.21.15.2:0/3908377098 wait complete. 2024-03-16T10:42:17.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T10:42:17.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 -- 172.21.15.2:0/1713457305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 msgr2=0x7f99a00fec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1713457305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a00fec30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9998009c20 tx=0x7f999802c1b0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 -- 172.21.15.2:0/1713457305 shutdown_connections 2024-03-16T10:42:17.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1713457305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a00ff170 0x7f99a00ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1713457305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a00fec30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 -- 172.21.15.2:0/1713457305 >> 172.21.15.2:0/1713457305 conn(0x7f99a00fa430 msgr2=0x7f99a00fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 -- 172.21.15.2:0/1713457305 shutdown_connections 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.709+0000 7f99a6fae640 1 -- 172.21.15.2:0/1713457305 wait complete. 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a6fae640 1 Processor -- start 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a6fae640 1 -- start start 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a6fae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a0197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a6fae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a00ff170 0x7f99a0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a6fae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0198840 con 0x7f99a00fe810 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a6fae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a01989b0 con 0x7f99a00ff170 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a4d4a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a0197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a4d4a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a0197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46586/0 (socket says 172.21.15.2:46586) 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a4d4a640 1 -- 172.21.15.2:0/1552031674 learned_addr learned my addr 172.21.15.2:0/1552031674 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f999ffff640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a00ff170 0x7f99a0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a4d4a640 1 -- 172.21.15.2:0/1552031674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a00ff170 msgr2=0x7f99a0198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a4d4a640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a00ff170 0x7f99a0198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f99a4d4a640 1 -- 172.21.15.2:0/1552031674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998009900 con 0x7f99a00fe810 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.710+0000 7f999ffff640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a00ff170 0x7f99a0198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.711+0000 7f99a4d4a640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a0197d30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9998000f80 tx=0x7f9998032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.711+0000 7f999dffb640 1 -- 172.21.15.2:0/1552031674 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9998034650 con 0x7f99a00fe810 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.711+0000 7f999dffb640 1 -- 172.21.15.2:0/1552031674 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9998003e30 con 0x7f99a00fe810 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.711+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a019d2f0 con 0x7f99a00fe810 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.711+0000 7f999dffb640 1 -- 172.21.15.2:0/1552031674 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99980047d0 con 0x7f99a00fe810 2024-03-16T10:42:17.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.711+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a019d870 con 0x7f99a00fe810 2024-03-16T10:42:17.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.712+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0191d10 con 0x7f99a00fe810 2024-03-16T10:42:17.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.712+0000 7f999dffb640 1 -- 172.21.15.2:0/1552031674 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f999803b040 con 0x7f99a00fe810 2024-03-16T10:42:17.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.713+0000 7f999dffb640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f998806c7f0 0x7f998806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:17.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.713+0000 7f999dffb640 1 -- 172.21.15.2:0/1552031674 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f99980aea90 con 0x7f99a00fe810 2024-03-16T10:42:17.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.713+0000 7f999ffff640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f998806c7f0 0x7f998806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:17.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.713+0000 7f999ffff640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f998806c7f0 0x7f998806ecb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9994009c20 tx=0x7f9994005f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:17.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.717+0000 7f999dffb640 1 -- 172.21.15.2:0/1552031674 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f99980790c0 con 0x7f99a00fe810 2024-03-16T10:42:17.859 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:17 smithi002 ceph-mon[29248]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-16T10:42:17.859 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:17 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1412191716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:42:17.859 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:17 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/222960109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:42:17.859 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:17 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3908377098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:42:17.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.900+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f99a010afc0 con 0x7f99a00fe810 2024-03-16T10:42:17.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.902+0000 7f999dffb640 1 -- 172.21.15.2:0/1552031674 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f999807c6e0 con 0x7f99a00fe810 2024-03-16T10:42:17.903 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f998806c7f0 msgr2=0x7f998806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f998806c7f0 0x7f998806ecb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9994009c20 tx=0x7f9994005f90 comp rx=0 tx=0).stop 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 msgr2=0x7f99a0197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a0197d30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9998000f80 tx=0x7f9998032b60 comp rx=0 tx=0).stop 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 shutdown_connections 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f998806c7f0 0x7f998806ecb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a00ff170 0x7f99a0198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 --2- 172.21.15.2:0/1552031674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99a00fe810 0x7f99a0197d30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 >> 172.21.15.2:0/1552031674 conn(0x7f99a00fa430 msgr2=0x7f99a0107300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 shutdown_connections 2024-03-16T10:42:17.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:17.905+0000 7f99a6fae640 1 -- 172.21.15.2:0/1552031674 wait complete. 2024-03-16T10:42:17.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:42:17.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:17 smithi026 ceph-mon[37041]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-16T10:42:17.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:17 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1412191716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:42:17.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:17 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/222960109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:42:17.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:17 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3908377098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 -- 172.21.15.2:0/911656058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 msgr2=0x7f9d08102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/911656058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08102e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80098b0 tx=0x7f9cf802bcd0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 -- 172.21.15.2:0/911656058 shutdown_connections 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/911656058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d08103370 0x7f9d08105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/911656058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08102e30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 -- 172.21.15.2:0/911656058 >> 172.21.15.2:0/911656058 conn(0x7f9d080fa450 msgr2=0x7f9d080fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 -- 172.21.15.2:0/911656058 shutdown_connections 2024-03-16T10:42:18.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 -- 172.21.15.2:0/911656058 wait complete. 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 Processor -- start 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.019+0000 7f9d0e81f640 1 -- start start 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d0e81f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d0e81f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d08103370 0x7f9d08196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d0e81f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08196600 con 0x7f9d08100a40 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d0e81f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08196770 con 0x7f9d08103370 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d07fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d07fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08195af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46594/0 (socket says 172.21.15.2:46594) 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d07fff640 1 -- 172.21.15.2:0/1986279053 learned_addr learned my addr 172.21.15.2:0/1986279053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:18.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d077fe640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d08103370 0x7f9d08196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d07fff640 1 -- 172.21.15.2:0/1986279053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d08103370 msgr2=0x7f9d08196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d07fff640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d08103370 0x7f9d08196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d07fff640 1 -- 172.21.15.2:0/1986279053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8009560 con 0x7f9d08100a40 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d07fff640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08195af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009430 tx=0x7f9cf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d057fa640 1 -- 172.21.15.2:0/1986279053 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cf80039d0 con 0x7f9d08100a40 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0819b150 con 0x7f9d08100a40 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d057fa640 1 -- 172.21.15.2:0/1986279053 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cf80044e0 con 0x7f9d08100a40 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d057fa640 1 -- 172.21.15.2:0/1986279053 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cf8034440 con 0x7f9d08100a40 2024-03-16T10:42:18.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.020+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0819b6d0 con 0x7f9d08100a40 2024-03-16T10:42:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.021+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d08108be0 con 0x7f9d08100a40 2024-03-16T10:42:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.022+0000 7f9d057fa640 1 -- 172.21.15.2:0/1986279053 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f9cf8004ff0 con 0x7f9d08100a40 2024-03-16T10:42:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.022+0000 7f9d057fa640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9cf006c7f0 0x7f9cf006ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.022+0000 7f9d057fa640 1 -- 172.21.15.2:0/1986279053 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9cf80ae1c0 con 0x7f9d08100a40 2024-03-16T10:42:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.022+0000 7f9d077fe640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9cf006c7f0 0x7f9cf006ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.023+0000 7f9d077fe640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9cf006c7f0 0x7f9cf006ecb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9d08196fb0 tx=0x7f9cfc00a250 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:18.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.027+0000 7f9d057fa640 1 -- 172.21.15.2:0/1986279053 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9cf8078820 con 0x7f9d08100a40 2024-03-16T10:42:18.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.250+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d08108da0 con 0x7f9d08100a40 2024-03-16T10:42:18.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.250+0000 7f9d057fa640 1 -- 172.21.15.2:0/1986279053 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4819 (secure 0 0 0) 0x7f9cf807be40 con 0x7f9d08100a40 2024-03-16T10:42:18.251 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:18.251 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14492,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.2:6829/2260596709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2260596709},{"type":"v1","addr":"172.21.15.2:6829","nonce":2260596709}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":14504,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/1847480131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1847480131},{"type":"v1","addr":"172.21.15.2:6827","nonce":1847480131}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:42:09.214374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi026.dzzagf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6825/2074466615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2074466615},{"type":"v1","addr":"172.21.15.26:6825","nonce":2074466615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-16T10:42:18.252 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:42:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9cf006c7f0 msgr2=0x7f9cf006ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9cf006c7f0 0x7f9cf006ecb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9d08196fb0 tx=0x7f9cfc00a250 comp rx=0 tx=0).stop 2024-03-16T10:42:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 msgr2=0x7f9d08195af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08195af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009430 tx=0x7f9cf80042f0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 shutdown_connections 2024-03-16T10:42:18.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9cf006c7f0 0x7f9cf006ecb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d08103370 0x7f9d08196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 --2- 172.21.15.2:0/1986279053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d08100a40 0x7f9d08195af0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:18.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 >> 172.21.15.2:0/1986279053 conn(0x7f9d080fa450 msgr2=0x7f9d080fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:18.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 shutdown_connections 2024-03-16T10:42:18.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:18.252+0000 7f9d0e81f640 1 -- 172.21.15.2:0/1986279053 wait complete. 2024-03-16T10:42:18.264 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 1, 'flags': 18} 2024-03-16T10:42:18.264 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-16T10:42:18.289 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-16T10:42:18.289 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-16T10:42:18.289 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-16T10:42:18.290 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T10:42:18.290 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:18.290 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-16T10:42:18.290 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T10:42:18.290 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:18.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:18.290 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-03-16T10:42:18.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:18.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-03-16T10:42:18.378 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-03-16T10:42:18.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:42:18.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:18.380 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-03-16T10:42:18.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:18.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-03-16T10:42:18.464 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2024-03-16T10:42:18.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:42:18.465 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-16T10:42:18.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T10:42:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:19.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:18 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:19.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:18 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1552031674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:42:19.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:18 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:19.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:18 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1986279053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:42:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:18 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:19.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:18 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1552031674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:42:19.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:18 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:19.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:18 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1986279053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:42:20.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:19 smithi002 ceph-mon[29248]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-16T10:42:20.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:19 smithi026 ceph-mon[37041]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-16T10:42:21.913 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:21 smithi002 ceph-mon[29248]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-16T10:42:22.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.017+0000 7f4f9a847700 1 -- 172.21.15.2:0/2974669978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 msgr2=0x7f4f94106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.017+0000 7f4f9a847700 1 --2- 172.21.15.2:0/2974669978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f94106e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009910 tx=0x7f4f84009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 -- 172.21.15.2:0/2974669978 shutdown_connections 2024-03-16T10:42:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 --2- 172.21.15.2:0/2974669978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f94100a70 0x7f4f94100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 --2- 172.21.15.2:0/2974669978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f94106e50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 -- 172.21.15.2:0/2974669978 >> 172.21.15.2:0/2974669978 conn(0x7f4f940fc5a0 msgr2=0x7f4f940fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 -- 172.21.15.2:0/2974669978 shutdown_connections 2024-03-16T10:42:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 -- 172.21.15.2:0/2974669978 wait complete. 2024-03-16T10:42:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 Processor -- start 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.018+0000 7f4f9a847700 1 -- start start 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f9a847700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f94100a70 0x7f4f9419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f9a847700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f9419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f9a847700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419d310 con 0x7f4f94106a70 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f9a847700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f941968f0 con 0x7f4f94100a70 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f937fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f9419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f937fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f9419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46612/0 (socket says 172.21.15.2:46612) 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f937fe700 1 -- 172.21.15.2:0/1228592347 learned_addr learned my addr 172.21.15.2:0/1228592347 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f93fff700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f94100a70 0x7f4f9419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f937fe700 1 -- 172.21.15.2:0/1228592347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f94100a70 msgr2=0x7f4f9419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f937fe700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f94100a70 0x7f4f9419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f937fe700 1 -- 172.21.15.2:0/1228592347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f840095f0 con 0x7f4f94106a70 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f937fe700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f9419ccf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b4b0 tx=0x7f4f8800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f917fa700 1 -- 172.21.15.2:0/1228592347 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f88011800 con 0x7f4f94106a70 2024-03-16T10:42:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f917fa700 1 -- 172.21.15.2:0/1228592347 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f88011e20 con 0x7f4f94106a70 2024-03-16T10:42:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f94196bd0 con 0x7f4f94106a70 2024-03-16T10:42:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f917fa700 1 -- 172.21.15.2:0/1228592347 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8800f2e0 con 0x7f4f94106a70 2024-03-16T10:42:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.019+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f94197120 con 0x7f4f94106a70 2024-03-16T10:42:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.020+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f94109070 con 0x7f4f94106a70 2024-03-16T10:42:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.021+0000 7f4f917fa700 1 -- 172.21.15.2:0/1228592347 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f4f88010380 con 0x7f4f94106a70 2024-03-16T10:42:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.021+0000 7f4f917fa700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7c06c880 0x7f4f7c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.021+0000 7f4f93fff700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7c06c880 0x7f4f7c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.021+0000 7f4f917fa700 1 -- 172.21.15.2:0/1228592347 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4f8808b6f0 con 0x7f4f94106a70 2024-03-16T10:42:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.022+0000 7f4f93fff700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7c06c880 0x7f4f7c06ed40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f8401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.028+0000 7f4f917fa700 1 -- 172.21.15.2:0/1228592347 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4f88059860 con 0x7f4f94106a70 2024-03-16T10:42:22.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:21 smithi026 ceph-mon[37041]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-16T10:42:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.299+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f941021b0 con 0x7f4f94106a70 2024-03-16T10:42:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.300+0000 7f4f917fa700 1 -- 172.21.15.2:0/1228592347 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4f880593f0 con 0x7f4f94106a70 2024-03-16T10:42:22.301 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7c06c880 msgr2=0x7f4f7c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7c06c880 0x7f4f7c06ed40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f8401a040 comp rx=0 tx=0).stop 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 msgr2=0x7f4f9419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f9419ccf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b4b0 tx=0x7f4f8800b7c0 comp rx=0 tx=0).stop 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 shutdown_connections 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f7c06c880 0x7f4f7c06ed40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f94100a70 0x7f4f9419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 --2- 172.21.15.2:0/1228592347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f94106a70 0x7f4f9419ccf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 >> 172.21.15.2:0/1228592347 conn(0x7f4f940fc5a0 msgr2=0x7f4f940fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 shutdown_connections 2024-03-16T10:42:22.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:22.302+0000 7f4f9a847700 1 -- 172.21.15.2:0/1228592347 wait complete. 2024-03-16T10:42:22.966 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T10:42:22.966 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T10:42:22.967 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-03-16T10:42:22.967 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T10:42:22.967 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:22.967 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T10:42:22.967 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T10:42:22.967 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T10:42:22.967 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T10:42:22.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:22.967 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-03-16T10:42:22.983 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T10:42:22.984 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42274sec preferred_lft 42274sec 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:22.985 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-03-16T10:42:22.985 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T10:42:22.986 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:22.986 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-16T10:42:22.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-03-16T10:42:22.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-03-16T10:42:22.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-03-16T10:42:22.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T10:42:22.986 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-16T10:42:23.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:23.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:23.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:23.199 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T10:42:23.226 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-03-16T10:42:23.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:23.228 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-03-16T10:42:23.300 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-03-16T10:42:23.300 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T10:42:23.300 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-16T10:42:23.300 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-16T10:42:23.300 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T10:42:23.301 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:23.301 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-16T10:42:23.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T10:42:23.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T10:42:23.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T10:42:23.301 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-16T10:42:23.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:23.510 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:23 smithi002 ceph-mon[29248]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 5.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-16T10:42:23.510 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:23.510 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:23.510 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:23.510 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:23.510 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:23 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:23.510 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:23 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1228592347' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:42:23.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:23.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:23.605 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-03-16T10:42:23.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:23.620 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-03-16T10:42:23.674 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:23.674 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-16T10:42:23.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:42:23.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-16T10:42:23.674 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-16T10:42:23.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:23 smithi026 ceph-mon[37041]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 5.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-16T10:42:23.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:23.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:23.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:23.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:23.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:23 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:23.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:23 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1228592347' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:42:23.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:23.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:23.796 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-16T10:42:23.797 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-16T10:42:23.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:23.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:23.950 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:23.950 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-03-16T10:42:23.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-03-16T10:42:23.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T10:42:23.951 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-03-16T10:42:23.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:24.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:24.032 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-16T10:42:24.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:42:24.032 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:24.046 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T10:42:24.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:42:24.047 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:24.100 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-03-16T10:42:24.168 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:42:24.224 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-03-16T10:42:24.224 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-03-16T10:42:24.224 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-03-16T10:42:24.224 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-03-16T10:42:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/4b73102206a896d4ed3f859b54cee8e852c83e506009da8f235ac2535a0835c6/merged 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/852c73a564c00f46b07ab75e50e1665cc56d245129a1ab36e04ca46fca407e4f/merged 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:24.226 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/a3fb9ee006f84853d95d5a3756615bfb9aab148a85502cffc0bde6a2481ab94f/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f3e4effddf7c27999b7950904b9be16ae7abe9eaa036a4ddf0c74a6e823c3daf/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/d9292463e1e6e3b43c34d6661fac2654bad3879d4e220b7e6923176a9c74ee93/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/14190dc4cd06ced56615f4cb1f6828b4c6c4a0433c6f59b9be8fb5d2fe865f89/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/318b989df57f6767e7e90510e87fc5f82450fdd4adb686c37f1bff1076a1142c/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e137698922625626ee14e1655bb7f184ea8bc9e231835b035636fa669942c564/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/5008f679de8e6ee9bcbfb8dffbe03d460cd83dab3f108d2f889756c0b9cb5a25/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/826c6e020afe786c8aa3e07615867d4dd3041f04d25f9c5cbe736ac393f65d6b/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/b7fc4248bf934f386f8abea74528fd07e395d9fc69803449deca93f5e38dd680/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/fd5c4cc023fba20d72f6f49df2fd89d064f4920226f12a6e7f9b331243ca2cff/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1f517258d839c90383e4ec8eb537ed67ac9ffcecd5d9f0596c2c976f75bf908b/merged 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:42:24.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:24.228 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-03-16T10:42:24.279 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-16T10:42:24.279 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-16T10:42:24.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2024-03-16T10:42:24.348 DEBUG:teuthology.orchestra.run.smithi002:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/proc 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/sys 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/dev 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/security 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/shm 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/pts 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/run 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/cgroup 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/pstore 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/bpf 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/ 2024-03-16T10:42:24.404 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/selinux 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/debug 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/mqueue 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/dev/hugepages 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/tracing 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/fs/fuse/connections 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/sys/kernel/config 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-sysctl.service 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/nfs/rpc_pipefs 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/1000 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/ceph 2024-03-16T10:42:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/4b73102206a896d4ed3f859b54cee8e852c83e506009da8f235ac2535a0835c6/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/852c73a564c00f46b07ab75e50e1665cc56d245129a1ab36e04ca46fca407e4f/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/run/user/0 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/a3fb9ee006f84853d95d5a3756615bfb9aab148a85502cffc0bde6a2481ab94f/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/f3e4effddf7c27999b7950904b9be16ae7abe9eaa036a4ddf0c74a6e823c3daf/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/d9292463e1e6e3b43c34d6661fac2654bad3879d4e220b7e6923176a9c74ee93/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/14190dc4cd06ced56615f4cb1f6828b4c6c4a0433c6f59b9be8fb5d2fe865f89/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/318b989df57f6767e7e90510e87fc5f82450fdd4adb686c37f1bff1076a1142c/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/e137698922625626ee14e1655bb7f184ea8bc9e231835b035636fa669942c564/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/5008f679de8e6ee9bcbfb8dffbe03d460cd83dab3f108d2f889756c0b9cb5a25/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/826c6e020afe786c8aa3e07615867d4dd3041f04d25f9c5cbe736ac393f65d6b/merged 2024-03-16T10:42:24.406 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/b7fc4248bf934f386f8abea74528fd07e395d9fc69803449deca93f5e38dd680/merged 2024-03-16T10:42:24.407 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/fd5c4cc023fba20d72f6f49df2fd89d064f4920226f12a6e7f9b331243ca2cff/merged 2024-03-16T10:42:24.407 INFO:teuthology.orchestra.run.smithi002.stdout:/var/lib/containers/storage/overlay/1f517258d839c90383e4ec8eb537ed67ac9ffcecd5d9f0596c2c976f75bf908b/merged 2024-03-16T10:42:24.407 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns 2024-03-16T10:42:24.407 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:42:24.407 INFO:teuthology.orchestra.run.smithi002.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:42:24.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:24.407 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2024-03-16T10:42:24.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:2024-03-16T10:42:24.426+0000 7f4a5d23b480 -1 init, newargv = 0x55f89a035e30 newargc=15 2024-03-16T10:42:24.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[77951]: starting ceph client 2024-03-16T10:42:24.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[77951]: starting fuse 2024-03-16T10:42:24.445 INFO:teuthology.orchestra.run.smithi002.stdout:159 2024-03-16T10:42:24.445 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [159] 2024-03-16T10:42:24.445 DEBUG:teuthology.orchestra.run.smithi002:> sudo stdin-killer -- python3 -c ' 2024-03-16T10:42:24.445 DEBUG:teuthology.orchestra.run.smithi002:> import glob 2024-03-16T10:42:24.445 DEBUG:teuthology.orchestra.run.smithi002:> import re 2024-03-16T10:42:24.445 DEBUG:teuthology.orchestra.run.smithi002:> import os 2024-03-16T10:42:24.445 DEBUG:teuthology.orchestra.run.smithi002:> import subprocess 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> def _find_admin_socket(client_name): 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> return files[0] 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> for f in files: 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-16T10:42:24.446 DEBUG:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> contents = proc_f.read() 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> if mountpoint in contents: 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> return f 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> print(_find_admin_socket("client.0")) 2024-03-16T10:42:24.447 DEBUG:teuthology.orchestra.run.smithi002:> ' 2024-03-16T10:42:24.566 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.0.77951.asok 2024-03-16T10:42:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:24.576 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77951.asok 2024-03-16T10:42:24.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:24.576 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77951.asok status 2024-03-16T10:42:24.718 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "a752c2f7898f53056d3000f2350a5865d1d67887", 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)", 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "0", 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "77951", 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/" 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 14544, 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2024-03-16T10:42:24.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 14544 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 676582552 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 676582552 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.14544 192.168.0.1:0/676582552", 2024-03-16T10:42:24.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "192.168.0.1:0/676582552", 2024-03-16T10:42:24.721 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 1, 2024-03-16T10:42:24.721 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 16, 2024-03-16T10:42:24.721 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 47, 2024-03-16T10:42:24.721 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2024-03-16T10:42:24.721 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklisted": false, 2024-03-16T10:42:24.721 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs" 2024-03-16T10:42:24.721 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:42:24.727 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T10:42:24.936 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:25.372 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:25 smithi002 ceph-mon[29248]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T10:42:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:25 smithi026 ceph-mon[37041]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-16T10:42:26.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1365069775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 msgr2=0x7fb7ec106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:26.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1365069775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec106e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009910 tx=0x7fb7dc009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1365069775 shutdown_connections 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1365069775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ec100a70 0x7fb7ec100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1365069775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec106e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1365069775 >> 172.21.15.2:0/1365069775 conn(0x7fb7ec0fc5a0 msgr2=0x7fb7ec0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1365069775 shutdown_connections 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.650+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1365069775 wait complete. 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7f29c6700 1 Processor -- start 2024-03-16T10:42:26.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7f29c6700 1 -- start start 2024-03-16T10:42:26.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7f29c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ec100a70 0x7fb7ec073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7f29c6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7f29c6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec073af0 con 0x7fb7ec106a70 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7f29c6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec073c30 con 0x7fb7ec100a70 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7eb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec0735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46708/0 (socket says 172.21.15.2:46708) 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.651+0000 7fb7eb7fe700 1 -- 172.21.15.2:0/1513971673 learned_addr learned my addr 172.21.15.2:0/1513971673 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7ebfff700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ec100a70 0x7fb7ec073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7eb7fe700 1 -- 172.21.15.2:0/1513971673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ec100a70 msgr2=0x7fb7ec073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7eb7fe700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ec100a70 0x7fb7ec073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7eb7fe700 1 -- 172.21.15.2:0/1513971673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc0095f0 con 0x7fb7ec106a70 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7eb7fe700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec0735b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b4b0 tx=0x7fb7e000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7e97fa700 1 -- 172.21.15.2:0/1513971673 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e0011800 con 0x7fb7ec106a70 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7e97fa700 1 -- 172.21.15.2:0/1513971673 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7e0011e20 con 0x7fb7ec106a70 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7e97fa700 1 -- 172.21.15.2:0/1513971673 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e000f2e0 con 0x7fb7ec106a70 2024-03-16T10:42:26.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ec1a2cb0 con 0x7fb7ec106a70 2024-03-16T10:42:26.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.652+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec1a3010 con 0x7fb7ec106a70 2024-03-16T10:42:26.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.653+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec109070 con 0x7fb7ec106a70 2024-03-16T10:42:26.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.654+0000 7fb7e97fa700 1 -- 172.21.15.2:0/1513971673 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fb7e0010380 con 0x7fb7ec106a70 2024-03-16T10:42:26.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.654+0000 7fb7e97fa700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7d406c880 0x7fb7d406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:26.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.654+0000 7fb7e97fa700 1 -- 172.21.15.2:0/1513971673 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb7e008b890 con 0x7fb7ec106a70 2024-03-16T10:42:26.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.654+0000 7fb7ebfff700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7d406c880 0x7fb7d406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:26.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.655+0000 7fb7ebfff700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7d406c880 0x7fb7d406ed40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00b5c0 tx=0x7fb7dc01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:26.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.659+0000 7fb7e97fa700 1 -- 172.21.15.2:0/1513971673 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb7e0055e40 con 0x7fb7ec106a70 2024-03-16T10:42:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.907+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb7ec1021b0 con 0x7fb7ec106a70 2024-03-16T10:42:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.907+0000 7fb7e97fa700 1 -- 172.21.15.2:0/1513971673 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7fb7e0059460 con 0x7fb7ec106a70 2024-03-16T10:42:26.908 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-16T10:42:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.909+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7d406c880 msgr2=0x7fb7d406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.909+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7d406c880 0x7fb7d406ed40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00b5c0 tx=0x7fb7dc01a040 comp rx=0 tx=0).stop 2024-03-16T10:42:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.909+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 msgr2=0x7fb7ec0735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.909+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec0735b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000b4b0 tx=0x7fb7e000b870 comp rx=0 tx=0).stop 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.910+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 shutdown_connections 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.910+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7d406c880 0x7fb7d406ed40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.910+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7ec100a70 0x7fb7ec073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.910+0000 7fb7f29c6700 1 --2- 172.21.15.2:0/1513971673 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7ec106a70 0x7fb7ec0735b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.910+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 >> 172.21.15.2:0/1513971673 conn(0x7fb7ec0fc5a0 msgr2=0x7fb7ec0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.910+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 shutdown_connections 2024-03-16T10:42:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:26.910+0000 7fb7f29c6700 1 -- 172.21.15.2:0/1513971673 wait complete. 2024-03-16T10:42:27.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:27 smithi026 ceph-mon[37041]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-03-16T10:42:27.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:27 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1513971673' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:42:27.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:27 smithi002 ceph-mon[29248]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-03-16T10:42:27.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:27 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1513971673' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T10:42:27.580 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T10:42:27.580 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T10:42:27.580 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-03-16T10:42:27.580 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-16T10:42:27.581 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:27.581 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T10:42:27.581 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T10:42:27.581 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T10:42:27.581 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-16T10:42:27.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:27.581 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:42:27.599 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42256sec preferred_lft 42256sec 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:42:27.600 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-03-16T10:42:27.601 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T10:42:27.601 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:27.601 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-16T10:42:27.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-03-16T10:42:27.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-03-16T10:42:27.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-03-16T10:42:27.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T10:42:27.601 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-16T10:42:27.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:27.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:27.825 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T10:42:27.852 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-03-16T10:42:27.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:27.854 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-03-16T10:42:27.920 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-03-16T10:42:27.920 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T10:42:27.920 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-16T10:42:27.920 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-16T10:42:27.920 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T10:42:27.920 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:27.920 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-16T10:42:27.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T10:42:27.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T10:42:27.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T10:42:27.921 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-16T10:42:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:28.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:28.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:28.230 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-03-16T10:42:28.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:28.245 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-03-16T10:42:28.299 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:28.299 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-16T10:42:28.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T10:42:28.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-16T10:42:28.299 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-16T10:42:28.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:28.441 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-16T10:42:28.441 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:28.441 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-16T10:42:28.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-16T10:42:28.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T10:42:28.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-16T10:42:28.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-16T10:42:28.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-16T10:42:28.442 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-16T10:42:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:28.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:28.588 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:42:28.588 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-03-16T10:42:28.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-03-16T10:42:28.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T10:42:28.588 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-03-16T10:42:28.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:42:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:28.670 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-16T10:42:28.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:42:28.670 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:28.685 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-16T10:42:28.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:42:28.685 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:28.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-03-16T10:42:28.807 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:42:28.863 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-03-16T10:42:28.863 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-03-16T10:42:28.863 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-03-16T10:42:28.863 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-03-16T10:42:28.863 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-03-16T10:42:28.863 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-16T10:42:28.864 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/0 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/b6115a4221403d074e6b6dc3c817b59db6a57c00b4fdb966ab1d62ba48485703/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/63d11319b0498451dfaa076a6685b5a3db28ab031cb9c0f3faba52507e166c4e/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/92af3bd2f771f7ef731ff2ddd6af5487872593a4c4d17dcbaffc0a2184c8365a/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/2d791c1d7c170cfa6e8b3bd8dc07855df09d9f72f61160359917033bd02160f8/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/784f7dcd0c4287bbeb58ff23935a1a3c9640ffbc07fe11404562f52cc319870f/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/19148c2fc2ccd9b9673e4ced3e2a681e1325fbfc4a7572ff7ed75faaf5a58b02/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1b7851bd1994c4f26ae2341bc1b5eabcc9a3fa453838cf364f3ab9a01953a009/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/fd5a0bddf328128e74b6a7b11329fc591f4482e025a425541e91cf2d61b31ad7/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/2479dedf392be4addd99cecd0e532a333f07cb3c1e0f78db22e02227f6a2d51a/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/d974ed46da69940203dcc5c1de5bec9d826cada9ae725d5624bffb99f93ac37f/merged 2024-03-16T10:42:28.865 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-03-16T10:42:28.866 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T10:42:28.866 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T10:42:28.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:28.866 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-03-16T10:42:28.917 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-16T10:42:28.918 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-16T10:42:28.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-03-16T10:42:28.986 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/bpf 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-03-16T10:42:29.042 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/0 2024-03-16T10:42:29.043 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/b6115a4221403d074e6b6dc3c817b59db6a57c00b4fdb966ab1d62ba48485703/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/63d11319b0498451dfaa076a6685b5a3db28ab031cb9c0f3faba52507e166c4e/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/92af3bd2f771f7ef731ff2ddd6af5487872593a4c4d17dcbaffc0a2184c8365a/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/2d791c1d7c170cfa6e8b3bd8dc07855df09d9f72f61160359917033bd02160f8/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/784f7dcd0c4287bbeb58ff23935a1a3c9640ffbc07fe11404562f52cc319870f/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/19148c2fc2ccd9b9673e4ced3e2a681e1325fbfc4a7572ff7ed75faaf5a58b02/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/1b7851bd1994c4f26ae2341bc1b5eabcc9a3fa453838cf364f3ab9a01953a009/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/fd5a0bddf328128e74b6a7b11329fc591f4482e025a425541e91cf2d61b31ad7/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/2479dedf392be4addd99cecd0e532a333f07cb3c1e0f78db22e02227f6a2d51a/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/containers/storage/overlay/d974ed46da69940203dcc5c1de5bec9d826cada9ae725d5624bffb99f93ac37f/merged 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T10:42:29.044 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T10:42:29.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:29.045 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-03-16T10:42:29.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi026.stderr:2024-03-16T10:42:29.067+0000 7f00f95db480 -1 init, newargv = 0x55c733827d80 newargc=15 2024-03-16T10:42:29.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi026.stderr:ceph-fuse[62980]: starting ceph client 2024-03-16T10:42:29.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi026.stderr:ceph-fuse[62980]: starting fuse 2024-03-16T10:42:29.085 INFO:teuthology.orchestra.run.smithi026.stdout:126 2024-03-16T10:42:29.085 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-03-16T10:42:29.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-03-16T10:42:29.085 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-03-16T10:42:29.085 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-03-16T10:42:29.085 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-03-16T10:42:29.085 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-03-16T10:42:29.085 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-16T10:42:29.085 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-03-16T10:42:29.086 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-03-16T10:42:29.087 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-16T10:42:29.087 DEBUG:teuthology.orchestra.run.smithi026:> 2024-03-16T10:42:29.087 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.1")) 2024-03-16T10:42:29.087 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-03-16T10:42:29.205 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.1.62980.asok 2024-03-16T10:42:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-03-16T10:42:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:42:29.215 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.62980.asok 2024-03-16T10:42:29.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:29.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.62980.asok status 2024-03-16T10:42:29.353 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-03-16T10:42:29.353 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-03-16T10:42:29.353 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "a752c2f7898f53056d3000f2350a5865d1d67887", 2024-03-16T10:42:29.353 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)", 2024-03-16T10:42:29.353 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "1", 2024-03-16T10:42:29.353 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "62980", 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24305, 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 24305 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-03-16T10:42:29.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-03-16T10:42:29.355 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:42:29.355 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3141458606 2024-03-16T10:42:29.355 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-03-16T10:42:29.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3141458606 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.24305 192.168.0.1:0/3141458606", 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/3141458606", 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 16, 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 47, 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-03-16T10:42:29.356 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-03-16T10:42:29.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:29.362 DEBUG:teuthology.orchestra.run.smithi002:> stat --file-system '--printf=%T 2024-03-16T10:42:29.362 DEBUG:teuthology.orchestra.run.smithi002:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:29.381 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2024-03-16T10:42:29.381 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:29.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:29.381 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:29.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:29.456 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-03-16T10:42:29.456 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:29.472 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-03-16T10:42:29.472 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:29.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:42:29.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:29.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:29 smithi026 ceph-mon[37041]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:42:29.510 INFO:teuthology.run_tasks:Running task print... 2024-03-16T10:42:29.520 INFO:teuthology.task.print:**** done client 2024-03-16T10:42:29.521 INFO:teuthology.run_tasks:Running task parallel... 2024-03-16T10:42:29.531 INFO:teuthology.task.parallel:starting parallel... 2024-03-16T10:42:29.531 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-16T10:42:29.532 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-16T10:42:29.532 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:42:29.532 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-03-16T10:42:29.533 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-16T10:42:29.533 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-16T10:42:29.536 INFO:tasks.workunit:Pulling workunits from ref 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:29.537 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-16T10:42:29.537 INFO:tasks.workunit:timeout=3h 2024-03-16T10:42:29.537 INFO:tasks.workunit:cleanup=True 2024-03-16T10:42:29.537 DEBUG:teuthology.orchestra.run.smithi002:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:29.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:29 smithi002 ceph-mon[29248]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 9fh/159d Inode: 1 Links: 2 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-03-16 10:41:55.366669150 +0000 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-03-16 10:42:29.453496134 +0000 2024-03-16T10:42:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2024-03-16T10:42:29.558 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-16T10:42:29.558 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-16T10:42:29.631 DEBUG:teuthology.orchestra.run.smithi026:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:29.647 INFO:teuthology.orchestra.run.smithi026.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:29.647 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-16T10:42:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 7eh/126d Inode: 1 Links: 3 2024-03-16T10:42:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:42:29.647 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-16T10:42:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-16T10:42:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-03-16 10:42:29.626231693 +0000 2024-03-16T10:42:29.648 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-03-16 10:42:29.626231693 +0000 2024-03-16T10:42:29.648 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-03-16T10:42:29.648 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-16T10:42:29.648 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-16T10:42:29.719 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:29.719 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:29.735 INFO:tasks.workunit.client.0.smithi002.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-16T10:42:29.759 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:29.774 INFO:tasks.workunit.client.1.smithi026.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-16T10:42:30.423 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:30 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:30.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:30 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:31.527 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:31 smithi002 ceph-mon[29248]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:42:31.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 -- 172.21.15.2:0/1271854343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04100ad0 msgr2=0x7f8c04100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:31.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 --2- 172.21.15.2:0/1271854343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04100ad0 0x7f8c04100ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009910 tx=0x7f8bf4009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:31.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 -- 172.21.15.2:0/1271854343 shutdown_connections 2024-03-16T10:42:31.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 --2- 172.21.15.2:0/1271854343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c04101cd0 0x7f8c04102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:31.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 --2- 172.21.15.2:0/1271854343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04100ad0 0x7f8c04100ef0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:31.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 -- 172.21.15.2:0/1271854343 >> 172.21.15.2:0/1271854343 conn(0x7f8c040fc090 msgr2=0x7f8c040fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:31.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 -- 172.21.15.2:0/1271854343 shutdown_connections 2024-03-16T10:42:31.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.530+0000 7f8c0a515700 1 -- 172.21.15.2:0/1271854343 wait complete. 2024-03-16T10:42:31.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c0a515700 1 Processor -- start 2024-03-16T10:42:31.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c0a515700 1 -- start start 2024-03-16T10:42:31.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c0a515700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c04100ad0 0x7f8c041964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:31.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c0a515700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04101cd0 0x7f8c04196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:31.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c0a515700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04197050 con 0x7f8c04101cd0 2024-03-16T10:42:31.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c0a515700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04197190 con 0x7f8c04100ad0 2024-03-16T10:42:31.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04101cd0 0x7f8c04196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04101cd0 0x7f8c04196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46720/0 (socket says 172.21.15.2:46720) 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c037fe700 1 -- 172.21.15.2:0/2301273722 learned_addr learned my addr 172.21.15.2:0/2301273722 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c03fff700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c04100ad0 0x7f8c041964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c037fe700 1 -- 172.21.15.2:0/2301273722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c04100ad0 msgr2=0x7f8c041964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c037fe700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c04100ad0 0x7f8c041964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.531+0000 7f8c037fe700 1 -- 172.21.15.2:0/2301273722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40095f0 con 0x7f8c04101cd0 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.532+0000 7f8c037fe700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04101cd0 0x7f8c04196a30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d8d0 tx=0x7f8bf800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.532+0000 7f8c017fa700 1 -- 172.21.15.2:0/2301273722 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf80098a0 con 0x7f8c04101cd0 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.532+0000 7f8c017fa700 1 -- 172.21.15.2:0/2301273722 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bf8010460 con 0x7f8c04101cd0 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.532+0000 7f8c017fa700 1 -- 172.21.15.2:0/2301273722 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf800f5f0 con 0x7f8c04101cd0 2024-03-16T10:42:31.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.532+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0419bbf0 con 0x7f8c04101cd0 2024-03-16T10:42:31.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.532+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0419c1c0 con 0x7f8c04101cd0 2024-03-16T10:42:31.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.533+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c04190650 con 0x7f8c04101cd0 2024-03-16T10:42:31.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.533+0000 7f8c017fa700 1 -- 172.21.15.2:0/2301273722 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f8bf8009a00 con 0x7f8c04101cd0 2024-03-16T10:42:31.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.534+0000 7f8c017fa700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8bec06c880 0x7f8bec06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:31.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.534+0000 7f8c017fa700 1 -- 172.21.15.2:0/2301273722 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8bf808bba0 con 0x7f8c04101cd0 2024-03-16T10:42:31.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.534+0000 7f8c03fff700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8bec06c880 0x7f8bec06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:31.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.534+0000 7f8c03fff700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8bec06c880 0x7f8bec06ed40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4000c00 tx=0x7f8bf4005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:31.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.538+0000 7f8c017fa700 1 -- 172.21.15.2:0/2301273722 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8bf8059be0 con 0x7f8c04101cd0 2024-03-16T10:42:31.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:31 smithi026 ceph-mon[37041]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:42:31.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.740+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f8c0404ea80 con 0x7f8c04101cd0 2024-03-16T10:42:31.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.744+0000 7f8c017fa700 1 -- 172.21.15.2:0/2301273722 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v22)=0 v22) v1 ==== 126+0+0 (secure 0 0 0) 0x7f8bf8059770 con 0x7f8c04101cd0 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8bec06c880 msgr2=0x7f8bec06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8bec06c880 0x7f8bec06ed40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4000c00 tx=0x7f8bf4005960 comp rx=0 tx=0).stop 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04101cd0 msgr2=0x7f8c04196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04101cd0 0x7f8c04196a30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800d8d0 tx=0x7f8bf800dbe0 comp rx=0 tx=0).stop 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 shutdown_connections 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8bec06c880 0x7f8bec06ed40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c04100ad0 0x7f8c041964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:31.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 --2- 172.21.15.2:0/2301273722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c04101cd0 0x7f8c04196a30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 >> 172.21.15.2:0/2301273722 conn(0x7f8c040fc090 msgr2=0x7f8c041052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 shutdown_connections 2024-03-16T10:42:31.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:31.747+0000 7f8c0a515700 1 -- 172.21.15.2:0/2301273722 wait complete. 2024-03-16T10:42:32.419 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-16T10:42:32.420 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:42:32.420 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-16T10:42:32.671 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:33.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:32 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2301273722' entity='client.admin' 2024-03-16T10:42:33.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:33.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:33.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:33.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:32 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:32 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2301273722' entity='client.admin' 2024-03-16T10:42:33.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:33.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:33.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:33.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:32 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:33.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:33 smithi002 ceph-mon[29248]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:42:34.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:33 smithi026 ceph-mon[37041]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 -- 172.21.15.2:0/1538491005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 msgr2=0x7f235c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 --2- 172.21.15.2:0/1538491005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c102140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2344009910 tx=0x7f2344009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 -- 172.21.15.2:0/1538491005 shutdown_connections 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 --2- 172.21.15.2:0/1538491005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c102140 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 --2- 172.21.15.2:0/1538491005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f235c100ae0 0x7f235c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 -- 172.21.15.2:0/1538491005 >> 172.21.15.2:0/1538491005 conn(0x7f235c0fc0a0 msgr2=0x7f235c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 -- 172.21.15.2:0/1538491005 shutdown_connections 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 -- 172.21.15.2:0/1538491005 wait complete. 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.381+0000 7f2362979700 1 Processor -- start 2024-03-16T10:42:34.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2362979700 1 -- start start 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2362979700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f235c100ae0 0x7f235c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2362979700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2362979700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c197130 con 0x7f235c101ce0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2362979700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c197270 con 0x7f235c100ae0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2353fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2353fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55226/0 (socket says 172.21.15.2:55226) 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2353fff700 1 -- 172.21.15.2:0/2562174872 learned_addr learned my addr 172.21.15.2:0/2562174872 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f235bfff700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f235c100ae0 0x7f235c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2353fff700 1 -- 172.21.15.2:0/2562174872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f235c100ae0 msgr2=0x7f235c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2353fff700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f235c100ae0 0x7f235c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.382+0000 7f2353fff700 1 -- 172.21.15.2:0/2562174872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440095f0 con 0x7f235c101ce0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.383+0000 7f2353fff700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c196b10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23440092e0 tx=0x7f234400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.383+0000 7f2359ffb700 1 -- 172.21.15.2:0/2562174872 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234401d070 con 0x7f235c101ce0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.383+0000 7f2359ffb700 1 -- 172.21.15.2:0/2562174872 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f234400f460 con 0x7f235c101ce0 2024-03-16T10:42:34.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.383+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c19bc70 con 0x7f235c101ce0 2024-03-16T10:42:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.383+0000 7f2359ffb700 1 -- 172.21.15.2:0/2562174872 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2344021640 con 0x7f235c101ce0 2024-03-16T10:42:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.383+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c19c1e0 con 0x7f235c101ce0 2024-03-16T10:42:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.384+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c190700 con 0x7f235c101ce0 2024-03-16T10:42:34.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.385+0000 7f2359ffb700 1 -- 172.21.15.2:0/2562174872 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f234400f5d0 con 0x7f235c101ce0 2024-03-16T10:42:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.385+0000 7f2359ffb700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f234806c880 0x7f234806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.385+0000 7f2359ffb700 1 -- 172.21.15.2:0/2562174872 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f234408cf20 con 0x7f235c101ce0 2024-03-16T10:42:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.385+0000 7f235bfff700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f234806c880 0x7f234806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:34.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.386+0000 7f235bfff700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f234806c880 0x7f234806ed40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f235c101b40 tx=0x7f234c009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:34.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.391+0000 7f2359ffb700 1 -- 172.21.15.2:0/2562174872 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f23440c09f0 con 0x7f235c101ce0 2024-03-16T10:42:34.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.590+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f235c04ea80 con 0x7f235c101ce0 2024-03-16T10:42:34.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.590+0000 7f2359ffb700 1 -- 172.21.15.2:0/2562174872 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v22)=0 v22) v1 ==== 155+0+0 (secure 0 0 0) 0x7f234405afe0 con 0x7f235c101ce0 2024-03-16T10:42:34.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f234806c880 msgr2=0x7f234806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:34.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f234806c880 0x7f234806ed40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f235c101b40 tx=0x7f234c009190 comp rx=0 tx=0).stop 2024-03-16T10:42:34.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 msgr2=0x7f235c196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:34.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c196b10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23440092e0 tx=0x7f234400bd70 comp rx=0 tx=0).stop 2024-03-16T10:42:34.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 shutdown_connections 2024-03-16T10:42:34.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f234806c880 0x7f234806ed40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:34.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f235c100ae0 0x7f235c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:34.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 --2- 172.21.15.2:0/2562174872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f235c101ce0 0x7f235c196b10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:34.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 >> 172.21.15.2:0/2562174872 conn(0x7f235c0fc0a0 msgr2=0x7f235c105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:34.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 shutdown_connections 2024-03-16T10:42:34.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:34.592+0000 7f2362979700 1 -- 172.21.15.2:0/2562174872 wait complete. 2024-03-16T10:42:35.232 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-16T10:42:35.445 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:36.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:35 smithi002 ceph-mon[29248]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T10:42:36.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:35 smithi026 ceph-mon[37041]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 -- 172.21.15.2:0/2452698663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 msgr2=0x7f80b0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 --2- 172.21.15.2:0/2452698663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0102140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009910 tx=0x7f80a0009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 -- 172.21.15.2:0/2452698663 shutdown_connections 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 --2- 172.21.15.2:0/2452698663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0102140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 --2- 172.21.15.2:0/2452698663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b0100ae0 0x7f80b0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 -- 172.21.15.2:0/2452698663 >> 172.21.15.2:0/2452698663 conn(0x7f80b00fc0a0 msgr2=0x7f80b00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 -- 172.21.15.2:0/2452698663 shutdown_connections 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.076+0000 7f80b79af700 1 -- 172.21.15.2:0/2452698663 wait complete. 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b79af700 1 Processor -- start 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b79af700 1 -- start start 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b79af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b0100ae0 0x7f80b0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b79af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:37.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b79af700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0199280 con 0x7f80b0101ce0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b79af700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01993c0 con 0x7f80b0100ae0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b4f4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b4f4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55244/0 (socket says 172.21.15.2:55244) 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b4f4a700 1 -- 172.21.15.2:0/2213178227 learned_addr learned my addr 172.21.15.2:0/2213178227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.077+0000 7f80b574b700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b0100ae0 0x7f80b0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80b4f4a700 1 -- 172.21.15.2:0/2213178227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b0100ae0 msgr2=0x7f80b0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80b4f4a700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b0100ae0 0x7f80b0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80b4f4a700 1 -- 172.21.15.2:0/2213178227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00095f0 con 0x7f80b0101ce0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80b4f4a700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0198c60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80a00092e0 tx=0x7f80a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80a67fc700 1 -- 172.21.15.2:0/2213178227 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a001d070 con 0x7f80b0101ce0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80a67fc700 1 -- 172.21.15.2:0/2213178227 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80a000f460 con 0x7f80b0101ce0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b019dd70 con 0x7f80b0101ce0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80a67fc700 1 -- 172.21.15.2:0/2213178227 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a0021620 con 0x7f80b0101ce0 2024-03-16T10:42:37.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.078+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b019e2e0 con 0x7f80b0101ce0 2024-03-16T10:42:37.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.079+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b0106140 con 0x7f80b0101ce0 2024-03-16T10:42:37.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.079+0000 7f80a67fc700 1 -- 172.21.15.2:0/2213178227 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f80a000f5d0 con 0x7f80b0101ce0 2024-03-16T10:42:37.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.080+0000 7f80a67fc700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f809c06c880 0x7f809c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:37.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.080+0000 7f80a67fc700 1 -- 172.21.15.2:0/2213178227 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f80a008ce30 con 0x7f80b0101ce0 2024-03-16T10:42:37.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.080+0000 7f80b574b700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f809c06c880 0x7f809c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:37.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.080+0000 7f80b574b700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f809c06c880 0x7f809c06ed40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f80b0101b40 tx=0x7f80ac00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:37.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.085+0000 7f80a67fc700 1 -- 172.21.15.2:0/2213178227 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f80a005aef0 con 0x7f80b0101ce0 2024-03-16T10:42:37.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.285+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f80b00619c0 con 0x7f80b0101ce0 2024-03-16T10:42:37.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.286+0000 7f80a67fc700 1 -- 172.21.15.2:0/2213178227 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v22)=0 v22) v1 ==== 163+0+0 (secure 0 0 0) 0x7f80a005aa80 con 0x7f80b0101ce0 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f809c06c880 msgr2=0x7f809c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f809c06c880 0x7f809c06ed40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f80b0101b40 tx=0x7f80ac00b410 comp rx=0 tx=0).stop 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 msgr2=0x7f80b0198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0198c60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80a00092e0 tx=0x7f80a000bd70 comp rx=0 tx=0).stop 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 shutdown_connections 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f809c06c880 0x7f809c06ed40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b0100ae0 0x7f80b0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 --2- 172.21.15.2:0/2213178227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80b0101ce0 0x7f80b0198c60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 >> 172.21.15.2:0/2213178227 conn(0x7f80b00fc0a0 msgr2=0x7f80b0103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 shutdown_connections 2024-03-16T10:42:37.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:37.288+0000 7f80b79af700 1 -- 172.21.15.2:0/2213178227 wait complete. 2024-03-16T10:42:37.918 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:37 smithi002 ceph-mon[29248]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-16T10:42:37.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-16T10:42:38.154 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:38.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:37 smithi026 ceph-mon[37041]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-16T10:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 -- 172.21.15.2:0/2092102649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d40fedd0 msgr2=0x7f74d40ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 --2- 172.21.15.2:0/2092102649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d40fedd0 0x7f74d40ff1f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f74c400b0a0 tx=0x7f74c400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 -- 172.21.15.2:0/2092102649 shutdown_connections 2024-03-16T10:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 --2- 172.21.15.2:0/2092102649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d4101db0 0x7f74d4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 --2- 172.21.15.2:0/2092102649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d40fedd0 0x7f74d40ff1f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 -- 172.21.15.2:0/2092102649 >> 172.21.15.2:0/2092102649 conn(0x7f74d40fa730 msgr2=0x7f74d40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 -- 172.21.15.2:0/2092102649 shutdown_connections 2024-03-16T10:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 -- 172.21.15.2:0/2092102649 wait complete. 2024-03-16T10:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.877+0000 7f74d9208700 1 Processor -- start 2024-03-16T10:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d9208700 1 -- start start 2024-03-16T10:42:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d9208700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d40fedd0 0x7f74d41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d9208700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d4101db0 0x7f74d4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d9208700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4199340 con 0x7f74d4101db0 2024-03-16T10:42:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d9208700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4199480 con 0x7f74d40fedd0 2024-03-16T10:42:39.881 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:39 smithi002 ceph-mon[29248]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-16T10:42:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d4101db0 0x7f74d4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d4101db0 0x7f74d4198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55272/0 (socket says 172.21.15.2:55272) 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d259c700 1 -- 172.21.15.2:0/1827656695 learned_addr learned my addr 172.21.15.2:0/1827656695 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d2d9d700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d40fedd0 0x7f74d41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d259c700 1 -- 172.21.15.2:0/1827656695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d40fedd0 msgr2=0x7f74d41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d259c700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d40fedd0 0x7f74d41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d259c700 1 -- 172.21.15.2:0/1827656695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4009d00 con 0x7f74d4101db0 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.878+0000 7f74d259c700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d4101db0 0x7f74d4198d20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b4b0 tx=0x7f74bc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.879+0000 7f74cbfff700 1 -- 172.21.15.2:0/1827656695 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74bc011800 con 0x7f74d4101db0 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.879+0000 7f74cbfff700 1 -- 172.21.15.2:0/1827656695 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74bc011e20 con 0x7f74d4101db0 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.879+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d419dee0 con 0x7f74d4101db0 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.879+0000 7f74cbfff700 1 -- 172.21.15.2:0/1827656695 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74bc00f2e0 con 0x7f74d4101db0 2024-03-16T10:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.879+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d419e480 con 0x7f74d4101db0 2024-03-16T10:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.880+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d4106780 con 0x7f74d4101db0 2024-03-16T10:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.880+0000 7f74cbfff700 1 -- 172.21.15.2:0/1827656695 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f74bc010380 con 0x7f74d4101db0 2024-03-16T10:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.881+0000 7f74cbfff700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f74c006c880 0x7f74c006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.881+0000 7f74cbfff700 1 -- 172.21.15.2:0/1827656695 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f74bc08b800 con 0x7f74d4101db0 2024-03-16T10:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.881+0000 7f74d2d9d700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f74c006c880 0x7f74c006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.881+0000 7f74d2d9d700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f74c006c880 0x7f74c006ed40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009fd0 tx=0x7f74c4006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:39.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:39.885+0000 7f74cbfff700 1 -- 172.21.15.2:0/1827656695 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f74bc059840 con 0x7f74d4101db0 2024-03-16T10:42:40.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.084+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f74d404ea80 con 0x7f74d4101db0 2024-03-16T10:42:40.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.085+0000 7f74cbfff700 1 -- 172.21.15.2:0/1827656695 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v22)=0 v22) v1 ==== 135+0+0 (secure 0 0 0) 0x7f74bc0593d0 con 0x7f74d4101db0 2024-03-16T10:42:40.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f74c006c880 msgr2=0x7f74c006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:40.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f74c006c880 0x7f74c006ed40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009fd0 tx=0x7f74c4006000 comp rx=0 tx=0).stop 2024-03-16T10:42:40.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d4101db0 msgr2=0x7f74d4198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:40.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d4101db0 0x7f74d4198d20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b4b0 tx=0x7f74bc00b7c0 comp rx=0 tx=0).stop 2024-03-16T10:42:40.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 shutdown_connections 2024-03-16T10:42:40.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f74c006c880 0x7f74c006ed40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:40.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74d40fedd0 0x7f74d41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:40.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 --2- 172.21.15.2:0/1827656695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74d4101db0 0x7f74d4198d20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:40.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 >> 172.21.15.2:0/1827656695 conn(0x7f74d40fa730 msgr2=0x7f74d40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:40.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 shutdown_connections 2024-03-16T10:42:40.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:40.087+0000 7f74d9208700 1 -- 172.21.15.2:0/1827656695 wait complete. 2024-03-16T10:42:40.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:39 smithi026 ceph-mon[37041]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-16T10:42:40.760 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-16T10:42:40.970 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:42.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:41 smithi002 ceph-mon[29248]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T10:42:42.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:41 smithi026 ceph-mon[37041]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 -- 172.21.15.2:0/1269767956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 msgr2=0x7f8d440ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/1269767956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d440ffe00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009910 tx=0x7f8d34009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 -- 172.21.15.2:0/1269767956 shutdown_connections 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/1269767956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d44100be0 0x7f8d44101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/1269767956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d440ffe00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 -- 172.21.15.2:0/1269767956 >> 172.21.15.2:0/1269767956 conn(0x7f8d440faf80 msgr2=0x7f8d440fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 -- 172.21.15.2:0/1269767956 shutdown_connections 2024-03-16T10:42:42.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.767+0000 7f8d48bfb700 1 -- 172.21.15.2:0/1269767956 wait complete. 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d48bfb700 1 Processor -- start 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d48bfb700 1 -- start start 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d48bfb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d44196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d48bfb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d44100be0 0x7f8d44196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d48bfb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44197000 con 0x7f8d440ff9e0 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d48bfb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44197140 con 0x7f8d44100be0 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d4259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d44196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d4259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d44196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33056/0 (socket says 172.21.15.2:33056) 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d4259c700 1 -- 172.21.15.2:0/3663308340 learned_addr learned my addr 172.21.15.2:0/3663308340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d41d9b700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d44100be0 0x7f8d44196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d4259c700 1 -- 172.21.15.2:0/3663308340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d44100be0 msgr2=0x7f8d44196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:42.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d4259c700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d44100be0 0x7f8d44196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:42.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.768+0000 7f8d4259c700 1 -- 172.21.15.2:0/3663308340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340095f0 con 0x7f8d440ff9e0 2024-03-16T10:42:42.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.769+0000 7f8d4259c700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d44196530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8d34000c00 tx=0x7f8d3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:42.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.769+0000 7f8d3b7fe700 1 -- 172.21.15.2:0/3663308340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3401d070 con 0x7f8d440ff9e0 2024-03-16T10:42:42.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.769+0000 7f8d3b7fe700 1 -- 172.21.15.2:0/3663308340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d3400f460 con 0x7f8d440ff9e0 2024-03-16T10:42:42.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.769+0000 7f8d3b7fe700 1 -- 172.21.15.2:0/3663308340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d34021640 con 0x7f8d440ff9e0 2024-03-16T10:42:42.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.769+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4419bb00 con 0x7f8d440ff9e0 2024-03-16T10:42:42.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.769+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4419c040 con 0x7f8d440ff9e0 2024-03-16T10:42:42.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.770+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d44190660 con 0x7f8d440ff9e0 2024-03-16T10:42:42.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.770+0000 7f8d3b7fe700 1 -- 172.21.15.2:0/3663308340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f8d3400fab0 con 0x7f8d440ff9e0 2024-03-16T10:42:42.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.771+0000 7f8d3b7fe700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d3006c890 0x7f8d3006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:42.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.771+0000 7f8d3b7fe700 1 -- 172.21.15.2:0/3663308340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8d3408cfe0 con 0x7f8d440ff9e0 2024-03-16T10:42:42.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.771+0000 7f8d41d9b700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d3006c890 0x7f8d3006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:42.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.771+0000 7f8d41d9b700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d3006c890 0x7f8d3006ed50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005ea0 tx=0x7f8d2c005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:42.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.776+0000 7f8d3b7fe700 1 -- 172.21.15.2:0/3663308340 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d3405b0a0 con 0x7f8d440ff9e0 2024-03-16T10:42:42.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:42.997+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d440611f0 con 0x7f8d3006c890 2024-03-16T10:42:43.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.002+0000 7f8d3b7fe700 1 -- 172.21.15.2:0/3663308340 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f8d440611f0 con 0x7f8d3006c890 2024-03-16T10:42:43.003 INFO:teuthology.orchestra.run.smithi002.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d3006c890 msgr2=0x7f8d3006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d3006c890 0x7f8d3006ed50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005ea0 tx=0x7f8d2c005e30 comp rx=0 tx=0).stop 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 msgr2=0x7f8d44196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d44196530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8d34000c00 tx=0x7f8d3400be20 comp rx=0 tx=0).stop 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 shutdown_connections 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8d3006c890 0x7f8d3006ed50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d440ff9e0 0x7f8d44196530 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 --2- 172.21.15.2:0/3663308340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d44100be0 0x7f8d44196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 >> 172.21.15.2:0/3663308340 conn(0x7f8d440faf80 msgr2=0x7f8d441041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 shutdown_connections 2024-03-16T10:42:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:43.004+0000 7f8d48bfb700 1 -- 172.21.15.2:0/3663308340 wait complete. 2024-03-16T10:42:43.784 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-16T10:42:43.784 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:42:43.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-16T10:42:44.002 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:42:44.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:44 smithi002 ceph-mon[29248]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T10:42:44.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:44 smithi002 ceph-mon[29248]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:44.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:44 smithi002 ceph-mon[29248]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:44.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:44 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:44.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:44 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:44.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:44 smithi026 ceph-mon[37041]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-16T10:42:44.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:44 smithi026 ceph-mon[37041]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:44.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:44 smithi026 ceph-mon[37041]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:44.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:44 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:44.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:44 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:42:45.278 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:45.278 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:45.278 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:45 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:45.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:42:45.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:42:45.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:45 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:42:45.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.819+0000 7f9d68337700 1 -- 172.21.15.2:0/2469626547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60101d10 msgr2=0x7f9d60102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:45.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.819+0000 7f9d68337700 1 --2- 172.21.15.2:0/2469626547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60101d10 0x7f9d60102170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009910 tx=0x7f9d50009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 -- 172.21.15.2:0/2469626547 shutdown_connections 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 --2- 172.21.15.2:0/2469626547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60101d10 0x7f9d60102170 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 --2- 172.21.15.2:0/2469626547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d60100b10 0x7f9d60100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 -- 172.21.15.2:0/2469626547 >> 172.21.15.2:0/2469626547 conn(0x7f9d600fc090 msgr2=0x7f9d600fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 -- 172.21.15.2:0/2469626547 shutdown_connections 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 -- 172.21.15.2:0/2469626547 wait complete. 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 Processor -- start 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 -- start start 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60100b10 0x7f9d60074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:45.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d60101d10 0x7f9d600731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600736e0 con 0x7f9d60100b10 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.820+0000 7f9d68337700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60073820 con 0x7f9d60101d10 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d660d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60100b10 0x7f9d60074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d660d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60100b10 0x7f9d60074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33086/0 (socket says 172.21.15.2:33086) 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d660d3700 1 -- 172.21.15.2:0/3179406246 learned_addr learned my addr 172.21.15.2:0/3179406246 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d660d3700 1 -- 172.21.15.2:0/3179406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d60101d10 msgr2=0x7f9d600731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d658d2700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d60101d10 0x7f9d600731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d660d3700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d60101d10 0x7f9d600731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d660d3700 1 -- 172.21.15.2:0/3179406246 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d500095f0 con 0x7f9d60100b10 2024-03-16T10:42:45.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d660d3700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60100b10 0x7f9d60074b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b480 tx=0x7f9d5c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:45.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d577fe700 1 -- 172.21.15.2:0/3179406246 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5c0118c0 con 0x7f9d60100b10 2024-03-16T10:42:45.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d60073b00 con 0x7f9d60100b10 2024-03-16T10:42:45.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d577fe700 1 -- 172.21.15.2:0/3179406246 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d5c010380 con 0x7f9d60100b10 2024-03-16T10:42:45.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d577fe700 1 -- 172.21.15.2:0/3179406246 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5c00f4f0 con 0x7f9d60100b10 2024-03-16T10:42:45.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.821+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6019e630 con 0x7f9d60100b10 2024-03-16T10:42:45.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.822+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d60190730 con 0x7f9d60100b10 2024-03-16T10:42:45.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.823+0000 7f9d577fe700 1 -- 172.21.15.2:0/3179406246 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f9d5c0104f0 con 0x7f9d60100b10 2024-03-16T10:42:45.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.823+0000 7f9d577fe700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9d4c06c890 0x7f9d4c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:45.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.823+0000 7f9d577fe700 1 -- 172.21.15.2:0/3179406246 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9d5c08b960 con 0x7f9d60100b10 2024-03-16T10:42:45.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.823+0000 7f9d658d2700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9d4c06c890 0x7f9d4c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:45.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.824+0000 7f9d658d2700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9d4c06c890 0x7f9d4c06ed50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000b5c0 tx=0x7f9d50005160 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:45.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:45.828+0000 7f9d577fe700 1 -- 172.21.15.2:0/3179406246 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9d5c0599a0 con 0x7f9d60100b10 2024-03-16T10:42:46.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.080+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d600611f0 con 0x7f9d4c06c890 2024-03-16T10:42:46.082 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:46 smithi002 ceph-mon[29248]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:46.082 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:46 smithi002 ceph-mon[29248]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T10:42:46.082 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:46.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.082+0000 7f9d577fe700 1 -- 172.21.15.2:0/3179406246 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f9d600611f0 con 0x7f9d4c06c890 2024-03-16T10:42:46.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9d4c06c890 msgr2=0x7f9d4c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9d4c06c890 0x7f9d4c06ed50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000b5c0 tx=0x7f9d50005160 comp rx=0 tx=0).stop 2024-03-16T10:42:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60100b10 msgr2=0x7f9d60074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60100b10 0x7f9d60074b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00b480 tx=0x7f9d5c00b840 comp rx=0 tx=0).stop 2024-03-16T10:42:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 shutdown_connections 2024-03-16T10:42:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9d4c06c890 0x7f9d4c06ed50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d60100b10 0x7f9d60074b00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 --2- 172.21.15.2:0/3179406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d60101d10 0x7f9d600731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 >> 172.21.15.2:0/3179406246 conn(0x7f9d600fc090 msgr2=0x7f9d60105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 shutdown_connections 2024-03-16T10:42:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.084+0000 7f9d68337700 1 -- 172.21.15.2:0/3179406246 wait complete. 2024-03-16T10:42:46.095 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:42:46.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.200+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2431792957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf540ff9d0 msgr2=0x7fcf540ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.200+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2431792957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf540ff9d0 0x7fcf540ffdf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009910 tx=0x7fcf44009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.200+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2431792957 shutdown_connections 2024-03-16T10:42:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.200+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2431792957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf54100bd0 0x7fcf54101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.200+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2431792957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf540ff9d0 0x7fcf540ffdf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.200+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2431792957 >> 172.21.15.2:0/2431792957 conn(0x7fcf540faf70 msgr2=0x7fcf540fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.200+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2431792957 shutdown_connections 2024-03-16T10:42:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.201+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2431792957 wait complete. 2024-03-16T10:42:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.201+0000 7fcf58fa1700 1 Processor -- start 2024-03-16T10:42:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.201+0000 7fcf58fa1700 1 -- start start 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.201+0000 7fcf58fa1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf540ff9d0 0x7fcf54103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf58fa1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf54100bd0 0x7fcf54101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf58fa1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf541024f0 con 0x7fcf54100bd0 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf58fa1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54102630 con 0x7fcf540ff9d0 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf51d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf54100bd0 0x7fcf54101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf51d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf54100bd0 0x7fcf54101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33114/0 (socket says 172.21.15.2:33114) 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf51d9b700 1 -- 172.21.15.2:0/2688575772 learned_addr learned my addr 172.21.15.2:0/2688575772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf5259c700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf540ff9d0 0x7fcf54103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf51d9b700 1 -- 172.21.15.2:0/2688575772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf540ff9d0 msgr2=0x7fcf54103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf51d9b700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf540ff9d0 0x7fcf54103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf51d9b700 1 -- 172.21.15.2:0/2688575772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440095f0 con 0x7fcf54100bd0 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf51d9b700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf54100bd0 0x7fcf54101fb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800d8d0 tx=0x7fcf4800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf437fe700 1 -- 172.21.15.2:0/2688575772 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf480098a0 con 0x7fcf54100bd0 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.202+0000 7fcf437fe700 1 -- 172.21.15.2:0/2688575772 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf48010460 con 0x7fcf54100bd0 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.203+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf54102910 con 0x7fcf54100bd0 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.203+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5419e540 con 0x7fcf54100bd0 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.203+0000 7fcf437fe700 1 -- 172.21.15.2:0/2688575772 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4800f5f0 con 0x7fcf54100bd0 2024-03-16T10:42:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.204+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5404ea80 con 0x7fcf54100bd0 2024-03-16T10:42:46.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.204+0000 7fcf437fe700 1 -- 172.21.15.2:0/2688575772 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fcf48009a00 con 0x7fcf54100bd0 2024-03-16T10:42:46.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.204+0000 7fcf437fe700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fcf3c06c890 0x7fcf3c06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.205+0000 7fcf437fe700 1 -- 172.21.15.2:0/2688575772 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcf4808bc50 con 0x7fcf54100bd0 2024-03-16T10:42:46.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.205+0000 7fcf5259c700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fcf3c06c890 0x7fcf3c06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.205+0000 7fcf5259c700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fcf3c06c890 0x7fcf3c06ed50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005090 tx=0x7fcf440058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:46.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.209+0000 7fcf437fe700 1 -- 172.21.15.2:0/2688575772 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcf48059bf0 con 0x7fcf54100bd0 2024-03-16T10:42:46.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.447+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf54069510 con 0x7fcf3c06c890 2024-03-16T10:42:46.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.448+0000 7fcf437fe700 1 -- 172.21.15.2:0/2688575772 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fcf54069510 con 0x7fcf3c06c890 2024-03-16T10:42:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fcf3c06c890 msgr2=0x7fcf3c06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fcf3c06c890 0x7fcf3c06ed50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005090 tx=0x7fcf440058e0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf54100bd0 msgr2=0x7fcf54101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf54100bd0 0x7fcf54101fb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800d8d0 tx=0x7fcf4800dbe0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 shutdown_connections 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fcf3c06c890 0x7fcf3c06ed50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf540ff9d0 0x7fcf54103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 --2- 172.21.15.2:0/2688575772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf54100bd0 0x7fcf54101fb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.450+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 >> 172.21.15.2:0/2688575772 conn(0x7fcf540faf70 msgr2=0x7fcf54068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.451+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 shutdown_connections 2024-03-16T10:42:46.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.451+0000 7fcf58fa1700 1 -- 172.21.15.2:0/2688575772 wait complete. 2024-03-16T10:42:46.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:46 smithi026 ceph-mon[37041]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:42:46.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:46 smithi026 ceph-mon[37041]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-16T10:42:46.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:42:46.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 -- 172.21.15.2:0/2190806683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 msgr2=0x7fef6c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 --2- 172.21.15.2:0/2190806683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c102210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef60009910 tx=0x7fef60009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 -- 172.21.15.2:0/2190806683 shutdown_connections 2024-03-16T10:42:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 --2- 172.21.15.2:0/2190806683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c102210 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 --2- 172.21.15.2:0/2190806683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef6c0fedd0 0x7fef6c0ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 -- 172.21.15.2:0/2190806683 >> 172.21.15.2:0/2190806683 conn(0x7fef6c0fa730 msgr2=0x7fef6c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 -- 172.21.15.2:0/2190806683 shutdown_connections 2024-03-16T10:42:46.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.574+0000 7fef70c73700 1 -- 172.21.15.2:0/2190806683 wait complete. 2024-03-16T10:42:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef70c73700 1 Processor -- start 2024-03-16T10:42:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef70c73700 1 -- start start 2024-03-16T10:42:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef70c73700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef6c0fedd0 0x7fef6c196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef70c73700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef70c73700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c1970c0 con 0x7fef6c101db0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef70c73700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c197200 con 0x7fef6c0fedd0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef69d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef69d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33130/0 (socket says 172.21.15.2:33130) 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef69d9b700 1 -- 172.21.15.2:0/2809834309 learned_addr learned my addr 172.21.15.2:0/2809834309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef6a59c700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef6c0fedd0 0x7fef6c196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef69d9b700 1 -- 172.21.15.2:0/2809834309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef6c0fedd0 msgr2=0x7fef6c196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef69d9b700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef6c0fedd0 0x7fef6c196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.575+0000 7fef69d9b700 1 -- 172.21.15.2:0/2809834309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef600095f0 con 0x7fef6c101db0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.576+0000 7fef69d9b700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c196aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef60006010 tx=0x7fef6000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.576+0000 7fef5b7fe700 1 -- 172.21.15.2:0/2809834309 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6001d070 con 0x7fef6c101db0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.576+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c19bb60 con 0x7fef6c101db0 2024-03-16T10:42:46.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.576+0000 7fef5b7fe700 1 -- 172.21.15.2:0/2809834309 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef6000f460 con 0x7fef6c101db0 2024-03-16T10:42:46.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.576+0000 7fef5b7fe700 1 -- 172.21.15.2:0/2809834309 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef60021620 con 0x7fef6c101db0 2024-03-16T10:42:46.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.576+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c19c050 con 0x7fef6c101db0 2024-03-16T10:42:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.577+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c190690 con 0x7fef6c101db0 2024-03-16T10:42:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.577+0000 7fef5b7fe700 1 -- 172.21.15.2:0/2809834309 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fef6000f5d0 con 0x7fef6c101db0 2024-03-16T10:42:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.578+0000 7fef5b7fe700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fef5406c880 0x7fef5406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.578+0000 7fef5b7fe700 1 -- 172.21.15.2:0/2809834309 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fef6008cf10 con 0x7fef6c101db0 2024-03-16T10:42:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.578+0000 7fef6a59c700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fef5406c880 0x7fef5406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.578+0000 7fef6a59c700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fef5406c880 0x7fef5406ed40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef6c068db0 tx=0x7fef5c007400 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:46.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.583+0000 7fef5b7fe700 1 -- 172.21.15.2:0/2809834309 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fef6005af50 con 0x7fef6c101db0 2024-03-16T10:42:46.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.809+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef6c0611f0 con 0x7fef5406c880 2024-03-16T10:42:46.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.820+0000 7fef5b7fe700 1 -- 172.21.15.2:0/2809834309 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fef6c0611f0 con 0x7fef5406c880 2024-03-16T10:42:46.821 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:42:46.821 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (4m) 24s ago 6m 25.1M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:42:46.821 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (6m) 24s ago 6m 7436k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:42:46.821 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (5m) 28s ago 5m 7356k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:42:46.821 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (6m) 24s ago 6m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:42:46.821 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (5m) 28s ago 5m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (4m) 24s ago 6m 84.9M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (44s) 24s ago 44s 14.7M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (52s) 24s ago 51s 11.5M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (49s) 28s ago 48s 15.7M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (41s) 28s ago 41s 13.3M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:9283,8765,8443 running (7m) 24s ago 7m 497M - 18.2.2-878-ga752c2f7 f692adb9a265 1b8a582e9d84 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (5m) 28s ago 5m 449M - 18.2.2-878-ga752c2f7 f692adb9a265 11a47ca222e7 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (8m) 24s ago 8m 45.3M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (5m) 28s ago 5m 35.1M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:42:46.822 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (6m) 24s ago 6m 16.3M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (5m) 28s ago 5m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (4m) 24s ago 4m 46.9M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (4m) 24s ago 4m 47.8M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (3m) 24s ago 3m 48.6M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (3m) 28s ago 3m 46.9M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (2m) 28s ago 2m 44.2M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (2m) 28s ago 2m 46.2M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:42:46.823 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (4m) 24s ago 5m 59.0M - 2.43.0 a07b618ecd1d 02e5ed409aab 2024-03-16T10:42:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fef5406c880 msgr2=0x7fef5406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fef5406c880 0x7fef5406ed40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef6c068db0 tx=0x7fef5c007400 comp rx=0 tx=0).stop 2024-03-16T10:42:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 msgr2=0x7fef6c196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c196aa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef60006010 tx=0x7fef6000bd70 comp rx=0 tx=0).stop 2024-03-16T10:42:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 shutdown_connections 2024-03-16T10:42:46.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fef5406c880 0x7fef5406ed40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef6c0fedd0 0x7fef6c196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 --2- 172.21.15.2:0/2809834309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef6c101db0 0x7fef6c196aa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.822+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 >> 172.21.15.2:0/2809834309 conn(0x7fef6c0fa730 msgr2=0x7fef6c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:46.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.823+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 shutdown_connections 2024-03-16T10:42:46.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.823+0000 7fef70c73700 1 -- 172.21.15.2:0/2809834309 wait complete. 2024-03-16T10:42:46.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 -- 172.21.15.2:0/581089529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 msgr2=0x7f814c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 --2- 172.21.15.2:0/581089529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c101030 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8140009910 tx=0x7f8140009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 -- 172.21.15.2:0/581089529 shutdown_connections 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 --2- 172.21.15.2:0/581089529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c101030 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 --2- 172.21.15.2:0/581089529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f814c0ff9d0 0x7f814c0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 -- 172.21.15.2:0/581089529 >> 172.21.15.2:0/581089529 conn(0x7f814c0faf70 msgr2=0x7f814c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 -- 172.21.15.2:0/581089529 shutdown_connections 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 -- 172.21.15.2:0/581089529 wait complete. 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.929+0000 7f8151825700 1 Processor -- start 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f8151825700 1 -- start start 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f8151825700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f814c0ff9d0 0x7f814c103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f8151825700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c1022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f8151825700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c102810 con 0x7f814c100bd0 2024-03-16T10:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f8151825700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c102950 con 0x7f814c0ff9d0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c1022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c1022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33160/0 (socket says 172.21.15.2:33160) 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814a7fc700 1 -- 172.21.15.2:0/2160751871 learned_addr learned my addr 172.21.15.2:0/2160751871 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814affd700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f814c0ff9d0 0x7f814c103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814a7fc700 1 -- 172.21.15.2:0/2160751871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f814c0ff9d0 msgr2=0x7f814c103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814a7fc700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f814c0ff9d0 0x7f814c103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814a7fc700 1 -- 172.21.15.2:0/2160751871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400095f0 con 0x7f814c100bd0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.930+0000 7f814a7fc700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c1022d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f814000b800 tx=0x7f8140005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.931+0000 7f8150823700 1 -- 172.21.15.2:0/2160751871 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f814001d070 con 0x7f814c100bd0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.931+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c102bd0 con 0x7f814c100bd0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.931+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c103140 con 0x7f814c100bd0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.931+0000 7f8150823700 1 -- 172.21.15.2:0/2160751871 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81400052a0 con 0x7f814c100bd0 2024-03-16T10:42:46.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.931+0000 7f8150823700 1 -- 172.21.15.2:0/2160751871 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8140017400 con 0x7f814c100bd0 2024-03-16T10:42:46.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.932+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81380052f0 con 0x7f814c100bd0 2024-03-16T10:42:46.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.933+0000 7f8150823700 1 -- 172.21.15.2:0/2160751871 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f8140017670 con 0x7f814c100bd0 2024-03-16T10:42:46.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.933+0000 7f8150823700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f813406c880 0x7f813406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:46.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.933+0000 7f8150823700 1 -- 172.21.15.2:0/2160751871 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f814008ce40 con 0x7f814c100bd0 2024-03-16T10:42:46.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.933+0000 7f814affd700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f813406c880 0x7f813406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:46.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.933+0000 7f814affd700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f813406c880 0x7f813406ed40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f814c100a30 tx=0x7f813c009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:46.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:46.938+0000 7f8150823700 1 -- 172.21.15.2:0/2160751871 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f814005af00 con 0x7f814c100bd0 2024-03-16T10:42:47.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.245+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81380061d0 con 0x7f814c100bd0 2024-03-16T10:42:47.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.246+0000 7f8150823700 1 -- 172.21.15.2:0/2160751871 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f814005aa90 con 0x7f814c100bd0 2024-03-16T10:42:47.247 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:42:47.247 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:42:47.247 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:42:47.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:47.247 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:42:47.247 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 14 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:42:47.248 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f813406c880 msgr2=0x7f813406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f813406c880 0x7f813406ed40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f814c100a30 tx=0x7f813c009260 comp rx=0 tx=0).stop 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 msgr2=0x7f814c1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c1022d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f814000b800 tx=0x7f8140005f00 comp rx=0 tx=0).stop 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 shutdown_connections 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f813406c880 0x7f813406ed40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f814c0ff9d0 0x7f814c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 --2- 172.21.15.2:0/2160751871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f814c100bd0 0x7f814c1022d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 >> 172.21.15.2:0/2160751871 conn(0x7f814c0faf70 msgr2=0x7f814c068480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:47.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 shutdown_connections 2024-03-16T10:42:47.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.248+0000 7f8151825700 1 -- 172.21.15.2:0/2160751871 wait complete. 2024-03-16T10:42:47.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 -- 172.21.15.2:0/1439340182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344101640 msgr2=0x7f7344103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 --2- 172.21.15.2:0/1439340182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344101640 0x7f7344103a30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7340009910 tx=0x7f7340009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:47.360 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:47 smithi002 ceph-mon[29248]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 -- 172.21.15.2:0/1439340182 shutdown_connections 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 --2- 172.21.15.2:0/1439340182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7344103f70 0x7f7344106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 --2- 172.21.15.2:0/1439340182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344101640 0x7f7344103a30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 -- 172.21.15.2:0/1439340182 >> 172.21.15.2:0/1439340182 conn(0x7f73440fafb0 msgr2=0x7f73440fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 -- 172.21.15.2:0/1439340182 shutdown_connections 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.359+0000 7f734c1b2700 1 -- 172.21.15.2:0/1439340182 wait complete. 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.360+0000 7f734c1b2700 1 Processor -- start 2024-03-16T10:42:47.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.360+0000 7f734c1b2700 1 -- start start 2024-03-16T10:42:47.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734c1b2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7344101640 0x7f7344100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:47.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734c1b2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344103f70 0x7f73440ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734c1b2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734419a100 con 0x7f7344103f70 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734c1b2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734419a240 con 0x7f7344101640 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734974d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344103f70 0x7f73440ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734974d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344103f70 0x7f73440ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33174/0 (socket says 172.21.15.2:33174) 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734974d700 1 -- 172.21.15.2:0/837289393 learned_addr learned my addr 172.21.15.2:0/837289393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f7349f4e700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7344101640 0x7f7344100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734974d700 1 -- 172.21.15.2:0/837289393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7344101640 msgr2=0x7f7344100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734974d700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7344101640 0x7f7344100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f734974d700 1 -- 172.21.15.2:0/837289393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400095f0 con 0x7f7344103f70 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.361+0000 7f7349f4e700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7344101640 0x7f7344100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:42:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.362+0000 7f734974d700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344103f70 0x7f73440ff5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f733400e920 tx=0x7f733400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.362+0000 7f733affd700 1 -- 172.21.15.2:0/837289393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f733400c9d0 con 0x7f7344103f70 2024-03-16T10:42:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.362+0000 7f733affd700 1 -- 172.21.15.2:0/837289393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7334010430 con 0x7f7344103f70 2024-03-16T10:42:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.362+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73440ffb30 con 0x7f7344103f70 2024-03-16T10:42:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.362+0000 7f733affd700 1 -- 172.21.15.2:0/837289393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7334018630 con 0x7f7344103f70 2024-03-16T10:42:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.362+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73441000d0 con 0x7f7344103f70 2024-03-16T10:42:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.364+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734404ea80 con 0x7f7344103f70 2024-03-16T10:42:47.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.365+0000 7f733affd700 1 -- 172.21.15.2:0/837289393 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f73340105a0 con 0x7f7344103f70 2024-03-16T10:42:47.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.365+0000 7f733affd700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f733006c890 0x7f733006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:47.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.365+0000 7f733affd700 1 -- 172.21.15.2:0/837289393 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7334014070 con 0x7f7344103f70 2024-03-16T10:42:47.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.365+0000 7f7349f4e700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f733006c890 0x7f733006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:47.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.366+0000 7f7349f4e700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f733006c890 0x7f733006ed50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f734000b800 tx=0x7f7340009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:47.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.370+0000 7f733affd700 1 -- 172.21.15.2:0/837289393 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f73340529f0 con 0x7f7344103f70 2024-03-16T10:42:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:47 smithi026 ceph-mon[37041]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:47.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.625+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f73441003b0 con 0x7f7344103f70 2024-03-16T10:42:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.625+0000 7f733affd700 1 -- 172.21.15.2:0/837289393 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f7334054e60 con 0x7f7344103f70 2024-03-16T10:42:47.626 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:42:47.626 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:42:47.626 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:42:47.626 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:42:47.626 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:47.626 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:42:47.627 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:47.628 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:47.629 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:42:47.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f733006c890 msgr2=0x7f733006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f733006c890 0x7f733006ed50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f734000b800 tx=0x7f7340009f70 comp rx=0 tx=0).stop 2024-03-16T10:42:47.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344103f70 msgr2=0x7f73440ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344103f70 0x7f73440ff5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f733400e920 tx=0x7f733400ec30 comp rx=0 tx=0).stop 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 shutdown_connections 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f733006c890 0x7f733006ed50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7344101640 0x7f7344100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 --2- 172.21.15.2:0/837289393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7344103f70 0x7f73440ff5f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.627+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 >> 172.21.15.2:0/837289393 conn(0x7f73440fafb0 msgr2=0x7f73440fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.628+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 shutdown_connections 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.628+0000 7f734c1b2700 1 -- 172.21.15.2:0/837289393 wait complete. 2024-03-16T10:42:47.630 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 -- 172.21.15.2:0/517518716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 msgr2=0x7f8ee8103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/517518716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8103950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009910 tx=0x7f8ed8009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 -- 172.21.15.2:0/517518716 shutdown_connections 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/517518716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8103e90 0x7f8ee8106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/517518716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8103950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 -- 172.21.15.2:0/517518716 >> 172.21.15.2:0/517518716 conn(0x7f8ee80faf70 msgr2=0x7f8ee80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 -- 172.21.15.2:0/517518716 shutdown_connections 2024-03-16T10:42:47.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 -- 172.21.15.2:0/517518716 wait complete. 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.738+0000 7f8eed2fe700 1 Processor -- start 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8eed2fe700 1 -- start start 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8eed2fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8eed2fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8103e90 0x7f8ee80ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8eed2fe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80ffa50 con 0x7f8ee8101560 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8eed2fe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80ffb90 con 0x7f8ee8103e90 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8ee6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8ee6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33192/0 (socket says 172.21.15.2:33192) 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8ee6ffd700 1 -- 172.21.15.2:0/476373629 learned_addr learned my addr 172.21.15.2:0/476373629 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8ee67fc700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8103e90 0x7f8ee80ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8ee6ffd700 1 -- 172.21.15.2:0/476373629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8103e90 msgr2=0x7f8ee80ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8ee6ffd700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8103e90 0x7f8ee80ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.739+0000 7f8ee6ffd700 1 -- 172.21.15.2:0/476373629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80095f0 con 0x7f8ee8101560 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.740+0000 7f8ee6ffd700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8100e70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80092e0 tx=0x7f8ed800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.740+0000 7f8ecffff700 1 -- 172.21.15.2:0/476373629 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ed801d070 con 0x7f8ee8101560 2024-03-16T10:42:47.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.740+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee80ffe10 con 0x7f8ee8101560 2024-03-16T10:42:47.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.740+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee8100380 con 0x7f8ee8101560 2024-03-16T10:42:47.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.740+0000 7f8ecffff700 1 -- 172.21.15.2:0/476373629 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ed800f460 con 0x7f8ee8101560 2024-03-16T10:42:47.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.740+0000 7f8ecffff700 1 -- 172.21.15.2:0/476373629 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ed8021620 con 0x7f8ee8101560 2024-03-16T10:42:47.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.741+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed4005320 con 0x7f8ee8101560 2024-03-16T10:42:47.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.742+0000 7f8ecffff700 1 -- 172.21.15.2:0/476373629 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f8ed8003a60 con 0x7f8ee8101560 2024-03-16T10:42:47.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.742+0000 7f8ecffff700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8ed006c880 0x7f8ed006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:47.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.742+0000 7f8ecffff700 1 -- 172.21.15.2:0/476373629 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8ed808cc60 con 0x7f8ee8101560 2024-03-16T10:42:47.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.742+0000 7f8ee67fc700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8ed006c880 0x7f8ed006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:47.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.743+0000 7f8ee67fc700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8ed006c880 0x7f8ed006ed40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005950 tx=0x7f8edc00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:47.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:47.747+0000 7f8ecffff700 1 -- 172.21.15.2:0/476373629 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8ed805ad20 con 0x7f8ee8101560 2024-03-16T10:42:48.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.005+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ed4000bf0 con 0x7f8ed006c880 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.006+0000 7f8ecffff700 1 -- 172.21.15.2:0/476373629 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f8ed4000bf0 con 0x7f8ed006c880 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [], 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "", 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af image", 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:42:48.007 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8ed006c880 msgr2=0x7f8ed006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8ed006c880 0x7f8ed006ed40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005950 tx=0x7f8edc00b410 comp rx=0 tx=0).stop 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 msgr2=0x7f8ee8100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8100e70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80092e0 tx=0x7f8ed800bd70 comp rx=0 tx=0).stop 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 shutdown_connections 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f8ed006c880 0x7f8ed006ed40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ee8101560 0x7f8ee8100e70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 --2- 172.21.15.2:0/476373629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ee8103e90 0x7f8ee80ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 >> 172.21.15.2:0/476373629 conn(0x7f8ee80faf70 msgr2=0x7f8ee80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 shutdown_connections 2024-03-16T10:42:48.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.009+0000 7f8eed2fe700 1 -- 172.21.15.2:0/476373629 wait complete. 2024-03-16T10:42:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.124+0000 7f9b38168700 1 -- 172.21.15.2:0/909763816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 msgr2=0x7f9b300ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.124+0000 7f9b38168700 1 --2- 172.21.15.2:0/909763816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b300ffdf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009910 tx=0x7f9b2c009c20 comp rx=0 tx=0).stop 2024-03-16T10:42:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.124+0000 7f9b38168700 1 -- 172.21.15.2:0/909763816 shutdown_connections 2024-03-16T10:42:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 --2- 172.21.15.2:0/909763816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b30100bd0 0x7f9b30101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 --2- 172.21.15.2:0/909763816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b300ffdf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 -- 172.21.15.2:0/909763816 >> 172.21.15.2:0/909763816 conn(0x7f9b300faf70 msgr2=0x7f9b300fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 -- 172.21.15.2:0/909763816 shutdown_connections 2024-03-16T10:42:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 -- 172.21.15.2:0/909763816 wait complete. 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 Processor -- start 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 -- start start 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b30196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b30100bd0 0x7f9b30196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b301970e0 con 0x7f9b300ff9d0 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.125+0000 7f9b38168700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30197220 con 0x7f9b30100bd0 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35f04700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b30196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35f04700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b30196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33210/0 (socket says 172.21.15.2:33210) 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35f04700 1 -- 172.21.15.2:0/1094389942 learned_addr learned my addr 172.21.15.2:0/1094389942 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35703700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b30100bd0 0x7f9b30196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35f04700 1 -- 172.21.15.2:0/1094389942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b30100bd0 msgr2=0x7f9b30196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35f04700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b30100bd0 0x7f9b30196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35f04700 1 -- 172.21.15.2:0/1094389942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0095f0 con 0x7f9b300ff9d0 2024-03-16T10:42:48.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b35f04700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b30196580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0059d0 tx=0x7f9b2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:48.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b26ffd700 1 -- 172.21.15.2:0/1094389942 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b2c01d070 con 0x7f9b300ff9d0 2024-03-16T10:42:48.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b26ffd700 1 -- 172.21.15.2:0/1094389942 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b2c00f460 con 0x7f9b300ff9d0 2024-03-16T10:42:48.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3019bbd0 con 0x7f9b300ff9d0 2024-03-16T10:42:48.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b26ffd700 1 -- 172.21.15.2:0/1094389942 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b2c021640 con 0x7f9b300ff9d0 2024-03-16T10:42:48.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.126+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3019c0c0 con 0x7f9b300ff9d0 2024-03-16T10:42:48.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.128+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3004ea80 con 0x7f9b300ff9d0 2024-03-16T10:42:48.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.128+0000 7f9b26ffd700 1 -- 172.21.15.2:0/1094389942 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f9b2c00f5d0 con 0x7f9b300ff9d0 2024-03-16T10:42:48.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.128+0000 7f9b26ffd700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9b1c06c880 0x7f9b1c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:42:48.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.128+0000 7f9b26ffd700 1 -- 172.21.15.2:0/1094389942 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9b2c08cfe0 con 0x7f9b300ff9d0 2024-03-16T10:42:48.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.128+0000 7f9b35703700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9b1c06c880 0x7f9b1c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:42:48.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.129+0000 7f9b35703700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9b1c06c880 0x7f9b1c06ed40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000bbd0 tx=0x7f9b2000b5b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:42:48.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.133+0000 7f9b26ffd700 1 -- 172.21.15.2:0/1094389942 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9b2c05b020 con 0x7f9b300ff9d0 2024-03-16T10:42:48.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:48 smithi002 ceph-mon[29248]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T10:42:48.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:48 smithi002 ceph-mon[29248]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:48.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:48 smithi002 ceph-mon[29248]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:48.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:48 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2160751871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:42:48.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:48 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/837289393' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:42:48.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.460+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9b3019c400 con 0x7f9b300ff9d0 2024-03-16T10:42:48.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.461+0000 7f9b26ffd700 1 -- 172.21.15.2:0/1094389942 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9b2c055e60 con 0x7f9b300ff9d0 2024-03-16T10:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9b1c06c880 msgr2=0x7f9b1c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9b1c06c880 0x7f9b1c06ed40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000bbd0 tx=0x7f9b2000b5b0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 msgr2=0x7f9b30196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b30196580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0059d0 tx=0x7f9b2c00be20 comp rx=0 tx=0).stop 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 shutdown_connections 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f9b1c06c880 0x7f9b1c06ed40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b300ff9d0 0x7f9b30196580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 --2- 172.21.15.2:0/1094389942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b30100bd0 0x7f9b30196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 >> 172.21.15.2:0/1094389942 conn(0x7f9b300faf70 msgr2=0x7f9b30068530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:42:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 shutdown_connections 2024-03-16T10:42:48.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:42:48.463+0000 7f9b38168700 1 -- 172.21.15.2:0/1094389942 wait complete. 2024-03-16T10:42:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:48 smithi026 ceph-mon[37041]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-16T10:42:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:48 smithi026 ceph-mon[37041]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:48 smithi026 ceph-mon[37041]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:48 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2160751871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:42:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:48 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/837289393' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:42:49.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:49 smithi026 ceph-mon[37041]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:49.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:49 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1094389942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:42:49.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:49 smithi002 ceph-mon[29248]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:42:49.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:49 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1094389942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:42:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:50 smithi026 ceph-mon[37041]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T10:42:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:50 smithi002 ceph-mon[29248]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T10:42:52.367 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:52 smithi026 ceph-mon[37041]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T10:42:52.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:52 smithi002 ceph-mon[29248]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-16T10:42:54.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:54 smithi026 ceph-mon[37041]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:42:54.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:54 smithi002 ceph-mon[29248]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:42:56.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:56 smithi026 ceph-mon[37041]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:42:56.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:56 smithi002 ceph-mon[29248]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-16T10:42:58.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:42:58 smithi026 ceph-mon[37041]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:42:58.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:42:58 smithi002 ceph-mon[29248]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:00 smithi026 ceph-mon[37041]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:00.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:00 smithi002 ceph-mon[29248]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:01.383 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:01 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:01.542 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:01 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:02.468 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:02 smithi026 ceph-mon[37041]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:02.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:02 smithi002 ceph-mon[29248]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:04.350 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:04 smithi002 ceph-mon[29248]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:04.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:04 smithi026 ceph-mon[37041]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:06.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:06 smithi026 ceph-mon[37041]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:06.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:06 smithi002 ceph-mon[29248]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:08.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:08 smithi026 ceph-mon[37041]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:08.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:08 smithi002 ceph-mon[29248]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:10.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:10 smithi026 ceph-mon[37041]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:10.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:10 smithi002 ceph-mon[29248]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:12.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:12 smithi002 ceph-mon[29248]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:43:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:12 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:12.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:12 smithi026 ceph-mon[37041]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:12.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:12.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:43:12.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:12 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:13 smithi026 ceph-mon[37041]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (unknown) 2024-03-16T10:43:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:13 smithi026 ceph-mon[37041]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-16T10:43:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:13 smithi026 ceph-mon[37041]: Upgrade: Need to upgrade myself (mgr.smithi002.zevrtm) 2024-03-16T10:43:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:13 smithi002 ceph-mon[29248]: Upgrade: Target is version 19.0.0-2162-g3d78f5ba (unknown) 2024-03-16T10:43:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:13 smithi002 ceph-mon[29248]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513'] 2024-03-16T10:43:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:13 smithi002 ceph-mon[29248]: Upgrade: Need to upgrade myself (mgr.smithi002.zevrtm) 2024-03-16T10:43:14.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:14 smithi026 ceph-mon[37041]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:14.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:14 smithi026 ceph-mon[37041]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 on smithi026 2024-03-16T10:43:14.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:14 smithi002 ceph-mon[29248]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:14.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:14 smithi002 ceph-mon[29248]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 on smithi026 2024-03-16T10:43:16.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:16 smithi026 ceph-mon[37041]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:16.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:16 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:16.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:16 smithi002 ceph-mon[29248]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:16.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:16 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:18 smithi026 ceph-mon[37041]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:18 smithi002 ceph-mon[29248]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 -- 172.21.15.2:0/4236028544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 msgr2=0x7f81c8101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 --2- 172.21.15.2:0/4236028544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8101030 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009910 tx=0x7f81b8009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 -- 172.21.15.2:0/4236028544 shutdown_connections 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 --2- 172.21.15.2:0/4236028544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8101030 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 --2- 172.21.15.2:0/4236028544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81c80ff9d0 0x7f81c80ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 -- 172.21.15.2:0/4236028544 >> 172.21.15.2:0/4236028544 conn(0x7f81c80faf70 msgr2=0x7f81c80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 -- 172.21.15.2:0/4236028544 shutdown_connections 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.578+0000 7f81cf496700 1 -- 172.21.15.2:0/4236028544 wait complete. 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cf496700 1 Processor -- start 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cf496700 1 -- start start 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cf496700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81c80ff9d0 0x7f81c8196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cf496700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cf496700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c81970e0 con 0x7f81c8100bd0 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cf496700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c8197220 con 0x7f81c80ff9d0 2024-03-16T10:43:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cca31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cca31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50392/0 (socket says 172.21.15.2:50392) 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cca31700 1 -- 172.21.15.2:0/1180389129 learned_addr learned my addr 172.21.15.2:0/1180389129 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.579+0000 7f81cd232700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81c80ff9d0 0x7f81c8196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81cca31700 1 -- 172.21.15.2:0/1180389129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81c80ff9d0 msgr2=0x7f81c8196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81cca31700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81c80ff9d0 0x7f81c8196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81cca31700 1 -- 172.21.15.2:0/1180389129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80095f0 con 0x7f81c8100bd0 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81cca31700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8196ac0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005b40 tx=0x7f81b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81be7fc700 1 -- 172.21.15.2:0/1180389129 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b801d070 con 0x7f81c8100bd0 2024-03-16T10:43:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c819bbd0 con 0x7f81c8100bd0 2024-03-16T10:43:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81be7fc700 1 -- 172.21.15.2:0/1180389129 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b80050d0 con 0x7f81c8100bd0 2024-03-16T10:43:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81be7fc700 1 -- 172.21.15.2:0/1180389129 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b8022470 con 0x7f81c8100bd0 2024-03-16T10:43:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.580+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c8102110 con 0x7f81c8100bd0 2024-03-16T10:43:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.581+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c804ea80 con 0x7f81c8100bd0 2024-03-16T10:43:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.582+0000 7f81be7fc700 1 -- 172.21.15.2:0/1180389129 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f81b80037f0 con 0x7f81c8100bd0 2024-03-16T10:43:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.582+0000 7f81be7fc700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f81b4070ca0 0x7f81b4073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.582+0000 7f81be7fc700 1 -- 172.21.15.2:0/1180389129 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f81b808d250 con 0x7f81c8100bd0 2024-03-16T10:43:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.583+0000 7f81cd232700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f81b4070ca0 0x7f81b4073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.583+0000 7f81cd232700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f81b4070ca0 0x7f81b4073160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f81c8100a30 tx=0x7f81c4009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:18.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.587+0000 7f81be7fc700 1 -- 172.21.15.2:0/1180389129 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f81b805b340 con 0x7f81c8100bd0 2024-03-16T10:43:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.820+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81c8068b90 con 0x7f81b4070ca0 2024-03-16T10:43:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.821+0000 7f81be7fc700 1 -- 172.21.15.2:0/1180389129 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f81c8068b90 con 0x7f81b4070ca0 2024-03-16T10:43:18.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f81b4070ca0 msgr2=0x7f81b4073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f81b4070ca0 0x7f81b4073160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f81c8100a30 tx=0x7f81c4009260 comp rx=0 tx=0).stop 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 msgr2=0x7f81c8196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8196ac0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005b40 tx=0x7f81b800f690 comp rx=0 tx=0).stop 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 shutdown_connections 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f81b4070ca0 0x7f81b4073160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81c80ff9d0 0x7f81c8196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 --2- 172.21.15.2:0/1180389129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81c8100bd0 0x7f81c8196ac0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.823+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 >> 172.21.15.2:0/1180389129 conn(0x7f81c80faf70 msgr2=0x7f81c8068480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.824+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 shutdown_connections 2024-03-16T10:43:18.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.824+0000 7f81cf496700 1 -- 172.21.15.2:0/1180389129 wait complete. 2024-03-16T10:43:18.835 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 -- 172.21.15.2:0/3693034598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 msgr2=0x7f68e01062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/3693034598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e01062e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009910 tx=0x7f68cc009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 -- 172.21.15.2:0/3693034598 shutdown_connections 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/3693034598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e01062e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/3693034598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e01015c0 0x7f68e01039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 -- 172.21.15.2:0/3693034598 >> 172.21.15.2:0/3693034598 conn(0x7f68e00faf70 msgr2=0x7f68e00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 -- 172.21.15.2:0/3693034598 shutdown_connections 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.934+0000 7f68e6c0b700 1 -- 172.21.15.2:0/3693034598 wait complete. 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68e6c0b700 1 Processor -- start 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68e6c0b700 1 -- start start 2024-03-16T10:43:18.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68e6c0b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e01015c0 0x7f68e0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68e6c0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e00ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68e6c0b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00ffab0 con 0x7f68e0103ef0 2024-03-16T10:43:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68e6c0b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00ffbf0 con 0x7f68e01015c0 2024-03-16T10:43:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68dffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e00ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68dffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e00ff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50410/0 (socket says 172.21.15.2:50410) 2024-03-16T10:43:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68dffff700 1 -- 172.21.15.2:0/392807846 learned_addr learned my addr 172.21.15.2:0/392807846 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68e49a7700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e01015c0 0x7f68e0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68dffff700 1 -- 172.21.15.2:0/392807846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e01015c0 msgr2=0x7f68e0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68dffff700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e01015c0 0x7f68e0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.935+0000 7f68dffff700 1 -- 172.21.15.2:0/392807846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc0095f0 con 0x7f68e0103ef0 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.936+0000 7f68dffff700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e00ff570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b800 tx=0x7f68cc005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.936+0000 7f68ddffb700 1 -- 172.21.15.2:0/392807846 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68cc01d070 con 0x7f68e0103ef0 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.936+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e00ffe70 con 0x7f68e0103ef0 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.936+0000 7f68ddffb700 1 -- 172.21.15.2:0/392807846 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68cc0052a0 con 0x7f68e0103ef0 2024-03-16T10:43:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.936+0000 7f68ddffb700 1 -- 172.21.15.2:0/392807846 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68cc017400 con 0x7f68e0103ef0 2024-03-16T10:43:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.936+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e01003e0 con 0x7f68e0103ef0 2024-03-16T10:43:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.937+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e004ea80 con 0x7f68e0103ef0 2024-03-16T10:43:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.937+0000 7f68ddffb700 1 -- 172.21.15.2:0/392807846 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f68cc003680 con 0x7f68e0103ef0 2024-03-16T10:43:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.938+0000 7f68ddffb700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f68d006c880 0x7f68d006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.938+0000 7f68ddffb700 1 -- 172.21.15.2:0/392807846 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f68cc08cfa0 con 0x7f68e0103ef0 2024-03-16T10:43:18.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.938+0000 7f68e49a7700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f68d006c880 0x7f68d006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:18.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.938+0000 7f68e49a7700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f68d006c880 0x7f68d006ed40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009c60 tx=0x7f68d4009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:18.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:18.943+0000 7f68ddffb700 1 -- 172.21.15.2:0/392807846 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f68cc05afe0 con 0x7f68e0103ef0 2024-03-16T10:43:19.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.175+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68e019e4d0 con 0x7f68d006c880 2024-03-16T10:43:19.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.176+0000 7f68ddffb700 1 -- 172.21.15.2:0/392807846 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f68e019e4d0 con 0x7f68d006c880 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f68d006c880 msgr2=0x7f68d006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f68d006c880 0x7f68d006ed40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009c60 tx=0x7f68d4009260 comp rx=0 tx=0).stop 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 msgr2=0x7f68e00ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e00ff570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b800 tx=0x7f68cc005f00 comp rx=0 tx=0).stop 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 shutdown_connections 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f68d006c880 0x7f68d006ed40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e01015c0 0x7f68e0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 --2- 172.21.15.2:0/392807846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0103ef0 0x7f68e00ff570 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 >> 172.21.15.2:0/392807846 conn(0x7f68e00faf70 msgr2=0x7f68e00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:19.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 shutdown_connections 2024-03-16T10:43:19.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.178+0000 7f68e6c0b700 1 -- 172.21.15.2:0/392807846 wait complete. 2024-03-16T10:43:19.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.287+0000 7f494d699700 1 -- 172.21.15.2:0/284502569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 msgr2=0x7f49480ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.287+0000 7f494d699700 1 --2- 172.21.15.2:0/284502569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f49480ff580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f493800aa20 tx=0x7f493800ad30 comp rx=0 tx=0).stop 2024-03-16T10:43:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.287+0000 7f494d699700 1 -- 172.21.15.2:0/284502569 shutdown_connections 2024-03-16T10:43:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.287+0000 7f494d699700 1 --2- 172.21.15.2:0/284502569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948100360 0x7f49481007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.287+0000 7f494d699700 1 --2- 172.21.15.2:0/284502569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f49480ff580 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.287+0000 7f494d699700 1 -- 172.21.15.2:0/284502569 >> 172.21.15.2:0/284502569 conn(0x7f49480fa700 msgr2=0x7f49480fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.288+0000 7f494d699700 1 -- 172.21.15.2:0/284502569 shutdown_connections 2024-03-16T10:43:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.288+0000 7f494d699700 1 -- 172.21.15.2:0/284502569 wait complete. 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.288+0000 7f494d699700 1 Processor -- start 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.288+0000 7f494d699700 1 -- start start 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f494d699700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f4948196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f494d699700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948100360 0x7f4948196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f494d699700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49481970d0 con 0x7f49480ff160 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f494d699700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4948197210 con 0x7f4948100360 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f4946ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f4948196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f4946ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f4948196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50440/0 (socket says 172.21.15.2:50440) 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f4946ffd700 1 -- 172.21.15.2:0/3728214506 learned_addr learned my addr 172.21.15.2:0/3728214506 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f49467fc700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948100360 0x7f4948196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f4946ffd700 1 -- 172.21.15.2:0/3728214506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948100360 msgr2=0x7f4948196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f4946ffd700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948100360 0x7f4948196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f4946ffd700 1 -- 172.21.15.2:0/3728214506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493800a700 con 0x7f49480ff160 2024-03-16T10:43:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f4946ffd700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f4948196570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4938006010 tx=0x7f493800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f492ffff700 1 -- 172.21.15.2:0/3728214506 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4938004af0 con 0x7f49480ff160 2024-03-16T10:43:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f492ffff700 1 -- 172.21.15.2:0/3728214506 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4938004c50 con 0x7f49480ff160 2024-03-16T10:43:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.289+0000 7f492ffff700 1 -- 172.21.15.2:0/3728214506 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49380166f0 con 0x7f49480ff160 2024-03-16T10:43:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.290+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494819bc10 con 0x7f49480ff160 2024-03-16T10:43:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.290+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494819c100 con 0x7f49480ff160 2024-03-16T10:43:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.291+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4948190460 con 0x7f49480ff160 2024-03-16T10:43:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.291+0000 7f492ffff700 1 -- 172.21.15.2:0/3728214506 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f493800eaa0 con 0x7f49480ff160 2024-03-16T10:43:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.292+0000 7f492ffff700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f493006c880 0x7f493006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.292+0000 7f49467fc700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f493006c880 0x7f493006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.292+0000 7f492ffff700 1 -- 172.21.15.2:0/3728214506 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4938012070 con 0x7f49480ff160 2024-03-16T10:43:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.292+0000 7f49467fc700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f493006c880 0x7f493006ed40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f493c009c60 tx=0x7f493c009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:19.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.296+0000 7f492ffff700 1 -- 172.21.15.2:0/3728214506 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f493805bc00 con 0x7f49480ff160 2024-03-16T10:43:19.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.517+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f49480611f0 con 0x7f493006c880 2024-03-16T10:43:19.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.528+0000 7f492ffff700 1 -- 172.21.15.2:0/3728214506 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f49480611f0 con 0x7f493006c880 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (5m) 57s ago 6m 25.1M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (7m) 57s ago 7m 7436k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (6m) 61s ago 6m 7356k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (7m) 57s ago 7m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (6m) 61s ago 6m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (5m) 57s ago 6m 84.9M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (77s) 57s ago 76s 14.7M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (84s) 57s ago 84s 11.5M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (81s) 61s ago 81s 15.7M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:43:19.529 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (74s) 61s ago 74s 13.3M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:9283,8765,8443 running (8m) 57s ago 8m 497M - 18.2.2-878-ga752c2f7 f692adb9a265 1b8a582e9d84 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (5m) 61s ago 5m 449M - 18.2.2-878-ga752c2f7 f692adb9a265 11a47ca222e7 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (8m) 57s ago 8m 45.3M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (5m) 61s ago 5m 35.1M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (7m) 57s ago 7m 16.3M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (6m) 61s ago 6m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (5m) 57s ago 5m 46.9M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (4m) 57s ago 4m 47.8M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (4m) 57s ago 4m 48.6M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (3m) 61s ago 3m 46.9M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (3m) 61s ago 3m 44.2M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (2m) 61s ago 2m 46.2M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:43:19.530 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (5m) 57s ago 6m 59.0M - 2.43.0 a07b618ecd1d 02e5ed409aab 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f493006c880 msgr2=0x7f493006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f493006c880 0x7f493006ed40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f493c009c60 tx=0x7f493c009260 comp rx=0 tx=0).stop 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 msgr2=0x7f4948196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f4948196570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4938006010 tx=0x7f493800cd20 comp rx=0 tx=0).stop 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 shutdown_connections 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f493006c880 0x7f493006ed40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49480ff160 0x7f4948196570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 --2- 172.21.15.2:0/3728214506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4948100360 0x7f4948196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 >> 172.21.15.2:0/3728214506 conn(0x7f49480fa700 msgr2=0x7f4948103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:19.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 shutdown_connections 2024-03-16T10:43:19.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.530+0000 7f494d699700 1 -- 172.21.15.2:0/3728214506 wait complete. 2024-03-16T10:43:19.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.635+0000 7f47ca9a9700 1 -- 172.21.15.2:0/2466065730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 msgr2=0x7f47c4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.635+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/2466065730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c4100f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009910 tx=0x7f47b4009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:19.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 -- 172.21.15.2:0/2466065730 shutdown_connections 2024-03-16T10:43:19.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/2466065730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c4101ce0 0x7f47c4102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/2466065730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c4100f00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 -- 172.21.15.2:0/2466065730 >> 172.21.15.2:0/2466065730 conn(0x7f47c40fc0a0 msgr2=0x7f47c40fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:19.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 -- 172.21.15.2:0/2466065730 shutdown_connections 2024-03-16T10:43:19.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 -- 172.21.15.2:0/2466065730 wait complete. 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 Processor -- start 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.636+0000 7f47ca9a9700 1 -- start start 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47ca9a9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c41964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47ca9a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c4101ce0 0x7f47c4196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47ca9a9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4197020 con 0x7f47c4100ae0 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47ca9a9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4197160 con 0x7f47c4101ce0 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c41964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c41964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50460/0 (socket says 172.21.15.2:50460) 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c3fff700 1 -- 172.21.15.2:0/3110326266 learned_addr learned my addr 172.21.15.2:0/3110326266 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:19.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c37fe700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c4101ce0 0x7f47c4196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c3fff700 1 -- 172.21.15.2:0/3110326266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c4101ce0 msgr2=0x7f47c4196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c3fff700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c4101ce0 0x7f47c4196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c3fff700 1 -- 172.21.15.2:0/3110326266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40095f0 con 0x7f47c4100ae0 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c3fff700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c41964c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b620 tx=0x7f47b4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c17fa700 1 -- 172.21.15.2:0/3110326266 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b401d070 con 0x7f47c4100ae0 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c419bb60 con 0x7f47c4100ae0 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c17fa700 1 -- 172.21.15.2:0/3110326266 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47b40052a0 con 0x7f47c4100ae0 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.637+0000 7f47c17fa700 1 -- 172.21.15.2:0/3110326266 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b4017400 con 0x7f47c4100ae0 2024-03-16T10:43:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.638+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c419c050 con 0x7f47c4100ae0 2024-03-16T10:43:19.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.639+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c4190660 con 0x7f47c4100ae0 2024-03-16T10:43:19.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.639+0000 7f47c17fa700 1 -- 172.21.15.2:0/3110326266 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f47b4003680 con 0x7f47c4100ae0 2024-03-16T10:43:19.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.639+0000 7f47c17fa700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f47ac06c890 0x7f47ac06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:19.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.640+0000 7f47c37fe700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f47ac06c890 0x7f47ac06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:19.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.640+0000 7f47c17fa700 1 -- 172.21.15.2:0/3110326266 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f47b408d010 con 0x7f47c4100ae0 2024-03-16T10:43:19.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.640+0000 7f47c37fe700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f47ac06c890 0x7f47ac06ed50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009c60 tx=0x7f47b8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:19.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.644+0000 7f47c17fa700 1 -- 172.21.15.2:0/3110326266 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f47b405b0d0 con 0x7f47c4100ae0 2024-03-16T10:43:19.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.958+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f47c404ea80 con 0x7f47c4100ae0 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.959+0000 7f47c17fa700 1 -- 172.21.15.2:0/3110326266 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f47b405ac60 con 0x7f47c4100ae0 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:43:19.961 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 14 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:43:19.962 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f47ac06c890 msgr2=0x7f47ac06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f47ac06c890 0x7f47ac06ed50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009c60 tx=0x7f47b8009260 comp rx=0 tx=0).stop 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 msgr2=0x7f47c41964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c41964c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b620 tx=0x7f47b4005f00 comp rx=0 tx=0).stop 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 shutdown_connections 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f47ac06c890 0x7f47ac06ed50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47c4100ae0 0x7f47c41964c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 --2- 172.21.15.2:0/3110326266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c4101ce0 0x7f47c4196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 >> 172.21.15.2:0/3110326266 conn(0x7f47c40fc0a0 msgr2=0x7f47c41052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:19.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 shutdown_connections 2024-03-16T10:43:19.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:19.961+0000 7f47ca9a9700 1 -- 172.21.15.2:0/3110326266 wait complete. 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 -- 172.21.15.2:0/586538778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 msgr2=0x7f89781060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 --2- 172.21.15.2:0/586538778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f89781060e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8968009910 tx=0x7f8968009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 -- 172.21.15.2:0/586538778 shutdown_connections 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 --2- 172.21.15.2:0/586538778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f89781060e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 --2- 172.21.15.2:0/586538778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89781013c0 0x7f89781037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 -- 172.21.15.2:0/586538778 >> 172.21.15.2:0/586538778 conn(0x7f89780fadf0 msgr2=0x7f89780fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 -- 172.21.15.2:0/586538778 shutdown_connections 2024-03-16T10:43:20.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.071+0000 7f897e99e700 1 -- 172.21.15.2:0/586538778 wait complete. 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897e99e700 1 Processor -- start 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897e99e700 1 -- start start 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897e99e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89781013c0 0x7f89781962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897e99e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f8978196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897e99e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978196e50 con 0x7f8978103cf0 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897e99e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978196f90 con 0x7f89781013c0 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897d19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f8978196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897d19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f8978196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50482/0 (socket says 172.21.15.2:50482) 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897d19b700 1 -- 172.21.15.2:0/3003601752 learned_addr learned my addr 172.21.15.2:0/3003601752 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897d99c700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89781013c0 0x7f89781962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897d19b700 1 -- 172.21.15.2:0/3003601752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89781013c0 msgr2=0x7f89781962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897d19b700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89781013c0 0x7f89781962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.072+0000 7f897d19b700 1 -- 172.21.15.2:0/3003601752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89680095f0 con 0x7f8978103cf0 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.073+0000 7f897d19b700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f8978196830 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f896800b800 tx=0x7f8968005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.073+0000 7f896effd700 1 -- 172.21.15.2:0/3003601752 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f896801d070 con 0x7f8978103cf0 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.073+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897819b990 con 0x7f8978103cf0 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.073+0000 7f896effd700 1 -- 172.21.15.2:0/3003601752 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89680052a0 con 0x7f8978103cf0 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.073+0000 7f896effd700 1 -- 172.21.15.2:0/3003601752 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8968017400 con 0x7f8978103cf0 2024-03-16T10:43:20.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.073+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897819bf00 con 0x7f8978103cf0 2024-03-16T10:43:20.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.074+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897804f2d0 con 0x7f8978103cf0 2024-03-16T10:43:20.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.074+0000 7f896effd700 1 -- 172.21.15.2:0/3003601752 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f8968003780 con 0x7f8978103cf0 2024-03-16T10:43:20.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.075+0000 7f896effd700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f896406c880 0x7f896406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.075+0000 7f896effd700 1 -- 172.21.15.2:0/3003601752 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f896808d150 con 0x7f8978103cf0 2024-03-16T10:43:20.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.075+0000 7f897d99c700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f896406c880 0x7f896406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.075+0000 7f897d99c700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f896406c880 0x7f896406ed40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c60 tx=0x7f8974009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:20.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.083+0000 7f896effd700 1 -- 172.21.15.2:0/3003601752 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f896805b190 con 0x7f8978103cf0 2024-03-16T10:43:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:20 smithi002 ceph-mon[29248]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:20 smithi002 ceph-mon[29248]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:20 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3110326266' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:43:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.356+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f897819c7b0 con 0x7f8978103cf0 2024-03-16T10:43:20.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.356+0000 7f896effd700 1 -- 172.21.15.2:0/3003601752 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f896805ad20 con 0x7f8978103cf0 2024-03-16T10:43:20.358 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:43:20.358 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:43:20.358 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:43:20.358 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:43:20.359 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:43:20.360 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:20.361 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:20.361 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:20.361 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:43:20.361 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:20.361 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:20.361 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:20.361 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f896406c880 msgr2=0x7f896406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f896406c880 0x7f896406ed40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c60 tx=0x7f8974009260 comp rx=0 tx=0).stop 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 msgr2=0x7f8978196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f8978196830 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f896800b800 tx=0x7f8968005f00 comp rx=0 tx=0).stop 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 shutdown_connections 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f896406c880 0x7f896406ed40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89781013c0 0x7f89781962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 --2- 172.21.15.2:0/3003601752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8978103cf0 0x7f8978196830 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.358+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 >> 172.21.15.2:0/3003601752 conn(0x7f89780fadf0 msgr2=0x7f89780fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.359+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 shutdown_connections 2024-03-16T10:43:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.359+0000 7f897e99e700 1 -- 172.21.15.2:0/3003601752 wait complete. 2024-03-16T10:43:20.363 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:43:20.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 -- 172.21.15.2:0/1525587082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810101560 msgr2=0x7fb810103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 --2- 172.21.15.2:0/1525587082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810101560 0x7fb810103950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009910 tx=0x7fb80c009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:20.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 -- 172.21.15.2:0/1525587082 shutdown_connections 2024-03-16T10:43:20.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 --2- 172.21.15.2:0/1525587082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810103e90 0x7fb810106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 --2- 172.21.15.2:0/1525587082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810101560 0x7fb810103950 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 -- 172.21.15.2:0/1525587082 >> 172.21.15.2:0/1525587082 conn(0x7fb8100faf70 msgr2=0x7fb8100fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:20.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 -- 172.21.15.2:0/1525587082 shutdown_connections 2024-03-16T10:43:20.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.471+0000 7fb817370700 1 -- 172.21.15.2:0/1525587082 wait complete. 2024-03-16T10:43:20.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb817370700 1 Processor -- start 2024-03-16T10:43:20.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb817370700 1 -- start start 2024-03-16T10:43:20.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb817370700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810101560 0x7fb810100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb817370700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810103e90 0x7fb8100ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb817370700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100ffa50 con 0x7fb810103e90 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb817370700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100ffb90 con 0x7fb810101560 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81490b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810103e90 0x7fb8100ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81490b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810103e90 0x7fb8100ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50496/0 (socket says 172.21.15.2:50496) 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81510c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810101560 0x7fb810100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81490b700 1 -- 172.21.15.2:0/1677528539 learned_addr learned my addr 172.21.15.2:0/1677528539 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81510c700 1 -- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810103e90 msgr2=0x7fb8100ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81510c700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810103e90 0x7fb8100ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81510c700 1 -- 172.21.15.2:0/1677528539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c0095f0 con 0x7fb810101560 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.472+0000 7fb81490b700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810103e90 0x7fb8100ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.473+0000 7fb81510c700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810101560 0x7fb810100e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb80c006010 tx=0x7fb80c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.473+0000 7fb8067fc700 1 -- 172.21.15.2:0/1677528539 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80c01d070 con 0x7fb810101560 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.473+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8100ffe10 con 0x7fb810101560 2024-03-16T10:43:20.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.473+0000 7fb8067fc700 1 -- 172.21.15.2:0/1677528539 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb80c0052a0 con 0x7fb810101560 2024-03-16T10:43:20.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.473+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb810100380 con 0x7fb810101560 2024-03-16T10:43:20.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.473+0000 7fb8067fc700 1 -- 172.21.15.2:0/1677528539 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80c017400 con 0x7fb810101560 2024-03-16T10:43:20.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.474+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81004ea80 con 0x7fb810101560 2024-03-16T10:43:20.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.475+0000 7fb8067fc700 1 -- 172.21.15.2:0/1677528539 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fb80c003780 con 0x7fb810101560 2024-03-16T10:43:20.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.475+0000 7fb8067fc700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7fc06c880 0x7fb7fc06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.475+0000 7fb8067fc700 1 -- 172.21.15.2:0/1677528539 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb80c08d120 con 0x7fb810101560 2024-03-16T10:43:20.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.475+0000 7fb81490b700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7fc06c880 0x7fb7fc06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.476+0000 7fb81490b700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7fc06c880 0x7fb7fc06ed40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb800011550 tx=0x7fb800009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:20.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:20 smithi026 ceph-mon[37041]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:20.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:20 smithi026 ceph-mon[37041]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:20.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:20 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3110326266' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:43:20.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.483+0000 7fb8067fc700 1 -- 172.21.15.2:0/1677528539 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb80c05b330 con 0x7fb810101560 2024-03-16T10:43:20.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.743+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb810100380 con 0x7fb7fc06c880 2024-03-16T10:43:20.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.745+0000 7fb8067fc700 1 -- 172.21.15.2:0/1677528539 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fb810100380 con 0x7fb7fc06c880 2024-03-16T10:43:20.746 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:43:20.746 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:43:20.746 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:43:20.747 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:43:20.747 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [], 2024-03-16T10:43:20.747 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "0/23 daemons upgraded", 2024-03-16T10:43:20.747 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 image on host smithi026", 2024-03-16T10:43:20.747 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:43:20.747 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7fc06c880 msgr2=0x7fb7fc06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7fc06c880 0x7fb7fc06ed40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb800011550 tx=0x7fb800009260 comp rx=0 tx=0).stop 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810101560 msgr2=0x7fb810100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810101560 0x7fb810100e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb80c006010 tx=0x7fb80c005f00 comp rx=0 tx=0).stop 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 shutdown_connections 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb7fc06c880 0x7fb7fc06ed40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb810101560 0x7fb810100e70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 --2- 172.21.15.2:0/1677528539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb810103e90 0x7fb8100ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 >> 172.21.15.2:0/1677528539 conn(0x7fb8100faf70 msgr2=0x7fb8100fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:20.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 shutdown_connections 2024-03-16T10:43:20.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.747+0000 7fb817370700 1 -- 172.21.15.2:0/1677528539 wait complete. 2024-03-16T10:43:20.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 -- 172.21.15.2:0/304338618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72040ff9e0 msgr2=0x7f72040ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 --2- 172.21.15.2:0/304338618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72040ff9e0 0x7f72040ffe00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009910 tx=0x7f71f4009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:20.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 -- 172.21.15.2:0/304338618 shutdown_connections 2024-03-16T10:43:20.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 --2- 172.21.15.2:0/304338618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7204100be0 0x7f7204101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 --2- 172.21.15.2:0/304338618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72040ff9e0 0x7f72040ffe00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 -- 172.21.15.2:0/304338618 >> 172.21.15.2:0/304338618 conn(0x7f72040faf80 msgr2=0x7f72040fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 -- 172.21.15.2:0/304338618 shutdown_connections 2024-03-16T10:43:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 -- 172.21.15.2:0/304338618 wait complete. 2024-03-16T10:43:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.856+0000 7f720941d700 1 Processor -- start 2024-03-16T10:43:20.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f720941d700 1 -- start start 2024-03-16T10:43:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f720941d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72040ff9e0 0x7f7204069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f720941d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7204100be0 0x7f720419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f720941d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204197070 con 0x7f7204100be0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f720941d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720419dbd0 con 0x7f72040ff9e0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f72027fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7204100be0 0x7f720419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f72027fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7204100be0 0x7f720419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50522/0 (socket says 172.21.15.2:50522) 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f72027fc700 1 -- 172.21.15.2:0/2574022423 learned_addr learned my addr 172.21.15.2:0/2574022423 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f7202ffd700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72040ff9e0 0x7f7204069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f72027fc700 1 -- 172.21.15.2:0/2574022423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72040ff9e0 msgr2=0x7f7204069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f72027fc700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72040ff9e0 0x7f7204069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f72027fc700 1 -- 172.21.15.2:0/2574022423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40095f0 con 0x7f7204100be0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.857+0000 7f72027fc700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7204100be0 0x7f720419d690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e920 tx=0x7f71f800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.858+0000 7f71ebfff700 1 -- 172.21.15.2:0/2574022423 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f800c9d0 con 0x7f7204100be0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.858+0000 7f71ebfff700 1 -- 172.21.15.2:0/2574022423 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71f8010430 con 0x7f7204100be0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.858+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720419ddd0 con 0x7f7204100be0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.858+0000 7f71ebfff700 1 -- 172.21.15.2:0/2574022423 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71f8018650 con 0x7f7204100be0 2024-03-16T10:43:20.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.858+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720419e320 con 0x7f7204100be0 2024-03-16T10:43:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.859+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720418f560 con 0x7f7204100be0 2024-03-16T10:43:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.859+0000 7f71ebfff700 1 -- 172.21.15.2:0/2574022423 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f71f80105a0 con 0x7f7204100be0 2024-03-16T10:43:20.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.860+0000 7f71ebfff700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f71ec06c890 0x7f71ec06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.860+0000 7f71ebfff700 1 -- 172.21.15.2:0/2574022423 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f71f8014070 con 0x7f7204100be0 2024-03-16T10:43:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.860+0000 7f7202ffd700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f71ec06c890 0x7f71ec06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:20.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.861+0000 7f7202ffd700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f71ec06c890 0x7f71ec06ed50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71f400b800 tx=0x7f71f4009f70 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:20.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:20.865+0000 7f71ebfff700 1 -- 172.21.15.2:0/2574022423 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f71f80529f0 con 0x7f7204100be0 2024-03-16T10:43:21.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.199+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f720404ea80 con 0x7f7204100be0 2024-03-16T10:43:21.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.199+0000 7f71ebfff700 1 -- 172.21.15.2:0/2574022423 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f71f805a000 con 0x7f7204100be0 2024-03-16T10:43:21.201 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:43:21.202 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:43:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f71ec06c890 msgr2=0x7f71ec06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f71ec06c890 0x7f71ec06ed50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f71f400b800 tx=0x7f71f4009f70 comp rx=0 tx=0).stop 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7204100be0 msgr2=0x7f720419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7204100be0 0x7f720419d690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e920 tx=0x7f71f800ec30 comp rx=0 tx=0).stop 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 shutdown_connections 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f71ec06c890 0x7f71ec06ed50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f72040ff9e0 0x7f7204069ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 --2- 172.21.15.2:0/2574022423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7204100be0 0x7f720419d690 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 >> 172.21.15.2:0/2574022423 conn(0x7f72040faf80 msgr2=0x7f72041041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 shutdown_connections 2024-03-16T10:43:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:21.202+0000 7f720941d700 1 -- 172.21.15.2:0/2574022423 wait complete. 2024-03-16T10:43:21.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:21 smithi026 ceph-mon[37041]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:21.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:21 smithi026 ceph-mon[37041]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:21.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:21 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3003601752' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:43:21.542 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:21 smithi002 ceph-mon[29248]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:21.542 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:21 smithi002 ceph-mon[29248]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:21.542 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:21 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3003601752' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:43:22.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:22 smithi026 ceph-mon[37041]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:22.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:22 smithi026 ceph-mon[37041]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:22.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:22 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2574022423' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:43:22.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:22 smithi002 ceph-mon[29248]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:22.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:22 smithi002 ceph-mon[29248]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:22.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:22 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2574022423' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:43:24.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:24 smithi026 ceph-mon[37041]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:24 smithi002 ceph-mon[29248]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:26.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:26 smithi002 ceph-mon[29248]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:26.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:26 smithi026 ceph-mon[37041]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:28.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:28 smithi026 ceph-mon[37041]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:28.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:28 smithi002 ceph-mon[29248]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:30.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:30 smithi026 ceph-mon[37041]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:30.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:30 smithi002 ceph-mon[29248]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:31 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:31.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:31 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:32.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:32 smithi026 ceph-mon[37041]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:32.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:32 smithi002 ceph-mon[29248]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:34.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:34 smithi026 ceph-mon[37041]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:34.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:34 smithi002 ceph-mon[29248]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:36 smithi026 ceph-mon[37041]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:36 smithi002 ceph-mon[29248]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:38.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:38 smithi026 ceph-mon[37041]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:38.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:38 smithi002 ceph-mon[29248]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:40.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:40 smithi026 ceph-mon[37041]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:40.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:40 smithi002 ceph-mon[29248]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:42.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:42 smithi026 ceph-mon[37041]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:42.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:42 smithi002 ceph-mon[29248]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:44.404 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:44 smithi026 ceph-mon[37041]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:44.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:44 smithi002 ceph-mon[29248]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:46.471 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:46 smithi026 ceph-mon[37041]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:46.471 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:46.471 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:46.471 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:43:46.471 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:43:46.471 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:46 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:43:46.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:46 smithi002 ceph-mon[29248]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:46.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:43:46.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:46.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:43:46.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:43:46.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:46 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:43:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:47 smithi026 ceph-mon[37041]: Upgrade: Updating mgr.smithi026.omjkpo 2024-03-16T10:43:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:47 smithi026 ceph-mon[37041]: Deploying daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:43:47.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:47 smithi002 ceph-mon[29248]: Upgrade: Updating mgr.smithi026.omjkpo 2024-03-16T10:43:47.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:47 smithi002 ceph-mon[29248]: Deploying daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:43:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:48 smithi026 ceph-mon[37041]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:48.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:48 smithi002 ceph-mon[29248]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:49.279 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:49 smithi026 ceph-mon[37041]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:49.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:49 smithi002 ceph-mon[29248]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:50.853 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:50.854 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:50.854 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:50 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:43:51.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:51.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:51.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:50 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 -- 172.21.15.2:0/2774065211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 msgr2=0x7faf681021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2774065211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf681021e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf54009910 tx=0x7faf54009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 -- 172.21.15.2:0/2774065211 shutdown_connections 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2774065211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf681021e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2774065211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf680feda0 0x7faf680ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 -- 172.21.15.2:0/2774065211 >> 172.21.15.2:0/2774065211 conn(0x7faf680fa700 msgr2=0x7faf680fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 -- 172.21.15.2:0/2774065211 shutdown_connections 2024-03-16T10:43:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 -- 172.21.15.2:0/2774065211 wait complete. 2024-03-16T10:43:51.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.316+0000 7faf6ea68700 1 Processor -- start 2024-03-16T10:43:51.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf6ea68700 1 -- start start 2024-03-16T10:43:51.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf6ea68700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf680feda0 0x7faf68198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:51.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf6ea68700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf68198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:51.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf6ea68700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681992c0 con 0x7faf68101d80 2024-03-16T10:43:51.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf6ea68700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf68199400 con 0x7faf680feda0 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf67fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf68198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf67fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf68198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60284/0 (socket says 172.21.15.2:60284) 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf67fff700 1 -- 172.21.15.2:0/2702502049 learned_addr learned my addr 172.21.15.2:0/2702502049 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf6c804700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf680feda0 0x7faf68198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf67fff700 1 -- 172.21.15.2:0/2702502049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf680feda0 msgr2=0x7faf68198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf67fff700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf680feda0 0x7faf68198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf67fff700 1 -- 172.21.15.2:0/2702502049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540095f0 con 0x7faf68101d80 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.317+0000 7faf67fff700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf68198ca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faf5400bd70 tx=0x7faf5400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.318+0000 7faf65ffb700 1 -- 172.21.15.2:0/2702502049 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf5401d070 con 0x7faf68101d80 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.318+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6819de00 con 0x7faf68101d80 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.318+0000 7faf65ffb700 1 -- 172.21.15.2:0/2702502049 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf5400f460 con 0x7faf68101d80 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.318+0000 7faf65ffb700 1 -- 172.21.15.2:0/2702502049 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf54017630 con 0x7faf68101d80 2024-03-16T10:43:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.318+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6819e340 con 0x7faf68101d80 2024-03-16T10:43:51.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.319+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf68106750 con 0x7faf68101d80 2024-03-16T10:43:51.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.320+0000 7faf65ffb700 1 -- 172.21.15.2:0/2702502049 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7faf540178a0 con 0x7faf68101d80 2024-03-16T10:43:51.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.320+0000 7faf65ffb700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faf5806c880 0x7faf5806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:51.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.320+0000 7faf65ffb700 1 -- 172.21.15.2:0/2702502049 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7faf5408d0d0 con 0x7faf68101d80 2024-03-16T10:43:51.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.320+0000 7faf6c804700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faf5806c880 0x7faf5806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:51.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.321+0000 7faf6c804700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faf5806c880 0x7faf5806ed40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf68068280 tx=0x7faf5c00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:51.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.328+0000 7faf65ffb700 1 -- 172.21.15.2:0/2702502049 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faf5405b180 con 0x7faf68101d80 2024-03-16T10:43:51.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.553+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf680611f0 con 0x7faf5806c880 2024-03-16T10:43:51.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.554+0000 7faf65ffb700 1 -- 172.21.15.2:0/2702502049 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7faf680611f0 con 0x7faf5806c880 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faf5806c880 msgr2=0x7faf5806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faf5806c880 0x7faf5806ed40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf68068280 tx=0x7faf5c00b410 comp rx=0 tx=0).stop 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 msgr2=0x7faf68198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf68198ca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faf5400bd70 tx=0x7faf5400bda0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 shutdown_connections 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7faf5806c880 0x7faf5806ed40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf680feda0 0x7faf68198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 --2- 172.21.15.2:0/2702502049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf68101d80 0x7faf68198ca0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 >> 172.21.15.2:0/2702502049 conn(0x7faf680fa700 msgr2=0x7faf680fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 shutdown_connections 2024-03-16T10:43:51.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.556+0000 7faf6ea68700 1 -- 172.21.15.2:0/2702502049 wait complete. 2024-03-16T10:43:51.568 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 -- 172.21.15.2:0/83074013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 msgr2=0x7f3a14101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/83074013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14101030 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009910 tx=0x7f3a00009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 -- 172.21.15.2:0/83074013 shutdown_connections 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/83074013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14101030 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/83074013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a140ff9d0 0x7f3a140ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 -- 172.21.15.2:0/83074013 >> 172.21.15.2:0/83074013 conn(0x7f3a140faf70 msgr2=0x7f3a140fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 -- 172.21.15.2:0/83074013 shutdown_connections 2024-03-16T10:43:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.666+0000 7f3a1afb4700 1 -- 172.21.15.2:0/83074013 wait complete. 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a1afb4700 1 Processor -- start 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a1afb4700 1 -- start start 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a1afb4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a140ff9d0 0x7f3a14196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a1afb4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a1afb4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a141970e0 con 0x7f3a14100bd0 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a1afb4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14197220 con 0x7f3a140ff9d0 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a13fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a13fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60304/0 (socket says 172.21.15.2:60304) 2024-03-16T10:43:51.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a13fff700 1 -- 172.21.15.2:0/1151116070 learned_addr learned my addr 172.21.15.2:0/1151116070 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a18d50700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a140ff9d0 0x7f3a14196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a13fff700 1 -- 172.21.15.2:0/1151116070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a140ff9d0 msgr2=0x7f3a14196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.667+0000 7f3a13fff700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a140ff9d0 0x7f3a14196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.668+0000 7f3a13fff700 1 -- 172.21.15.2:0/1151116070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a000095f0 con 0x7f3a14100bd0 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.668+0000 7f3a13fff700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14196ac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3a000092e0 tx=0x7f3a0000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.668+0000 7f3a11ffb700 1 -- 172.21.15.2:0/1151116070 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a0001d070 con 0x7f3a14100bd0 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.668+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1419bbd0 con 0x7f3a14100bd0 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.668+0000 7f3a11ffb700 1 -- 172.21.15.2:0/1151116070 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a0000f460 con 0x7f3a14100bd0 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.668+0000 7f3a11ffb700 1 -- 172.21.15.2:0/1151116070 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a00021620 con 0x7f3a14100bd0 2024-03-16T10:43:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.668+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1419c140 con 0x7f3a14100bd0 2024-03-16T10:43:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.669+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1404ea80 con 0x7f3a14100bd0 2024-03-16T10:43:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.670+0000 7f3a11ffb700 1 -- 172.21.15.2:0/1151116070 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f3a0000f5d0 con 0x7f3a14100bd0 2024-03-16T10:43:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.670+0000 7f3a11ffb700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a0406c880 0x7f3a0406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.670+0000 7f3a11ffb700 1 -- 172.21.15.2:0/1151116070 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3a0008cf90 con 0x7f3a14100bd0 2024-03-16T10:43:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.670+0000 7f3a18d50700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a0406c880 0x7f3a0406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.671+0000 7f3a18d50700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a0406c880 0x7f3a0406ed40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3a14100a30 tx=0x7f3a0800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.675+0000 7f3a11ffb700 1 -- 172.21.15.2:0/1151116070 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3a0005af30 con 0x7f3a14100bd0 2024-03-16T10:43:51.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.898+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a14068c40 con 0x7f3a0406c880 2024-03-16T10:43:51.899 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:51 smithi002 ceph-mon[29248]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:51.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.899+0000 7f3a11ffb700 1 -- 172.21.15.2:0/1151116070 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f3a14068c40 con 0x7f3a0406c880 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.901+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a0406c880 msgr2=0x7f3a0406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.901+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a0406c880 0x7f3a0406ed40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3a14100a30 tx=0x7f3a0800b410 comp rx=0 tx=0).stop 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.901+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 msgr2=0x7f3a14196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.901+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14196ac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3a000092e0 tx=0x7f3a0000bd70 comp rx=0 tx=0).stop 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.902+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 shutdown_connections 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.902+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f3a0406c880 0x7f3a0406ed40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.902+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a140ff9d0 0x7f3a14196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.902+0000 7f3a1afb4700 1 --2- 172.21.15.2:0/1151116070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a14100bd0 0x7f3a14196ac0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.902+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 >> 172.21.15.2:0/1151116070 conn(0x7f3a140faf70 msgr2=0x7f3a14068530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.902+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 shutdown_connections 2024-03-16T10:43:51.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:51.902+0000 7f3a1afb4700 1 -- 172.21.15.2:0/1151116070 wait complete. 2024-03-16T10:43:51.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:51 smithi026 ceph-mon[37041]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 -- 172.21.15.2:0/1042440059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 msgr2=0x7fb928101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/1042440059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928101410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb924009910 tx=0x7fb924009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 -- 172.21.15.2:0/1042440059 shutdown_connections 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/1042440059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280ffc50 0x7fb9281000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/1042440059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928101410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 -- 172.21.15.2:0/1042440059 >> 172.21.15.2:0/1042440059 conn(0x7fb9280fa700 msgr2=0x7fb9280fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 -- 172.21.15.2:0/1042440059 shutdown_connections 2024-03-16T10:43:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.009+0000 7fb92f8cc700 1 -- 172.21.15.2:0/1042440059 wait complete. 2024-03-16T10:43:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92f8cc700 1 Processor -- start 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92f8cc700 1 -- start start 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92f8cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280ffc50 0x7fb928196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92f8cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92f8cc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9281970c0 con 0x7fb928100ff0 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92f8cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb928197200 con 0x7fb9280ffc50 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92ce67700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92ce67700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60334/0 (socket says 172.21.15.2:60334) 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92ce67700 1 -- 172.21.15.2:0/2348598124 learned_addr learned my addr 172.21.15.2:0/2348598124 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92d668700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280ffc50 0x7fb928196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92ce67700 1 -- 172.21.15.2:0/2348598124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280ffc50 msgr2=0x7fb928196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92ce67700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280ffc50 0x7fb928196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.010+0000 7fb92ce67700 1 -- 172.21.15.2:0/2348598124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9240095f0 con 0x7fb928100ff0 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.011+0000 7fb92ce67700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928196aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d8d0 tx=0x7fb91c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.011+0000 7fb91a7fc700 1 -- 172.21.15.2:0/2348598124 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91c009940 con 0x7fb928100ff0 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.011+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92819bc10 con 0x7fb928100ff0 2024-03-16T10:43:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.011+0000 7fb91a7fc700 1 -- 172.21.15.2:0/2348598124 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb91c010460 con 0x7fb928100ff0 2024-03-16T10:43:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.011+0000 7fb91a7fc700 1 -- 172.21.15.2:0/2348598124 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91c00f5f0 con 0x7fb928100ff0 2024-03-16T10:43:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.011+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92819c1e0 con 0x7fb928100ff0 2024-03-16T10:43:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.012+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92804ea80 con 0x7fb928100ff0 2024-03-16T10:43:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.012+0000 7fb91a7fc700 1 -- 172.21.15.2:0/2348598124 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fb91c0105d0 con 0x7fb928100ff0 2024-03-16T10:43:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.013+0000 7fb91a7fc700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb91406c880 0x7fb91406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.013+0000 7fb91a7fc700 1 -- 172.21.15.2:0/2348598124 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb91c08bd20 con 0x7fb928100ff0 2024-03-16T10:43:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.013+0000 7fb92d668700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb91406c880 0x7fb91406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.013+0000 7fb92d668700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb91406c880 0x7fb91406ed40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9240051c0 tx=0x7fb92401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:52.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.017+0000 7fb91a7fc700 1 -- 172.21.15.2:0/2348598124 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb91c059d60 con 0x7fb928100ff0 2024-03-16T10:43:52.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.243+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9280ff250 con 0x7fb91406c880 2024-03-16T10:43:52.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.253+0000 7fb91a7fc700 1 -- 172.21.15.2:0/2348598124 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb9280ff250 con 0x7fb91406c880 2024-03-16T10:43:52.254 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:43:52.254 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (6m) 89s ago 7m 25.1M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:43:52.254 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (7m) 89s ago 7m 7436k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (6m) 94s ago 6m 7356k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (7m) 89s ago 7m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (6m) 94s ago 6m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (5m) 89s ago 7m 84.9M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (110s) 89s ago 109s 14.7M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (117s) 89s ago 116s 11.5M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (114s) 94s ago 114s 15.7M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (107s) 94s ago 106s 13.3M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:9283,8765,8443 running (9m) 89s ago 9m 497M - 18.2.2-878-ga752c2f7 f692adb9a265 1b8a582e9d84 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 starting - - - - 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (9m) 89s ago 9m 45.3M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (6m) 94s ago 6m 35.1M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (7m) 89s ago 7m 16.3M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (6m) 94s ago 6m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (5m) 89s ago 5m 46.9M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:43:52.255 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (5m) 89s ago 5m 47.8M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:43:52.256 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (4m) 89s ago 4m 48.6M 3948M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:43:52.256 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (4m) 94s ago 4m 46.9M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:43:52.256 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (3m) 94s ago 3m 44.2M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:43:52.256 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (3m) 94s ago 3m 46.2M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:43:52.256 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (5m) 89s ago 7m 59.0M - 2.43.0 a07b618ecd1d 02e5ed409aab 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb91406c880 msgr2=0x7fb91406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb91406c880 0x7fb91406ed40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9240051c0 tx=0x7fb92401a040 comp rx=0 tx=0).stop 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 msgr2=0x7fb928196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928196aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d8d0 tx=0x7fb91c00dc90 comp rx=0 tx=0).stop 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 shutdown_connections 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fb91406c880 0x7fb91406ed40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280ffc50 0x7fb928196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 --2- 172.21.15.2:0/2348598124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb928100ff0 0x7fb928196aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 >> 172.21.15.2:0/2348598124 conn(0x7fb9280fa700 msgr2=0x7fb9280feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 shutdown_connections 2024-03-16T10:43:52.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.256+0000 7fb92f8cc700 1 -- 172.21.15.2:0/2348598124 wait complete. 2024-03-16T10:43:52.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 -- 172.21.15.2:0/223972986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89600ff9f0 msgr2=0x7f89600ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 --2- 172.21.15.2:0/223972986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89600ff9f0 0x7f89600ffe10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8950009910 tx=0x7f8950009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:52.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 -- 172.21.15.2:0/223972986 shutdown_connections 2024-03-16T10:43:52.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 --2- 172.21.15.2:0/223972986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8960100bf0 0x7f8960101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 --2- 172.21.15.2:0/223972986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89600ff9f0 0x7f89600ffe10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 -- 172.21.15.2:0/223972986 >> 172.21.15.2:0/223972986 conn(0x7f89600faf90 msgr2=0x7f89600fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 -- 172.21.15.2:0/223972986 shutdown_connections 2024-03-16T10:43:52.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.359+0000 7f89664cd700 1 -- 172.21.15.2:0/223972986 wait complete. 2024-03-16T10:43:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.360+0000 7f89664cd700 1 Processor -- start 2024-03-16T10:43:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.360+0000 7f89664cd700 1 -- start start 2024-03-16T10:43:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.360+0000 7f89664cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89600ff9f0 0x7f89601038e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.360+0000 7f89664cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8960100bf0 0x7f8960101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.360+0000 7f89664cd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8960102510 con 0x7f8960100bf0 2024-03-16T10:43:52.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.360+0000 7f89664cd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8960102650 con 0x7f89600ff9f0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8960100bf0 0x7f8960101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8960100bf0 0x7f8960101fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60364/0 (socket says 172.21.15.2:60364) 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895f7fe700 1 -- 172.21.15.2:0/3159994394 learned_addr learned my addr 172.21.15.2:0/3159994394 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895ffff700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89600ff9f0 0x7f89601038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895f7fe700 1 -- 172.21.15.2:0/3159994394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89600ff9f0 msgr2=0x7f89601038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895f7fe700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89600ff9f0 0x7f89601038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895f7fe700 1 -- 172.21.15.2:0/3159994394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89500095f0 con 0x7f8960100bf0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895f7fe700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8960100bf0 0x7f8960101fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f895400d8d0 tx=0x7f895400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895d7fa700 1 -- 172.21.15.2:0/3159994394 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89540098a0 con 0x7f8960100bf0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8960102930 con 0x7f8960100bf0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895d7fa700 1 -- 172.21.15.2:0/3159994394 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8954010460 con 0x7f8960100bf0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f895d7fa700 1 -- 172.21.15.2:0/3159994394 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f895400f5f0 con 0x7f8960100bf0 2024-03-16T10:43:52.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.361+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896019e620 con 0x7f8960100bf0 2024-03-16T10:43:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.362+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896004ea80 con 0x7f8960100bf0 2024-03-16T10:43:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.363+0000 7f895d7fa700 1 -- 172.21.15.2:0/3159994394 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f8954009a00 con 0x7f8960100bf0 2024-03-16T10:43:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.363+0000 7f895d7fa700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f894806c880 0x7f894806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.363+0000 7f895d7fa700 1 -- 172.21.15.2:0/3159994394 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f895408bba0 con 0x7f8960100bf0 2024-03-16T10:43:52.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.363+0000 7f895ffff700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f894806c880 0x7f894806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.364+0000 7f895ffff700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f894806c880 0x7f894806ed40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89500051c0 tx=0x7f895001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:52.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.368+0000 7f895d7fa700 1 -- 172.21.15.2:0/3159994394 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8954059be0 con 0x7f8960100bf0 2024-03-16T10:43:52.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.655+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f896019e7b0 con 0x7f8960100bf0 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.656+0000 7f895d7fa700 1 -- 172.21.15.2:0/3159994394 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f8954059770 con 0x7f8960100bf0 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:43:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:43:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:43:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:43:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:43:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 14 2024-03-16T10:43:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:43:52.658 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f894806c880 msgr2=0x7f894806ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f894806c880 0x7f894806ed40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89500051c0 tx=0x7f895001a040 comp rx=0 tx=0).stop 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8960100bf0 msgr2=0x7f8960101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8960100bf0 0x7f8960101fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f895400d8d0 tx=0x7f895400dbe0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 shutdown_connections 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f894806c880 0x7f894806ed40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89600ff9f0 0x7f89601038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 --2- 172.21.15.2:0/3159994394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8960100bf0 0x7f8960101fd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 >> 172.21.15.2:0/3159994394 conn(0x7f89600faf90 msgr2=0x7f89600690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 shutdown_connections 2024-03-16T10:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.658+0000 7f89664cd700 1 -- 172.21.15.2:0/3159994394 wait complete. 2024-03-16T10:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 -- 172.21.15.2:0/3111009558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 msgr2=0x7f4f500ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/3111009558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f500ffdf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009910 tx=0x7f4f44009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 -- 172.21.15.2:0/3111009558 shutdown_connections 2024-03-16T10:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/3111009558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f50100bd0 0x7f4f50101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/3111009558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f500ffdf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 -- 172.21.15.2:0/3111009558 >> 172.21.15.2:0/3111009558 conn(0x7f4f500faf70 msgr2=0x7f4f500fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 -- 172.21.15.2:0/3111009558 shutdown_connections 2024-03-16T10:43:52.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.767+0000 7f4f56fa0700 1 -- 172.21.15.2:0/3111009558 wait complete. 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f56fa0700 1 Processor -- start 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f56fa0700 1 -- start start 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f56fa0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f50195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f56fa0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f50100bd0 0x7f4f50195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f56fa0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f50195fa0 con 0x7f4f500ff9d0 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f56fa0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501960e0 con 0x7f4f50100bd0 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f54d3c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f50195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f54d3c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f50195440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33636/0 (socket says 172.21.15.2:33636) 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f54d3c700 1 -- 172.21.15.2:0/1816649621 learned_addr learned my addr 172.21.15.2:0/1816649621 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.768+0000 7f4f4ffff700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f50100bd0 0x7f4f50195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f54d3c700 1 -- 172.21.15.2:0/1816649621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f50100bd0 msgr2=0x7f4f50195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f54d3c700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f50100bd0 0x7f4f50195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f54d3c700 1 -- 172.21.15.2:0/1816649621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f440095f0 con 0x7f4f500ff9d0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f54d3c700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f50195440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4f440059d0 tx=0x7f4f4400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f4dffb700 1 -- 172.21.15.2:0/1816649621 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f4401d070 con 0x7f4f500ff9d0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5019aa90 con 0x7f4f500ff9d0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f4dffb700 1 -- 172.21.15.2:0/1816649621 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f4400f460 con 0x7f4f500ff9d0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f4dffb700 1 -- 172.21.15.2:0/1816649621 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f44021640 con 0x7f4f500ff9d0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.769+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f500686d0 con 0x7f4f500ff9d0 2024-03-16T10:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.770+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5004ea80 con 0x7f4f500ff9d0 2024-03-16T10:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.771+0000 7f4f4dffb700 1 -- 172.21.15.2:0/1816649621 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f4f44003a60 con 0x7f4f500ff9d0 2024-03-16T10:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.772+0000 7f4f4dffb700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f4006ea90 0x7f4f40070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.772+0000 7f4f4dffb700 1 -- 172.21.15.2:0/1816649621 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4f4408cf30 con 0x7f4f500ff9d0 2024-03-16T10:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.772+0000 7f4f4ffff700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f4006ea90 0x7f4f40070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:52.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.772+0000 7f4f4ffff700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f4006ea90 0x7f4f40070f50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005950 tx=0x7f4f3c00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:52.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:52.776+0000 7f4f4dffb700 1 -- 172.21.15.2:0/1816649621 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4f440c09f0 con 0x7f4f500ff9d0 2024-03-16T10:43:52.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:52 smithi026 ceph-mon[37041]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:52.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:52 smithi026 ceph-mon[37041]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:52.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:52 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3159994394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:43:53.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.022+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4f50069330 con 0x7f4f500ff9d0 2024-03-16T10:43:53.023 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:52 smithi002 ceph-mon[29248]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:53.023 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:52 smithi002 ceph-mon[29248]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:53.023 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:52 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3159994394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.023+0000 7f4f4dffb700 1 -- 172.21.15.2:0/1816649621 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f4f4405af70 con 0x7f4f500ff9d0 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:43:53.024 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:43:53.025 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:53.026 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:53.027 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:43:53.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f4006ea90 msgr2=0x7f4f40070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f4006ea90 0x7f4f40070f50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005950 tx=0x7f4f3c00b410 comp rx=0 tx=0).stop 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 msgr2=0x7f4f50195440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f50195440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4f440059d0 tx=0x7f4f4400bd70 comp rx=0 tx=0).stop 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 shutdown_connections 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f4f4006ea90 0x7f4f40070f50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f500ff9d0 0x7f4f50195440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 --2- 172.21.15.2:0/1816649621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f50100bd0 0x7f4f50195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 >> 172.21.15.2:0/1816649621 conn(0x7f4f500faf70 msgr2=0x7f4f50104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 shutdown_connections 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.025+0000 7f4f56fa0700 1 -- 172.21.15.2:0/1816649621 wait complete. 2024-03-16T10:43:53.028 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/3651316277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8101d00 msgr2=0x7fc9a8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/3651316277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8101d00 0x7fc9a8102160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc990009910 tx=0x7fc990009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/3651316277 shutdown_connections 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/3651316277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8101d00 0x7fc9a8102160 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/3651316277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a8100b00 0x7fc9a8100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/3651316277 >> 172.21.15.2:0/3651316277 conn(0x7fc9a80fc0a0 msgr2=0x7fc9a80fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/3651316277 shutdown_connections 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.140+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/3651316277 wait complete. 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9ae5a9700 1 Processor -- start 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9ae5a9700 1 -- start start 2024-03-16T10:43:53.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9ae5a9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8100b00 0x7fc9a8196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:53.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9ae5a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a8101d00 0x7fc9a8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:53.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9ae5a9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8197070 con 0x7fc9a8100b00 2024-03-16T10:43:53.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9ae5a9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81971b0 con 0x7fc9a8101d00 2024-03-16T10:43:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9a7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8100b00 0x7fc9a8196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a8101d00 0x7fc9a8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a8101d00 0x7fc9a8196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:57798/0 (socket says 172.21.15.2:57798) 2024-03-16T10:43:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.141+0000 7fc9a77fe700 1 -- 172.21.15.2:0/1828132066 learned_addr learned my addr 172.21.15.2:0/1828132066 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9a7fff700 1 -- 172.21.15.2:0/1828132066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a8101d00 msgr2=0x7fc9a8196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9a7fff700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a8101d00 0x7fc9a8196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9a7fff700 1 -- 172.21.15.2:0/1828132066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900095f0 con 0x7fc9a8100b00 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9a7fff700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8100b00 0x7fc9a8196510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc99800d6a0 tx=0x7fc99800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9a57fa700 1 -- 172.21.15.2:0/1828132066 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99800f860 con 0x7fc9a8100b00 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9a57fa700 1 -- 172.21.15.2:0/1828132066 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc99800fe80 con 0x7fc9a8100b00 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a819bbc0 con 0x7fc9a8100b00 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9a57fa700 1 -- 172.21.15.2:0/1828132066 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9980185f0 con 0x7fc9a8100b00 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.142+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a819c110 con 0x7fc9a8100b00 2024-03-16T10:43:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.143+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a8190680 con 0x7fc9a8100b00 2024-03-16T10:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.144+0000 7fc9a57fa700 1 -- 172.21.15.2:0/1828132066 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7fc998010460 con 0x7fc9a8100b00 2024-03-16T10:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.144+0000 7fc9a57fa700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc99406c880 0x7fc99406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.144+0000 7fc9a57fa700 1 -- 172.21.15.2:0/1828132066 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc99808bd10 con 0x7fc9a8100b00 2024-03-16T10:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.144+0000 7fc9a77fe700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc99406c880 0x7fc99406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.145+0000 7fc9a77fe700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc99406c880 0x7fc99406ed40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9900051c0 tx=0x7fc99000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.149+0000 7fc9a57fa700 1 -- 172.21.15.2:0/1828132066 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc998059cb0 con 0x7fc9a8100b00 2024-03-16T10:43:53.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.372+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 --> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9a80611f0 con 0x7fc99406c880 2024-03-16T10:43:53.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.374+0000 7fc9a57fa700 1 -- 172.21.15.2:0/1828132066 <== mgr.14213 v2:172.21.15.2:6800/3195213211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fc9a80611f0 con 0x7fc99406c880 2024-03-16T10:43:53.374 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [], 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "1/23 daemons upgraded", 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading mgr daemons", 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:43:53.375 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc99406c880 msgr2=0x7fc99406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc99406c880 0x7fc99406ed40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9900051c0 tx=0x7fc99000b540 comp rx=0 tx=0).stop 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8100b00 msgr2=0x7fc9a8196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8100b00 0x7fc9a8196510 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc99800d6a0 tx=0x7fc99800d9b0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 shutdown_connections 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7fc99406c880 0x7fc99406ed40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9a8100b00 0x7fc9a8196510 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 --2- 172.21.15.2:0/1828132066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9a8101d00 0x7fc9a8196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.376+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 >> 172.21.15.2:0/1828132066 conn(0x7fc9a80fc0a0 msgr2=0x7fc9a81052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:53.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.377+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 shutdown_connections 2024-03-16T10:43:53.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.377+0000 7fc9ae5a9700 1 -- 172.21.15.2:0/1828132066 wait complete. 2024-03-16T10:43:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.483+0000 7f329ae47700 1 -- 172.21.15.2:0/3602316038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294101ce0 msgr2=0x7f3294102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.483+0000 7f329ae47700 1 --2- 172.21.15.2:0/3602316038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294101ce0 0x7f3294102140 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3280009910 tx=0x7f3280009c20 comp rx=0 tx=0).stop 2024-03-16T10:43:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.483+0000 7f329ae47700 1 -- 172.21.15.2:0/3602316038 shutdown_connections 2024-03-16T10:43:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.483+0000 7f329ae47700 1 --2- 172.21.15.2:0/3602316038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294101ce0 0x7f3294102140 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.483+0000 7f329ae47700 1 --2- 172.21.15.2:0/3602316038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3294100ae0 0x7f3294100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.483+0000 7f329ae47700 1 -- 172.21.15.2:0/3602316038 >> 172.21.15.2:0/3602316038 conn(0x7f32940fc0a0 msgr2=0x7f32940fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 -- 172.21.15.2:0/3602316038 shutdown_connections 2024-03-16T10:43:53.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 -- 172.21.15.2:0/3602316038 wait complete. 2024-03-16T10:43:53.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 Processor -- start 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 -- start start 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294100ae0 0x7f32941964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3294101ce0 0x7f3294196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294197020 con 0x7f3294100ae0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f329ae47700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294197160 con 0x7f3294101ce0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f3298be3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294100ae0 0x7f32941964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f3298be3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294100ae0 0x7f32941964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33654/0 (socket says 172.21.15.2:33654) 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.484+0000 7f3298be3700 1 -- 172.21.15.2:0/2640689810 learned_addr learned my addr 172.21.15.2:0/2640689810 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3293fff700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3294101ce0 0x7f3294196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3298be3700 1 -- 172.21.15.2:0/2640689810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3294101ce0 msgr2=0x7f3294196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3298be3700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3294101ce0 0x7f3294196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3298be3700 1 -- 172.21.15.2:0/2640689810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800095f0 con 0x7f3294100ae0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3298be3700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294100ae0 0x7f32941964c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f328800d900 tx=0x7f328800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3291ffb700 1 -- 172.21.15.2:0/2640689810 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32880041f0 con 0x7f3294100ae0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3291ffb700 1 -- 172.21.15.2:0/2640689810 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3288004350 con 0x7f3294100ae0 2024-03-16T10:43:53.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329419bbc0 con 0x7f3294100ae0 2024-03-16T10:43:53.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f3291ffb700 1 -- 172.21.15.2:0/2640689810 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3288003cf0 con 0x7f3294100ae0 2024-03-16T10:43:53.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.485+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329419c110 con 0x7f3294100ae0 2024-03-16T10:43:53.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.486+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3294190660 con 0x7f3294100ae0 2024-03-16T10:43:53.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.487+0000 7f3291ffb700 1 -- 172.21.15.2:0/2640689810 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) v1 ==== 90522+0+0 (secure 0 0 0) 0x7f3288009730 con 0x7f3294100ae0 2024-03-16T10:43:53.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.488+0000 7f3291ffb700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f328406c880 0x7f328406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:43:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.488+0000 7f3291ffb700 1 -- 172.21.15.2:0/2640689810 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f328808baa0 con 0x7f3294100ae0 2024-03-16T10:43:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.488+0000 7f3293fff700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f328406c880 0x7f328406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:43:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.488+0000 7f3293fff700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f328406c880 0x7f328406ed40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3280005090 tx=0x7f3280005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:43:53.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.492+0000 7f3291ffb700 1 -- 172.21.15.2:0/2640689810 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3288059b60 con 0x7f3294100ae0 2024-03-16T10:43:53.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.782+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f329404ea80 con 0x7f3294100ae0 2024-03-16T10:43:53.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.783+0000 7f3291ffb700 1 -- 172.21.15.2:0/2640689810 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f32880596f0 con 0x7f3294100ae0 2024-03-16T10:43:53.784 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:43:53.784 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:43:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f328406c880 msgr2=0x7f328406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f328406c880 0x7f328406ed40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3280005090 tx=0x7f3280005960 comp rx=0 tx=0).stop 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294100ae0 msgr2=0x7f32941964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294100ae0 0x7f32941964c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f328800d900 tx=0x7f328800dc10 comp rx=0 tx=0).stop 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 shutdown_connections 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:6800/3195213211,v1:172.21.15.2:6801/3195213211] conn(0x7f328406c880 0x7f328406ed40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3294100ae0 0x7f32941964c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 --2- 172.21.15.2:0/2640689810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3294101ce0 0x7f3294196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:43:53.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 >> 172.21.15.2:0/2640689810 conn(0x7f32940fc0a0 msgr2=0x7f32941052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:43:53.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 shutdown_connections 2024-03-16T10:43:53.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:43:53.785+0000 7f329ae47700 1 -- 172.21.15.2:0/2640689810 wait complete. 2024-03-16T10:43:53.885 INFO:tasks.workunit.client.0.smithi002.stderr:Updating files: 93% (11429/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-16T10:43:54.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:53 smithi002 ceph-mon[29248]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:54.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:53 smithi002 ceph-mon[29248]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:54.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:53 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1816649621' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:43:54.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:53 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:54.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:53 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:54.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:53 smithi026 ceph-mon[37041]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:54.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:53 smithi026 ceph-mon[37041]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:54.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:53 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1816649621' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:43:54.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:53 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:54.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:53 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr:Note: switching to '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'. 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr:state without impacting any branches by switching back to a branch. 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr: git switch -c 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr:Or undo this operation with: 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-16T10:43:54.620 INFO:tasks.workunit.client.0.smithi002.stderr: git switch - 2024-03-16T10:43:54.621 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-16T10:43:54.621 INFO:tasks.workunit.client.0.smithi002.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-16T10:43:54.621 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-03-16T10:43:54.621 INFO:tasks.workunit.client.0.smithi002.stderr:HEAD is now at 3d78f5ba07b Merge PR #54581 into wip-batrick-testing-20240314.183243 2024-03-16T10:43:54.625 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-16T10:43:54.685 INFO:tasks.workunit.client.0.smithi002.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-16T10:43:54.687 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T10:43:54.687 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-16T10:43:54.769 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:54 smithi026 ceph-mon[37041]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:54.770 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:54 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2640689810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:43:54.770 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:54 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:54.770 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:54 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:55.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:54 smithi002 ceph-mon[29248]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:43:55.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:54 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2640689810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:43:55.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:54 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:55.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:54 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:55.444 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-16T10:43:55.509 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-16T10:43:55.548 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T10:43:55.549 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T10:43:55.549 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-16T10:43:55.587 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T10:43:55.591 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:43:55.591 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-16T10:43:55.645 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-16T10:43:55.648 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-16T10:43:55.648 DEBUG:teuthology.orchestra.run.smithi002:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-16T10:43:55.710 INFO:tasks.workunit.client.0.smithi002.stderr:+ mkdir -p fsstress 2024-03-16T10:43:55.713 INFO:tasks.workunit.client.0.smithi002.stderr:+ pushd fsstress 2024-03-16T10:43:55.714 INFO:tasks.workunit.client.0.smithi002.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-16T10:43:55.714 INFO:tasks.workunit.client.0.smithi002.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-16T10:43:56.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:56 smithi026 ceph-mon[37041]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:56.368 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:56 smithi002 ceph-mon[29248]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:57.738 INFO:tasks.workunit.client.0.smithi002.stderr:+ tar xzf ltp-full.tgz 2024-03-16T10:43:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr fail", "who": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: osdmap e48: 6 total, 6 up, 6 in 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "mgr fail", "who": "smithi002.zevrtm"}]': finished 2024-03-16T10:43:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:58 smithi002 ceph-mon[29248]: mgrmap e20: smithi026.omjkpo(active, starting, since 0.00704632s) 2024-03-16T10:43:58.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr fail", "who": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: osdmap e48: 6 total, 6 up, 6 in 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: from='mgr.14213 172.21.15.2:0/3438063715' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "mgr fail", "who": "smithi002.zevrtm"}]': finished 2024-03-16T10:43:58.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:58 smithi026 ceph-mon[37041]: mgrmap e20: smithi026.omjkpo(active, starting, since 0.00704632s) 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: Active manager daemon smithi026.omjkpo restarted 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: Activating manager daemon smithi026.omjkpo 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: osdmap e49: 6 total, 6 up, 6 in 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: mgrmap e21: smithi026.omjkpo(active, starting, since 0.0057578s) 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:43:59.593 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:43:59.594 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:43:59.595 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:43:59.595 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:43:59 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: Active manager daemon smithi026.omjkpo restarted 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: Activating manager daemon smithi026.omjkpo 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: osdmap e49: 6 total, 6 up, 6 in 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: mgrmap e21: smithi026.omjkpo(active, starting, since 0.0057578s) 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:43:59.649 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:43:59.650 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:43:59 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: Manager daemon smithi026.omjkpo is now available 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/trash_purge_schedule"}]: dispatch 2024-03-16T10:44:00.700 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:00 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/trash_purge_schedule"}]: dispatch 2024-03-16T10:44:00.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: Manager daemon smithi026.omjkpo is now available 2024-03-16T10:44:00.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:44:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:44:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/trash_purge_schedule"}]: dispatch 2024-03-16T10:44:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:00 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi026.omjkpo/trash_purge_schedule"}]: dispatch 2024-03-16T10:44:01.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:01 smithi026 ceph-mon[37041]: Deploying cephadm binary to smithi026 2024-03-16T10:44:01.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:01 smithi026 ceph-mon[37041]: Deploying cephadm binary to smithi002 2024-03-16T10:44:01.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:01 smithi026 ceph-mon[37041]: mgrmap e22: smithi026.omjkpo(active, since 1.06128s) 2024-03-16T10:44:01.633 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:01 smithi026 ceph-mon[37041]: pgmap v3: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:01.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:01 smithi002 ceph-mon[29248]: Deploying cephadm binary to smithi026 2024-03-16T10:44:01.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:01 smithi002 ceph-mon[29248]: Deploying cephadm binary to smithi002 2024-03-16T10:44:01.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:01 smithi002 ceph-mon[29248]: mgrmap e22: smithi026.omjkpo(active, since 1.06128s) 2024-03-16T10:44:01.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:01 smithi002 ceph-mon[29248]: pgmap v3: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:02.387 INFO:tasks.workunit.client.1.smithi026.stderr:Updating files: 82% (10086/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-16T10:44:02.664 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:02 smithi002 ceph-mon[29248]: pgmap v4: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:02.664 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:02 smithi002 ceph-mon[29248]: mgrmap e23: smithi026.omjkpo(active, since 3s) 2024-03-16T10:44:02.719 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:02 smithi026 ceph-mon[37041]: pgmap v4: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:02.719 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:02 smithi026 ceph-mon[37041]: mgrmap e23: smithi026.omjkpo(active, since 3s) 2024-03-16T10:44:02.956 INFO:tasks.workunit.client.1.smithi026.stderr:Note: switching to '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'. 2024-03-16T10:44:02.956 INFO:tasks.workunit.client.1.smithi026.stderr: 2024-03-16T10:44:02.956 INFO:tasks.workunit.client.1.smithi026.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-16T10:44:02.956 INFO:tasks.workunit.client.1.smithi026.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-16T10:44:02.956 INFO:tasks.workunit.client.1.smithi026.stderr:state without impacting any branches by switching back to a branch. 2024-03-16T10:44:02.956 INFO:tasks.workunit.client.1.smithi026.stderr: 2024-03-16T10:44:02.956 INFO:tasks.workunit.client.1.smithi026.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr: 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr: git switch -c 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr: 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr:Or undo this operation with: 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr: 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr: git switch - 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr: 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr: 2024-03-16T10:44:02.957 INFO:tasks.workunit.client.1.smithi026.stderr:HEAD is now at 3d78f5ba07b Merge PR #54581 into wip-batrick-testing-20240314.183243 2024-03-16T10:44:02.961 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-16T10:44:03.034 INFO:tasks.workunit.client.1.smithi026.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-16T10:44:03.036 INFO:tasks.workunit.client.1.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-16T10:44:03.036 INFO:tasks.workunit.client.1.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-16T10:44:03.471 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:03 smithi002 ceph-mon[29248]: [16/Mar/2024:10:44:01] ENGINE Bus STARTING 2024-03-16T10:44:03.471 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:03 smithi002 ceph-mon[29248]: [16/Mar/2024:10:44:01] ENGINE Serving on https://172.21.15.26:7150 2024-03-16T10:44:03.471 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:03 smithi002 ceph-mon[29248]: [16/Mar/2024:10:44:01] ENGINE Serving on http://172.21.15.26:8765 2024-03-16T10:44:03.471 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:03 smithi002 ceph-mon[29248]: [16/Mar/2024:10:44:01] ENGINE Bus STARTED 2024-03-16T10:44:03.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:03 smithi026 ceph-mon[37041]: [16/Mar/2024:10:44:01] ENGINE Bus STARTING 2024-03-16T10:44:03.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:03 smithi026 ceph-mon[37041]: [16/Mar/2024:10:44:01] ENGINE Serving on https://172.21.15.26:7150 2024-03-16T10:44:03.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:03 smithi026 ceph-mon[37041]: [16/Mar/2024:10:44:01] ENGINE Serving on http://172.21.15.26:8765 2024-03-16T10:44:03.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:03 smithi026 ceph-mon[37041]: [16/Mar/2024:10:44:01] ENGINE Bus STARTED 2024-03-16T10:44:03.797 INFO:tasks.workunit.client.1.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-16T10:44:03.887 INFO:tasks.workunit.client.1.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-16T10:44:03.938 INFO:tasks.workunit.client.1.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-16T10:44:03.939 INFO:tasks.workunit.client.1.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-16T10:44:03.939 INFO:tasks.workunit.client.1.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-16T10:44:03.986 INFO:tasks.workunit.client.1.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-16T10:44:03.989 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-03-16T10:44:03.989 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-16T10:44:04.053 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-16T10:44:04.054 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-16T10:44:04.054 DEBUG:teuthology.orchestra.run.smithi026:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-16T10:44:04.125 INFO:tasks.workunit.client.1.smithi026.stderr:+ mkdir -p fsstress 2024-03-16T10:44:04.127 INFO:tasks.workunit.client.1.smithi026.stderr:+ pushd fsstress 2024-03-16T10:44:04.128 INFO:tasks.workunit.client.1.smithi026.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-16T10:44:04.128 INFO:tasks.workunit.client.1.smithi026.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-16T10:44:04.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:04 smithi026 ceph-mon[37041]: pgmap v5: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:04.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:04 smithi026 ceph-mon[37041]: mgrmap e24: smithi026.omjkpo(active, since 4s) 2024-03-16T10:44:04.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:04 smithi002 ceph-mon[29248]: pgmap v5: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:04.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:04 smithi002 ceph-mon[29248]: mgrmap e24: smithi026.omjkpo(active, since 4s) 2024-03-16T10:44:05.616 INFO:tasks.workunit.client.1.smithi026.stderr:+ tar xzf ltp-full.tgz 2024-03-16T10:44:06.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:06 smithi002 ceph-mon[29248]: pgmap v6: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:06.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:06 smithi002 ceph-mon[29248]: Standby manager daemon smithi002.zevrtm started 2024-03-16T10:44:06.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:06 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/crt"}]: dispatch 2024-03-16T10:44:06.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:06 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:44:06.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:06 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/key"}]: dispatch 2024-03-16T10:44:06.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:06 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:44:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:06 smithi026 ceph-mon[37041]: pgmap v6: 65 pgs: 65 active+clean; 14 MiB data, 203 MiB used, 536 GiB / 536 GiB avail 2024-03-16T10:44:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:06 smithi026 ceph-mon[37041]: Standby manager daemon smithi002.zevrtm started 2024-03-16T10:44:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:06 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/crt"}]: dispatch 2024-03-16T10:44:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:06 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:44:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:06 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/key"}]: dispatch 2024-03-16T10:44:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:06 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/3740734389' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:44:07.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:07 smithi026 ceph-mon[37041]: mgrmap e25: smithi026.omjkpo(active, since 7s), standbys: smithi002.zevrtm 2024-03-16T10:44:07.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:07 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:44:07.771 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:07 smithi002 ceph-mon[29248]: mgrmap e25: smithi026.omjkpo(active, since 7s), standbys: smithi002.zevrtm 2024-03-16T10:44:07.771 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:07 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:44:08.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:08 smithi002 ceph-mon[29248]: pgmap v7: 65 pgs: 65 active+clean; 23 MiB data, 223 MiB used, 536 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-16T10:44:08.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:08 smithi026 ceph-mon[37041]: pgmap v7: 65 pgs: 65 active+clean; 23 MiB data, 223 MiB used, 536 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-16T10:44:10.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:10 smithi026 ceph-mon[37041]: pgmap v8: 65 pgs: 65 active+clean; 23 MiB data, 223 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-03-16T10:44:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:10 smithi002 ceph-mon[29248]: pgmap v8: 65 pgs: 65 active+clean; 23 MiB data, 223 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 76 op/s 2024-03-16T10:44:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:11 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:11 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:11 smithi002 ceph-mon[29248]: pgmap v9: 65 pgs: 65 active+clean; 54 MiB data, 344 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 154 op/s 2024-03-16T10:44:12.120 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:11 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:12.120 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:11 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:12.120 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:11 smithi026 ceph-mon[37041]: pgmap v9: 65 pgs: 65 active+clean; 54 MiB data, 344 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 3.7 MiB/s wr, 154 op/s 2024-03-16T10:44:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi002 to 1217M 2024-03-16T10:44:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:13 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi002 to 1217M 2024-03-16T10:44:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:13 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:14.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:14 smithi002 ceph-mon[29248]: pgmap v10: 65 pgs: 65 active+clean; 54 MiB data, 344 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-03-16T10:44:14.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:14 smithi026 ceph-mon[37041]: pgmap v10: 65 pgs: 65 active+clean; 54 MiB data, 344 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-03-16T10:44:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:15 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:15.388 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:15 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:16.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:16 smithi002 ceph-mon[29248]: pgmap v11: 65 pgs: 65 active+clean; 54 MiB data, 344 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-03-16T10:44:16.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:16 smithi026 ceph-mon[37041]: pgmap v11: 65 pgs: 65 active+clean; 54 MiB data, 344 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-03-16T10:44:17.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:17 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:17.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:17 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:17 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:17 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:18 smithi002 ceph-mon[29248]: pgmap v12: 65 pgs: 65 active+clean; 72 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 205 op/s 2024-03-16T10:44:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:18 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:18 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:18.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:18 smithi026 ceph-mon[37041]: pgmap v12: 65 pgs: 65 active+clean; 72 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 205 op/s 2024-03-16T10:44:18.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:18 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:18.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:18 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:20.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:20 smithi026 ceph-mon[37041]: pgmap v13: 65 pgs: 65 active+clean; 72 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 148 op/s 2024-03-16T10:44:20.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:20 smithi002 ceph-mon[29248]: pgmap v13: 65 pgs: 65 active+clean; 72 MiB data, 406 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.1 MiB/s wr, 148 op/s 2024-03-16T10:44:22.567 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:22 smithi026 ceph-mon[37041]: pgmap v14: 65 pgs: 65 active+clean; 89 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.6 MiB/s wr, 208 op/s 2024-03-16T10:44:22.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:22 smithi002 ceph-mon[29248]: pgmap v14: 65 pgs: 65 active+clean; 89 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.6 MiB/s wr, 208 op/s 2024-03-16T10:44:23.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.906+0000 7f71f558e700 1 -- 172.21.15.2:0/3076450067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 msgr2=0x7f71f0102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.906+0000 7f71f558e700 1 --2- 172.21.15.2:0/3076450067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0102130 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009910 tx=0x7f71e4009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.906+0000 7f71f558e700 1 -- 172.21.15.2:0/3076450067 shutdown_connections 2024-03-16T10:44:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.906+0000 7f71f558e700 1 --2- 172.21.15.2:0/3076450067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0102130 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.906+0000 7f71f558e700 1 --2- 172.21.15.2:0/3076450067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0100ad0 0x7f71f0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.906+0000 7f71f558e700 1 -- 172.21.15.2:0/3076450067 >> 172.21.15.2:0/3076450067 conn(0x7f71f00fc090 msgr2=0x7f71f00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.906+0000 7f71f558e700 1 -- 172.21.15.2:0/3076450067 shutdown_connections 2024-03-16T10:44:23.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.907+0000 7f71f558e700 1 -- 172.21.15.2:0/3076450067 wait complete. 2024-03-16T10:44:23.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.907+0000 7f71f558e700 1 Processor -- start 2024-03-16T10:44:23.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71f558e700 1 -- start start 2024-03-16T10:44:23.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71f558e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0100ad0 0x7f71f0196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:23.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71f558e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71f558e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0197080 con 0x7f71f0101cd0 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71f558e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01971c0 con 0x7f71f0100ad0 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71ee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71ee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43176/0 (socket says 172.21.15.2:43176) 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71ee7fc700 1 -- 172.21.15.2:0/54122375 learned_addr learned my addr 172.21.15.2:0/54122375 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.908+0000 7f71eeffd700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0100ad0 0x7f71f0196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71ee7fc700 1 -- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0100ad0 msgr2=0x7f71f0196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71ee7fc700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0100ad0 0x7f71f0196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:23.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71ee7fc700 1 -- 172.21.15.2:0/54122375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40095f0 con 0x7f71f0101cd0 2024-03-16T10:44:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71ee7fc700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0196a60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71e4005f30 tx=0x7f71e4005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71d7fff700 1 -- 172.21.15.2:0/54122375 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71e401d070 con 0x7f71f0101cd0 2024-03-16T10:44:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f019bbc0 con 0x7f71f0101cd0 2024-03-16T10:44:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f019c130 con 0x7f71f0101cd0 2024-03-16T10:44:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71d7fff700 1 -- 172.21.15.2:0/54122375 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71e40052a0 con 0x7f71f0101cd0 2024-03-16T10:44:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.909+0000 7f71d7fff700 1 -- 172.21.15.2:0/54122375 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71e4017400 con 0x7f71f0101cd0 2024-03-16T10:44:23.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.910+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc005320 con 0x7f71f0101cd0 2024-03-16T10:44:23.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.911+0000 7f71d7fff700 1 -- 172.21.15.2:0/54122375 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f71e4017670 con 0x7f71f0101cd0 2024-03-16T10:44:23.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.911+0000 7f71d7fff700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f71d806d390 0x7f71d806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:23.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.911+0000 7f71d7fff700 1 -- 172.21.15.2:0/54122375 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f71e408df10 con 0x7f71f0101cd0 2024-03-16T10:44:23.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.912+0000 7f71eeffd700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f71d806d390 0x7f71d806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:23.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.912+0000 7f71eeffd700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f71d806d390 0x7f71d806f850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71f0101b30 tx=0x7f71e0009260 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:23.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:23.918+0000 7f71d7fff700 1 -- 172.21.15.2:0/54122375 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f71e405b340 con 0x7f71f0101cd0 2024-03-16T10:44:24.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.177+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71dc000bf0 con 0x7f71d806d390 2024-03-16T10:44:24.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.179+0000 7f71d7fff700 1 -- 172.21.15.2:0/54122375 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f71dc000bf0 con 0x7f71d806d390 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f71d806d390 msgr2=0x7f71d806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f71d806d390 0x7f71d806f850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71f0101b30 tx=0x7f71e0009260 comp rx=0 tx=0).stop 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 msgr2=0x7f71f0196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0196a60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71e4005f30 tx=0x7f71e4005960 comp rx=0 tx=0).stop 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 shutdown_connections 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f71d806d390 0x7f71d806f850 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0100ad0 0x7f71f0196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 --2- 172.21.15.2:0/54122375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f0101cd0 0x7f71f0196a60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 >> 172.21.15.2:0/54122375 conn(0x7f71f00fc090 msgr2=0x7f71f0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:24.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 shutdown_connections 2024-03-16T10:44:24.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.182+0000 7f71f558e700 1 -- 172.21.15.2:0/54122375 wait complete. 2024-03-16T10:44:24.194 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:44:24.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 -- 172.21.15.2:0/1941242123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa340103ef0 msgr2=0x7fa3401062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 --2- 172.21.15.2:0/1941242123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa340103ef0 0x7fa3401062e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa330009910 tx=0x7fa330009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:24.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 -- 172.21.15.2:0/1941242123 shutdown_connections 2024-03-16T10:44:24.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 --2- 172.21.15.2:0/1941242123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa340103ef0 0x7fa3401062e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 --2- 172.21.15.2:0/1941242123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3401015c0 0x7fa3401039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 -- 172.21.15.2:0/1941242123 >> 172.21.15.2:0/1941242123 conn(0x7fa3400faf70 msgr2=0x7fa3400fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:24.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 -- 172.21.15.2:0/1941242123 shutdown_connections 2024-03-16T10:44:24.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.302+0000 7fa347394700 1 -- 172.21.15.2:0/1941242123 wait complete. 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa347394700 1 Processor -- start 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa347394700 1 -- start start 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa347394700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3401015c0 0x7fa340100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa347394700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa340103ef0 0x7fa3400ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa347394700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400ffab0 con 0x7fa3401015c0 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa347394700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400ffbf0 con 0x7fa340103ef0 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa345130700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3401015c0 0x7fa340100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa345130700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3401015c0 0x7fa340100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43192/0 (socket says 172.21.15.2:43192) 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa345130700 1 -- 172.21.15.2:0/2487655359 learned_addr learned my addr 172.21.15.2:0/2487655359 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.303+0000 7fa34492f700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa340103ef0 0x7fa3400ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:24.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa345130700 1 -- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa340103ef0 msgr2=0x7fa3400ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa345130700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa340103ef0 0x7fa3400ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa345130700 1 -- 172.21.15.2:0/2487655359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300095f0 con 0x7fa3401015c0 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa345130700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3401015c0 0x7fa340100ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00e920 tx=0x7fa33c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa3367fc700 1 -- 172.21.15.2:0/2487655359 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa33c00c9d0 con 0x7fa3401015c0 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa3367fc700 1 -- 172.21.15.2:0/2487655359 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa33c010430 con 0x7fa3401015c0 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa3367fc700 1 -- 172.21.15.2:0/2487655359 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa33c018630 con 0x7fa3401015c0 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3400ffed0 con 0x7fa3401015c0 2024-03-16T10:44:24.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.304+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34019e4f0 con 0x7fa3401015c0 2024-03-16T10:44:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.305+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34004ea80 con 0x7fa3401015c0 2024-03-16T10:44:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.306+0000 7fa3367fc700 1 -- 172.21.15.2:0/2487655359 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7fa33c0105a0 con 0x7fa3401015c0 2024-03-16T10:44:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.306+0000 7fa3367fc700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa32c06d390 0x7fa32c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.306+0000 7fa3367fc700 1 -- 172.21.15.2:0/2487655359 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa33c014070 con 0x7fa3401015c0 2024-03-16T10:44:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.306+0000 7fa34492f700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa32c06d390 0x7fa32c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:24.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.307+0000 7fa34492f700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa32c06d390 0x7fa32c06f850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa33000b800 tx=0x7fa330009f70 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:24.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.311+0000 7fa3367fc700 1 -- 172.21.15.2:0/2487655359 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fa33c0534c0 con 0x7fa3401015c0 2024-03-16T10:44:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:24 smithi002 ceph-mon[29248]: pgmap v15: 65 pgs: 65 active+clean; 89 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 125 op/s 2024-03-16T10:44:24.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.553+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa340105dd0 con 0x7fa32c06d390 2024-03-16T10:44:24.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.555+0000 7fa3367fc700 1 -- 172.21.15.2:0/2487655359 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fa340105dd0 con 0x7fa32c06d390 2024-03-16T10:44:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa32c06d390 msgr2=0x7fa32c06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa32c06d390 0x7fa32c06f850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa33000b800 tx=0x7fa330009f70 comp rx=0 tx=0).stop 2024-03-16T10:44:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3401015c0 msgr2=0x7fa340100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3401015c0 0x7fa340100ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00e920 tx=0x7fa33c00ec30 comp rx=0 tx=0).stop 2024-03-16T10:44:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 shutdown_connections 2024-03-16T10:44:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa32c06d390 0x7fa32c06f850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3401015c0 0x7fa340100ed0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 --2- 172.21.15.2:0/2487655359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa340103ef0 0x7fa3400ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 >> 172.21.15.2:0/2487655359 conn(0x7fa3400faf70 msgr2=0x7fa3400fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 shutdown_connections 2024-03-16T10:44:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.557+0000 7fa347394700 1 -- 172.21.15.2:0/2487655359 wait complete. 2024-03-16T10:44:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 -- 172.21.15.2:0/206748882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c100ad0 msgr2=0x7fa48c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 --2- 172.21.15.2:0/206748882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c100ad0 0x7fa48c100ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009910 tx=0x7fa47c009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 -- 172.21.15.2:0/206748882 shutdown_connections 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 --2- 172.21.15.2:0/206748882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c101cd0 0x7fa48c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 --2- 172.21.15.2:0/206748882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c100ad0 0x7fa48c100ef0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 -- 172.21.15.2:0/206748882 >> 172.21.15.2:0/206748882 conn(0x7fa48c0fc090 msgr2=0x7fa48c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 -- 172.21.15.2:0/206748882 shutdown_connections 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.676+0000 7fa49278a700 1 -- 172.21.15.2:0/206748882 wait complete. 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.677+0000 7fa49278a700 1 Processor -- start 2024-03-16T10:44:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.677+0000 7fa49278a700 1 -- start start 2024-03-16T10:44:24.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.677+0000 7fa49278a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c100ad0 0x7fa48c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.677+0000 7fa49278a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c101cd0 0x7fa48c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.677+0000 7fa49278a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0736e0 con 0x7fa48c101cd0 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.677+0000 7fa49278a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c073820 con 0x7fa48c100ad0 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.677+0000 7fa48b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c101cd0 0x7fa48c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa48bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c100ad0 0x7fa48c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa48b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c101cd0 0x7fa48c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43208/0 (socket says 172.21.15.2:43208) 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa48b7fe700 1 -- 172.21.15.2:0/184387804 learned_addr learned my addr 172.21.15.2:0/184387804 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa48b7fe700 1 -- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c100ad0 msgr2=0x7fa48c074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa48b7fe700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c100ad0 0x7fa48c074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa48b7fe700 1 -- 172.21.15.2:0/184387804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0095f0 con 0x7fa48c101cd0 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa48b7fe700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c101cd0 0x7fa48c0731a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e920 tx=0x7fa48000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa4897fa700 1 -- 172.21.15.2:0/184387804 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa48000c9d0 con 0x7fa48c101cd0 2024-03-16T10:44:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa4897fa700 1 -- 172.21.15.2:0/184387804 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa480010430 con 0x7fa48c101cd0 2024-03-16T10:44:24.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa4897fa700 1 -- 172.21.15.2:0/184387804 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa480018630 con 0x7fa48c101cd0 2024-03-16T10:44:24.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c073b00 con 0x7fa48c101cd0 2024-03-16T10:44:24.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.678+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c19e550 con 0x7fa48c101cd0 2024-03-16T10:44:24.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.679+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c190650 con 0x7fa48c101cd0 2024-03-16T10:44:24.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.680+0000 7fa4897fa700 1 -- 172.21.15.2:0/184387804 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7fa48000cb30 con 0x7fa48c101cd0 2024-03-16T10:44:24.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.680+0000 7fa4897fa700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa47406d390 0x7fa47406f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:24.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.680+0000 7fa4897fa700 1 -- 172.21.15.2:0/184387804 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa480014070 con 0x7fa48c101cd0 2024-03-16T10:44:24.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.680+0000 7fa48bfff700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa47406d390 0x7fa47406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:24.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.681+0000 7fa48bfff700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa47406d390 0x7fa47406f850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00b800 tx=0x7fa47c009f70 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:24.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.687+0000 7fa4897fa700 1 -- 172.21.15.2:0/184387804 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fa4800534c0 con 0x7fa48c101cd0 2024-03-16T10:44:24.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:24 smithi026 ceph-mon[37041]: pgmap v15: 65 pgs: 65 active+clean; 89 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 125 op/s 2024-03-16T10:44:24.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.931+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa48c0611f0 con 0x7fa47406d390 2024-03-16T10:44:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.949+0000 7fa4897fa700 1 -- 172.21.15.2:0/184387804 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa48c0611f0 con 0x7fa47406d390 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (6m) 14s ago 7m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (8m) 14s ago 8m 7583k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (7m) 8s ago 7m 7596k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (8m) 14s ago 8m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (7m) 8s ago 7m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (6m) 14s ago 7m 86.9M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (2m) 14s ago 2m 15.9M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (2m) 14s ago 2m 12.8M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:44:24.950 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (2m) 8s ago 2m 63.7M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (2m) 8s ago 2m 14.6M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:9283,8765,8443 running (9m) 14s ago 9m 445M - 18.2.2-878-ga752c2f7 f692adb9a265 1b8a582e9d84 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (35s) 8s ago 7m 492M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5df99c885681 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (9m) 14s ago 9m 51.5M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (6m) 8s ago 6m 39.1M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (8m) 14s ago 8m 17.0M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (7m) 8s ago 7m 15.5M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (6m) 14s ago 6m 56.3M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (5m) 14s ago 5m 58.7M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (5m) 14s ago 5m 56.0M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (4m) 8s ago 4m 60.3M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (4m) 8s ago 4m 55.9M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (3m) 8s ago 3m 57.2M 4972M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:44:24.951 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (6m) 14s ago 7m 61.2M - 2.43.0 a07b618ecd1d 02e5ed409aab 2024-03-16T10:44:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa47406d390 msgr2=0x7fa47406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa47406d390 0x7fa47406f850 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00b800 tx=0x7fa47c009f70 comp rx=0 tx=0).stop 2024-03-16T10:44:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c101cd0 msgr2=0x7fa48c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c101cd0 0x7fa48c0731a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa48000e920 tx=0x7fa48000ec30 comp rx=0 tx=0).stop 2024-03-16T10:44:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 shutdown_connections 2024-03-16T10:44:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fa47406d390 0x7fa47406f850 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa48c100ad0 0x7fa48c074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 --2- 172.21.15.2:0/184387804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa48c101cd0 0x7fa48c0731a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 >> 172.21.15.2:0/184387804 conn(0x7fa48c0fc090 msgr2=0x7fa48c1052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 shutdown_connections 2024-03-16T10:44:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:24.951+0000 7fa49278a700 1 -- 172.21.15.2:0/184387804 wait complete. 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 -- 172.21.15.2:0/1222162861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 msgr2=0x7f6ddc1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 --2- 172.21.15.2:0/1222162861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 0x7f6ddc1021e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009910 tx=0x7f6dcc009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 -- 172.21.15.2:0/1222162861 shutdown_connections 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 --2- 172.21.15.2:0/1222162861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 0x7f6ddc1021e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 --2- 172.21.15.2:0/1222162861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ddc0feda0 0x7f6ddc0ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 -- 172.21.15.2:0/1222162861 >> 172.21.15.2:0/1222162861 conn(0x7f6ddc0fa700 msgr2=0x7f6ddc0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 -- 172.21.15.2:0/1222162861 shutdown_connections 2024-03-16T10:44:25.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 -- 172.21.15.2:0/1222162861 wait complete. 2024-03-16T10:44:25.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.071+0000 7f6de455a700 1 Processor -- start 2024-03-16T10:44:25.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de455a700 1 -- start start 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de455a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ddc0feda0 0x7f6ddc196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de455a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 0x7f6ddc196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de455a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc1970f0 con 0x7f6ddc101d80 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de455a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc197230 con 0x7f6ddc0feda0 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de1af5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 0x7f6ddc196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de22f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ddc0feda0 0x7f6ddc196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de1af5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 0x7f6ddc196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43218/0 (socket says 172.21.15.2:43218) 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de1af5700 1 -- 172.21.15.2:0/3923617295 learned_addr learned my addr 172.21.15.2:0/3923617295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de22f6700 1 -- 172.21.15.2:0/3923617295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 msgr2=0x7f6ddc196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de22f6700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 0x7f6ddc196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de22f6700 1 -- 172.21.15.2:0/3923617295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0095f0 con 0x7f6ddc0feda0 2024-03-16T10:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.072+0000 7f6de22f6700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ddc0feda0 0x7f6ddc196590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e940 tx=0x7f6dd800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.073+0000 7f6dd37fe700 1 -- 172.21.15.2:0/3923617295 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd80041f0 con 0x7f6ddc0feda0 2024-03-16T10:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.073+0000 7f6dd37fe700 1 -- 172.21.15.2:0/3923617295 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dd8004350 con 0x7f6ddc0feda0 2024-03-16T10:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.073+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc0ffbd0 con 0x7f6ddc0feda0 2024-03-16T10:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.073+0000 7f6dd37fe700 1 -- 172.21.15.2:0/3923617295 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd8003c90 con 0x7f6ddc0feda0 2024-03-16T10:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.073+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc1001a0 con 0x7f6ddc0feda0 2024-03-16T10:44:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.074+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc190660 con 0x7f6ddc0feda0 2024-03-16T10:44:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.074+0000 7f6dd37fe700 1 -- 172.21.15.2:0/3923617295 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f6dd800c7c0 con 0x7f6ddc0feda0 2024-03-16T10:44:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.075+0000 7f6dd37fe700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f6dc806d390 0x7f6dc806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.075+0000 7f6dd37fe700 1 -- 172.21.15.2:0/3923617295 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6dd8014070 con 0x7f6ddc0feda0 2024-03-16T10:44:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.075+0000 7f6de1af5700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f6dc806d390 0x7f6dc806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.075+0000 7f6de1af5700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f6dc806d390 0x7f6dc806f850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b800 tx=0x7f6dcc009f90 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:25.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.080+0000 7f6dd37fe700 1 -- 172.21.15.2:0/3923617295 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f6dd805a3d0 con 0x7f6ddc0feda0 2024-03-16T10:44:25.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.417+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ddc02ccb0 con 0x7f6ddc0feda0 2024-03-16T10:44:25.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.418+0000 7f6dd37fe700 1 -- 172.21.15.2:0/3923617295 <== mon.1 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f6dd8056070 con 0x7f6ddc0feda0 2024-03-16T10:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 1, 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 13, 2024-03-16T10:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T10:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:44:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f6dc806d390 msgr2=0x7f6dc806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f6dc806d390 0x7f6dc806f850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b800 tx=0x7f6dcc009f90 comp rx=0 tx=0).stop 2024-03-16T10:44:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ddc0feda0 msgr2=0x7f6ddc196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ddc0feda0 0x7f6ddc196590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e940 tx=0x7f6dd800ec50 comp rx=0 tx=0).stop 2024-03-16T10:44:25.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 shutdown_connections 2024-03-16T10:44:25.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f6dc806d390 0x7f6dc806f850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ddc0feda0 0x7f6ddc196590 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 --2- 172.21.15.2:0/3923617295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ddc101d80 0x7f6ddc196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.420+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 >> 172.21.15.2:0/3923617295 conn(0x7f6ddc0fa700 msgr2=0x7f6ddc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:25.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.421+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 shutdown_connections 2024-03-16T10:44:25.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.421+0000 7f6de455a700 1 -- 172.21.15.2:0/3923617295 wait complete. 2024-03-16T10:44:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.538+0000 7f70beb6a700 1 -- 172.21.15.2:0/854247045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b8100380 msgr2=0x7f70b8100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.538+0000 7f70beb6a700 1 --2- 172.21.15.2:0/854247045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b8100380 0x7f70b8100800 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f70a400aa20 tx=0x7f70a400ad30 comp rx=0 tx=0).stop 2024-03-16T10:44:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 -- 172.21.15.2:0/854247045 shutdown_connections 2024-03-16T10:44:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 --2- 172.21.15.2:0/854247045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b8100380 0x7f70b8100800 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 --2- 172.21.15.2:0/854247045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b80ff2c0 0x7f70b80ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 -- 172.21.15.2:0/854247045 >> 172.21.15.2:0/854247045 conn(0x7f70b80fa2c0 msgr2=0x7f70b80fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 -- 172.21.15.2:0/854247045 shutdown_connections 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 -- 172.21.15.2:0/854247045 wait complete. 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 Processor -- start 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 -- start start 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b80ff2c0 0x7f70b81964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b8100380 0x7f70b8196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b8197050 con 0x7f70b80ff2c0 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.539+0000 7f70beb6a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b8197190 con 0x7f70b8100380 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70bc906700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b80ff2c0 0x7f70b81964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70bc906700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b80ff2c0 0x7f70b81964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43232/0 (socket says 172.21.15.2:43232) 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70bc906700 1 -- 172.21.15.2:0/724346697 learned_addr learned my addr 172.21.15.2:0/724346697 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b8100380 0x7f70b8196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70bc906700 1 -- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b8100380 msgr2=0x7f70b8196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70bc906700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b8100380 0x7f70b8196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70bc906700 1 -- 172.21.15.2:0/724346697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a400a700 con 0x7f70b80ff2c0 2024-03-16T10:44:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70bc906700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b80ff2c0 0x7f70b81964f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00d900 tx=0x7f70ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70b5ffb700 1 -- 172.21.15.2:0/724346697 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70ac0041f0 con 0x7f70b80ff2c0 2024-03-16T10:44:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70b5ffb700 1 -- 172.21.15.2:0/724346697 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70ac004cf0 con 0x7f70b80ff2c0 2024-03-16T10:44:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70b5ffb700 1 -- 172.21.15.2:0/724346697 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70ac00b750 con 0x7f70b80ff2c0 2024-03-16T10:44:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.540+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b819bbf0 con 0x7f70b80ff2c0 2024-03-16T10:44:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.541+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b819c140 con 0x7f70b80ff2c0 2024-03-16T10:44:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.542+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b8190470 con 0x7f70b80ff2c0 2024-03-16T10:44:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.542+0000 7f70b5ffb700 1 -- 172.21.15.2:0/724346697 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f70ac004750 con 0x7f70b80ff2c0 2024-03-16T10:44:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.542+0000 7f70b5ffb700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f70a806d380 0x7f70a806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.543+0000 7f70b5ffb700 1 -- 172.21.15.2:0/724346697 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f70ac08cb20 con 0x7f70b80ff2c0 2024-03-16T10:44:25.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.543+0000 7f70b7fff700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f70a806d380 0x7f70a806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.543+0000 7f70b7fff700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f70a806d380 0x7f70a806f840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006010 tx=0x7f70a400af70 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:25.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.548+0000 7f70b5ffb700 1 -- 172.21.15.2:0/724346697 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f70ac059f50 con 0x7f70b80ff2c0 2024-03-16T10:44:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:25 smithi026 ceph-mon[37041]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:25.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:25 smithi026 ceph-mon[37041]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:25.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:25 smithi002 ceph-mon[29248]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:25.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:25 smithi002 ceph-mon[29248]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:25.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.848+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f70b802d0d0 con 0x7f70b80ff2c0 2024-03-16T10:44:25.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.848+0000 7f70b5ffb700 1 -- 172.21.15.2:0/724346697 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f70ac016020 con 0x7f70b80ff2c0 2024-03-16T10:44:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:44:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:44:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:44:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f70a806d380 msgr2=0x7f70a806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f70a806d380 0x7f70a806f840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006010 tx=0x7f70a400af70 comp rx=0 tx=0).stop 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b80ff2c0 msgr2=0x7f70b81964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b80ff2c0 0x7f70b81964f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00d900 tx=0x7f70ac00dc10 comp rx=0 tx=0).stop 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 shutdown_connections 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f70a806d380 0x7f70a806f840 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b80ff2c0 0x7f70b81964f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 --2- 172.21.15.2:0/724346697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b8100380 0x7f70b8196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 >> 172.21.15.2:0/724346697 conn(0x7f70b80fa2c0 msgr2=0x7f70b8103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 shutdown_connections 2024-03-16T10:44:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.851+0000 7f70beb6a700 1 -- 172.21.15.2:0/724346697 wait complete. 2024-03-16T10:44:25.854 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/2113569681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 msgr2=0x7f4ae00ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/2113569681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae00ffc60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009910 tx=0x7f4ad0009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/2113569681 shutdown_connections 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/2113569681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae0100a40 0x7f4ae0100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/2113569681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae00ffc60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/2113569681 >> 172.21.15.2:0/2113569681 conn(0x7f4ae00fae00 msgr2=0x7f4ae00fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/2113569681 shutdown_connections 2024-03-16T10:44:25.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.957+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/2113569681 wait complete. 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4ae4b9d700 1 Processor -- start 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4ae4b9d700 1 -- start start 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4ae4b9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae0196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4ae4b9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae0100a40 0x7f4ae0196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4ae4b9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0196e90 con 0x7f4ae00ff840 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4ae4b9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0196fd0 con 0x7f4ae0100a40 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4adf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae0196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4adf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae0196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43252/0 (socket says 172.21.15.2:43252) 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.958+0000 7f4adf7fe700 1 -- 172.21.15.2:0/462104310 learned_addr learned my addr 172.21.15.2:0/462104310 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adeffd700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae0100a40 0x7f4ae0196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adf7fe700 1 -- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae0100a40 msgr2=0x7f4ae0196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adf7fe700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae0100a40 0x7f4ae0196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:25.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adf7fe700 1 -- 172.21.15.2:0/462104310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00095f0 con 0x7f4ae00ff840 2024-03-16T10:44:25.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adf7fe700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae0196330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00059d0 tx=0x7f4ad000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:25.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adcff9700 1 -- 172.21.15.2:0/462104310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad001d070 con 0x7f4ae00ff840 2024-03-16T10:44:25.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adcff9700 1 -- 172.21.15.2:0/462104310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ad000f460 con 0x7f4ae00ff840 2024-03-16T10:44:25.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae019b9d0 con 0x7f4ae00ff840 2024-03-16T10:44:25.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4adcff9700 1 -- 172.21.15.2:0/462104310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad0021640 con 0x7f4ae00ff840 2024-03-16T10:44:25.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.959+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae019bec0 con 0x7f4ae00ff840 2024-03-16T10:44:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.960+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae004f2d0 con 0x7f4ae00ff840 2024-03-16T10:44:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.961+0000 7f4adcff9700 1 -- 172.21.15.2:0/462104310 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f4ad0003a60 con 0x7f4ae00ff840 2024-03-16T10:44:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.961+0000 7f4adcff9700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4ac806d390 0x7f4ac806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.961+0000 7f4adcff9700 1 -- 172.21.15.2:0/462104310 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4ad008df10 con 0x7f4ae00ff840 2024-03-16T10:44:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.961+0000 7f4adeffd700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4ac806d390 0x7f4ac806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:25.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.962+0000 7f4adeffd700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4ac806d390 0x7f4ac806f850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4007c60 tx=0x7f4ad40073d0 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:25.966+0000 7f4adcff9700 1 -- 172.21.15.2:0/462104310 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f4ad005b2c0 con 0x7f4ae00ff840 2024-03-16T10:44:26.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.214+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ae00fcd40 con 0x7f4ac806d390 2024-03-16T10:44:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.215+0000 7f4adcff9700 1 -- 172.21.15.2:0/462104310 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f4ae00fcd40 con 0x7f4ac806d390 2024-03-16T10:44:26.216 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:44:26.216 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:44:26.216 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:44:26.216 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:44:26.217 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [], 2024-03-16T10:44:26.217 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "1/23 daemons upgraded", 2024-03-16T10:44:26.217 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "", 2024-03-16T10:44:26.217 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:44:26.217 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:44:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4ac806d390 msgr2=0x7f4ac806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4ac806d390 0x7f4ac806f850 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4007c60 tx=0x7f4ad40073d0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 msgr2=0x7f4ae0196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae0196330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00059d0 tx=0x7f4ad000bd70 comp rx=0 tx=0).stop 2024-03-16T10:44:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 shutdown_connections 2024-03-16T10:44:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4ac806d390 0x7f4ac806f850 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ae00ff840 0x7f4ae0196330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 --2- 172.21.15.2:0/462104310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae0100a40 0x7f4ae0196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.218+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 >> 172.21.15.2:0/462104310 conn(0x7f4ae00fae00 msgr2=0x7f4ae00fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.219+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 shutdown_connections 2024-03-16T10:44:26.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.219+0000 7f4ae4b9d700 1 -- 172.21.15.2:0/462104310 wait complete. 2024-03-16T10:44:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 -- 172.21.15.2:0/1820486270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42980ff9d0 msgr2=0x7f42980ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 --2- 172.21.15.2:0/1820486270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42980ff9d0 0x7f42980ffdf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4294009910 tx=0x7f4294009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 -- 172.21.15.2:0/1820486270 shutdown_connections 2024-03-16T10:44:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 --2- 172.21.15.2:0/1820486270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4298100bd0 0x7f4298101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 --2- 172.21.15.2:0/1820486270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42980ff9d0 0x7f42980ffdf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 -- 172.21.15.2:0/1820486270 >> 172.21.15.2:0/1820486270 conn(0x7f42980faf70 msgr2=0x7f42980fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:26.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 -- 172.21.15.2:0/1820486270 shutdown_connections 2024-03-16T10:44:26.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.337+0000 7f42a0625700 1 -- 172.21.15.2:0/1820486270 wait complete. 2024-03-16T10:44:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f42a0625700 1 Processor -- start 2024-03-16T10:44:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f42a0625700 1 -- start start 2024-03-16T10:44:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f42a0625700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42980ff9d0 0x7f4298103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f42a0625700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4298100bd0 0x7f4298101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f42a0625700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981024f0 con 0x7f4298100bd0 2024-03-16T10:44:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f42a0625700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298102630 con 0x7f42980ff9d0 2024-03-16T10:44:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f429dbc0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4298100bd0 0x7f4298101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f429dbc0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4298100bd0 0x7f4298101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43266/0 (socket says 172.21.15.2:43266) 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f429dbc0700 1 -- 172.21.15.2:0/2231087404 learned_addr learned my addr 172.21.15.2:0/2231087404 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.338+0000 7f429e3c1700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42980ff9d0 0x7f4298103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f429dbc0700 1 -- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42980ff9d0 msgr2=0x7f4298103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f429dbc0700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42980ff9d0 0x7f4298103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f429dbc0700 1 -- 172.21.15.2:0/2231087404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940095f0 con 0x7f4298100bd0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f429dbc0700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4298100bd0 0x7f4298101fb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f428800e940 tx=0x7f428800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f428f7fe700 1 -- 172.21.15.2:0/2231087404 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428800cba0 con 0x7f4298100bd0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f428f7fe700 1 -- 172.21.15.2:0/2231087404 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f428800ee90 con 0x7f4298100bd0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4298102910 con 0x7f4298100bd0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f428f7fe700 1 -- 172.21.15.2:0/2231087404 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42880055d0 con 0x7f4298100bd0 2024-03-16T10:44:26.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.339+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429819e5e0 con 0x7f4298100bd0 2024-03-16T10:44:26.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.340+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429804ea80 con 0x7f4298100bd0 2024-03-16T10:44:26.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.341+0000 7f428f7fe700 1 -- 172.21.15.2:0/2231087404 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f4288004750 con 0x7f4298100bd0 2024-03-16T10:44:26.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.341+0000 7f428f7fe700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f428406d390 0x7f428406f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:26.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.341+0000 7f428f7fe700 1 -- 172.21.15.2:0/2231087404 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4288014070 con 0x7f4298100bd0 2024-03-16T10:44:26.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.341+0000 7f429e3c1700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f428406d390 0x7f428406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:26.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.342+0000 7f429e3c1700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f428406d390 0x7f428406f850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4294006010 tx=0x7f4294009f70 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:26.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.346+0000 7f428f7fe700 1 -- 172.21.15.2:0/2231087404 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f428805a350 con 0x7f4298100bd0 2024-03-16T10:44:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:26 smithi002 ceph-mon[29248]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:26 smithi002 ceph-mon[29248]: pgmap v16: 65 pgs: 65 active+clean; 89 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 125 op/s 2024-03-16T10:44:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:26 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3923617295' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:44:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:26 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/724346697' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.682+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f429819e7d0 con 0x7f4298100bd0 2024-03-16T10:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.683+0000 7f428f7fe700 1 -- 172.21.15.2:0/2231087404 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f4288059ee0 con 0x7f4298100bd0 2024-03-16T10:44:26.684 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:44:26.684 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:44:26.684 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:44:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f428406d390 msgr2=0x7f428406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f428406d390 0x7f428406f850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4294006010 tx=0x7f4294009f70 comp rx=0 tx=0).stop 2024-03-16T10:44:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4298100bd0 msgr2=0x7f4298101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4298100bd0 0x7f4298101fb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f428800e940 tx=0x7f428800ec50 comp rx=0 tx=0).stop 2024-03-16T10:44:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 shutdown_connections 2024-03-16T10:44:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f428406d390 0x7f428406f850 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42980ff9d0 0x7f4298103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 --2- 172.21.15.2:0/2231087404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4298100bd0 0x7f4298101fb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 >> 172.21.15.2:0/2231087404 conn(0x7f42980faf70 msgr2=0x7f42980684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 shutdown_connections 2024-03-16T10:44:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:26.685+0000 7f42a0625700 1 -- 172.21.15.2:0/2231087404 wait complete. 2024-03-16T10:44:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:26 smithi026 ceph-mon[37041]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:26 smithi026 ceph-mon[37041]: pgmap v16: 65 pgs: 65 active+clean; 89 MiB data, 463 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 125 op/s 2024-03-16T10:44:26.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:26 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3923617295' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:44:26.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:26 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/724346697' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:44:27.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:27 smithi026 ceph-mon[37041]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:27.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:27 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2231087404' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:44:27.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:27 smithi002 ceph-mon[29248]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:27.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:27 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2231087404' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:44:28.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:28 smithi026 ceph-mon[37041]: pgmap v17: 65 pgs: 65 active+clean; 108 MiB data, 537 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 203 op/s 2024-03-16T10:44:28.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:28 smithi002 ceph-mon[29248]: pgmap v17: 65 pgs: 65 active+clean; 108 MiB data, 537 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 203 op/s 2024-03-16T10:44:30.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:30 smithi026 ceph-mon[37041]: pgmap v18: 65 pgs: 65 active+clean; 108 MiB data, 537 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.1 MiB/s wr, 138 op/s 2024-03-16T10:44:30.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:30 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:30.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:30 smithi002 ceph-mon[29248]: pgmap v18: 65 pgs: 65 active+clean; 108 MiB data, 537 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.1 MiB/s wr, 138 op/s 2024-03-16T10:44:30.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:30 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:32.696 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:32 smithi026 ceph-mon[37041]: pgmap v19: 65 pgs: 65 active+clean; 128 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.8 MiB/s wr, 231 op/s 2024-03-16T10:44:32.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:32 smithi002 ceph-mon[29248]: pgmap v19: 65 pgs: 65 active+clean; 128 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.8 MiB/s wr, 231 op/s 2024-03-16T10:44:34.655 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:34 smithi026 ceph-mon[37041]: pgmap v20: 65 pgs: 65 active+clean; 128 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 171 op/s 2024-03-16T10:44:34.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:34 smithi002 ceph-mon[29248]: pgmap v20: 65 pgs: 65 active+clean; 128 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 171 op/s 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Adjusting osd_memory_target on smithi026 to 2241M 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:44:35.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: pgmap v21: 65 pgs: 65 active+clean; 128 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 171 op/s 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:44:35.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:35 smithi026 ceph-mon[37041]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:44:35.996 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:35.996 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:35.996 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.996 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Adjusting osd_memory_target on smithi026 to 2241M 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: pgmap v21: 65 pgs: 65 active+clean; 128 MiB data, 606 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.3 MiB/s wr, 171 op/s 2024-03-16T10:44:35.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:44:35.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:35 smithi002 ceph-mon[29248]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:44:37.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:36 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:36 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:36 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:36 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:36 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:36 smithi002 ceph-mon[29248]: Reconfiguring prometheus.smithi002 (dependencies changed)... 2024-03-16T10:44:37.042 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:36 smithi002 ceph-mon[29248]: Reconfiguring daemon prometheus.smithi002 on smithi002 2024-03-16T10:44:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:36 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:36 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:36 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:36 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:36 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:37.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:36 smithi026 ceph-mon[37041]: Reconfiguring prometheus.smithi002 (dependencies changed)... 2024-03-16T10:44:37.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:36 smithi026 ceph-mon[37041]: Reconfiguring daemon prometheus.smithi002 on smithi002 2024-03-16T10:44:38.229 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:37 smithi002 ceph-mon[29248]: pgmap v22: 65 pgs: 65 active+clean; 140 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 252 op/s 2024-03-16T10:44:38.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:37 smithi026 ceph-mon[37041]: pgmap v22: 65 pgs: 65 active+clean; 140 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 252 op/s 2024-03-16T10:44:40.532 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:40 smithi002 ceph-mon[29248]: pgmap v23: 65 pgs: 65 active+clean; 140 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 174 op/s 2024-03-16T10:44:40.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:40 smithi026 ceph-mon[37041]: pgmap v23: 65 pgs: 65 active+clean; 140 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.7 MiB/s wr, 174 op/s 2024-03-16T10:44:42.615 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:42 smithi002 ceph-mon[29248]: pgmap v24: 65 pgs: 65 active+clean; 162 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 341 op/s 2024-03-16T10:44:42.616 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:42 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:42.616 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:42 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:42.616 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:42 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:44:42.616 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:42 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:42.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:42 smithi026 ceph-mon[37041]: pgmap v24: 65 pgs: 65 active+clean; 162 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 341 op/s 2024-03-16T10:44:42.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:42 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:42.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:42 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:42.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:42 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:44:42.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:42 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:43.653 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:43 smithi026 ceph-mon[37041]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:44:43.653 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:43 smithi026 ceph-mon[37041]: Upgrade: Need to upgrade myself (mgr.smithi026.omjkpo) 2024-03-16T10:44:43.653 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:43 smithi026 ceph-mon[37041]: Upgrade: Need to upgrade myself (mgr.smithi026.omjkpo) 2024-03-16T10:44:43.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:43 smithi002 ceph-mon[29248]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:44:43.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:43 smithi002 ceph-mon[29248]: Upgrade: Need to upgrade myself (mgr.smithi026.omjkpo) 2024-03-16T10:44:43.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:43 smithi002 ceph-mon[29248]: Upgrade: Need to upgrade myself (mgr.smithi026.omjkpo) 2024-03-16T10:44:44.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: pgmap v25: 65 pgs: 65 active+clean; 162 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-16T10:44:44.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: Upgrade: Updating mgr.smithi002.zevrtm 2024-03-16T10:44:44.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:44.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:44:44.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:44:44.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:44:44.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:44:44.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:44 smithi026 ceph-mon[37041]: Deploying daemon mgr.smithi002.zevrtm on smithi002 2024-03-16T10:44:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: pgmap v25: 65 pgs: 65 active+clean; 162 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-16T10:44:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: Upgrade: Updating mgr.smithi002.zevrtm 2024-03-16T10:44:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:44:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:44:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:44:44.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:44:44.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:44 smithi002 ceph-mon[29248]: Deploying daemon mgr.smithi002.zevrtm on smithi002 2024-03-16T10:44:45.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:45 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:45.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:45 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:44:45.924 INFO:tasks.workunit.client.0.smithi002.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-16T10:44:45.930 INFO:tasks.workunit.client.0.smithi002.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-16T10:44:45.931 INFO:tasks.workunit.client.0.smithi002.stderr:+ make 2024-03-16T10:44:46.708 INFO:tasks.workunit.client.0.smithi002.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-16T10:44:46.722 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:46 smithi002 ceph-mon[29248]: pgmap v26: 65 pgs: 65 active+clean; 162 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-16T10:44:46.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:46 smithi026 ceph-mon[37041]: pgmap v26: 65 pgs: 65 active+clean; 162 MiB data, 748 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-16T10:44:47.242 INFO:tasks.workunit.client.0.smithi002.stderr:++ readlink -f fsstress 2024-03-16T10:44:47.244 INFO:tasks.workunit.client.0.smithi002.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-16T10:44:47.244 INFO:tasks.workunit.client.0.smithi002.stderr:+ popd 2024-03-16T10:44:47.244 INFO:tasks.workunit.client.0.smithi002.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-16T10:44:47.245 INFO:tasks.workunit.client.0.smithi002.stderr:+ popd 2024-03-16T10:44:47.245 INFO:tasks.workunit.client.0.smithi002.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-16T10:44:47.245 INFO:tasks.workunit.client.0.smithi002.stderr:++ mktemp -d -p . 2024-03-16T10:44:47.248 INFO:tasks.workunit.client.0.smithi002.stderr:+ T=./tmp.QX75oORedg 2024-03-16T10:44:47.248 INFO:tasks.workunit.client.0.smithi002.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.QX75oORedg -l 1 -n 1000 -p 10 -v 2024-03-16T10:44:47.252 INFO:tasks.workunit.client.0.smithi002.stdout:seed = 1710662664 2024-03-16T10:44:47.258 INFO:tasks.workunit.client.0.smithi002.stdout:0/0: mknod c0 0 2024-03-16T10:44:47.259 INFO:tasks.workunit.client.0.smithi002.stdout:0/1: dwrite - no filename 2024-03-16T10:44:47.259 INFO:tasks.workunit.client.0.smithi002.stdout:0/2: dwrite - no filename 2024-03-16T10:44:47.259 INFO:tasks.workunit.client.0.smithi002.stdout:0/3: dwrite - no filename 2024-03-16T10:44:47.259 INFO:tasks.workunit.client.0.smithi002.stdout:0/4: stat c0 0 2024-03-16T10:44:47.259 INFO:tasks.workunit.client.0.smithi002.stdout:0/5: chown c0 978568 1 2024-03-16T10:44:47.259 INFO:tasks.workunit.client.0.smithi002.stdout:0/6: chown c0 1549966 1 2024-03-16T10:44:47.259 INFO:tasks.workunit.client.0.smithi002.stdout:0/7: dread - no filename 2024-03-16T10:44:47.260 INFO:tasks.workunit.client.0.smithi002.stdout:1/0: mknod c0 0 2024-03-16T10:44:47.262 INFO:tasks.workunit.client.0.smithi002.stdout:0/8: rename c0 to c1 0 2024-03-16T10:44:47.262 INFO:tasks.workunit.client.0.smithi002.stdout:0/9: write - no filename 2024-03-16T10:44:47.262 INFO:tasks.workunit.client.0.smithi002.stdout:0/10: write - no filename 2024-03-16T10:44:47.262 INFO:tasks.workunit.client.0.smithi002.stdout:0/11: dread - no filename 2024-03-16T10:44:47.263 INFO:tasks.workunit.client.0.smithi002.stdout:1/1: creat f1 x:0 0 0 2024-03-16T10:44:47.263 INFO:tasks.workunit.client.0.smithi002.stdout:1/2: dread - f1 zero size 2024-03-16T10:44:47.264 INFO:tasks.workunit.client.0.smithi002.stdout:2/0: creat f0 x:0 0 0 2024-03-16T10:44:47.265 INFO:tasks.workunit.client.0.smithi002.stdout:4/0: fdatasync - no filename 2024-03-16T10:44:47.266 INFO:tasks.workunit.client.0.smithi002.stdout:0/12: creat f2 x:0 0 0 2024-03-16T10:44:47.266 INFO:tasks.workunit.client.0.smithi002.stdout:4/1: readlink - no filename 2024-03-16T10:44:47.267 INFO:tasks.workunit.client.0.smithi002.stdout:3/0: creat f0 x:0 0 0 2024-03-16T10:44:47.268 INFO:tasks.workunit.client.0.smithi002.stdout:0/13: dread - f2 zero size 2024-03-16T10:44:47.269 INFO:tasks.workunit.client.0.smithi002.stdout:4/2: getdents . 0 2024-03-16T10:44:47.272 INFO:tasks.workunit.client.0.smithi002.stdout:3/1: truncate f0 619682 0 2024-03-16T10:44:47.272 INFO:tasks.workunit.client.0.smithi002.stdout:0/14: mkdir d3 0 2024-03-16T10:44:47.274 INFO:tasks.workunit.client.0.smithi002.stdout:4/3: symlink l0 0 2024-03-16T10:44:47.274 INFO:tasks.workunit.client.0.smithi002.stdout:4/4: fdatasync - no filename 2024-03-16T10:44:47.274 INFO:tasks.workunit.client.0.smithi002.stdout:4/5: dwrite - no filename 2024-03-16T10:44:47.276 INFO:tasks.workunit.client.0.smithi002.stdout:3/2: mkdir d1 0 2024-03-16T10:44:47.276 INFO:tasks.workunit.client.0.smithi002.stdout:1/3: dwrite f1 [0,4194304] 0 2024-03-16T10:44:47.276 INFO:tasks.workunit.client.0.smithi002.stdout:4/6: chown l0 29820341 1 2024-03-16T10:44:47.278 INFO:tasks.workunit.client.0.smithi002.stdout:2/1: dwrite f0 [0,4194304] 0 2024-03-16T10:44:47.278 INFO:tasks.workunit.client.0.smithi002.stdout:5/0: mkdir d0 0 2024-03-16T10:44:47.278 INFO:tasks.workunit.client.0.smithi002.stdout:3/3: write f0 [1387682,61415] 0 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/7: mknod c1 0 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/8: write - no filename 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/9: read - no filename 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/10: write - no filename 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/11: dread - no filename 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/12: dread - no filename 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/13: dwrite - no filename 2024-03-16T10:44:47.286 INFO:tasks.workunit.client.0.smithi002.stdout:8/0: readlink - no filename 2024-03-16T10:44:47.287 INFO:tasks.workunit.client.0.smithi002.stdout:4/14: chown l0 261257227 1 2024-03-16T10:44:47.287 INFO:tasks.workunit.client.0.smithi002.stdout:0/15: symlink d3/l4 0 2024-03-16T10:44:47.289 INFO:tasks.workunit.client.0.smithi002.stdout:2/2: symlink l1 0 2024-03-16T10:44:47.294 INFO:tasks.workunit.client.0.smithi002.stdout:7/0: mknod c0 0 2024-03-16T10:44:47.295 INFO:tasks.workunit.client.0.smithi002.stdout:7/1: fsync - no filename 2024-03-16T10:44:47.295 INFO:tasks.workunit.client.0.smithi002.stdout:0/16: dwrite f2 [0,4194304] 0 2024-03-16T10:44:47.298 INFO:tasks.workunit.client.0.smithi002.stdout:9/0: stat - no entries 2024-03-16T10:44:47.298 INFO:tasks.workunit.client.0.smithi002.stdout:9/1: fsync - no filename 2024-03-16T10:44:47.302 INFO:tasks.workunit.client.0.smithi002.stdout:4/15: mkdir d2 0 2024-03-16T10:44:47.302 INFO:tasks.workunit.client.0.smithi002.stdout:4/16: write - no filename 2024-03-16T10:44:47.308 INFO:tasks.workunit.client.0.smithi002.stdout:5/1: creat d0/f1 x:0 0 0 2024-03-16T10:44:47.310 INFO:tasks.workunit.client.0.smithi002.stdout:7/2: creat f1 x:0 0 0 2024-03-16T10:44:47.311 INFO:tasks.workunit.client.0.smithi002.stdout:7/3: fdatasync f1 0 2024-03-16T10:44:47.311 INFO:tasks.workunit.client.0.smithi002.stdout:7/4: dread - f1 zero size 2024-03-16T10:44:47.311 INFO:tasks.workunit.client.0.smithi002.stdout:7/5: dread - f1 zero size 2024-03-16T10:44:47.311 INFO:tasks.workunit.client.0.smithi002.stdout:0/17: mkdir d3/d5 0 2024-03-16T10:44:47.312 INFO:tasks.workunit.client.0.smithi002.stdout:7/6: dread - f1 zero size 2024-03-16T10:44:47.312 INFO:tasks.workunit.client.0.smithi002.stdout:0/18: stat d3 0 2024-03-16T10:44:47.314 INFO:tasks.workunit.client.0.smithi002.stdout:8/1: creat f0 x:0 0 0 2024-03-16T10:44:47.316 INFO:tasks.workunit.client.0.smithi002.stdout:5/2: mknod d0/c2 0 2024-03-16T10:44:47.317 INFO:tasks.workunit.client.0.smithi002.stdout:5/3: write d0/f1 [817205,82383] 0 2024-03-16T10:44:47.318 INFO:tasks.workunit.client.0.smithi002.stdout:7/7: symlink l2 0 2024-03-16T10:44:47.319 INFO:tasks.workunit.client.0.smithi002.stdout:9/2: creat f0 x:0 0 0 2024-03-16T10:44:47.320 INFO:tasks.workunit.client.0.smithi002.stdout:9/3: dread - f0 zero size 2024-03-16T10:44:47.321 INFO:tasks.workunit.client.0.smithi002.stdout:8/2: mknod c1 0 2024-03-16T10:44:47.322 INFO:tasks.workunit.client.0.smithi002.stdout:8/3: dread - f0 zero size 2024-03-16T10:44:47.322 INFO:tasks.workunit.client.0.smithi002.stdout:4/17: mknod d2/c3 0 2024-03-16T10:44:47.322 INFO:tasks.workunit.client.0.smithi002.stdout:4/18: fdatasync - no filename 2024-03-16T10:44:47.322 INFO:tasks.workunit.client.0.smithi002.stdout:4/19: dread - no filename 2024-03-16T10:44:47.323 INFO:tasks.workunit.client.0.smithi002.stdout:0/19: symlink d3/d5/l6 0 2024-03-16T10:44:47.324 INFO:tasks.workunit.client.0.smithi002.stdout:0/20: write f2 [1956864,110432] 0 2024-03-16T10:44:47.324 INFO:tasks.workunit.client.0.smithi002.stdout:0/21: stat d3 0 2024-03-16T10:44:47.326 INFO:tasks.workunit.client.0.smithi002.stdout:8/4: rename f0 to f2 0 2024-03-16T10:44:47.328 INFO:tasks.workunit.client.0.smithi002.stdout:4/20: symlink d2/l4 0 2024-03-16T10:44:47.328 INFO:tasks.workunit.client.0.smithi002.stdout:4/21: chown d2/c3 588739035 1 2024-03-16T10:44:47.328 INFO:tasks.workunit.client.0.smithi002.stdout:4/22: write - no filename 2024-03-16T10:44:47.328 INFO:tasks.workunit.client.0.smithi002.stdout:4/23: fdatasync - no filename 2024-03-16T10:44:47.328 INFO:tasks.workunit.client.0.smithi002.stdout:4/24: write - no filename 2024-03-16T10:44:47.331 INFO:tasks.workunit.client.0.smithi002.stdout:9/4: link f0 f1 0 2024-03-16T10:44:47.331 INFO:tasks.workunit.client.0.smithi002.stdout:9/5: chown f1 103903 1 2024-03-16T10:44:47.331 INFO:tasks.workunit.client.0.smithi002.stdout:0/22: mknod d3/c7 0 2024-03-16T10:44:47.334 INFO:tasks.workunit.client.0.smithi002.stdout:8/5: symlink l3 0 2024-03-16T10:44:47.335 INFO:tasks.workunit.client.0.smithi002.stdout:4/25: rmdir d2 39 2024-03-16T10:44:47.338 INFO:tasks.workunit.client.0.smithi002.stdout:8/6: creat f4 x:0 0 0 2024-03-16T10:44:47.341 INFO:tasks.workunit.client.0.smithi002.stdout:8/7: mknod c5 0 2024-03-16T10:44:47.341 INFO:tasks.workunit.client.0.smithi002.stdout:8/8: chown c1 60699 1 2024-03-16T10:44:47.343 INFO:tasks.workunit.client.0.smithi002.stdout:8/9: symlink l6 0 2024-03-16T10:44:47.345 INFO:tasks.workunit.client.0.smithi002.stdout:8/10: write f2 [741664,128209] 0 2024-03-16T10:44:47.346 INFO:tasks.workunit.client.0.smithi002.stdout:8/11: fdatasync f4 0 2024-03-16T10:44:47.347 INFO:tasks.workunit.client.0.smithi002.stdout:8/12: mkdir d7 0 2024-03-16T10:44:47.351 INFO:tasks.workunit.client.0.smithi002.stdout:8/13: creat d7/f8 x:0 0 0 2024-03-16T10:44:47.351 INFO:tasks.workunit.client.0.smithi002.stdout:8/14: dread - f4 zero size 2024-03-16T10:44:47.354 INFO:tasks.workunit.client.0.smithi002.stdout:8/15: mkdir d7/d9 0 2024-03-16T10:44:47.357 INFO:tasks.workunit.client.0.smithi002.stdout:8/16: creat d7/d9/fa x:0 0 0 2024-03-16T10:44:47.359 INFO:tasks.workunit.client.0.smithi002.stdout:8/17: creat d7/fb x:0 0 0 2024-03-16T10:44:47.359 INFO:tasks.workunit.client.0.smithi002.stdout:8/18: fdatasync f4 0 2024-03-16T10:44:47.359 INFO:tasks.workunit.client.0.smithi002.stdout:8/19: truncate f4 54956 0 2024-03-16T10:44:47.360 INFO:tasks.workunit.client.0.smithi002.stdout:8/20: truncate d7/fb 380604 0 2024-03-16T10:44:47.362 INFO:tasks.workunit.client.0.smithi002.stdout:8/21: creat d7/fc x:0 0 0 2024-03-16T10:44:47.366 INFO:tasks.workunit.client.0.smithi002.stdout:8/22: dread f2 [0,4194304] 0 2024-03-16T10:44:47.368 INFO:tasks.workunit.client.0.smithi002.stdout:8/23: creat d7/fd x:0 0 0 2024-03-16T10:44:47.369 INFO:tasks.workunit.client.0.smithi002.stdout:8/24: truncate d7/fd 601003 0 2024-03-16T10:44:47.376 INFO:tasks.workunit.client.0.smithi002.stdout:8/25: dwrite d7/fc [0,4194304] 0 2024-03-16T10:44:47.376 INFO:tasks.workunit.client.0.smithi002.stdout:8/26: readlink l3 0 2024-03-16T10:44:47.385 INFO:tasks.workunit.client.0.smithi002.stdout:8/27: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:47.385 INFO:tasks.workunit.client.0.smithi002.stdout:8/28: chown d7/d9/fa 1 1 2024-03-16T10:44:47.394 INFO:tasks.workunit.client.0.smithi002.stdout:8/29: dwrite d7/d9/fa [0,4194304] 0 2024-03-16T10:44:47.398 INFO:tasks.workunit.client.0.smithi002.stdout:8/30: mkdir d7/de 0 2024-03-16T10:44:47.595 INFO:tasks.workunit.client.0.smithi002.stdout:3/4: fsync f0 0 2024-03-16T10:44:47.595 INFO:tasks.workunit.client.0.smithi002.stdout:3/5: chown d1 49950 1 2024-03-16T10:44:47.595 INFO:tasks.workunit.client.0.smithi002.stdout:3/6: truncate f0 1732682 0 2024-03-16T10:44:47.600 INFO:tasks.workunit.client.0.smithi002.stdout:3/7: dread f0 [0,4194304] 0 2024-03-16T10:44:47.603 INFO:tasks.workunit.client.0.smithi002.stdout:3/8: getdents d1 0 2024-03-16T10:44:47.606 INFO:tasks.workunit.client.0.smithi002.stdout:3/9: unlink f0 0 2024-03-16T10:44:47.606 INFO:tasks.workunit.client.0.smithi002.stdout:3/10: dwrite - no filename 2024-03-16T10:44:47.606 INFO:tasks.workunit.client.0.smithi002.stdout:3/11: link - no file 2024-03-16T10:44:47.606 INFO:tasks.workunit.client.0.smithi002.stdout:2/3: dwrite f0 [4194304,4194304] 0 2024-03-16T10:44:47.610 INFO:tasks.workunit.client.0.smithi002.stdout:2/4: dread f0 [4194304,4194304] 0 2024-03-16T10:44:47.613 INFO:tasks.workunit.client.0.smithi002.stdout:3/12: mknod d1/c2 0 2024-03-16T10:44:47.613 INFO:tasks.workunit.client.0.smithi002.stdout:3/13: dread - no filename 2024-03-16T10:44:47.613 INFO:tasks.workunit.client.0.smithi002.stdout:1/4: truncate f1 3538036 0 2024-03-16T10:44:47.615 INFO:tasks.workunit.client.0.smithi002.stdout:2/5: creat f2 x:0 0 0 2024-03-16T10:44:47.618 INFO:tasks.workunit.client.0.smithi002.stdout:3/14: unlink d1/c2 0 2024-03-16T10:44:47.618 INFO:tasks.workunit.client.0.smithi002.stdout:3/15: truncate - no filename 2024-03-16T10:44:47.618 INFO:tasks.workunit.client.0.smithi002.stdout:3/16: chown d1 15895286 1 2024-03-16T10:44:47.618 INFO:tasks.workunit.client.0.smithi002.stdout:3/17: write - no filename 2024-03-16T10:44:47.619 INFO:tasks.workunit.client.0.smithi002.stdout:0/23: write f2 [5228978,71451] 0 2024-03-16T10:44:47.619 INFO:tasks.workunit.client.0.smithi002.stdout:3/18: chown d1 57881023 1 2024-03-16T10:44:47.619 INFO:tasks.workunit.client.0.smithi002.stdout:3/19: dread - no filename 2024-03-16T10:44:47.619 INFO:tasks.workunit.client.0.smithi002.stdout:0/24: chown d3/d5/l6 79464655 1 2024-03-16T10:44:47.622 INFO:tasks.workunit.client.0.smithi002.stdout:3/20: creat d1/f3 x:0 0 0 2024-03-16T10:44:47.624 INFO:tasks.workunit.client.0.smithi002.stdout:3/21: truncate d1/f3 366294 0 2024-03-16T10:44:47.625 INFO:tasks.workunit.client.0.smithi002.stdout:5/4: getdents d0 0 2024-03-16T10:44:47.627 INFO:tasks.workunit.client.0.smithi002.stdout:0/25: dwrite f2 [4194304,4194304] 0 2024-03-16T10:44:47.628 INFO:tasks.workunit.client.0.smithi002.stdout:3/22: unlink d1/f3 0 2024-03-16T10:44:47.628 INFO:tasks.workunit.client.0.smithi002.stdout:3/23: dread - no filename 2024-03-16T10:44:47.628 INFO:tasks.workunit.client.0.smithi002.stdout:3/24: dread - no filename 2024-03-16T10:44:47.629 INFO:tasks.workunit.client.0.smithi002.stdout:3/25: chown d1 21923 1 2024-03-16T10:44:47.629 INFO:tasks.workunit.client.0.smithi002.stdout:5/5: mknod d0/c3 0 2024-03-16T10:44:47.634 INFO:tasks.workunit.client.0.smithi002.stdout:0/26: symlink d3/l8 0 2024-03-16T10:44:47.636 INFO:tasks.workunit.client.0.smithi002.stdout:5/6: symlink d0/l4 0 2024-03-16T10:44:47.637 INFO:tasks.workunit.client.0.smithi002.stdout:5/7: chown d0/c3 6167820 1 2024-03-16T10:44:47.641 INFO:tasks.workunit.client.0.smithi002.stdout:5/8: dread d0/f1 [0,4194304] 0 2024-03-16T10:44:47.645 INFO:tasks.workunit.client.0.smithi002.stdout:0/27: link d3/c7 d3/d5/c9 0 2024-03-16T10:44:47.648 INFO:tasks.workunit.client.0.smithi002.stdout:0/28: mkdir d3/d5/da 0 2024-03-16T10:44:47.652 INFO:tasks.workunit.client.0.smithi002.stdout:0/29: rename f2 to d3/d5/fb 0 2024-03-16T10:44:47.660 INFO:tasks.workunit.client.0.smithi002.stdout:0/30: dwrite d3/d5/fb [4194304,4194304] 0 2024-03-16T10:44:47.664 INFO:tasks.workunit.client.0.smithi002.stdout:0/31: creat d3/fc x:0 0 0 2024-03-16T10:44:47.667 INFO:tasks.workunit.client.0.smithi002.stdout:0/32: creat d3/d5/fd x:0 0 0 2024-03-16T10:44:47.668 INFO:tasks.workunit.client.0.smithi002.stdout:0/33: write d3/d5/fd [270642,47897] 0 2024-03-16T10:44:47.672 INFO:tasks.workunit.client.0.smithi002.stdout:0/34: mkdir d3/de 0 2024-03-16T10:44:47.672 INFO:tasks.workunit.client.0.smithi002.stdout:8/31: rmdir d7 39 2024-03-16T10:44:47.678 INFO:tasks.workunit.client.0.smithi002.stdout:0/35: rename c1 to d3/d5/cf 0 2024-03-16T10:44:47.679 INFO:tasks.workunit.client.0.smithi002.stdout:0/36: stat d3/d5/da 0 2024-03-16T10:44:47.679 INFO:tasks.workunit.client.0.smithi002.stdout:8/32: dread d7/fc [0,4194304] 0 2024-03-16T10:44:47.679 INFO:tasks.workunit.client.0.smithi002.stdout:8/33: chown c1 364340000 1 2024-03-16T10:44:47.684 INFO:tasks.workunit.client.0.smithi002.stdout:0/37: dwrite d3/d5/fd [0,4194304] 0 2024-03-16T10:44:47.686 INFO:tasks.workunit.client.0.smithi002.stdout:8/34: symlink d7/d9/lf 0 2024-03-16T10:44:47.692 INFO:tasks.workunit.client.0.smithi002.stdout:0/38: getdents d3/d5/da 0 2024-03-16T10:44:47.695 INFO:tasks.workunit.client.0.smithi002.stdout:8/35: symlink d7/de/l10 0 2024-03-16T10:44:47.707 INFO:tasks.workunit.client.0.smithi002.stdout:8/36: getdents d7 0 2024-03-16T10:44:47.708 INFO:tasks.workunit.client.0.smithi002.stdout:8/37: write d7/fb [1333711,88721] 0 2024-03-16T10:44:47.712 INFO:tasks.workunit.client.0.smithi002.stdout:8/38: mknod d7/c11 0 2024-03-16T10:44:47.715 INFO:tasks.workunit.client.0.smithi002.stdout:8/39: mknod d7/c12 0 2024-03-16T10:44:47.715 INFO:tasks.workunit.client.0.smithi002.stdout:8/40: fsync f4 0 2024-03-16T10:44:47.721 INFO:tasks.workunit.client.0.smithi002.stdout:2/6: truncate f0 4534461 0 2024-03-16T10:44:47.721 INFO:tasks.workunit.client.0.smithi002.stdout:2/7: readlink l1 0 2024-03-16T10:44:47.722 INFO:tasks.workunit.client.0.smithi002.stdout:8/41: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:47.778 INFO:tasks.workunit.client.0.smithi002.stdout:0/39: chown d3/d5/cf 2052 1 2024-03-16T10:44:47.778 INFO:tasks.workunit.client.0.smithi002.stdout:0/40: truncate d3/fc 239386 0 2024-03-16T10:44:47.779 INFO:tasks.workunit.client.0.smithi002.stdout:0/41: write d3/d5/fb [4106322,124959] 0 2024-03-16T10:44:47.785 INFO:tasks.workunit.client.0.smithi002.stdout:0/42: creat d3/de/f10 x:0 0 0 2024-03-16T10:44:47.785 INFO:tasks.workunit.client.0.smithi002.stdout:8/42: getdents d7/d9 0 2024-03-16T10:44:47.785 INFO:tasks.workunit.client.0.smithi002.stdout:0/43: stat d3/d5/l6 0 2024-03-16T10:44:47.788 INFO:tasks.workunit.client.0.smithi002.stdout:8/43: rmdir d7 39 2024-03-16T10:44:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:8/44: rename d7/de to d7/d13 0 2024-03-16T10:44:47.798 INFO:tasks.workunit.client.0.smithi002.stdout:8/45: fdatasync f4 0 2024-03-16T10:44:47.803 INFO:tasks.workunit.client.0.smithi002.stdout:8/46: rename d7/fb to d7/f14 0 2024-03-16T10:44:47.804 INFO:tasks.workunit.client.0.smithi002.stdout:0/44: getdents d3 0 2024-03-16T10:44:47.808 INFO:tasks.workunit.client.0.smithi002.stdout:8/47: write d7/f14 [2202299,8226] 0 2024-03-16T10:44:47.811 INFO:tasks.workunit.client.0.smithi002.stdout:0/45: dwrite d3/fc [0,4194304] 0 2024-03-16T10:44:47.819 INFO:tasks.workunit.client.0.smithi002.stdout:8/48: dwrite d7/fd [0,4194304] 0 2024-03-16T10:44:47.823 INFO:tasks.workunit.client.0.smithi002.stdout:2/8: dread f0 [0,4194304] 0 2024-03-16T10:44:47.823 INFO:tasks.workunit.client.0.smithi002.stdout:0/46: rmdir d3/d5/da 0 2024-03-16T10:44:47.826 INFO:tasks.workunit.client.0.smithi002.stdout:2/9: creat f3 x:0 0 0 2024-03-16T10:44:47.828 INFO:tasks.workunit.client.0.smithi002.stdout:8/49: dwrite f4 [0,4194304] 0 2024-03-16T10:44:47.828 INFO:tasks.workunit.client.0.smithi002.stdout:2/10: write f2 [349013,71380] 0 2024-03-16T10:44:47.829 INFO:tasks.workunit.client.0.smithi002.stdout:8/50: stat c1 0 2024-03-16T10:44:47.829 INFO:tasks.workunit.client.0.smithi002.stdout:8/51: stat l6 0 2024-03-16T10:44:47.833 INFO:tasks.workunit.client.0.smithi002.stdout:2/11: rename f2 to f4 0 2024-03-16T10:44:47.836 INFO:tasks.workunit.client.0.smithi002.stdout:8/52: mknod d7/c15 0 2024-03-16T10:44:47.840 INFO:tasks.workunit.client.0.smithi002.stdout:2/12: rename f0 to f5 0 2024-03-16T10:44:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:2/13: write f4 [1372639,50025] 0 2024-03-16T10:44:47.844 INFO:tasks.workunit.client.0.smithi002.stdout:2/14: truncate f3 932769 0 2024-03-16T10:44:47.845 INFO:tasks.workunit.client.0.smithi002.stdout:2/15: write f4 [1752799,122441] 0 2024-03-16T10:44:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:1/5: truncate f1 127523 0 2024-03-16T10:44:47.847 INFO:tasks.workunit.client.0.smithi002.stdout:2/16: dread f3 [0,4194304] 0 2024-03-16T10:44:47.851 INFO:tasks.workunit.client.0.smithi002.stdout:1/6: mkdir d2 0 2024-03-16T10:44:47.852 INFO:tasks.workunit.client.0.smithi002.stdout:1/7: readlink - no filename 2024-03-16T10:44:47.858 INFO:tasks.workunit.client.0.smithi002.stdout:2/17: dwrite f5 [0,4194304] 0 2024-03-16T10:44:47.862 INFO:tasks.workunit.client.0.smithi002.stdout:2/18: creat f6 x:0 0 0 2024-03-16T10:44:47.870 INFO:tasks.workunit.client.0.smithi002.stdout:2/19: dwrite f3 [0,4194304] 0 2024-03-16T10:44:47.875 INFO:tasks.workunit.client.0.smithi002.stdout:2/20: dread f4 [0,4194304] 0 2024-03-16T10:44:47.882 INFO:tasks.workunit.client.0.smithi002.stdout:2/21: dwrite f6 [0,4194304] 0 2024-03-16T10:44:47.882 INFO:tasks.workunit.client.0.smithi002.stdout:2/22: chown f3 1895 1 2024-03-16T10:44:47.914 INFO:tasks.workunit.client.0.smithi002.stdout:8/53: truncate d7/f14 1857292 0 2024-03-16T10:44:47.919 INFO:tasks.workunit.client.0.smithi002.stdout:8/54: unlink d7/d9/lf 0 2024-03-16T10:44:47.920 INFO:tasks.workunit.client.0.smithi002.stdout:8/55: dread f2 [0,4194304] 0 2024-03-16T10:44:47.921 INFO:tasks.workunit.client.0.smithi002.stdout:8/56: dread f2 [0,4194304] 0 2024-03-16T10:44:47.929 INFO:tasks.workunit.client.0.smithi002.stdout:8/57: dwrite f2 [0,4194304] 0 2024-03-16T10:44:47.949 INFO:tasks.workunit.client.0.smithi002.stdout:2/23: truncate f3 3933222 0 2024-03-16T10:44:47.968 INFO:tasks.workunit.client.0.smithi002.stdout:2/24: mkdir d7 0 2024-03-16T10:44:47.968 INFO:tasks.workunit.client.0.smithi002.stdout:1/8: write f1 [589651,33994] 0 2024-03-16T10:44:47.969 INFO:tasks.workunit.client.0.smithi002.stdout:1/9: truncate f1 1338683 0 2024-03-16T10:44:47.972 INFO:tasks.workunit.client.0.smithi002.stdout:1/10: dread f1 [0,4194304] 0 2024-03-16T10:44:47.979 INFO:tasks.workunit.client.0.smithi002.stdout:1/11: dwrite f1 [0,4194304] 0 2024-03-16T10:44:47.979 INFO:tasks.workunit.client.0.smithi002.stdout:1/12: readlink - no filename 2024-03-16T10:44:47.986 INFO:tasks.workunit.client.0.smithi002.stdout:1/13: dwrite f1 [0,4194304] 0 2024-03-16T10:44:47.989 INFO:tasks.workunit.client.0.smithi002.stdout:8/58: getdents d7 0 2024-03-16T10:44:47.991 INFO:tasks.workunit.client.0.smithi002.stdout:1/14: mkdir d2/d3 0 2024-03-16T10:44:48.022 INFO:tasks.workunit.client.0.smithi002.stdout:8/59: rename d7/c15 to d7/d13/c16 0 2024-03-16T10:44:48.023 INFO:tasks.workunit.client.0.smithi002.stdout:1/15: rmdir d2 39 2024-03-16T10:44:48.024 INFO:tasks.workunit.client.0.smithi002.stdout:1/16: chown d2/d3 8 1 2024-03-16T10:44:48.027 INFO:tasks.workunit.client.0.smithi002.stdout:1/17: symlink d2/l4 0 2024-03-16T10:44:48.033 INFO:tasks.workunit.client.0.smithi002.stdout:1/18: dwrite f1 [4194304,4194304] 0 2024-03-16T10:44:48.328 INFO:tasks.workunit.client.0.smithi002.stdout:1/19: link f1 d2/f5 0 2024-03-16T10:44:48.330 INFO:tasks.workunit.client.0.smithi002.stdout:1/20: symlink d2/d3/l6 0 2024-03-16T10:44:48.332 INFO:tasks.workunit.client.0.smithi002.stdout:1/21: creat d2/d3/f7 x:0 0 0 2024-03-16T10:44:48.332 INFO:tasks.workunit.client.0.smithi002.stdout:1/22: dread - d2/d3/f7 zero size 2024-03-16T10:44:48.334 INFO:tasks.workunit.client.0.smithi002.stdout:1/23: creat d2/d3/f8 x:0 0 0 2024-03-16T10:44:48.443 INFO:tasks.workunit.client.0.smithi002.stdout:1/24: dwrite d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:48.483 INFO:tasks.workunit.client.0.smithi002.stdout:1/25: dwrite d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:48.484 INFO:tasks.workunit.client.0.smithi002.stdout:1/26: read d2/d3/f8 [4083325,73842] 0 2024-03-16T10:44:48.486 INFO:tasks.workunit.client.0.smithi002.stdout:1/27: mknod d2/d3/c9 0 2024-03-16T10:44:48.502 INFO:tasks.workunit.client.0.smithi002.stdout:1/28: dwrite d2/f5 [0,4194304] 0 2024-03-16T10:44:48.502 INFO:tasks.workunit.client.0.smithi002.stdout:1/29: chown d2/l4 118480 1 2024-03-16T10:44:48.505 INFO:tasks.workunit.client.0.smithi002.stdout:1/30: rename d2/l4 to d2/d3/la 0 2024-03-16T10:44:48.508 INFO:tasks.workunit.client.0.smithi002.stdout:1/31: symlink d2/d3/lb 0 2024-03-16T10:44:48.511 INFO:tasks.workunit.client.0.smithi002.stdout:1/32: dread f1 [4194304,4194304] 0 2024-03-16T10:44:48.518 INFO:tasks.workunit.client.0.smithi002.stdout:1/33: dwrite d2/d3/f8 [4194304,4194304] 0 2024-03-16T10:44:48.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:48 smithi026 ceph-mon[37041]: pgmap v27: 65 pgs: 65 active+clean; 186 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 372 op/s 2024-03-16T10:44:48.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:48 smithi002 ceph-mon[29248]: pgmap v27: 65 pgs: 65 active+clean; 186 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 372 op/s 2024-03-16T10:44:49.317 INFO:tasks.workunit.client.0.smithi002.stdout:7/8: sync 2024-03-16T10:44:49.317 INFO:tasks.workunit.client.0.smithi002.stdout:6/0: sync 2024-03-16T10:44:49.318 INFO:tasks.workunit.client.0.smithi002.stdout:7/9: mkdir d3 0 2024-03-16T10:44:49.319 INFO:tasks.workunit.client.0.smithi002.stdout:6/1: symlink l0 0 2024-03-16T10:44:49.319 INFO:tasks.workunit.client.0.smithi002.stdout:7/10: mkdir d3/d4 0 2024-03-16T10:44:49.320 INFO:tasks.workunit.client.0.smithi002.stdout:6/2: symlink l1 0 2024-03-16T10:44:49.326 INFO:tasks.workunit.client.0.smithi002.stdout:7/11: dwrite f1 [0,4194304] 0 2024-03-16T10:44:49.376 INFO:tasks.workunit.client.0.smithi002.stdout:3/26: sync 2024-03-16T10:44:49.376 INFO:tasks.workunit.client.0.smithi002.stdout:4/26: sync 2024-03-16T10:44:49.376 INFO:tasks.workunit.client.0.smithi002.stdout:3/27: truncate - no filename 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:4/27: fdatasync - no filename 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:0/47: sync 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:3/28: dwrite - no filename 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:8/60: sync 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:9/6: sync 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:2/25: sync 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:5/9: sync 2024-03-16T10:44:49.377 INFO:tasks.workunit.client.0.smithi002.stdout:1/34: sync 2024-03-16T10:44:49.378 INFO:tasks.workunit.client.0.smithi002.stdout:9/7: truncate f1 654169 0 2024-03-16T10:44:49.378 INFO:tasks.workunit.client.0.smithi002.stdout:4/28: readlink d2/l4 0 2024-03-16T10:44:49.379 INFO:tasks.workunit.client.0.smithi002.stdout:9/8: dread f1 [0,4194304] 0 2024-03-16T10:44:49.381 INFO:tasks.workunit.client.0.smithi002.stdout:3/29: symlink d1/l4 0 2024-03-16T10:44:49.381 INFO:tasks.workunit.client.0.smithi002.stdout:3/30: truncate - no filename 2024-03-16T10:44:49.382 INFO:tasks.workunit.client.0.smithi002.stdout:0/48: creat d3/d5/f11 x:0 0 0 2024-03-16T10:44:49.383 INFO:tasks.workunit.client.0.smithi002.stdout:0/49: chown d3/d5/fd 107801653 1 2024-03-16T10:44:49.386 INFO:tasks.workunit.client.0.smithi002.stdout:4/29: creat d2/f5 x:0 0 0 2024-03-16T10:44:49.388 INFO:tasks.workunit.client.0.smithi002.stdout:2/26: getdents d7 0 2024-03-16T10:44:49.391 INFO:tasks.workunit.client.0.smithi002.stdout:2/27: dread f6 [0,4194304] 0 2024-03-16T10:44:49.391 INFO:tasks.workunit.client.0.smithi002.stdout:3/31: symlink d1/l5 0 2024-03-16T10:44:49.391 INFO:tasks.workunit.client.0.smithi002.stdout:2/28: fdatasync f6 0 2024-03-16T10:44:49.391 INFO:tasks.workunit.client.0.smithi002.stdout:3/32: rename d1 to d1/d6 22 2024-03-16T10:44:49.392 INFO:tasks.workunit.client.0.smithi002.stdout:3/33: dread - no filename 2024-03-16T10:44:49.392 INFO:tasks.workunit.client.0.smithi002.stdout:3/34: dwrite - no filename 2024-03-16T10:44:49.392 INFO:tasks.workunit.client.0.smithi002.stdout:3/35: truncate - no filename 2024-03-16T10:44:49.392 INFO:tasks.workunit.client.0.smithi002.stdout:3/36: truncate - no filename 2024-03-16T10:44:49.392 INFO:tasks.workunit.client.0.smithi002.stdout:3/37: truncate - no filename 2024-03-16T10:44:49.392 INFO:tasks.workunit.client.0.smithi002.stdout:2/29: chown f5 866525022 1 2024-03-16T10:44:49.393 INFO:tasks.workunit.client.0.smithi002.stdout:2/30: write f5 [2750750,93792] 0 2024-03-16T10:44:49.394 INFO:tasks.workunit.client.0.smithi002.stdout:2/31: readlink l1 0 2024-03-16T10:44:49.396 INFO:tasks.workunit.client.0.smithi002.stdout:4/30: unlink c1 0 2024-03-16T10:44:49.399 INFO:tasks.workunit.client.0.smithi002.stdout:0/50: unlink d3/d5/c9 0 2024-03-16T10:44:49.401 INFO:tasks.workunit.client.0.smithi002.stdout:3/38: rename d1/l5 to d1/l7 0 2024-03-16T10:44:49.401 INFO:tasks.workunit.client.0.smithi002.stdout:3/39: dwrite - no filename 2024-03-16T10:44:49.401 INFO:tasks.workunit.client.0.smithi002.stdout:3/40: rename d1 to d1/d8 22 2024-03-16T10:44:49.401 INFO:tasks.workunit.client.0.smithi002.stdout:3/41: write - no filename 2024-03-16T10:44:49.404 INFO:tasks.workunit.client.0.smithi002.stdout:5/10: link d0/l4 d0/l5 0 2024-03-16T10:44:49.405 INFO:tasks.workunit.client.0.smithi002.stdout:5/11: read d0/f1 [193777,55032] 0 2024-03-16T10:44:49.405 INFO:tasks.workunit.client.0.smithi002.stdout:5/12: read d0/f1 [54412,94286] 0 2024-03-16T10:44:49.405 INFO:tasks.workunit.client.0.smithi002.stdout:4/31: mkdir d2/d6 0 2024-03-16T10:44:49.408 INFO:tasks.workunit.client.0.smithi002.stdout:0/51: creat d3/d5/f12 x:0 0 0 2024-03-16T10:44:49.409 INFO:tasks.workunit.client.0.smithi002.stdout:3/42: creat d1/f9 x:0 0 0 2024-03-16T10:44:49.410 INFO:tasks.workunit.client.0.smithi002.stdout:1/35: getdents d2/d3 0 2024-03-16T10:44:49.411 INFO:tasks.workunit.client.0.smithi002.stdout:1/36: chown d2/d3/lb 1 1 2024-03-16T10:44:49.412 INFO:tasks.workunit.client.0.smithi002.stdout:4/32: mkdir d2/d7 0 2024-03-16T10:44:49.412 INFO:tasks.workunit.client.0.smithi002.stdout:4/33: dread - d2/f5 zero size 2024-03-16T10:44:49.414 INFO:tasks.workunit.client.0.smithi002.stdout:1/37: dread f1 [4194304,4194304] 0 2024-03-16T10:44:49.420 INFO:tasks.workunit.client.0.smithi002.stdout:4/34: mknod d2/d6/c8 0 2024-03-16T10:44:49.421 INFO:tasks.workunit.client.0.smithi002.stdout:1/38: dwrite d2/d3/f7 [0,4194304] 0 2024-03-16T10:44:49.422 INFO:tasks.workunit.client.0.smithi002.stdout:4/35: fdatasync d2/f5 0 2024-03-16T10:44:49.429 INFO:tasks.workunit.client.0.smithi002.stdout:1/39: dwrite d2/d3/f7 [0,4194304] 0 2024-03-16T10:44:49.437 INFO:tasks.workunit.client.0.smithi002.stdout:1/40: dwrite d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:49.441 INFO:tasks.workunit.client.0.smithi002.stdout:1/41: dread d2/d3/f7 [0,4194304] 0 2024-03-16T10:44:49.445 INFO:tasks.workunit.client.0.smithi002.stdout:1/42: creat d2/d3/fc x:0 0 0 2024-03-16T10:44:49.448 INFO:tasks.workunit.client.0.smithi002.stdout:1/43: symlink d2/d3/ld 0 2024-03-16T10:44:49.451 INFO:tasks.workunit.client.0.smithi002.stdout:1/44: creat d2/d3/fe x:0 0 0 2024-03-16T10:44:49.558 INFO:tasks.workunit.client.0.smithi002.stdout:9/9: sync 2024-03-16T10:44:49.583 INFO:tasks.workunit.client.0.smithi002.stdout:6/3: getdents . 0 2024-03-16T10:44:49.586 INFO:tasks.workunit.client.0.smithi002.stdout:6/4: creat f2 x:0 0 0 2024-03-16T10:44:49.586 INFO:tasks.workunit.client.0.smithi002.stdout:6/5: read - f2 zero size 2024-03-16T10:44:49.588 INFO:tasks.workunit.client.0.smithi002.stdout:6/6: mknod c3 0 2024-03-16T10:44:49.589 INFO:tasks.workunit.client.0.smithi002.stdout:6/7: dread - f2 zero size 2024-03-16T10:44:49.589 INFO:tasks.workunit.client.0.smithi002.stdout:7/12: dwrite f1 [4194304,4194304] 0 2024-03-16T10:44:49.590 INFO:tasks.workunit.client.0.smithi002.stdout:6/8: dread - f2 zero size 2024-03-16T10:44:49.593 INFO:tasks.workunit.client.0.smithi002.stdout:6/9: creat f4 x:0 0 0 2024-03-16T10:44:49.596 INFO:tasks.workunit.client.0.smithi002.stdout:6/10: creat f5 x:0 0 0 2024-03-16T10:44:49.599 INFO:tasks.workunit.client.0.smithi002.stdout:0/52: fsync d3/d5/f11 0 2024-03-16T10:44:49.599 INFO:tasks.workunit.client.0.smithi002.stdout:7/13: dwrite f1 [0,4194304] 0 2024-03-16T10:44:49.602 INFO:tasks.workunit.client.0.smithi002.stdout:9/10: write f1 [1526247,24963] 0 2024-03-16T10:44:49.606 INFO:tasks.workunit.client.0.smithi002.stdout:5/13: getdents d0 0 2024-03-16T10:44:49.608 INFO:tasks.workunit.client.0.smithi002.stdout:0/53: creat d3/de/f13 x:0 0 0 2024-03-16T10:44:49.609 INFO:tasks.workunit.client.0.smithi002.stdout:2/32: sync 2024-03-16T10:44:49.609 INFO:tasks.workunit.client.0.smithi002.stdout:1/45: sync 2024-03-16T10:44:49.609 INFO:tasks.workunit.client.0.smithi002.stdout:4/36: sync 2024-03-16T10:44:49.612 INFO:tasks.workunit.client.0.smithi002.stdout:9/11: creat f2 x:0 0 0 2024-03-16T10:44:49.612 INFO:tasks.workunit.client.0.smithi002.stdout:0/54: chown d3/d5 16674526 1 2024-03-16T10:44:49.614 INFO:tasks.workunit.client.0.smithi002.stdout:6/11: dwrite f2 [0,4194304] 0 2024-03-16T10:44:49.614 INFO:tasks.workunit.client.0.smithi002.stdout:8/61: dwrite d7/fc [0,4194304] 0 2024-03-16T10:44:49.616 INFO:tasks.workunit.client.0.smithi002.stdout:1/46: mkdir d2/df 0 2024-03-16T10:44:49.617 INFO:tasks.workunit.client.0.smithi002.stdout:0/55: stat d3/d5/fd 0 2024-03-16T10:44:49.617 INFO:tasks.workunit.client.0.smithi002.stdout:4/37: truncate d2/f5 160830 0 2024-03-16T10:44:49.619 INFO:tasks.workunit.client.0.smithi002.stdout:2/33: rename f5 to d7/f8 0 2024-03-16T10:44:49.620 INFO:tasks.workunit.client.0.smithi002.stdout:5/14: write d0/f1 [298724,36933] 0 2024-03-16T10:44:49.622 INFO:tasks.workunit.client.0.smithi002.stdout:3/43: getdents d1 0 2024-03-16T10:44:49.622 INFO:tasks.workunit.client.0.smithi002.stdout:1/47: readlink d2/d3/ld 0 2024-03-16T10:44:49.624 INFO:tasks.workunit.client.0.smithi002.stdout:7/14: dwrite f1 [4194304,4194304] 0 2024-03-16T10:44:49.624 INFO:tasks.workunit.client.0.smithi002.stdout:4/38: write d2/f5 [656947,82259] 0 2024-03-16T10:44:49.624 INFO:tasks.workunit.client.0.smithi002.stdout:8/62: mkdir d7/d9/d17 0 2024-03-16T10:44:49.625 INFO:tasks.workunit.client.0.smithi002.stdout:3/44: dread - d1/f9 zero size 2024-03-16T10:44:49.630 INFO:tasks.workunit.client.0.smithi002.stdout:0/56: creat d3/d5/f14 x:0 0 0 2024-03-16T10:44:49.633 INFO:tasks.workunit.client.0.smithi002.stdout:5/15: readlink d0/l5 0 2024-03-16T10:44:49.633 INFO:tasks.workunit.client.0.smithi002.stdout:1/48: mknod d2/d3/c10 0 2024-03-16T10:44:49.635 INFO:tasks.workunit.client.0.smithi002.stdout:1/49: fdatasync d2/d3/fe 0 2024-03-16T10:44:49.635 INFO:tasks.workunit.client.0.smithi002.stdout:8/63: symlink d7/d13/l18 0 2024-03-16T10:44:49.636 INFO:tasks.workunit.client.0.smithi002.stdout:5/16: dread d0/f1 [0,4194304] 0 2024-03-16T10:44:49.636 INFO:tasks.workunit.client.0.smithi002.stdout:4/39: unlink d2/f5 0 2024-03-16T10:44:49.636 INFO:tasks.workunit.client.0.smithi002.stdout:4/40: dwrite - no filename 2024-03-16T10:44:49.636 INFO:tasks.workunit.client.0.smithi002.stdout:4/41: read - no filename 2024-03-16T10:44:49.636 INFO:tasks.workunit.client.0.smithi002.stdout:4/42: dwrite - no filename 2024-03-16T10:44:49.636 INFO:tasks.workunit.client.0.smithi002.stdout:4/43: dread - no filename 2024-03-16T10:44:49.637 INFO:tasks.workunit.client.0.smithi002.stdout:4/44: chown d2/d6 21189 1 2024-03-16T10:44:49.638 INFO:tasks.workunit.client.0.smithi002.stdout:4/45: chown l0 0 1 2024-03-16T10:44:49.638 INFO:tasks.workunit.client.0.smithi002.stdout:4/46: write - no filename 2024-03-16T10:44:49.639 INFO:tasks.workunit.client.0.smithi002.stdout:6/12: link f4 f6 0 2024-03-16T10:44:49.640 INFO:tasks.workunit.client.0.smithi002.stdout:4/47: chown d2 468 1 2024-03-16T10:44:49.640 INFO:tasks.workunit.client.0.smithi002.stdout:5/17: truncate d0/f1 1864215 0 2024-03-16T10:44:49.641 INFO:tasks.workunit.client.0.smithi002.stdout:0/57: creat d3/de/f15 x:0 0 0 2024-03-16T10:44:49.642 INFO:tasks.workunit.client.0.smithi002.stdout:4/48: chown l0 1751529 1 2024-03-16T10:44:49.643 INFO:tasks.workunit.client.0.smithi002.stdout:0/58: fdatasync d3/de/f10 0 2024-03-16T10:44:49.644 INFO:tasks.workunit.client.0.smithi002.stdout:1/50: dread f1 [0,4194304] 0 2024-03-16T10:44:49.645 INFO:tasks.workunit.client.0.smithi002.stdout:0/59: fdatasync d3/d5/fb 0 2024-03-16T10:44:49.645 INFO:tasks.workunit.client.0.smithi002.stdout:8/64: dwrite f4 [0,4194304] 0 2024-03-16T10:44:49.645 INFO:tasks.workunit.client.0.smithi002.stdout:0/60: chown d3/de 2518 1 2024-03-16T10:44:49.647 INFO:tasks.workunit.client.0.smithi002.stdout:0/61: truncate d3/d5/f11 808146 0 2024-03-16T10:44:49.652 INFO:tasks.workunit.client.0.smithi002.stdout:6/13: symlink l7 0 2024-03-16T10:44:49.652 INFO:tasks.workunit.client.0.smithi002.stdout:6/14: readlink l1 0 2024-03-16T10:44:49.653 INFO:tasks.workunit.client.0.smithi002.stdout:6/15: stat c3 0 2024-03-16T10:44:49.655 INFO:tasks.workunit.client.0.smithi002.stdout:5/18: creat d0/f6 x:0 0 0 2024-03-16T10:44:49.655 INFO:tasks.workunit.client.0.smithi002.stdout:5/19: dread - d0/f6 zero size 2024-03-16T10:44:49.656 INFO:tasks.workunit.client.0.smithi002.stdout:5/20: readlink d0/l5 0 2024-03-16T10:44:49.656 INFO:tasks.workunit.client.0.smithi002.stdout:2/34: link l1 d7/l9 0 2024-03-16T10:44:49.657 INFO:tasks.workunit.client.0.smithi002.stdout:2/35: read f4 [931776,116984] 0 2024-03-16T10:44:49.660 INFO:tasks.workunit.client.0.smithi002.stdout:4/49: unlink d2/c3 0 2024-03-16T10:44:49.660 INFO:tasks.workunit.client.0.smithi002.stdout:4/50: readlink d2/l4 0 2024-03-16T10:44:49.661 INFO:tasks.workunit.client.0.smithi002.stdout:4/51: stat d2/d6/c8 0 2024-03-16T10:44:49.661 INFO:tasks.workunit.client.0.smithi002.stdout:4/52: write - no filename 2024-03-16T10:44:49.661 INFO:tasks.workunit.client.0.smithi002.stdout:4/53: write - no filename 2024-03-16T10:44:49.671 INFO:tasks.workunit.client.0.smithi002.stdout:0/62: symlink d3/l16 0 2024-03-16T10:44:49.673 INFO:tasks.workunit.client.0.smithi002.stdout:1/51: dwrite f1 [4194304,4194304] 0 2024-03-16T10:44:49.674 INFO:tasks.workunit.client.0.smithi002.stdout:3/45: link d1/l7 d1/la 0 2024-03-16T10:44:49.679 INFO:tasks.workunit.client.0.smithi002.stdout:5/21: rename d0/f1 to d0/f7 0 2024-03-16T10:44:49.682 INFO:tasks.workunit.client.0.smithi002.stdout:9/12: write f0 [2319658,78303] 0 2024-03-16T10:44:49.685 INFO:tasks.workunit.client.0.smithi002.stdout:9/13: chown f2 287 1 2024-03-16T10:44:49.688 INFO:tasks.workunit.client.0.smithi002.stdout:0/63: unlink d3/d5/f11 0 2024-03-16T10:44:49.689 INFO:tasks.workunit.client.0.smithi002.stdout:6/16: dwrite f6 [0,4194304] 0 2024-03-16T10:44:49.691 INFO:tasks.workunit.client.0.smithi002.stdout:0/64: stat d3/d5/f12 0 2024-03-16T10:44:49.692 INFO:tasks.workunit.client.0.smithi002.stdout:3/46: creat d1/fb x:0 0 0 2024-03-16T10:44:49.693 INFO:tasks.workunit.client.0.smithi002.stdout:6/17: write f2 [42955,33454] 0 2024-03-16T10:44:49.696 INFO:tasks.workunit.client.0.smithi002.stdout:5/22: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:49.697 INFO:tasks.workunit.client.0.smithi002.stdout:4/54: mknod d2/d7/c9 0 2024-03-16T10:44:49.697 INFO:tasks.workunit.client.0.smithi002.stdout:2/36: dwrite f4 [0,4194304] 0 2024-03-16T10:44:49.699 INFO:tasks.workunit.client.0.smithi002.stdout:2/37: stat f6 0 2024-03-16T10:44:49.700 INFO:tasks.workunit.client.0.smithi002.stdout:2/38: rename d7 to d7/da 22 2024-03-16T10:44:49.700 INFO:tasks.workunit.client.0.smithi002.stdout:9/14: unlink f2 0 2024-03-16T10:44:49.707 INFO:tasks.workunit.client.0.smithi002.stdout:1/52: mknod d2/df/c11 0 2024-03-16T10:44:49.711 INFO:tasks.workunit.client.0.smithi002.stdout:0/65: dwrite d3/d5/f12 [0,4194304] 0 2024-03-16T10:44:49.713 INFO:tasks.workunit.client.0.smithi002.stdout:3/47: symlink d1/lc 0 2024-03-16T10:44:49.715 INFO:tasks.workunit.client.0.smithi002.stdout:1/53: dread d2/f5 [0,4194304] 0 2024-03-16T10:44:49.717 INFO:tasks.workunit.client.0.smithi002.stdout:9/15: dread f1 [0,4194304] 0 2024-03-16T10:44:49.717 INFO:tasks.workunit.client.0.smithi002.stdout:9/16: readlink - no filename 2024-03-16T10:44:49.732 INFO:tasks.workunit.client.0.smithi002.stdout:7/15: truncate f1 3549792 0 2024-03-16T10:44:49.732 INFO:tasks.workunit.client.0.smithi002.stdout:7/16: rename d3 to d3/d4/d5 22 2024-03-16T10:44:49.734 INFO:tasks.workunit.client.0.smithi002.stdout:0/66: mknod d3/d5/c17 0 2024-03-16T10:44:49.734 INFO:tasks.workunit.client.0.smithi002.stdout:0/67: chown d3/l8 373833 1 2024-03-16T10:44:49.737 INFO:tasks.workunit.client.0.smithi002.stdout:1/54: mkdir d2/df/d12 0 2024-03-16T10:44:49.738 INFO:tasks.workunit.client.0.smithi002.stdout:1/55: dread - d2/d3/fe zero size 2024-03-16T10:44:49.741 INFO:tasks.workunit.client.0.smithi002.stdout:2/39: stat l1 0 2024-03-16T10:44:49.751 INFO:tasks.workunit.client.0.smithi002.stdout:0/68: unlink d3/d5/l6 0 2024-03-16T10:44:49.752 INFO:tasks.workunit.client.0.smithi002.stdout:8/65: truncate d7/d9/fa 2428005 0 2024-03-16T10:44:49.754 INFO:tasks.workunit.client.0.smithi002.stdout:1/56: creat d2/f13 x:0 0 0 2024-03-16T10:44:49.758 INFO:tasks.workunit.client.0.smithi002.stdout:1/57: dread d2/d3/f7 [0,4194304] 0 2024-03-16T10:44:49.759 INFO:tasks.workunit.client.0.smithi002.stdout:2/40: mkdir d7/db 0 2024-03-16T10:44:49.759 INFO:tasks.workunit.client.0.smithi002.stdout:2/41: stat l1 0 2024-03-16T10:44:49.760 INFO:tasks.workunit.client.0.smithi002.stdout:6/18: write f4 [4421553,114565] 0 2024-03-16T10:44:49.761 INFO:tasks.workunit.client.0.smithi002.stdout:5/23: link d0/c3 d0/c8 0 2024-03-16T10:44:49.763 INFO:tasks.workunit.client.0.smithi002.stdout:0/69: creat d3/d5/f18 x:0 0 0 2024-03-16T10:44:49.764 INFO:tasks.workunit.client.0.smithi002.stdout:0/70: write d3/d5/fb [4872416,104230] 0 2024-03-16T10:44:49.765 INFO:tasks.workunit.client.0.smithi002.stdout:6/19: dwrite f4 [0,4194304] 0 2024-03-16T10:44:49.766 INFO:tasks.workunit.client.0.smithi002.stdout:0/71: chown d3/d5/fb 199967627 1 2024-03-16T10:44:49.767 INFO:tasks.workunit.client.0.smithi002.stdout:8/66: creat d7/f19 x:0 0 0 2024-03-16T10:44:49.768 INFO:tasks.workunit.client.0.smithi002.stdout:0/72: fsync d3/d5/fd 0 2024-03-16T10:44:49.768 INFO:tasks.workunit.client.0.smithi002.stdout:4/55: getdents d2/d7 0 2024-03-16T10:44:49.768 INFO:tasks.workunit.client.0.smithi002.stdout:4/56: write - no filename 2024-03-16T10:44:49.768 INFO:tasks.workunit.client.0.smithi002.stdout:4/57: fsync - no filename 2024-03-16T10:44:49.769 INFO:tasks.workunit.client.0.smithi002.stdout:4/58: write - no filename 2024-03-16T10:44:49.769 INFO:tasks.workunit.client.0.smithi002.stdout:4/59: dread - no filename 2024-03-16T10:44:49.775 INFO:tasks.workunit.client.0.smithi002.stdout:1/58: dwrite d2/d3/f7 [0,4194304] 0 2024-03-16T10:44:49.776 INFO:tasks.workunit.client.0.smithi002.stdout:1/59: chown d2/d3/fc 30381452 1 2024-03-16T10:44:49.777 INFO:tasks.workunit.client.0.smithi002.stdout:1/60: fdatasync d2/f13 0 2024-03-16T10:44:49.778 INFO:tasks.workunit.client.0.smithi002.stdout:3/48: getdents d1 0 2024-03-16T10:44:49.782 INFO:tasks.workunit.client.0.smithi002.stdout:6/20: rename f5 to f8 0 2024-03-16T10:44:49.782 INFO:tasks.workunit.client.0.smithi002.stdout:6/21: rmdir - no directory 2024-03-16T10:44:49.783 INFO:tasks.workunit.client.0.smithi002.stdout:6/22: readlink l1 0 2024-03-16T10:44:49.788 INFO:tasks.workunit.client.0.smithi002.stdout:4/60: mkdir d2/d7/da 0 2024-03-16T10:44:49.789 INFO:tasks.workunit.client.0.smithi002.stdout:4/61: fsync - no filename 2024-03-16T10:44:49.789 INFO:tasks.workunit.client.0.smithi002.stdout:4/62: fsync - no filename 2024-03-16T10:44:49.791 INFO:tasks.workunit.client.0.smithi002.stdout:7/17: write f1 [2863672,31085] 0 2024-03-16T10:44:49.794 INFO:tasks.workunit.client.0.smithi002.stdout:5/24: chown d0/c8 69 1 2024-03-16T10:44:49.797 INFO:tasks.workunit.client.0.smithi002.stdout:3/49: mkdir d1/dd 0 2024-03-16T10:44:49.798 INFO:tasks.workunit.client.0.smithi002.stdout:3/50: chown d1 615 1 2024-03-16T10:44:49.798 INFO:tasks.workunit.client.0.smithi002.stdout:7/18: dwrite f1 [0,4194304] 0 2024-03-16T10:44:49.798 INFO:tasks.workunit.client.0.smithi002.stdout:3/51: chown d1/l4 10500242 1 2024-03-16T10:44:49.800 INFO:tasks.workunit.client.0.smithi002.stdout:6/23: creat f9 x:0 0 0 2024-03-16T10:44:49.802 INFO:tasks.workunit.client.0.smithi002.stdout:4/63: mknod d2/d6/cb 0 2024-03-16T10:44:49.802 INFO:tasks.workunit.client.0.smithi002.stdout:4/64: truncate - no filename 2024-03-16T10:44:49.803 INFO:tasks.workunit.client.0.smithi002.stdout:1/61: truncate d2/f5 6628836 0 2024-03-16T10:44:49.806 INFO:tasks.workunit.client.0.smithi002.stdout:0/73: link d3/d5/fd d3/d5/f19 0 2024-03-16T10:44:49.807 INFO:tasks.workunit.client.0.smithi002.stdout:0/74: write d3/d5/f12 [878021,82707] 0 2024-03-16T10:44:49.808 INFO:tasks.workunit.client.0.smithi002.stdout:0/75: read d3/d5/f19 [110351,102949] 0 2024-03-16T10:44:49.811 INFO:tasks.workunit.client.0.smithi002.stdout:8/67: rmdir d7/d9 39 2024-03-16T10:44:49.811 INFO:tasks.workunit.client.0.smithi002.stdout:8/68: write f4 [998183,2902] 0 2024-03-16T10:44:49.812 INFO:tasks.workunit.client.0.smithi002.stdout:8/69: chown d7/d13/l18 0 1 2024-03-16T10:44:49.814 INFO:tasks.workunit.client.0.smithi002.stdout:8/70: write d7/fc [4666238,125087] 0 2024-03-16T10:44:49.814 INFO:tasks.workunit.client.0.smithi002.stdout:8/71: stat d7 0 2024-03-16T10:44:49.816 INFO:tasks.workunit.client.0.smithi002.stdout:9/17: truncate f1 1315110 0 2024-03-16T10:44:49.817 INFO:tasks.workunit.client.0.smithi002.stdout:7/19: creat d3/f6 x:0 0 0 2024-03-16T10:44:49.819 INFO:tasks.workunit.client.0.smithi002.stdout:8/72: dread d7/fc [0,4194304] 0 2024-03-16T10:44:49.824 INFO:tasks.workunit.client.0.smithi002.stdout:7/20: dwrite f1 [0,4194304] 0 2024-03-16T10:44:49.824 INFO:tasks.workunit.client.0.smithi002.stdout:7/21: readlink l2 0 2024-03-16T10:44:49.825 INFO:tasks.workunit.client.0.smithi002.stdout:7/22: stat l2 0 2024-03-16T10:44:49.826 INFO:tasks.workunit.client.0.smithi002.stdout:7/23: truncate d3/f6 31597 0 2024-03-16T10:44:49.827 INFO:tasks.workunit.client.0.smithi002.stdout:7/24: write f1 [1871545,3492] 0 2024-03-16T10:44:49.828 INFO:tasks.workunit.client.0.smithi002.stdout:7/25: write f1 [1622175,91100] 0 2024-03-16T10:44:49.833 INFO:tasks.workunit.client.0.smithi002.stdout:2/42: getdents d7 0 2024-03-16T10:44:49.836 INFO:tasks.workunit.client.0.smithi002.stdout:7/26: dwrite f1 [4194304,4194304] 0 2024-03-16T10:44:49.837 INFO:tasks.workunit.client.0.smithi002.stdout:7/27: write f1 [4538822,95031] 0 2024-03-16T10:44:49.837 INFO:tasks.workunit.client.0.smithi002.stdout:7/28: readlink l2 0 2024-03-16T10:44:49.843 INFO:tasks.workunit.client.0.smithi002.stdout:3/52: creat d1/dd/fe x:0 0 0 2024-03-16T10:44:49.844 INFO:tasks.workunit.client.0.smithi002.stdout:3/53: read - d1/f9 zero size 2024-03-16T10:44:49.845 INFO:tasks.workunit.client.0.smithi002.stdout:6/24: link f8 fa 0 2024-03-16T10:44:49.845 INFO:tasks.workunit.client.0.smithi002.stdout:6/25: fdatasync f4 0 2024-03-16T10:44:49.847 INFO:tasks.workunit.client.0.smithi002.stdout:4/65: mknod d2/d7/da/cc 0 2024-03-16T10:44:49.855 INFO:tasks.workunit.client.0.smithi002.stdout:2/43: mkdir d7/dc 0 2024-03-16T10:44:49.861 INFO:tasks.workunit.client.0.smithi002.stdout:5/25: dwrite d0/f7 [0,4194304] 0 2024-03-16T10:44:49.864 INFO:tasks.workunit.client.0.smithi002.stdout:3/54: symlink d1/lf 0 2024-03-16T10:44:49.864 INFO:tasks.workunit.client.0.smithi002.stdout:3/55: dread - d1/f9 zero size 2024-03-16T10:44:49.868 INFO:tasks.workunit.client.0.smithi002.stdout:0/76: truncate d3/d5/fd 4309654 0 2024-03-16T10:44:49.868 INFO:tasks.workunit.client.0.smithi002.stdout:0/77: read - d3/de/f13 zero size 2024-03-16T10:44:49.876 INFO:tasks.workunit.client.0.smithi002.stdout:7/29: symlink d3/d4/l7 0 2024-03-16T10:44:49.880 INFO:tasks.workunit.client.0.smithi002.stdout:7/30: dread f1 [4194304,4194304] 0 2024-03-16T10:44:49.881 INFO:tasks.workunit.client.0.smithi002.stdout:5/26: unlink d0/l4 0 2024-03-16T10:44:49.885 INFO:tasks.workunit.client.0.smithi002.stdout:5/27: dread d0/f7 [0,4194304] 0 2024-03-16T10:44:49.887 INFO:tasks.workunit.client.0.smithi002.stdout:4/66: rename d2/d7 to d2/dd 0 2024-03-16T10:44:49.890 INFO:tasks.workunit.client.0.smithi002.stdout:4/67: dwrite - no filename 2024-03-16T10:44:49.898 INFO:tasks.workunit.client.0.smithi002.stdout:9/18: dread f0 [0,4194304] 0 2024-03-16T10:44:49.899 INFO:tasks.workunit.client.0.smithi002.stdout:3/56: rename d1/dd/fe to d1/dd/f10 0 2024-03-16T10:44:49.902 INFO:tasks.workunit.client.0.smithi002.stdout:5/28: creat d0/f9 x:0 0 0 2024-03-16T10:44:49.903 INFO:tasks.workunit.client.0.smithi002.stdout:5/29: write d0/f9 [62013,10516] 0 2024-03-16T10:44:49.903 INFO:tasks.workunit.client.0.smithi002.stdout:4/68: creat d2/fe x:0 0 0 2024-03-16T10:44:49.905 INFO:tasks.workunit.client.0.smithi002.stdout:0/78: rmdir d3/d5 39 2024-03-16T10:44:49.906 INFO:tasks.workunit.client.0.smithi002.stdout:8/73: link d7/d13/l18 d7/l1a 0 2024-03-16T10:44:49.907 INFO:tasks.workunit.client.0.smithi002.stdout:8/74: dread - d7/f19 zero size 2024-03-16T10:44:49.909 INFO:tasks.workunit.client.0.smithi002.stdout:5/30: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:49.911 INFO:tasks.workunit.client.0.smithi002.stdout:4/69: dwrite d2/fe [0,4194304] 0 2024-03-16T10:44:49.911 INFO:tasks.workunit.client.0.smithi002.stdout:7/31: rmdir d3 39 2024-03-16T10:44:49.913 INFO:tasks.workunit.client.0.smithi002.stdout:9/19: creat f3 x:0 0 0 2024-03-16T10:44:49.914 INFO:tasks.workunit.client.0.smithi002.stdout:9/20: dread - f3 zero size 2024-03-16T10:44:49.915 INFO:tasks.workunit.client.0.smithi002.stdout:3/57: chown d1/l7 1941 1 2024-03-16T10:44:49.918 INFO:tasks.workunit.client.0.smithi002.stdout:6/26: getdents . 0 2024-03-16T10:44:49.919 INFO:tasks.workunit.client.0.smithi002.stdout:6/27: write f4 [580768,807] 0 2024-03-16T10:44:49.919 INFO:tasks.workunit.client.0.smithi002.stdout:1/62: sync 2024-03-16T10:44:49.924 INFO:tasks.workunit.client.0.smithi002.stdout:6/28: dread f4 [0,4194304] 0 2024-03-16T10:44:49.925 INFO:tasks.workunit.client.0.smithi002.stdout:2/44: write f3 [3991112,5394] 0 2024-03-16T10:44:49.934 INFO:tasks.workunit.client.0.smithi002.stdout:5/31: dread d0/f7 [0,4194304] 0 2024-03-16T10:44:49.935 INFO:tasks.workunit.client.0.smithi002.stdout:9/21: symlink l4 0 2024-03-16T10:44:49.937 INFO:tasks.workunit.client.0.smithi002.stdout:3/58: mkdir d1/dd/d11 0 2024-03-16T10:44:49.938 INFO:tasks.workunit.client.0.smithi002.stdout:7/32: dwrite f1 [0,4194304] 0 2024-03-16T10:44:49.938 INFO:tasks.workunit.client.0.smithi002.stdout:3/59: truncate d1/fb 528860 0 2024-03-16T10:44:49.939 INFO:tasks.workunit.client.0.smithi002.stdout:3/60: rename d1 to d1/d12 22 2024-03-16T10:44:49.948 INFO:tasks.workunit.client.0.smithi002.stdout:6/29: symlink lb 0 2024-03-16T10:44:49.956 INFO:tasks.workunit.client.0.smithi002.stdout:6/30: dwrite f2 [4194304,4194304] 0 2024-03-16T10:44:49.962 INFO:tasks.workunit.client.0.smithi002.stdout:9/22: unlink l4 0 2024-03-16T10:44:49.964 INFO:tasks.workunit.client.0.smithi002.stdout:6/31: dwrite f9 [0,4194304] 0 2024-03-16T10:44:49.968 INFO:tasks.workunit.client.0.smithi002.stdout:3/61: rmdir d1 39 2024-03-16T10:44:49.978 INFO:tasks.workunit.client.0.smithi002.stdout:0/79: creat d3/f1a x:0 0 0 2024-03-16T10:44:49.978 INFO:tasks.workunit.client.0.smithi002.stdout:2/45: write f6 [5181293,79724] 0 2024-03-16T10:44:49.980 INFO:tasks.workunit.client.0.smithi002.stdout:4/70: truncate d2/fe 3878422 0 2024-03-16T10:44:49.987 INFO:tasks.workunit.client.0.smithi002.stdout:2/46: dwrite f3 [0,4194304] 0 2024-03-16T10:44:49.988 INFO:tasks.workunit.client.0.smithi002.stdout:7/33: mknod d3/c8 0 2024-03-16T10:44:49.991 INFO:tasks.workunit.client.0.smithi002.stdout:1/63: truncate d2/d3/f8 4355619 0 2024-03-16T10:44:49.992 INFO:tasks.workunit.client.0.smithi002.stdout:7/34: dread f1 [4194304,4194304] 0 2024-03-16T10:44:49.993 INFO:tasks.workunit.client.0.smithi002.stdout:1/64: write d2/f13 [48384,33885] 0 2024-03-16T10:44:49.995 INFO:tasks.workunit.client.0.smithi002.stdout:0/80: rmdir d3/de 39 2024-03-16T10:44:50.004 INFO:tasks.workunit.client.0.smithi002.stdout:9/23: write f0 [499843,7985] 0 2024-03-16T10:44:50.006 INFO:tasks.workunit.client.0.smithi002.stdout:7/35: dwrite d3/f6 [0,4194304] 0 2024-03-16T10:44:50.007 INFO:tasks.workunit.client.0.smithi002.stdout:6/32: write f8 [405626,122727] 0 2024-03-16T10:44:50.012 INFO:tasks.workunit.client.0.smithi002.stdout:1/65: dwrite d2/d3/fe [0,4194304] 0 2024-03-16T10:44:50.012 INFO:tasks.workunit.client.0.smithi002.stdout:6/33: chown f9 32336432 1 2024-03-16T10:44:50.013 INFO:tasks.workunit.client.0.smithi002.stdout:5/32: unlink d0/c8 0 2024-03-16T10:44:50.013 INFO:tasks.workunit.client.0.smithi002.stdout:3/62: creat d1/dd/f13 x:0 0 0 2024-03-16T10:44:50.019 INFO:tasks.workunit.client.0.smithi002.stdout:8/75: dwrite d7/d9/fa [0,4194304] 0 2024-03-16T10:44:50.022 INFO:tasks.workunit.client.0.smithi002.stdout:9/24: unlink f3 0 2024-03-16T10:44:50.023 INFO:tasks.workunit.client.0.smithi002.stdout:7/36: rename d3/c8 to d3/c9 0 2024-03-16T10:44:50.023 INFO:tasks.workunit.client.0.smithi002.stdout:4/71: write d2/fe [70277,14043] 0 2024-03-16T10:44:50.023 INFO:tasks.workunit.client.0.smithi002.stdout:1/66: mknod d2/c14 0 2024-03-16T10:44:50.025 INFO:tasks.workunit.client.0.smithi002.stdout:9/25: write f0 [1637113,62082] 0 2024-03-16T10:44:50.027 INFO:tasks.workunit.client.0.smithi002.stdout:0/81: mknod d3/c1b 0 2024-03-16T10:44:50.027 INFO:tasks.workunit.client.0.smithi002.stdout:2/47: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:50.029 INFO:tasks.workunit.client.0.smithi002.stdout:9/26: write f0 [2535935,126781] 0 2024-03-16T10:44:50.031 INFO:tasks.workunit.client.0.smithi002.stdout:9/27: write f0 [2990463,111519] 0 2024-03-16T10:44:50.033 INFO:tasks.workunit.client.0.smithi002.stdout:5/33: mknod d0/ca 0 2024-03-16T10:44:50.034 INFO:tasks.workunit.client.0.smithi002.stdout:6/34: dread f4 [0,4194304] 0 2024-03-16T10:44:50.035 INFO:tasks.workunit.client.0.smithi002.stdout:6/35: write f9 [2754326,114570] 0 2024-03-16T10:44:50.037 INFO:tasks.workunit.client.0.smithi002.stdout:6/36: write f2 [4307507,26798] 0 2024-03-16T10:44:50.037 INFO:tasks.workunit.client.0.smithi002.stdout:4/72: creat d2/dd/da/ff x:0 0 0 2024-03-16T10:44:50.038 INFO:tasks.workunit.client.0.smithi002.stdout:7/37: dread f1 [4194304,4194304] 0 2024-03-16T10:44:50.043 INFO:tasks.workunit.client.0.smithi002.stdout:3/63: link d1/f9 d1/dd/d11/f14 0 2024-03-16T10:44:50.050 INFO:tasks.workunit.client.0.smithi002.stdout:1/67: rename d2/d3/la to d2/df/d12/l15 0 2024-03-16T10:44:50.056 INFO:tasks.workunit.client.0.smithi002.stdout:9/28: mkdir d5 0 2024-03-16T10:44:50.056 INFO:tasks.workunit.client.0.smithi002.stdout:9/29: chown f0 6391 1 2024-03-16T10:44:50.063 INFO:tasks.workunit.client.0.smithi002.stdout:4/73: creat d2/d6/f10 x:0 0 0 2024-03-16T10:44:50.069 INFO:tasks.workunit.client.0.smithi002.stdout:1/68: rmdir d2/df 39 2024-03-16T10:44:50.070 INFO:tasks.workunit.client.0.smithi002.stdout:3/64: dwrite d1/f9 [0,4194304] 0 2024-03-16T10:44:50.070 INFO:tasks.workunit.client.0.smithi002.stdout:2/48: mkdir d7/dc/dd 0 2024-03-16T10:44:50.071 INFO:tasks.workunit.client.0.smithi002.stdout:2/49: write f6 [4422173,22109] 0 2024-03-16T10:44:50.162 INFO:tasks.workunit.client.0.smithi002.stdout:2/50: fdatasync f6 0 2024-03-16T10:44:50.162 INFO:tasks.workunit.client.0.smithi002.stdout:5/34: unlink d0/c3 0 2024-03-16T10:44:50.167 INFO:tasks.workunit.client.0.smithi002.stdout:3/65: dread d1/dd/d11/f14 [0,4194304] 0 2024-03-16T10:44:50.169 INFO:tasks.workunit.client.0.smithi002.stdout:0/82: write d3/d5/fd [5048974,25281] 0 2024-03-16T10:44:50.171 INFO:tasks.workunit.client.0.smithi002.stdout:8/76: dwrite d7/fc [4194304,4194304] 0 2024-03-16T10:44:50.172 INFO:tasks.workunit.client.0.smithi002.stdout:0/83: chown d3/l16 12988537 1 2024-03-16T10:44:50.173 INFO:tasks.workunit.client.0.smithi002.stdout:2/51: dwrite f3 [0,4194304] 0 2024-03-16T10:44:50.179 INFO:tasks.workunit.client.0.smithi002.stdout:0/84: dwrite d3/fc [0,4194304] 0 2024-03-16T10:44:50.183 INFO:tasks.workunit.client.0.smithi002.stdout:6/37: sync 2024-03-16T10:44:50.236 INFO:tasks.workunit.client.0.smithi002.stdout:4/74: sync 2024-03-16T10:44:50.236 INFO:tasks.workunit.client.0.smithi002.stdout:7/38: sync 2024-03-16T10:44:50.236 INFO:tasks.workunit.client.0.smithi002.stdout:4/75: dread - d2/d6/f10 zero size 2024-03-16T10:44:50.237 INFO:tasks.workunit.client.0.smithi002.stdout:4/76: dread - d2/d6/f10 zero size 2024-03-16T10:44:50.244 INFO:tasks.workunit.client.0.smithi002.stdout:4/77: dwrite d2/fe [0,4194304] 0 2024-03-16T10:44:50.252 INFO:tasks.workunit.client.0.smithi002.stdout:4/78: dwrite d2/d6/f10 [0,4194304] 0 2024-03-16T10:44:50.253 INFO:tasks.workunit.client.0.smithi002.stdout:4/79: truncate d2/fe 4569802 0 2024-03-16T10:44:50.328 INFO:tasks.workunit.client.0.smithi002.stdout:9/30: symlink d5/l6 0 2024-03-16T10:44:50.331 INFO:tasks.workunit.client.0.smithi002.stdout:1/69: mknod d2/c16 0 2024-03-16T10:44:50.334 INFO:tasks.workunit.client.0.smithi002.stdout:8/77: symlink d7/d13/l1b 0 2024-03-16T10:44:50.336 INFO:tasks.workunit.client.0.smithi002.stdout:0/85: creat d3/d5/f1c x:0 0 0 2024-03-16T10:44:50.337 INFO:tasks.workunit.client.0.smithi002.stdout:6/38: creat fc x:0 0 0 2024-03-16T10:44:50.338 INFO:tasks.workunit.client.0.smithi002.stdout:7/39: creat d3/fa x:0 0 0 2024-03-16T10:44:50.339 INFO:tasks.workunit.client.0.smithi002.stdout:1/70: dwrite d2/d3/fe [0,4194304] 0 2024-03-16T10:44:50.341 INFO:tasks.workunit.client.0.smithi002.stdout:1/71: chown d2/d3/fc 1811487404 1 2024-03-16T10:44:50.343 INFO:tasks.workunit.client.0.smithi002.stdout:4/80: symlink d2/dd/l11 0 2024-03-16T10:44:50.345 INFO:tasks.workunit.client.0.smithi002.stdout:9/31: rename f0 to d5/f7 0 2024-03-16T10:44:50.345 INFO:tasks.workunit.client.0.smithi002.stdout:7/40: dread f1 [4194304,4194304] 0 2024-03-16T10:44:50.345 INFO:tasks.workunit.client.0.smithi002.stdout:9/32: chown f1 16275182 1 2024-03-16T10:44:50.349 INFO:tasks.workunit.client.0.smithi002.stdout:3/66: mkdir d1/dd/d11/d15 0 2024-03-16T10:44:50.351 INFO:tasks.workunit.client.0.smithi002.stdout:2/52: symlink d7/dc/dd/le 0 2024-03-16T10:44:50.353 INFO:tasks.workunit.client.0.smithi002.stdout:7/41: dread d3/f6 [0,4194304] 0 2024-03-16T10:44:50.355 INFO:tasks.workunit.client.0.smithi002.stdout:2/53: dread f4 [0,4194304] 0 2024-03-16T10:44:50.357 INFO:tasks.workunit.client.0.smithi002.stdout:7/42: dread f1 [0,4194304] 0 2024-03-16T10:44:50.357 INFO:tasks.workunit.client.0.smithi002.stdout:7/43: readlink d3/d4/l7 0 2024-03-16T10:44:50.358 INFO:tasks.workunit.client.0.smithi002.stdout:6/39: creat fd x:0 0 0 2024-03-16T10:44:50.359 INFO:tasks.workunit.client.0.smithi002.stdout:6/40: chown f9 30864 1 2024-03-16T10:44:50.360 INFO:tasks.workunit.client.0.smithi002.stdout:6/41: chown f8 8378 1 2024-03-16T10:44:50.360 INFO:tasks.workunit.client.0.smithi002.stdout:6/42: chown f9 0 1 2024-03-16T10:44:50.360 INFO:tasks.workunit.client.0.smithi002.stdout:6/43: chown l7 120138 1 2024-03-16T10:44:50.362 INFO:tasks.workunit.client.0.smithi002.stdout:1/72: creat d2/d3/f17 x:0 0 0 2024-03-16T10:44:50.364 INFO:tasks.workunit.client.0.smithi002.stdout:4/81: mknod d2/dd/c12 0 2024-03-16T10:44:50.366 INFO:tasks.workunit.client.0.smithi002.stdout:9/33: write f1 [2406704,104484] 0 2024-03-16T10:44:50.367 INFO:tasks.workunit.client.0.smithi002.stdout:9/34: write d5/f7 [1486697,111180] 0 2024-03-16T10:44:50.369 INFO:tasks.workunit.client.0.smithi002.stdout:4/82: dread d2/d6/f10 [0,4194304] 0 2024-03-16T10:44:50.370 INFO:tasks.workunit.client.0.smithi002.stdout:4/83: write d2/dd/da/ff [794761,90132] 0 2024-03-16T10:44:50.372 INFO:tasks.workunit.client.0.smithi002.stdout:8/78: unlink d7/d13/c16 0 2024-03-16T10:44:50.374 INFO:tasks.workunit.client.0.smithi002.stdout:2/54: mknod d7/dc/cf 0 2024-03-16T10:44:50.375 INFO:tasks.workunit.client.0.smithi002.stdout:0/86: creat d3/de/f1d x:0 0 0 2024-03-16T10:44:50.376 INFO:tasks.workunit.client.0.smithi002.stdout:0/87: dread - d3/d5/f18 zero size 2024-03-16T10:44:50.376 INFO:tasks.workunit.client.0.smithi002.stdout:0/88: stat d3/fc 0 2024-03-16T10:44:50.377 INFO:tasks.workunit.client.0.smithi002.stdout:7/44: unlink d3/fa 0 2024-03-16T10:44:50.378 INFO:tasks.workunit.client.0.smithi002.stdout:6/44: symlink le 0 2024-03-16T10:44:50.379 INFO:tasks.workunit.client.0.smithi002.stdout:6/45: chown le 8765822 1 2024-03-16T10:44:50.382 INFO:tasks.workunit.client.0.smithi002.stdout:1/73: dread f1 [4194304,4194304] 0 2024-03-16T10:44:50.384 INFO:tasks.workunit.client.0.smithi002.stdout:1/74: write d2/d3/f17 [355598,101906] 0 2024-03-16T10:44:50.385 INFO:tasks.workunit.client.0.smithi002.stdout:9/35: symlink d5/l8 0 2024-03-16T10:44:50.389 INFO:tasks.workunit.client.0.smithi002.stdout:4/84: creat d2/d6/f13 x:0 0 0 2024-03-16T10:44:50.389 INFO:tasks.workunit.client.0.smithi002.stdout:4/85: chown d2/d6/f13 85518 1 2024-03-16T10:44:50.390 INFO:tasks.workunit.client.0.smithi002.stdout:4/86: read - d2/d6/f13 zero size 2024-03-16T10:44:50.391 INFO:tasks.workunit.client.0.smithi002.stdout:2/55: mknod d7/dc/c10 0 2024-03-16T10:44:50.392 INFO:tasks.workunit.client.0.smithi002.stdout:2/56: truncate d7/f8 5410546 0 2024-03-16T10:44:50.393 INFO:tasks.workunit.client.0.smithi002.stdout:2/57: write f3 [373965,1944] 0 2024-03-16T10:44:50.393 INFO:tasks.workunit.client.0.smithi002.stdout:0/89: creat d3/de/f1e x:0 0 0 2024-03-16T10:44:50.394 INFO:tasks.workunit.client.0.smithi002.stdout:2/58: readlink d7/l9 0 2024-03-16T10:44:50.394 INFO:tasks.workunit.client.0.smithi002.stdout:2/59: chown d7/dc/dd 1166043985 1 2024-03-16T10:44:50.396 INFO:tasks.workunit.client.0.smithi002.stdout:7/45: creat d3/d4/fb x:0 0 0 2024-03-16T10:44:50.397 INFO:tasks.workunit.client.0.smithi002.stdout:6/46: symlink lf 0 2024-03-16T10:44:50.397 INFO:tasks.workunit.client.0.smithi002.stdout:6/47: dread - fc zero size 2024-03-16T10:44:50.401 INFO:tasks.workunit.client.0.smithi002.stdout:9/36: mkdir d5/d9 0 2024-03-16T10:44:50.403 INFO:tasks.workunit.client.0.smithi002.stdout:9/37: write d5/f7 [2738317,63475] 0 2024-03-16T10:44:50.404 INFO:tasks.workunit.client.0.smithi002.stdout:6/48: dwrite fa [0,4194304] 0 2024-03-16T10:44:50.405 INFO:tasks.workunit.client.0.smithi002.stdout:6/49: chown l7 52727524 1 2024-03-16T10:44:50.405 INFO:tasks.workunit.client.0.smithi002.stdout:4/87: fsync d2/d6/f10 0 2024-03-16T10:44:50.406 INFO:tasks.workunit.client.0.smithi002.stdout:9/38: rename d5/d9 to d5/d9/da 22 2024-03-16T10:44:50.407 INFO:tasks.workunit.client.0.smithi002.stdout:6/50: write f8 [3072854,91887] 0 2024-03-16T10:44:50.408 INFO:tasks.workunit.client.0.smithi002.stdout:4/88: dread - d2/d6/f13 zero size 2024-03-16T10:44:50.408 INFO:tasks.workunit.client.0.smithi002.stdout:6/51: dread - fd zero size 2024-03-16T10:44:50.409 INFO:tasks.workunit.client.0.smithi002.stdout:2/60: write f4 [2108381,98550] 0 2024-03-16T10:44:50.415 INFO:tasks.workunit.client.0.smithi002.stdout:1/75: mknod d2/df/d12/c18 0 2024-03-16T10:44:50.415 INFO:tasks.workunit.client.0.smithi002.stdout:1/76: chown d2/d3/l6 1733876 1 2024-03-16T10:44:50.421 INFO:tasks.workunit.client.0.smithi002.stdout:0/90: symlink d3/l1f 0 2024-03-16T10:44:50.423 INFO:tasks.workunit.client.0.smithi002.stdout:6/52: creat f10 x:0 0 0 2024-03-16T10:44:50.423 INFO:tasks.workunit.client.0.smithi002.stdout:4/89: mknod d2/dd/c14 0 2024-03-16T10:44:50.424 INFO:tasks.workunit.client.0.smithi002.stdout:4/90: readlink d2/l4 0 2024-03-16T10:44:50.424 INFO:tasks.workunit.client.0.smithi002.stdout:6/53: chown fa 164 1 2024-03-16T10:44:50.426 INFO:tasks.workunit.client.0.smithi002.stdout:4/91: write d2/d6/f13 [59821,61416] 0 2024-03-16T10:44:50.427 INFO:tasks.workunit.client.0.smithi002.stdout:1/77: dwrite d2/f13 [0,4194304] 0 2024-03-16T10:44:50.427 INFO:tasks.workunit.client.0.smithi002.stdout:9/39: symlink d5/d9/lb 0 2024-03-16T10:44:50.429 INFO:tasks.workunit.client.0.smithi002.stdout:4/92: chown d2/dd/c12 1 1 2024-03-16T10:44:50.435 INFO:tasks.workunit.client.0.smithi002.stdout:6/54: rename f10 to f11 0 2024-03-16T10:44:50.437 INFO:tasks.workunit.client.0.smithi002.stdout:1/78: creat d2/d3/f19 x:0 0 0 2024-03-16T10:44:50.439 INFO:tasks.workunit.client.0.smithi002.stdout:0/91: link d3/d5/f12 d3/de/f20 0 2024-03-16T10:44:50.440 INFO:tasks.workunit.client.0.smithi002.stdout:1/79: dread d2/f5 [4194304,4194304] 0 2024-03-16T10:44:50.443 INFO:tasks.workunit.client.0.smithi002.stdout:4/93: rename l0 to d2/dd/da/l15 0 2024-03-16T10:44:50.445 INFO:tasks.workunit.client.0.smithi002.stdout:6/55: unlink f2 0 2024-03-16T10:44:50.456 INFO:tasks.workunit.client.0.smithi002.stdout:4/94: link d2/l4 d2/d6/l16 0 2024-03-16T10:44:50.457 INFO:tasks.workunit.client.0.smithi002.stdout:6/56: link l7 l12 0 2024-03-16T10:44:50.464 INFO:tasks.workunit.client.0.smithi002.stdout:6/57: dread f4 [0,4194304] 0 2024-03-16T10:44:50.465 INFO:tasks.workunit.client.0.smithi002.stdout:4/95: dwrite d2/d6/f13 [0,4194304] 0 2024-03-16T10:44:50.468 INFO:tasks.workunit.client.0.smithi002.stdout:6/58: rename f6 to f13 0 2024-03-16T10:44:50.475 INFO:tasks.workunit.client.0.smithi002.stdout:6/59: rename l12 to l14 0 2024-03-16T10:44:50.481 INFO:tasks.workunit.client.0.smithi002.stdout:6/60: truncate f4 3464613 0 2024-03-16T10:44:50.483 INFO:tasks.workunit.client.0.smithi002.stdout:6/61: mkdir d15 0 2024-03-16T10:44:50.571 INFO:tasks.workunit.client.0.smithi002.stdout:0/92: sync 2024-03-16T10:44:50.573 INFO:tasks.workunit.client.0.smithi002.stdout:0/93: symlink d3/d5/l21 0 2024-03-16T10:44:50.573 INFO:tasks.workunit.client.0.smithi002.stdout:0/94: fsync d3/f1a 0 2024-03-16T10:44:50.574 INFO:tasks.workunit.client.0.smithi002.stdout:0/95: dread - d3/de/f15 zero size 2024-03-16T10:44:50.581 INFO:tasks.workunit.client.0.smithi002.stdout:0/96: dwrite d3/de/f13 [0,4194304] 0 2024-03-16T10:44:50.584 INFO:tasks.workunit.client.0.smithi002.stdout:0/97: write d3/d5/fb [1238982,118526] 0 2024-03-16T10:44:50.586 INFO:tasks.workunit.client.0.smithi002.stdout:0/98: write d3/de/f1e [754407,101780] 0 2024-03-16T10:44:50.588 INFO:tasks.workunit.client.0.smithi002.stdout:5/35: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:50.592 INFO:tasks.workunit.client.0.smithi002.stdout:7/46: write f1 [9132572,88000] 0 2024-03-16T10:44:50.601 INFO:tasks.workunit.client.0.smithi002.stdout:0/99: dwrite d3/d5/f19 [4194304,4194304] 0 2024-03-16T10:44:50.602 INFO:tasks.workunit.client.0.smithi002.stdout:3/67: write d1/dd/f10 [997837,74322] 0 2024-03-16T10:44:50.604 INFO:tasks.workunit.client.0.smithi002.stdout:3/68: stat d1/dd/f13 0 2024-03-16T10:44:50.604 INFO:tasks.workunit.client.0.smithi002.stdout:7/47: unlink d3/c9 0 2024-03-16T10:44:50.608 INFO:tasks.workunit.client.0.smithi002.stdout:0/100: unlink d3/d5/f12 0 2024-03-16T10:44:50.610 INFO:tasks.workunit.client.0.smithi002.stdout:7/48: read d3/f6 [3088747,52053] 0 2024-03-16T10:44:50.610 INFO:tasks.workunit.client.0.smithi002.stdout:5/36: dwrite d0/f7 [0,4194304] 0 2024-03-16T10:44:50.619 INFO:tasks.workunit.client.0.smithi002.stdout:7/49: symlink d3/d4/lc 0 2024-03-16T10:44:50.621 INFO:tasks.workunit.client.0.smithi002.stdout:5/37: mknod d0/cb 0 2024-03-16T10:44:50.622 INFO:tasks.workunit.client.0.smithi002.stdout:0/101: creat d3/f22 x:0 0 0 2024-03-16T10:44:50.623 INFO:tasks.workunit.client.0.smithi002.stdout:0/102: truncate d3/fc 4867619 0 2024-03-16T10:44:50.627 INFO:tasks.workunit.client.0.smithi002.stdout:0/103: creat d3/d5/f23 x:0 0 0 2024-03-16T10:44:50.627 INFO:tasks.workunit.client.0.smithi002.stdout:8/79: write d7/f14 [2764036,79364] 0 2024-03-16T10:44:50.629 INFO:tasks.workunit.client.0.smithi002.stdout:0/104: dread - d3/f1a zero size 2024-03-16T10:44:50.629 INFO:tasks.workunit.client.0.smithi002.stdout:8/80: chown d7/d13 828177 1 2024-03-16T10:44:50.629 INFO:tasks.workunit.client.0.smithi002.stdout:8/81: chown f4 1243 1 2024-03-16T10:44:50.629 INFO:tasks.workunit.client.0.smithi002.stdout:0/105: chown d3/l8 7037366 1 2024-03-16T10:44:50.630 INFO:tasks.workunit.client.0.smithi002.stdout:8/82: write d7/f19 [242223,53364] 0 2024-03-16T10:44:50.633 INFO:tasks.workunit.client.0.smithi002.stdout:5/38: link d0/l5 d0/lc 0 2024-03-16T10:44:50.634 INFO:tasks.workunit.client.0.smithi002.stdout:8/83: unlink c5 0 2024-03-16T10:44:50.635 INFO:tasks.workunit.client.0.smithi002.stdout:8/84: rename d7/d9 to d7/d9/d1c 22 2024-03-16T10:44:50.635 INFO:tasks.workunit.client.0.smithi002.stdout:8/85: fdatasync f2 0 2024-03-16T10:44:50.641 INFO:tasks.workunit.client.0.smithi002.stdout:8/86: dread f4 [0,4194304] 0 2024-03-16T10:44:50.645 INFO:tasks.workunit.client.0.smithi002.stdout:6/62: dwrite f8 [4194304,4194304] 0 2024-03-16T10:44:50.645 INFO:tasks.workunit.client.0.smithi002.stdout:2/61: truncate f4 351072 0 2024-03-16T10:44:50.646 INFO:tasks.workunit.client.0.smithi002.stdout:6/63: truncate f11 417267 0 2024-03-16T10:44:50.646 INFO:tasks.workunit.client.0.smithi002.stdout:8/87: rename f2 to d7/d9/d17/f1d 0 2024-03-16T10:44:50.647 INFO:tasks.workunit.client.0.smithi002.stdout:8/88: chown f4 75512550 1 2024-03-16T10:44:50.648 INFO:tasks.workunit.client.0.smithi002.stdout:8/89: stat d7/d9/d17/f1d 0 2024-03-16T10:44:50.655 INFO:tasks.workunit.client.0.smithi002.stdout:8/90: creat d7/d9/d17/f1e x:0 0 0 2024-03-16T10:44:50.655 INFO:tasks.workunit.client.0.smithi002.stdout:9/40: truncate d5/f7 2444175 0 2024-03-16T10:44:50.658 INFO:tasks.workunit.client.0.smithi002.stdout:8/91: creat d7/d9/d17/f1f x:0 0 0 2024-03-16T10:44:50.659 INFO:tasks.workunit.client.0.smithi002.stdout:9/41: mkdir d5/d9/dc 0 2024-03-16T10:44:50.660 INFO:tasks.workunit.client.0.smithi002.stdout:4/96: getdents d2/d6 0 2024-03-16T10:44:50.662 INFO:tasks.workunit.client.0.smithi002.stdout:8/92: mkdir d7/d9/d17/d20 0 2024-03-16T10:44:50.662 INFO:tasks.workunit.client.0.smithi002.stdout:4/97: fsync d2/dd/da/ff 0 2024-03-16T10:44:50.662 INFO:tasks.workunit.client.0.smithi002.stdout:8/93: stat d7 0 2024-03-16T10:44:50.663 INFO:tasks.workunit.client.0.smithi002.stdout:1/80: dwrite f1 [4194304,4194304] 0 2024-03-16T10:44:50.663 INFO:tasks.workunit.client.0.smithi002.stdout:4/98: fsync d2/dd/da/ff 0 2024-03-16T10:44:50.667 INFO:tasks.workunit.client.0.smithi002.stdout:1/81: unlink c0 0 2024-03-16T10:44:50.671 INFO:tasks.workunit.client.0.smithi002.stdout:4/99: dread d2/dd/da/ff [0,4194304] 0 2024-03-16T10:44:50.673 INFO:tasks.workunit.client.0.smithi002.stdout:1/82: dread d2/f5 [0,4194304] 0 2024-03-16T10:44:50.680 INFO:tasks.workunit.client.0.smithi002.stdout:8/94: creat d7/f21 x:0 0 0 2024-03-16T10:44:50.684 INFO:tasks.workunit.client.0.smithi002.stdout:1/83: mknod d2/df/d12/c1a 0 2024-03-16T10:44:50.685 INFO:tasks.workunit.client.0.smithi002.stdout:1/84: chown d2/d3/f17 5 1 2024-03-16T10:44:50.686 INFO:tasks.workunit.client.0.smithi002.stdout:1/85: truncate d2/d3/fe 4812253 0 2024-03-16T10:44:50.686 INFO:tasks.workunit.client.0.smithi002.stdout:1/86: dread - d2/d3/fc zero size 2024-03-16T10:44:50.691 INFO:tasks.workunit.client.0.smithi002.stdout:4/100: getdents d2/d6 0 2024-03-16T10:44:50.692 INFO:tasks.workunit.client.0.smithi002.stdout:1/87: symlink d2/df/d12/l1b 0 2024-03-16T10:44:50.694 INFO:tasks.workunit.client.0.smithi002.stdout:1/88: write d2/d3/fe [2056702,73377] 0 2024-03-16T10:44:50.695 INFO:tasks.workunit.client.0.smithi002.stdout:4/101: readlink d2/dd/da/l15 0 2024-03-16T10:44:50.696 INFO:tasks.workunit.client.0.smithi002.stdout:8/95: dwrite d7/d9/d17/f1d [4194304,4194304] 0 2024-03-16T10:44:50.701 INFO:tasks.workunit.client.0.smithi002.stdout:1/89: dread d2/d3/fe [0,4194304] 0 2024-03-16T10:44:50.703 INFO:tasks.workunit.client.0.smithi002.stdout:8/96: dread f4 [0,4194304] 0 2024-03-16T10:44:50.704 INFO:tasks.workunit.client.0.smithi002.stdout:4/102: dwrite d2/fe [0,4194304] 0 2024-03-16T10:44:50.707 INFO:tasks.workunit.client.0.smithi002.stdout:1/90: dread f1 [0,4194304] 0 2024-03-16T10:44:50.709 INFO:tasks.workunit.client.0.smithi002.stdout:8/97: rename d7/c11 to d7/d9/d17/d20/c22 0 2024-03-16T10:44:50.709 INFO:tasks.workunit.client.0.smithi002.stdout:8/98: read d7/fd [3092416,82973] 0 2024-03-16T10:44:50.714 INFO:tasks.workunit.client.0.smithi002.stdout:1/91: mkdir d2/d3/d1c 0 2024-03-16T10:44:50.723 INFO:tasks.workunit.client.0.smithi002.stdout:4/103: link d2/d6/f13 d2/d6/f17 0 2024-03-16T10:44:50.724 INFO:tasks.workunit.client.0.smithi002.stdout:4/104: write d2/fe [4167124,88787] 0 2024-03-16T10:44:50.727 INFO:tasks.workunit.client.0.smithi002.stdout:3/69: truncate d1/dd/f10 882846 0 2024-03-16T10:44:50.729 INFO:tasks.workunit.client.0.smithi002.stdout:7/50: rmdir d3 39 2024-03-16T10:44:50.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:50 smithi026 ceph-mon[37041]: pgmap v28: 65 pgs: 65 active+clean; 186 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 291 op/s 2024-03-16T10:44:50.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:50 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:50.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:50 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:50.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:50 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:50.734 INFO:tasks.workunit.client.0.smithi002.stdout:5/39: getdents d0 0 2024-03-16T10:44:50.738 INFO:tasks.workunit.client.0.smithi002.stdout:0/106: truncate d3/d5/f19 3181025 0 2024-03-16T10:44:50.739 INFO:tasks.workunit.client.0.smithi002.stdout:4/105: mknod d2/c18 0 2024-03-16T10:44:50.740 INFO:tasks.workunit.client.0.smithi002.stdout:4/106: chown d2/fe 49568310 1 2024-03-16T10:44:50.741 INFO:tasks.workunit.client.0.smithi002.stdout:1/92: mknod d2/c1d 0 2024-03-16T10:44:50.743 INFO:tasks.workunit.client.0.smithi002.stdout:3/70: creat d1/f16 x:0 0 0 2024-03-16T10:44:50.744 INFO:tasks.workunit.client.0.smithi002.stdout:2/62: fsync f4 0 2024-03-16T10:44:50.746 INFO:tasks.workunit.client.0.smithi002.stdout:0/107: dwrite d3/d5/f23 [0,4194304] 0 2024-03-16T10:44:50.747 INFO:tasks.workunit.client.0.smithi002.stdout:7/51: write d3/d4/fb [1046572,31375] 0 2024-03-16T10:44:50.752 INFO:tasks.workunit.client.0.smithi002.stdout:4/107: mknod d2/dd/da/c19 0 2024-03-16T10:44:50.754 INFO:tasks.workunit.client.0.smithi002.stdout:9/42: dread d5/f7 [0,4194304] 0 2024-03-16T10:44:50.754 INFO:tasks.workunit.client.0.smithi002.stdout:1/93: mknod d2/df/d12/c1e 0 2024-03-16T10:44:50.755 INFO:tasks.workunit.client.0.smithi002.stdout:1/94: chown d2/d3 480 1 2024-03-16T10:44:50.757 INFO:tasks.workunit.client.0.smithi002.stdout:9/43: dread d5/f7 [0,4194304] 0 2024-03-16T10:44:50.763 INFO:tasks.workunit.client.0.smithi002.stdout:2/63: creat d7/dc/dd/f11 x:0 0 0 2024-03-16T10:44:50.764 INFO:tasks.workunit.client.0.smithi002.stdout:2/64: chown f4 39209 1 2024-03-16T10:44:50.764 INFO:tasks.workunit.client.0.smithi002.stdout:2/65: chown d7/f8 19 1 2024-03-16T10:44:50.767 INFO:tasks.workunit.client.0.smithi002.stdout:7/52: rename d3/f6 to d3/fd 0 2024-03-16T10:44:50.768 INFO:tasks.workunit.client.0.smithi002.stdout:7/53: chown l2 118 1 2024-03-16T10:44:50.771 INFO:tasks.workunit.client.0.smithi002.stdout:5/40: rename d0/lc to d0/ld 0 2024-03-16T10:44:50.771 INFO:tasks.workunit.client.0.smithi002.stdout:6/64: dread f13 [0,4194304] 0 2024-03-16T10:44:50.771 INFO:tasks.workunit.client.0.smithi002.stdout:7/54: read d3/fd [790594,27747] 0 2024-03-16T10:44:50.772 INFO:tasks.workunit.client.0.smithi002.stdout:2/66: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:50.772 INFO:tasks.workunit.client.0.smithi002.stdout:6/65: chown lb 621 1 2024-03-16T10:44:50.773 INFO:tasks.workunit.client.0.smithi002.stdout:7/55: write f1 [8712268,43940] 0 2024-03-16T10:44:50.774 INFO:tasks.workunit.client.0.smithi002.stdout:9/44: creat d5/fd x:0 0 0 2024-03-16T10:44:50.776 INFO:tasks.workunit.client.0.smithi002.stdout:3/71: creat d1/dd/d11/d15/f17 x:0 0 0 2024-03-16T10:44:50.791 INFO:tasks.workunit.client.0.smithi002.stdout:7/56: unlink c0 0 2024-03-16T10:44:50.793 INFO:tasks.workunit.client.0.smithi002.stdout:9/45: creat d5/d9/fe x:0 0 0 2024-03-16T10:44:50.794 INFO:tasks.workunit.client.0.smithi002.stdout:9/46: stat d5/fd 0 2024-03-16T10:44:50.795 INFO:tasks.workunit.client.0.smithi002.stdout:3/72: symlink d1/dd/d11/l18 0 2024-03-16T10:44:50.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:50 smithi002 ceph-mon[29248]: pgmap v28: 65 pgs: 65 active+clean; 186 MiB data, 785 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.9 MiB/s wr, 291 op/s 2024-03-16T10:44:50.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:50 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:50.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:50 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:44:50.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:50 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:44:50.799 INFO:tasks.workunit.client.0.smithi002.stdout:4/108: truncate d2/d6/f13 213060 0 2024-03-16T10:44:50.802 INFO:tasks.workunit.client.0.smithi002.stdout:7/57: rename f1 to d3/d4/fe 0 2024-03-16T10:44:50.804 INFO:tasks.workunit.client.0.smithi002.stdout:7/58: write d3/d4/fb [1394662,88753] 0 2024-03-16T10:44:50.806 INFO:tasks.workunit.client.0.smithi002.stdout:4/109: dwrite d2/fe [0,4194304] 0 2024-03-16T10:44:50.806 INFO:tasks.workunit.client.0.smithi002.stdout:8/99: truncate d7/d9/d17/f1d 7466297 0 2024-03-16T10:44:50.814 INFO:tasks.workunit.client.0.smithi002.stdout:0/108: read d3/d5/f19 [1101866,52453] 0 2024-03-16T10:44:50.819 INFO:tasks.workunit.client.0.smithi002.stdout:7/59: mknod d3/cf 0 2024-03-16T10:44:50.822 INFO:tasks.workunit.client.0.smithi002.stdout:4/110: creat d2/d6/f1a x:0 0 0 2024-03-16T10:44:50.824 INFO:tasks.workunit.client.0.smithi002.stdout:8/100: unlink d7/d9/d17/f1e 0 2024-03-16T10:44:50.826 INFO:tasks.workunit.client.0.smithi002.stdout:6/66: link l14 d15/l16 0 2024-03-16T10:44:50.829 INFO:tasks.workunit.client.0.smithi002.stdout:6/67: write f9 [3074792,115439] 0 2024-03-16T10:44:50.830 INFO:tasks.workunit.client.0.smithi002.stdout:2/67: getdents d7/dc 0 2024-03-16T10:44:50.831 INFO:tasks.workunit.client.0.smithi002.stdout:7/60: fdatasync d3/fd 0 2024-03-16T10:44:50.832 INFO:tasks.workunit.client.0.smithi002.stdout:7/61: chown d3/d4 2295403 1 2024-03-16T10:44:50.835 INFO:tasks.workunit.client.0.smithi002.stdout:4/111: creat d2/dd/f1b x:0 0 0 2024-03-16T10:44:50.843 INFO:tasks.workunit.client.0.smithi002.stdout:9/47: truncate d5/f7 2925358 0 2024-03-16T10:44:50.846 INFO:tasks.workunit.client.0.smithi002.stdout:2/68: rename d7/dc/dd to d7/db/d12 0 2024-03-16T10:44:50.847 INFO:tasks.workunit.client.0.smithi002.stdout:2/69: read - d7/db/d12/f11 zero size 2024-03-16T10:44:50.848 INFO:tasks.workunit.client.0.smithi002.stdout:0/109: symlink d3/l24 0 2024-03-16T10:44:50.849 INFO:tasks.workunit.client.0.smithi002.stdout:0/110: write d3/f22 [864901,74084] 0 2024-03-16T10:44:50.850 INFO:tasks.workunit.client.0.smithi002.stdout:7/62: mkdir d3/d4/d10 0 2024-03-16T10:44:50.850 INFO:tasks.workunit.client.0.smithi002.stdout:0/111: write d3/d5/f23 [4548243,52586] 0 2024-03-16T10:44:50.851 INFO:tasks.workunit.client.0.smithi002.stdout:7/63: write d3/fd [1934461,101391] 0 2024-03-16T10:44:50.852 INFO:tasks.workunit.client.0.smithi002.stdout:3/73: getdents d1/dd 0 2024-03-16T10:44:50.854 INFO:tasks.workunit.client.0.smithi002.stdout:4/112: mkdir d2/dd/d1c 0 2024-03-16T10:44:50.859 INFO:tasks.workunit.client.0.smithi002.stdout:6/68: symlink d15/l17 0 2024-03-16T10:44:50.861 INFO:tasks.workunit.client.0.smithi002.stdout:9/48: mkdir d5/df 0 2024-03-16T10:44:50.861 INFO:tasks.workunit.client.0.smithi002.stdout:9/49: write d5/fd [824003,37422] 0 2024-03-16T10:44:50.869 INFO:tasks.workunit.client.0.smithi002.stdout:0/112: mknod d3/d5/c25 0 2024-03-16T10:44:50.870 INFO:tasks.workunit.client.0.smithi002.stdout:7/64: fsync d3/d4/fe 0 2024-03-16T10:44:50.876 INFO:tasks.workunit.client.0.smithi002.stdout:7/65: dwrite d3/d4/fe [4194304,4194304] 0 2024-03-16T10:44:50.877 INFO:tasks.workunit.client.0.smithi002.stdout:5/41: unlink d0/ld 0 2024-03-16T10:44:50.889 INFO:tasks.workunit.client.0.smithi002.stdout:8/101: getdents d7/d9/d17/d20 0 2024-03-16T10:44:50.889 INFO:tasks.workunit.client.0.smithi002.stdout:8/102: stat l6 0 2024-03-16T10:44:50.890 INFO:tasks.workunit.client.0.smithi002.stdout:7/66: creat d3/f11 x:0 0 0 2024-03-16T10:44:50.891 INFO:tasks.workunit.client.0.smithi002.stdout:7/67: write d3/d4/fb [625957,9693] 0 2024-03-16T10:44:50.897 INFO:tasks.workunit.client.0.smithi002.stdout:7/68: dwrite d3/d4/fe [8388608,4194304] 0 2024-03-16T10:44:50.899 INFO:tasks.workunit.client.0.smithi002.stdout:7/69: write d3/d4/fe [7028835,84074] 0 2024-03-16T10:44:50.900 INFO:tasks.workunit.client.0.smithi002.stdout:7/70: write d3/fd [2036343,61965] 0 2024-03-16T10:44:50.901 INFO:tasks.workunit.client.0.smithi002.stdout:7/71: fdatasync d3/d4/fb 0 2024-03-16T10:44:50.902 INFO:tasks.workunit.client.0.smithi002.stdout:9/50: dread d5/f7 [0,4194304] 0 2024-03-16T10:44:50.903 INFO:tasks.workunit.client.0.smithi002.stdout:7/72: write d3/d4/fb [1751538,114938] 0 2024-03-16T10:44:50.903 INFO:tasks.workunit.client.0.smithi002.stdout:9/51: chown d5/l8 4 1 2024-03-16T10:44:50.905 INFO:tasks.workunit.client.0.smithi002.stdout:5/42: stat d0/l5 0 2024-03-16T10:44:50.909 INFO:tasks.workunit.client.0.smithi002.stdout:0/113: creat d3/f26 x:0 0 0 2024-03-16T10:44:50.910 INFO:tasks.workunit.client.0.smithi002.stdout:4/113: getdents d2 0 2024-03-16T10:44:50.913 INFO:tasks.workunit.client.0.smithi002.stdout:4/114: dread d2/dd/da/ff [0,4194304] 0 2024-03-16T10:44:50.915 INFO:tasks.workunit.client.0.smithi002.stdout:4/115: write d2/dd/da/ff [538859,46749] 0 2024-03-16T10:44:50.915 INFO:tasks.workunit.client.0.smithi002.stdout:0/114: dwrite d3/fc [0,4194304] 0 2024-03-16T10:44:50.915 INFO:tasks.workunit.client.0.smithi002.stdout:2/70: write f4 [537745,70846] 0 2024-03-16T10:44:50.916 INFO:tasks.workunit.client.0.smithi002.stdout:2/71: readlink d7/db/d12/le 0 2024-03-16T10:44:50.920 INFO:tasks.workunit.client.0.smithi002.stdout:7/73: unlink d3/d4/fb 0 2024-03-16T10:44:50.922 INFO:tasks.workunit.client.0.smithi002.stdout:7/74: chown d3/d4/fe 1513856440 1 2024-03-16T10:44:50.922 INFO:tasks.workunit.client.0.smithi002.stdout:9/52: rename f1 to d5/f10 0 2024-03-16T10:44:50.924 INFO:tasks.workunit.client.0.smithi002.stdout:5/43: stat d0/f9 0 2024-03-16T10:44:50.924 INFO:tasks.workunit.client.0.smithi002.stdout:7/75: read d3/d4/fe [12410664,35365] 0 2024-03-16T10:44:50.927 INFO:tasks.workunit.client.0.smithi002.stdout:6/69: write f13 [3550236,18852] 0 2024-03-16T10:44:50.928 INFO:tasks.workunit.client.0.smithi002.stdout:2/72: dwrite d7/db/d12/f11 [0,4194304] 0 2024-03-16T10:44:50.932 INFO:tasks.workunit.client.0.smithi002.stdout:6/70: write f8 [9001950,31612] 0 2024-03-16T10:44:50.932 INFO:tasks.workunit.client.0.smithi002.stdout:8/103: unlink d7/d9/d17/f1d 0 2024-03-16T10:44:50.935 INFO:tasks.workunit.client.0.smithi002.stdout:3/74: truncate d1/dd/f10 207766 0 2024-03-16T10:44:50.936 INFO:tasks.workunit.client.0.smithi002.stdout:8/104: chown d7/d13/l1b 32 1 2024-03-16T10:44:50.938 INFO:tasks.workunit.client.0.smithi002.stdout:9/53: dwrite d5/d9/fe [0,4194304] 0 2024-03-16T10:44:50.940 INFO:tasks.workunit.client.0.smithi002.stdout:9/54: chown d5/fd 1 1 2024-03-16T10:44:50.940 INFO:tasks.workunit.client.0.smithi002.stdout:7/76: dwrite d3/fd [4194304,4194304] 0 2024-03-16T10:44:50.946 INFO:tasks.workunit.client.0.smithi002.stdout:0/115: read d3/de/f13 [1618160,126964] 0 2024-03-16T10:44:50.948 INFO:tasks.workunit.client.0.smithi002.stdout:0/116: read - d3/d5/f14 zero size 2024-03-16T10:44:50.949 INFO:tasks.workunit.client.0.smithi002.stdout:5/44: dread d0/f7 [0,4194304] 0 2024-03-16T10:44:50.951 INFO:tasks.workunit.client.0.smithi002.stdout:6/71: creat d15/f18 x:0 0 0 2024-03-16T10:44:50.956 INFO:tasks.workunit.client.0.smithi002.stdout:8/105: mkdir d7/d9/d17/d23 0 2024-03-16T10:44:50.956 INFO:tasks.workunit.client.0.smithi002.stdout:8/106: chown d7 5457 1 2024-03-16T10:44:50.960 INFO:tasks.workunit.client.0.smithi002.stdout:7/77: symlink d3/l12 0 2024-03-16T10:44:50.960 INFO:tasks.workunit.client.0.smithi002.stdout:7/78: fdatasync d3/f11 0 2024-03-16T10:44:50.966 INFO:tasks.workunit.client.0.smithi002.stdout:0/117: write d3/de/f1e [1732102,126243] 0 2024-03-16T10:44:50.968 INFO:tasks.workunit.client.0.smithi002.stdout:6/72: creat d15/f19 x:0 0 0 2024-03-16T10:44:50.974 INFO:tasks.workunit.client.0.smithi002.stdout:6/73: dwrite fc [0,4194304] 0 2024-03-16T10:44:50.977 INFO:tasks.workunit.client.0.smithi002.stdout:8/107: dread d7/fd [0,4194304] 0 2024-03-16T10:44:50.981 INFO:tasks.workunit.client.0.smithi002.stdout:9/55: creat d5/d9/dc/f11 x:0 0 0 2024-03-16T10:44:50.981 INFO:tasks.workunit.client.0.smithi002.stdout:9/56: chown d5 21 1 2024-03-16T10:44:50.990 INFO:tasks.workunit.client.0.smithi002.stdout:0/118: write d3/de/f10 [465624,50794] 0 2024-03-16T10:44:50.991 INFO:tasks.workunit.client.0.smithi002.stdout:2/73: link d7/db/d12/le d7/dc/l13 0 2024-03-16T10:44:50.993 INFO:tasks.workunit.client.0.smithi002.stdout:2/74: dread f4 [0,4194304] 0 2024-03-16T10:44:50.999 INFO:tasks.workunit.client.0.smithi002.stdout:9/57: symlink d5/l12 0 2024-03-16T10:44:51.000 INFO:tasks.workunit.client.0.smithi002.stdout:5/45: link d0/f9 d0/fe 0 2024-03-16T10:44:51.005 INFO:tasks.workunit.client.0.smithi002.stdout:2/75: symlink d7/db/d12/l14 0 2024-03-16T10:44:51.005 INFO:tasks.workunit.client.0.smithi002.stdout:2/76: stat f3 0 2024-03-16T10:44:51.006 INFO:tasks.workunit.client.0.smithi002.stdout:2/77: chown f4 1408 1 2024-03-16T10:44:51.006 INFO:tasks.workunit.client.0.smithi002.stdout:2/78: fsync f6 0 2024-03-16T10:44:51.006 INFO:tasks.workunit.client.0.smithi002.stdout:2/79: write f3 [2048234,14944] 0 2024-03-16T10:44:51.007 INFO:tasks.workunit.client.0.smithi002.stdout:3/75: truncate d1/f9 2786818 0 2024-03-16T10:44:51.008 INFO:tasks.workunit.client.0.smithi002.stdout:2/80: write f3 [3918840,37858] 0 2024-03-16T10:44:51.008 INFO:tasks.workunit.client.0.smithi002.stdout:2/81: chown f3 182352 1 2024-03-16T10:44:51.012 INFO:tasks.workunit.client.0.smithi002.stdout:8/108: symlink d7/l24 0 2024-03-16T10:44:51.012 INFO:tasks.workunit.client.0.smithi002.stdout:2/82: dread d7/f8 [0,4194304] 0 2024-03-16T10:44:51.014 INFO:tasks.workunit.client.0.smithi002.stdout:7/79: link l2 d3/l13 0 2024-03-16T10:44:51.016 INFO:tasks.workunit.client.0.smithi002.stdout:9/58: creat d5/d9/f13 x:0 0 0 2024-03-16T10:44:51.018 INFO:tasks.workunit.client.0.smithi002.stdout:2/83: dwrite f6 [0,4194304] 0 2024-03-16T10:44:51.019 INFO:tasks.workunit.client.0.smithi002.stdout:0/119: mkdir d3/d27 0 2024-03-16T10:44:51.019 INFO:tasks.workunit.client.0.smithi002.stdout:0/120: chown d3/d5/f18 369 1 2024-03-16T10:44:51.030 INFO:tasks.workunit.client.0.smithi002.stdout:8/109: mkdir d7/d9/d25 0 2024-03-16T10:44:51.031 INFO:tasks.workunit.client.0.smithi002.stdout:8/110: chown d7/d13/l1b 48884155 1 2024-03-16T10:44:51.032 INFO:tasks.workunit.client.0.smithi002.stdout:8/111: truncate d7/f19 1312853 0 2024-03-16T10:44:51.032 INFO:tasks.workunit.client.0.smithi002.stdout:7/80: unlink d3/d4/fe 0 2024-03-16T10:44:51.033 INFO:tasks.workunit.client.0.smithi002.stdout:7/81: write d3/fd [1115459,14253] 0 2024-03-16T10:44:51.038 INFO:tasks.workunit.client.0.smithi002.stdout:9/59: dread d5/f10 [0,4194304] 0 2024-03-16T10:44:51.039 INFO:tasks.workunit.client.0.smithi002.stdout:7/82: dread d3/fd [0,4194304] 0 2024-03-16T10:44:51.040 INFO:tasks.workunit.client.0.smithi002.stdout:2/84: creat d7/dc/f15 x:0 0 0 2024-03-16T10:44:51.040 INFO:tasks.workunit.client.0.smithi002.stdout:8/112: dread d7/f14 [0,4194304] 0 2024-03-16T10:44:51.040 INFO:tasks.workunit.client.0.smithi002.stdout:7/83: fsync d3/f11 0 2024-03-16T10:44:51.041 INFO:tasks.workunit.client.0.smithi002.stdout:6/74: truncate f8 8813039 0 2024-03-16T10:44:51.041 INFO:tasks.workunit.client.0.smithi002.stdout:8/113: chown d7/d9/d17/d20 3169452 1 2024-03-16T10:44:51.042 INFO:tasks.workunit.client.0.smithi002.stdout:0/121: dread - d3/de/f1d zero size 2024-03-16T10:44:51.042 INFO:tasks.workunit.client.0.smithi002.stdout:8/114: truncate d7/f21 700140 0 2024-03-16T10:44:51.044 INFO:tasks.workunit.client.0.smithi002.stdout:8/115: write d7/fc [3231815,36199] 0 2024-03-16T10:44:51.050 INFO:tasks.workunit.client.0.smithi002.stdout:7/84: dwrite d3/f11 [0,4194304] 0 2024-03-16T10:44:51.051 INFO:tasks.workunit.client.0.smithi002.stdout:0/122: dwrite d3/f26 [0,4194304] 0 2024-03-16T10:44:51.052 INFO:tasks.workunit.client.0.smithi002.stdout:0/123: stat d3/d5/l21 0 2024-03-16T10:44:51.056 INFO:tasks.workunit.client.0.smithi002.stdout:2/85: creat d7/dc/f16 x:0 0 0 2024-03-16T10:44:51.059 INFO:tasks.workunit.client.0.smithi002.stdout:1/95: sync 2024-03-16T10:44:51.060 INFO:tasks.workunit.client.0.smithi002.stdout:6/75: chown d15/l16 173718 1 2024-03-16T10:44:51.061 INFO:tasks.workunit.client.0.smithi002.stdout:2/86: dread f6 [0,4194304] 0 2024-03-16T10:44:51.066 INFO:tasks.workunit.client.0.smithi002.stdout:1/96: dwrite d2/d3/fc [0,4194304] 0 2024-03-16T10:44:51.077 INFO:tasks.workunit.client.0.smithi002.stdout:7/85: creat d3/f14 x:0 0 0 2024-03-16T10:44:51.079 INFO:tasks.workunit.client.0.smithi002.stdout:5/46: rename d0/c2 to d0/cf 0 2024-03-16T10:44:51.087 INFO:tasks.workunit.client.0.smithi002.stdout:6/76: write f11 [549106,6357] 0 2024-03-16T10:44:51.087 INFO:tasks.workunit.client.0.smithi002.stdout:3/76: write d1/dd/f10 [867861,52386] 0 2024-03-16T10:44:51.089 INFO:tasks.workunit.client.0.smithi002.stdout:2/87: dread d7/f8 [4194304,4194304] 0 2024-03-16T10:44:51.089 INFO:tasks.workunit.client.0.smithi002.stdout:3/77: write d1/dd/f10 [486790,21663] 0 2024-03-16T10:44:51.090 INFO:tasks.workunit.client.0.smithi002.stdout:2/88: rename d7 to d7/db/d12/d17 22 2024-03-16T10:44:51.091 INFO:tasks.workunit.client.0.smithi002.stdout:2/89: rename d7/db to d7/db/d12/d18 22 2024-03-16T10:44:51.091 INFO:tasks.workunit.client.0.smithi002.stdout:1/97: creat d2/d3/f1f x:0 0 0 2024-03-16T10:44:51.092 INFO:tasks.workunit.client.0.smithi002.stdout:2/90: dread - d7/dc/f16 zero size 2024-03-16T10:44:51.092 INFO:tasks.workunit.client.0.smithi002.stdout:1/98: readlink d2/df/d12/l1b 0 2024-03-16T10:44:51.095 INFO:tasks.workunit.client.0.smithi002.stdout:7/86: creat d3/f15 x:0 0 0 2024-03-16T10:44:51.098 INFO:tasks.workunit.client.0.smithi002.stdout:0/124: mkdir d3/d28 0 2024-03-16T10:44:51.098 INFO:tasks.workunit.client.0.smithi002.stdout:0/125: stat d3/l1f 0 2024-03-16T10:44:51.099 INFO:tasks.workunit.client.0.smithi002.stdout:0/126: fdatasync d3/de/f1d 0 2024-03-16T10:44:51.103 INFO:tasks.workunit.client.0.smithi002.stdout:6/77: creat d15/f1a x:0 0 0 2024-03-16T10:44:51.106 INFO:tasks.workunit.client.0.smithi002.stdout:3/78: symlink d1/dd/d11/d15/l19 0 2024-03-16T10:44:51.108 INFO:tasks.workunit.client.0.smithi002.stdout:2/91: symlink d7/dc/l19 0 2024-03-16T10:44:51.110 INFO:tasks.workunit.client.0.smithi002.stdout:8/116: creat d7/f26 x:0 0 0 2024-03-16T10:44:51.112 INFO:tasks.workunit.client.0.smithi002.stdout:1/99: dread d2/d3/fe [0,4194304] 0 2024-03-16T10:44:51.113 INFO:tasks.workunit.client.0.smithi002.stdout:7/87: creat d3/f16 x:0 0 0 2024-03-16T10:44:51.113 INFO:tasks.workunit.client.0.smithi002.stdout:7/88: readlink d3/l12 0 2024-03-16T10:44:51.115 INFO:tasks.workunit.client.0.smithi002.stdout:0/127: dread - d3/de/f15 zero size 2024-03-16T10:44:51.115 INFO:tasks.workunit.client.0.smithi002.stdout:0/128: stat d3/d5/f1c 0 2024-03-16T10:44:51.115 INFO:tasks.workunit.client.0.smithi002.stdout:0/129: read d3/fc [753866,101589] 0 2024-03-16T10:44:51.116 INFO:tasks.workunit.client.0.smithi002.stdout:9/60: getdents d5 0 2024-03-16T10:44:51.118 INFO:tasks.workunit.client.0.smithi002.stdout:6/78: mkdir d15/d1b 0 2024-03-16T10:44:51.121 INFO:tasks.workunit.client.0.smithi002.stdout:1/100: mknod d2/d3/c20 0 2024-03-16T10:44:51.122 INFO:tasks.workunit.client.0.smithi002.stdout:2/92: dread f6 [0,4194304] 0 2024-03-16T10:44:51.122 INFO:tasks.workunit.client.0.smithi002.stdout:7/89: symlink d3/d4/l17 0 2024-03-16T10:44:51.125 INFO:tasks.workunit.client.0.smithi002.stdout:9/61: mknod d5/d9/dc/c14 0 2024-03-16T10:44:51.127 INFO:tasks.workunit.client.0.smithi002.stdout:6/79: creat d15/f1c x:0 0 0 2024-03-16T10:44:51.127 INFO:tasks.workunit.client.0.smithi002.stdout:6/80: read fc [2274531,9095] 0 2024-03-16T10:44:51.127 INFO:tasks.workunit.client.0.smithi002.stdout:6/81: chown lf 557963 1 2024-03-16T10:44:51.128 INFO:tasks.workunit.client.0.smithi002.stdout:6/82: write f13 [3240638,74112] 0 2024-03-16T10:44:51.130 INFO:tasks.workunit.client.0.smithi002.stdout:8/117: link f4 d7/d9/d17/f27 0 2024-03-16T10:44:51.130 INFO:tasks.workunit.client.0.smithi002.stdout:8/118: dread - d7/d9/d17/f1f zero size 2024-03-16T10:44:51.135 INFO:tasks.workunit.client.0.smithi002.stdout:2/93: creat d7/dc/f1a x:0 0 0 2024-03-16T10:44:51.136 INFO:tasks.workunit.client.0.smithi002.stdout:8/119: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:51.137 INFO:tasks.workunit.client.0.smithi002.stdout:8/120: read d7/f19 [1167067,60375] 0 2024-03-16T10:44:51.138 INFO:tasks.workunit.client.0.smithi002.stdout:0/130: creat d3/d28/f29 x:0 0 0 2024-03-16T10:44:51.141 INFO:tasks.workunit.client.0.smithi002.stdout:9/62: mknod d5/d9/dc/c15 0 2024-03-16T10:44:51.143 INFO:tasks.workunit.client.0.smithi002.stdout:5/47: rename d0/cf to d0/c10 0 2024-03-16T10:44:51.145 INFO:tasks.workunit.client.0.smithi002.stdout:9/63: dread d5/fd [0,4194304] 0 2024-03-16T10:44:51.145 INFO:tasks.workunit.client.0.smithi002.stdout:5/48: dread d0/fe [0,4194304] 0 2024-03-16T10:44:51.151 INFO:tasks.workunit.client.0.smithi002.stdout:4/116: sync 2024-03-16T10:44:51.152 INFO:tasks.workunit.client.0.smithi002.stdout:9/64: dwrite d5/d9/dc/f11 [0,4194304] 0 2024-03-16T10:44:51.157 INFO:tasks.workunit.client.0.smithi002.stdout:9/65: dread d5/d9/dc/f11 [0,4194304] 0 2024-03-16T10:44:51.173 INFO:tasks.workunit.client.0.smithi002.stdout:0/131: mkdir d3/d28/d2a 0 2024-03-16T10:44:51.174 INFO:tasks.workunit.client.0.smithi002.stdout:3/79: write d1/dd/d11/f14 [1793294,88530] 0 2024-03-16T10:44:51.175 INFO:tasks.workunit.client.0.smithi002.stdout:6/83: getdents d15/d1b 0 2024-03-16T10:44:51.175 INFO:tasks.workunit.client.0.smithi002.stdout:0/132: chown d3/l8 3511433 1 2024-03-16T10:44:51.176 INFO:tasks.workunit.client.0.smithi002.stdout:5/49: truncate d0/f6 4488613 0 2024-03-16T10:44:51.182 INFO:tasks.workunit.client.0.smithi002.stdout:9/66: unlink d5/f10 0 2024-03-16T10:44:51.182 INFO:tasks.workunit.client.0.smithi002.stdout:3/80: dwrite d1/dd/d11/f14 [0,4194304] 0 2024-03-16T10:44:51.186 INFO:tasks.workunit.client.0.smithi002.stdout:1/101: truncate d2/d3/fc 2424110 0 2024-03-16T10:44:51.189 INFO:tasks.workunit.client.0.smithi002.stdout:9/67: dwrite d5/d9/fe [0,4194304] 0 2024-03-16T10:44:51.251 INFO:tasks.workunit.client.0.smithi002.stdout:7/90: sync 2024-03-16T10:44:51.252 INFO:tasks.workunit.client.0.smithi002.stdout:7/91: truncate d3/f11 4282268 0 2024-03-16T10:44:51.252 INFO:tasks.workunit.client.0.smithi002.stdout:7/92: fsync d3/f14 0 2024-03-16T10:44:51.253 INFO:tasks.workunit.client.0.smithi002.stdout:7/93: chown d3/cf 4 1 2024-03-16T10:44:51.343 INFO:tasks.workunit.client.0.smithi002.stdout:8/121: link d7/l24 d7/d13/l28 0 2024-03-16T10:44:51.345 INFO:tasks.workunit.client.0.smithi002.stdout:3/81: creat d1/f1a x:0 0 0 2024-03-16T10:44:51.349 INFO:tasks.workunit.client.0.smithi002.stdout:1/102: dread d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:51.356 INFO:tasks.workunit.client.0.smithi002.stdout:2/94: getdents d7/db 0 2024-03-16T10:44:51.357 INFO:tasks.workunit.client.0.smithi002.stdout:2/95: fdatasync d7/dc/f1a 0 2024-03-16T10:44:51.357 INFO:tasks.workunit.client.0.smithi002.stdout:8/122: mknod d7/d13/c29 0 2024-03-16T10:44:51.358 INFO:tasks.workunit.client.0.smithi002.stdout:8/123: read d7/f21 [158827,88614] 0 2024-03-16T10:44:51.359 INFO:tasks.workunit.client.0.smithi002.stdout:3/82: rename d1/f9 to d1/dd/d11/d15/f1b 0 2024-03-16T10:44:51.362 INFO:tasks.workunit.client.0.smithi002.stdout:2/96: dwrite f3 [0,4194304] 0 2024-03-16T10:44:51.363 INFO:tasks.workunit.client.0.smithi002.stdout:2/97: chown d7 1673683 1 2024-03-16T10:44:51.364 INFO:tasks.workunit.client.0.smithi002.stdout:2/98: stat d7/dc/f15 0 2024-03-16T10:44:51.366 INFO:tasks.workunit.client.0.smithi002.stdout:2/99: write d7/dc/f15 [914356,60844] 0 2024-03-16T10:44:51.366 INFO:tasks.workunit.client.0.smithi002.stdout:3/83: dwrite d1/f1a [0,4194304] 0 2024-03-16T10:44:51.368 INFO:tasks.workunit.client.0.smithi002.stdout:3/84: dread - d1/dd/d11/d15/f17 zero size 2024-03-16T10:44:51.372 INFO:tasks.workunit.client.0.smithi002.stdout:3/85: dread d1/dd/f10 [0,4194304] 0 2024-03-16T10:44:51.373 INFO:tasks.workunit.client.0.smithi002.stdout:3/86: dread - d1/dd/f13 zero size 2024-03-16T10:44:51.374 INFO:tasks.workunit.client.0.smithi002.stdout:9/68: rmdir d5/df 0 2024-03-16T10:44:51.379 INFO:tasks.workunit.client.0.smithi002.stdout:6/84: getdents d15 0 2024-03-16T10:44:51.381 INFO:tasks.workunit.client.0.smithi002.stdout:5/50: rename d0/l5 to d0/l11 0 2024-03-16T10:44:51.382 INFO:tasks.workunit.client.0.smithi002.stdout:2/100: symlink d7/db/l1b 0 2024-03-16T10:44:51.383 INFO:tasks.workunit.client.0.smithi002.stdout:2/101: chown d7/l9 34081 1 2024-03-16T10:44:51.387 INFO:tasks.workunit.client.0.smithi002.stdout:3/87: symlink d1/dd/d11/d15/l1c 0 2024-03-16T10:44:51.388 INFO:tasks.workunit.client.0.smithi002.stdout:9/69: creat d5/d9/dc/f16 x:0 0 0 2024-03-16T10:44:51.388 INFO:tasks.workunit.client.0.smithi002.stdout:9/70: stat d5/l8 0 2024-03-16T10:44:51.390 INFO:tasks.workunit.client.0.smithi002.stdout:1/103: creat d2/f21 x:0 0 0 2024-03-16T10:44:51.392 INFO:tasks.workunit.client.0.smithi002.stdout:5/51: unlink d0/fe 0 2024-03-16T10:44:51.393 INFO:tasks.workunit.client.0.smithi002.stdout:2/102: unlink d7/dc/cf 0 2024-03-16T10:44:51.393 INFO:tasks.workunit.client.0.smithi002.stdout:2/103: stat d7/db/l1b 0 2024-03-16T10:44:51.396 INFO:tasks.workunit.client.0.smithi002.stdout:3/88: creat d1/f1d x:0 0 0 2024-03-16T10:44:51.397 INFO:tasks.workunit.client.0.smithi002.stdout:9/71: write d5/fd [1552398,130390] 0 2024-03-16T10:44:51.398 INFO:tasks.workunit.client.0.smithi002.stdout:6/85: symlink d15/d1b/l1d 0 2024-03-16T10:44:51.399 INFO:tasks.workunit.client.0.smithi002.stdout:9/72: write d5/d9/fe [3655765,93679] 0 2024-03-16T10:44:51.399 INFO:tasks.workunit.client.0.smithi002.stdout:6/86: fsync d15/f19 0 2024-03-16T10:44:51.400 INFO:tasks.workunit.client.0.smithi002.stdout:1/104: mknod d2/df/d12/c22 0 2024-03-16T10:44:51.401 INFO:tasks.workunit.client.0.smithi002.stdout:6/87: write d15/f19 [758272,48175] 0 2024-03-16T10:44:51.402 INFO:tasks.workunit.client.0.smithi002.stdout:2/104: unlink d7/dc/l19 0 2024-03-16T10:44:51.407 INFO:tasks.workunit.client.0.smithi002.stdout:9/73: symlink d5/d9/dc/l17 0 2024-03-16T10:44:51.409 INFO:tasks.workunit.client.0.smithi002.stdout:1/105: symlink d2/df/d12/l23 0 2024-03-16T10:44:51.411 INFO:tasks.workunit.client.0.smithi002.stdout:3/89: dwrite d1/dd/f10 [0,4194304] 0 2024-03-16T10:44:51.412 INFO:tasks.workunit.client.0.smithi002.stdout:6/88: rename l14 to d15/l1e 0 2024-03-16T10:44:51.413 INFO:tasks.workunit.client.0.smithi002.stdout:2/105: unlink d7/dc/f15 0 2024-03-16T10:44:51.418 INFO:tasks.workunit.client.0.smithi002.stdout:9/74: write d5/d9/dc/f11 [4140074,127062] 0 2024-03-16T10:44:51.419 INFO:tasks.workunit.client.0.smithi002.stdout:3/90: mkdir d1/dd/d11/d15/d1e 0 2024-03-16T10:44:51.420 INFO:tasks.workunit.client.0.smithi002.stdout:1/106: dwrite d2/f21 [0,4194304] 0 2024-03-16T10:44:51.422 INFO:tasks.workunit.client.0.smithi002.stdout:5/52: rename d0/cb to d0/c12 0 2024-03-16T10:44:51.424 INFO:tasks.workunit.client.0.smithi002.stdout:6/89: dread fc [0,4194304] 0 2024-03-16T10:44:51.426 INFO:tasks.workunit.client.0.smithi002.stdout:2/106: truncate d7/f8 2033291 0 2024-03-16T10:44:51.427 INFO:tasks.workunit.client.0.smithi002.stdout:2/107: fsync d7/db/d12/f11 0 2024-03-16T10:44:51.428 INFO:tasks.workunit.client.0.smithi002.stdout:9/75: mkdir d5/d9/d18 0 2024-03-16T10:44:51.429 INFO:tasks.workunit.client.0.smithi002.stdout:9/76: write d5/d9/f13 [354473,7978] 0 2024-03-16T10:44:51.431 INFO:tasks.workunit.client.0.smithi002.stdout:9/77: write d5/fd [297453,122033] 0 2024-03-16T10:44:51.432 INFO:tasks.workunit.client.0.smithi002.stdout:3/91: mknod d1/dd/d11/c1f 0 2024-03-16T10:44:51.432 INFO:tasks.workunit.client.0.smithi002.stdout:3/92: dread - d1/f1d zero size 2024-03-16T10:44:51.432 INFO:tasks.workunit.client.0.smithi002.stdout:9/78: fdatasync d5/d9/dc/f16 0 2024-03-16T10:44:51.433 INFO:tasks.workunit.client.0.smithi002.stdout:3/93: readlink d1/l4 0 2024-03-16T10:44:51.434 INFO:tasks.workunit.client.0.smithi002.stdout:6/90: chown fa 40447 1 2024-03-16T10:44:51.438 INFO:tasks.workunit.client.0.smithi002.stdout:1/107: mknod d2/d3/d1c/c24 0 2024-03-16T10:44:51.438 INFO:tasks.workunit.client.0.smithi002.stdout:1/108: fdatasync d2/f21 0 2024-03-16T10:44:51.440 INFO:tasks.workunit.client.0.smithi002.stdout:5/53: unlink d0/f9 0 2024-03-16T10:44:51.441 INFO:tasks.workunit.client.0.smithi002.stdout:5/54: write d0/f6 [4857045,116594] 0 2024-03-16T10:44:51.441 INFO:tasks.workunit.client.0.smithi002.stdout:9/79: mknod d5/d9/c19 0 2024-03-16T10:44:51.442 INFO:tasks.workunit.client.0.smithi002.stdout:1/109: dread f1 [0,4194304] 0 2024-03-16T10:44:51.442 INFO:tasks.workunit.client.0.smithi002.stdout:9/80: readlink d5/d9/lb 0 2024-03-16T10:44:51.448 INFO:tasks.workunit.client.0.smithi002.stdout:5/55: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:51.451 INFO:tasks.workunit.client.0.smithi002.stdout:1/110: mknod d2/d3/d1c/c25 0 2024-03-16T10:44:51.453 INFO:tasks.workunit.client.0.smithi002.stdout:3/94: link d1/dd/d11/d15/f1b d1/dd/d11/f20 0 2024-03-16T10:44:51.459 INFO:tasks.workunit.client.0.smithi002.stdout:9/81: fsync d5/f7 0 2024-03-16T10:44:51.463 INFO:tasks.workunit.client.0.smithi002.stdout:3/95: write d1/dd/d11/d15/f1b [4389250,8916] 0 2024-03-16T10:44:51.466 INFO:tasks.workunit.client.0.smithi002.stdout:5/56: symlink d0/l13 0 2024-03-16T10:44:51.470 INFO:tasks.workunit.client.0.smithi002.stdout:3/96: symlink d1/dd/d11/d15/l21 0 2024-03-16T10:44:51.471 INFO:tasks.workunit.client.0.smithi002.stdout:3/97: chown d1/dd/d11/d15/l1c 578160 1 2024-03-16T10:44:51.473 INFO:tasks.workunit.client.0.smithi002.stdout:5/57: unlink d0/f7 0 2024-03-16T10:44:51.478 INFO:tasks.workunit.client.0.smithi002.stdout:3/98: mkdir d1/dd/d11/d15/d1e/d22 0 2024-03-16T10:44:51.483 INFO:tasks.workunit.client.0.smithi002.stdout:3/99: dwrite d1/dd/d11/f20 [0,4194304] 0 2024-03-16T10:44:51.489 INFO:tasks.workunit.client.0.smithi002.stdout:3/100: dwrite d1/dd/d11/f20 [0,4194304] 0 2024-03-16T10:44:51.493 INFO:tasks.workunit.client.0.smithi002.stdout:3/101: mknod d1/dd/d11/d15/c23 0 2024-03-16T10:44:51.503 INFO:tasks.workunit.client.0.smithi002.stdout:3/102: dwrite d1/dd/d11/d15/f1b [4194304,4194304] 0 2024-03-16T10:44:51.537 INFO:tasks.workunit.client.0.smithi002.stdout:0/133: write d3/fc [3779930,105888] 0 2024-03-16T10:44:51.541 INFO:tasks.workunit.client.0.smithi002.stdout:0/134: mknod d3/c2b 0 2024-03-16T10:44:51.542 INFO:tasks.workunit.client.0.smithi002.stdout:7/94: write d3/fd [1078720,8217] 0 2024-03-16T10:44:51.543 INFO:tasks.workunit.client.0.smithi002.stdout:4/117: dwrite d2/d6/f17 [0,4194304] 0 2024-03-16T10:44:51.543 INFO:tasks.workunit.client.0.smithi002.stdout:7/95: fsync d3/f15 0 2024-03-16T10:44:51.544 INFO:tasks.workunit.client.0.smithi002.stdout:4/118: chown d2/dd/c12 793793 1 2024-03-16T10:44:51.544 INFO:tasks.workunit.client.0.smithi002.stdout:7/96: fsync d3/f16 0 2024-03-16T10:44:51.546 INFO:tasks.workunit.client.0.smithi002.stdout:0/135: creat d3/d28/d2a/f2c x:0 0 0 2024-03-16T10:44:51.553 INFO:tasks.workunit.client.0.smithi002.stdout:7/97: mknod d3/d4/d10/c18 0 2024-03-16T10:44:51.554 INFO:tasks.workunit.client.0.smithi002.stdout:4/119: creat d2/dd/d1c/f1d x:0 0 0 2024-03-16T10:44:51.555 INFO:tasks.workunit.client.0.smithi002.stdout:8/124: write f4 [2431694,65416] 0 2024-03-16T10:44:51.556 INFO:tasks.workunit.client.0.smithi002.stdout:0/136: creat d3/f2d x:0 0 0 2024-03-16T10:44:51.558 INFO:tasks.workunit.client.0.smithi002.stdout:0/137: fdatasync d3/d5/f1c 0 2024-03-16T10:44:51.559 INFO:tasks.workunit.client.0.smithi002.stdout:8/125: write d7/f21 [1099809,2893] 0 2024-03-16T10:44:51.561 INFO:tasks.workunit.client.0.smithi002.stdout:8/126: stat d7/f14 0 2024-03-16T10:44:51.563 INFO:tasks.workunit.client.0.smithi002.stdout:0/138: dread - d3/d5/f14 zero size 2024-03-16T10:44:51.564 INFO:tasks.workunit.client.0.smithi002.stdout:8/127: dread - d7/f26 zero size 2024-03-16T10:44:51.565 INFO:tasks.workunit.client.0.smithi002.stdout:7/98: dwrite d3/f15 [0,4194304] 0 2024-03-16T10:44:51.566 INFO:tasks.workunit.client.0.smithi002.stdout:6/91: getdents d15/d1b 0 2024-03-16T10:44:51.567 INFO:tasks.workunit.client.0.smithi002.stdout:0/139: write d3/de/f15 [259481,113165] 0 2024-03-16T10:44:51.569 INFO:tasks.workunit.client.0.smithi002.stdout:4/120: dwrite d2/dd/f1b [0,4194304] 0 2024-03-16T10:44:51.569 INFO:tasks.workunit.client.0.smithi002.stdout:8/128: chown d7/d13/l28 249 1 2024-03-16T10:44:51.573 INFO:tasks.workunit.client.0.smithi002.stdout:4/121: write d2/d6/f10 [1247533,7464] 0 2024-03-16T10:44:51.576 INFO:tasks.workunit.client.0.smithi002.stdout:7/99: dread d3/f11 [0,4194304] 0 2024-03-16T10:44:51.577 INFO:tasks.workunit.client.0.smithi002.stdout:4/122: write d2/d6/f1a [592822,8682] 0 2024-03-16T10:44:51.579 INFO:tasks.workunit.client.0.smithi002.stdout:6/92: dread fc [0,4194304] 0 2024-03-16T10:44:51.579 INFO:tasks.workunit.client.0.smithi002.stdout:6/93: readlink le 0 2024-03-16T10:44:51.581 INFO:tasks.workunit.client.0.smithi002.stdout:0/140: dwrite d3/d28/d2a/f2c [0,4194304] 0 2024-03-16T10:44:51.583 INFO:tasks.workunit.client.0.smithi002.stdout:0/141: write d3/d5/f23 [1176393,44429] 0 2024-03-16T10:44:51.584 INFO:tasks.workunit.client.0.smithi002.stdout:6/94: creat d15/d1b/f1f x:0 0 0 2024-03-16T10:44:51.584 INFO:tasks.workunit.client.0.smithi002.stdout:0/142: fsync d3/d28/f29 0 2024-03-16T10:44:51.592 INFO:tasks.workunit.client.0.smithi002.stdout:0/143: fdatasync d3/d5/f19 0 2024-03-16T10:44:51.592 INFO:tasks.workunit.client.0.smithi002.stdout:7/100: link d3/l12 d3/l19 0 2024-03-16T10:44:51.593 INFO:tasks.workunit.client.0.smithi002.stdout:0/144: write d3/f1a [547001,76835] 0 2024-03-16T10:44:51.596 INFO:tasks.workunit.client.0.smithi002.stdout:4/123: getdents d2/dd 0 2024-03-16T10:44:51.599 INFO:tasks.workunit.client.0.smithi002.stdout:4/124: truncate d2/d6/f1a 1340338 0 2024-03-16T10:44:51.601 INFO:tasks.workunit.client.0.smithi002.stdout:0/145: dread d3/d5/f23 [0,4194304] 0 2024-03-16T10:44:51.602 INFO:tasks.workunit.client.0.smithi002.stdout:7/101: dwrite d3/f14 [0,4194304] 0 2024-03-16T10:44:51.602 INFO:tasks.workunit.client.0.smithi002.stdout:4/125: readlink d2/d6/l16 0 2024-03-16T10:44:51.602 INFO:tasks.workunit.client.0.smithi002.stdout:0/146: readlink d3/l8 0 2024-03-16T10:44:51.603 INFO:tasks.workunit.client.0.smithi002.stdout:7/102: readlink d3/d4/lc 0 2024-03-16T10:44:51.604 INFO:tasks.workunit.client.0.smithi002.stdout:4/126: write d2/dd/d1c/f1d [249190,56706] 0 2024-03-16T10:44:51.605 INFO:tasks.workunit.client.0.smithi002.stdout:0/147: truncate d3/d5/f18 321677 0 2024-03-16T10:44:51.608 INFO:tasks.workunit.client.0.smithi002.stdout:0/148: write d3/de/f10 [842757,111173] 0 2024-03-16T10:44:51.608 INFO:tasks.workunit.client.0.smithi002.stdout:4/127: creat d2/d6/f1e x:0 0 0 2024-03-16T10:44:51.612 INFO:tasks.workunit.client.0.smithi002.stdout:0/149: chown d3/l16 3 1 2024-03-16T10:44:51.612 INFO:tasks.workunit.client.0.smithi002.stdout:4/128: chown d2/dd/da/cc 4753621 1 2024-03-16T10:44:51.614 INFO:tasks.workunit.client.0.smithi002.stdout:5/58: rmdir d0 39 2024-03-16T10:44:51.616 INFO:tasks.workunit.client.0.smithi002.stdout:7/103: dread d3/f11 [0,4194304] 0 2024-03-16T10:44:51.616 INFO:tasks.workunit.client.0.smithi002.stdout:1/111: dwrite d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:51.618 INFO:tasks.workunit.client.0.smithi002.stdout:9/82: dwrite d5/f7 [0,4194304] 0 2024-03-16T10:44:51.618 INFO:tasks.workunit.client.0.smithi002.stdout:0/150: truncate d3/de/f10 1496867 0 2024-03-16T10:44:51.621 INFO:tasks.workunit.client.0.smithi002.stdout:5/59: rmdir d0 39 2024-03-16T10:44:51.623 INFO:tasks.workunit.client.0.smithi002.stdout:0/151: mknod d3/d28/d2a/c2e 0 2024-03-16T10:44:51.626 INFO:tasks.workunit.client.0.smithi002.stdout:1/112: dread f1 [4194304,4194304] 0 2024-03-16T10:44:51.627 INFO:tasks.workunit.client.0.smithi002.stdout:3/103: truncate d1/f1a 1360391 0 2024-03-16T10:44:51.629 INFO:tasks.workunit.client.0.smithi002.stdout:0/152: truncate d3/d5/fb 8973480 0 2024-03-16T10:44:51.631 INFO:tasks.workunit.client.0.smithi002.stdout:4/129: dread d2/d6/f1a [0,4194304] 0 2024-03-16T10:44:51.632 INFO:tasks.workunit.client.0.smithi002.stdout:5/60: write d0/f6 [5584443,37599] 0 2024-03-16T10:44:51.633 INFO:tasks.workunit.client.0.smithi002.stdout:9/83: dwrite d5/d9/f13 [0,4194304] 0 2024-03-16T10:44:51.634 INFO:tasks.workunit.client.0.smithi002.stdout:4/130: chown d2/dd/da/ff 3 1 2024-03-16T10:44:51.637 INFO:tasks.workunit.client.0.smithi002.stdout:3/104: creat d1/dd/d11/d15/d1e/f24 x:0 0 0 2024-03-16T10:44:51.639 INFO:tasks.workunit.client.0.smithi002.stdout:3/105: chown d1/dd/d11/d15/d1e/d22 1 1 2024-03-16T10:44:51.639 INFO:tasks.workunit.client.0.smithi002.stdout:4/131: mknod d2/d6/c1f 0 2024-03-16T10:44:51.640 INFO:tasks.workunit.client.0.smithi002.stdout:1/113: fsync d2/d3/f8 0 2024-03-16T10:44:51.642 INFO:tasks.workunit.client.0.smithi002.stdout:4/132: fsync d2/dd/d1c/f1d 0 2024-03-16T10:44:51.644 INFO:tasks.workunit.client.0.smithi002.stdout:4/133: stat d2/fe 0 2024-03-16T10:44:51.645 INFO:tasks.workunit.client.0.smithi002.stdout:1/114: truncate d2/d3/f1f 871394 0 2024-03-16T10:44:51.645 INFO:tasks.workunit.client.0.smithi002.stdout:5/61: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:51.648 INFO:tasks.workunit.client.0.smithi002.stdout:1/115: stat d2/d3/lb 0 2024-03-16T10:44:51.648 INFO:tasks.workunit.client.0.smithi002.stdout:9/84: dwrite d5/d9/fe [0,4194304] 0 2024-03-16T10:44:51.652 INFO:tasks.workunit.client.0.smithi002.stdout:9/85: write d5/d9/dc/f11 [2199989,65765] 0 2024-03-16T10:44:51.653 INFO:tasks.workunit.client.0.smithi002.stdout:5/62: rename d0 to d0/d14 22 2024-03-16T10:44:51.656 INFO:tasks.workunit.client.0.smithi002.stdout:7/104: dwrite d3/f15 [4194304,4194304] 0 2024-03-16T10:44:51.658 INFO:tasks.workunit.client.0.smithi002.stdout:6/95: rmdir d15/d1b 39 2024-03-16T10:44:51.659 INFO:tasks.workunit.client.0.smithi002.stdout:2/108: write d7/f8 [403636,75682] 0 2024-03-16T10:44:51.661 INFO:tasks.workunit.client.0.smithi002.stdout:5/63: chown d0/l11 8 1 2024-03-16T10:44:51.663 INFO:tasks.workunit.client.0.smithi002.stdout:7/105: creat d3/f1a x:0 0 0 2024-03-16T10:44:51.663 INFO:tasks.workunit.client.0.smithi002.stdout:7/106: stat d3/f15 0 2024-03-16T10:44:51.663 INFO:tasks.workunit.client.0.smithi002.stdout:9/86: dwrite d5/f7 [0,4194304] 0 2024-03-16T10:44:51.664 INFO:tasks.workunit.client.0.smithi002.stdout:7/107: fsync d3/fd 0 2024-03-16T10:44:51.665 INFO:tasks.workunit.client.0.smithi002.stdout:9/87: fsync d5/fd 0 2024-03-16T10:44:51.670 INFO:tasks.workunit.client.0.smithi002.stdout:2/109: unlink l1 0 2024-03-16T10:44:51.680 INFO:tasks.workunit.client.0.smithi002.stdout:2/110: creat d7/dc/f1c x:0 0 0 2024-03-16T10:44:51.680 INFO:tasks.workunit.client.0.smithi002.stdout:0/153: write d3/de/f13 [2989839,126534] 0 2024-03-16T10:44:51.681 INFO:tasks.workunit.client.0.smithi002.stdout:9/88: link d5/d9/fe d5/d9/f1a 0 2024-03-16T10:44:51.682 INFO:tasks.workunit.client.0.smithi002.stdout:6/96: getdents d15 0 2024-03-16T10:44:51.683 INFO:tasks.workunit.client.0.smithi002.stdout:6/97: chown d15/f1a 8 1 2024-03-16T10:44:51.683 INFO:tasks.workunit.client.0.smithi002.stdout:4/134: rmdir d2/d6 39 2024-03-16T10:44:51.684 INFO:tasks.workunit.client.0.smithi002.stdout:2/111: chown d7/db/d12/le 2186571 1 2024-03-16T10:44:51.687 INFO:tasks.workunit.client.0.smithi002.stdout:0/154: creat d3/d5/f2f x:0 0 0 2024-03-16T10:44:51.690 INFO:tasks.workunit.client.0.smithi002.stdout:3/106: truncate d1/dd/d11/f14 4146171 0 2024-03-16T10:44:51.691 INFO:tasks.workunit.client.0.smithi002.stdout:6/98: rmdir d15 39 2024-03-16T10:44:51.695 INFO:tasks.workunit.client.0.smithi002.stdout:2/112: readlink d7/l9 0 2024-03-16T10:44:51.695 INFO:tasks.workunit.client.0.smithi002.stdout:2/113: chown d7/db 0 1 2024-03-16T10:44:51.696 INFO:tasks.workunit.client.0.smithi002.stdout:1/116: dwrite d2/f13 [4194304,4194304] 0 2024-03-16T10:44:51.697 INFO:tasks.workunit.client.0.smithi002.stdout:2/114: write d7/dc/f1a [312296,129988] 0 2024-03-16T10:44:51.701 INFO:tasks.workunit.client.0.smithi002.stdout:3/107: mkdir d1/dd/d11/d15/d1e/d25 0 2024-03-16T10:44:51.706 INFO:tasks.workunit.client.0.smithi002.stdout:5/64: truncate d0/f6 1618380 0 2024-03-16T10:44:51.711 INFO:tasks.workunit.client.0.smithi002.stdout:1/117: chown d2/d3/fc 378018 1 2024-03-16T10:44:51.713 INFO:tasks.workunit.client.0.smithi002.stdout:2/115: chown d7/l9 47602 1 2024-03-16T10:44:51.720 INFO:tasks.workunit.client.0.smithi002.stdout:6/99: creat d15/d1b/f20 x:0 0 0 2024-03-16T10:44:51.723 INFO:tasks.workunit.client.0.smithi002.stdout:4/135: creat d2/f20 x:0 0 0 2024-03-16T10:44:51.729 INFO:tasks.workunit.client.0.smithi002.stdout:1/118: symlink d2/df/l26 0 2024-03-16T10:44:51.730 INFO:tasks.workunit.client.0.smithi002.stdout:1/119: write d2/d3/f19 [270905,63838] 0 2024-03-16T10:44:51.731 INFO:tasks.workunit.client.0.smithi002.stdout:1/120: chown d2/df/d12/c1e 3430912 1 2024-03-16T10:44:51.732 INFO:tasks.workunit.client.0.smithi002.stdout:9/89: getdents d5/d9 0 2024-03-16T10:44:51.733 INFO:tasks.workunit.client.0.smithi002.stdout:9/90: fdatasync d5/d9/f13 0 2024-03-16T10:44:51.733 INFO:tasks.workunit.client.0.smithi002.stdout:9/91: chown d5/l6 110 1 2024-03-16T10:44:51.737 INFO:tasks.workunit.client.0.smithi002.stdout:6/100: symlink d15/l21 0 2024-03-16T10:44:51.744 INFO:tasks.workunit.client.0.smithi002.stdout:0/155: getdents d3 0 2024-03-16T10:44:51.744 INFO:tasks.workunit.client.0.smithi002.stdout:4/136: dwrite d2/d6/f13 [0,4194304] 0 2024-03-16T10:44:51.746 INFO:tasks.workunit.client.0.smithi002.stdout:0/156: dread d3/de/f15 [0,4194304] 0 2024-03-16T10:44:51.752 INFO:tasks.workunit.client.0.smithi002.stdout:5/65: fdatasync d0/f6 0 2024-03-16T10:44:51.757 INFO:tasks.workunit.client.0.smithi002.stdout:9/92: symlink d5/d9/d18/l1b 0 2024-03-16T10:44:51.758 INFO:tasks.workunit.client.0.smithi002.stdout:3/108: getdents d1 0 2024-03-16T10:44:51.763 INFO:tasks.workunit.client.0.smithi002.stdout:0/157: link d3/d28/d2a/f2c d3/d5/f30 0 2024-03-16T10:44:51.764 INFO:tasks.workunit.client.0.smithi002.stdout:9/93: symlink d5/d9/dc/l1c 0 2024-03-16T10:44:51.766 INFO:tasks.workunit.client.0.smithi002.stdout:5/66: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:51.767 INFO:tasks.workunit.client.0.smithi002.stdout:3/109: unlink d1/l4 0 2024-03-16T10:44:51.770 INFO:tasks.workunit.client.0.smithi002.stdout:6/101: dread f11 [0,4194304] 0 2024-03-16T10:44:51.770 INFO:tasks.workunit.client.0.smithi002.stdout:1/121: getdents d2/df 0 2024-03-16T10:44:51.771 INFO:tasks.workunit.client.0.smithi002.stdout:1/122: chown d2/df/d12/c1a 1915133113 1 2024-03-16T10:44:51.774 INFO:tasks.workunit.client.0.smithi002.stdout:1/123: dread d2/d3/fc [0,4194304] 0 2024-03-16T10:44:51.776 INFO:tasks.workunit.client.0.smithi002.stdout:0/158: mkdir d3/d31 0 2024-03-16T10:44:51.778 INFO:tasks.workunit.client.0.smithi002.stdout:9/94: symlink d5/d9/d18/l1d 0 2024-03-16T10:44:51.779 INFO:tasks.workunit.client.0.smithi002.stdout:4/137: fdatasync d2/d6/f13 0 2024-03-16T10:44:51.786 INFO:tasks.workunit.client.0.smithi002.stdout:1/124: rmdir d2/df/d12 39 2024-03-16T10:44:51.792 INFO:tasks.workunit.client.0.smithi002.stdout:9/95: mkdir d5/d9/dc/d1e 0 2024-03-16T10:44:51.796 INFO:tasks.workunit.client.0.smithi002.stdout:4/138: mkdir d2/dd/da/d21 0 2024-03-16T10:44:51.797 INFO:tasks.workunit.client.0.smithi002.stdout:4/139: write d2/d6/f17 [2290161,106979] 0 2024-03-16T10:44:51.799 INFO:tasks.workunit.client.0.smithi002.stdout:3/110: link d1/f1d d1/dd/d11/d15/d1e/d25/f26 0 2024-03-16T10:44:51.801 INFO:tasks.workunit.client.0.smithi002.stdout:6/102: write d15/f1c [686253,66992] 0 2024-03-16T10:44:51.802 INFO:tasks.workunit.client.0.smithi002.stdout:4/140: dread d2/fe [0,4194304] 0 2024-03-16T10:44:51.805 INFO:tasks.workunit.client.0.smithi002.stdout:9/96: creat d5/d9/dc/f1f x:0 0 0 2024-03-16T10:44:51.809 INFO:tasks.workunit.client.0.smithi002.stdout:6/103: dwrite f9 [0,4194304] 0 2024-03-16T10:44:51.811 INFO:tasks.workunit.client.0.smithi002.stdout:4/141: dwrite d2/dd/da/ff [0,4194304] 0 2024-03-16T10:44:51.814 INFO:tasks.workunit.client.0.smithi002.stdout:3/111: dread d1/dd/d11/d15/f1b [0,4194304] 0 2024-03-16T10:44:51.814 INFO:tasks.workunit.client.0.smithi002.stdout:2/116: mkdir d7/dc/d1d 0 2024-03-16T10:44:51.822 INFO:tasks.workunit.client.0.smithi002.stdout:6/104: write d15/f18 [10767,110543] 0 2024-03-16T10:44:51.823 INFO:tasks.workunit.client.0.smithi002.stdout:1/125: creat d2/df/f27 x:0 0 0 2024-03-16T10:44:51.826 INFO:tasks.workunit.client.0.smithi002.stdout:4/142: creat d2/d6/f22 x:0 0 0 2024-03-16T10:44:51.826 INFO:tasks.workunit.client.0.smithi002.stdout:6/105: dread f13 [0,4194304] 0 2024-03-16T10:44:51.827 INFO:tasks.workunit.client.0.smithi002.stdout:6/106: truncate fd 807142 0 2024-03-16T10:44:51.827 INFO:tasks.workunit.client.0.smithi002.stdout:5/67: truncate d0/f6 1531075 0 2024-03-16T10:44:51.834 INFO:tasks.workunit.client.0.smithi002.stdout:3/112: link d1/fb d1/dd/d11/f27 0 2024-03-16T10:44:51.834 INFO:tasks.workunit.client.0.smithi002.stdout:3/113: chown d1/dd/d11/l18 25 1 2024-03-16T10:44:51.835 INFO:tasks.workunit.client.0.smithi002.stdout:6/107: write d15/d1b/f20 [846765,122035] 0 2024-03-16T10:44:51.836 INFO:tasks.workunit.client.0.smithi002.stdout:2/117: symlink d7/dc/d1d/l1e 0 2024-03-16T10:44:51.838 INFO:tasks.workunit.client.0.smithi002.stdout:9/97: link d5/d9/fe d5/d9/dc/f20 0 2024-03-16T10:44:51.839 INFO:tasks.workunit.client.0.smithi002.stdout:9/98: truncate d5/d9/dc/f1f 660100 0 2024-03-16T10:44:51.839 INFO:tasks.workunit.client.0.smithi002.stdout:9/99: chown d5/l6 77857 1 2024-03-16T10:44:51.839 INFO:tasks.workunit.client.0.smithi002.stdout:9/100: readlink d5/d9/d18/l1b 0 2024-03-16T10:44:51.840 INFO:tasks.workunit.client.0.smithi002.stdout:5/68: symlink d0/l15 0 2024-03-16T10:44:51.842 INFO:tasks.workunit.client.0.smithi002.stdout:3/114: write d1/dd/d11/d15/d1e/d25/f26 [654660,30667] 0 2024-03-16T10:44:51.843 INFO:tasks.workunit.client.0.smithi002.stdout:8/129: sync 2024-03-16T10:44:51.843 INFO:tasks.workunit.client.0.smithi002.stdout:6/108: creat d15/d1b/f22 x:0 0 0 2024-03-16T10:44:51.844 INFO:tasks.workunit.client.0.smithi002.stdout:2/118: rmdir d7 39 2024-03-16T10:44:51.844 INFO:tasks.workunit.client.0.smithi002.stdout:6/109: chown f8 81886 1 2024-03-16T10:44:51.845 INFO:tasks.workunit.client.0.smithi002.stdout:0/159: mknod d3/de/c32 0 2024-03-16T10:44:51.846 INFO:tasks.workunit.client.0.smithi002.stdout:0/160: write d3/de/f1d [881329,41591] 0 2024-03-16T10:44:51.848 INFO:tasks.workunit.client.0.smithi002.stdout:1/126: rename f1 to d2/f28 0 2024-03-16T10:44:51.848 INFO:tasks.workunit.client.0.smithi002.stdout:4/143: rename d2 to d2/dd/da/d21/d23 22 2024-03-16T10:44:51.854 INFO:tasks.workunit.client.0.smithi002.stdout:3/115: write d1/fb [1102830,93070] 0 2024-03-16T10:44:51.855 INFO:tasks.workunit.client.0.smithi002.stdout:1/127: dwrite d2/d3/f17 [0,4194304] 0 2024-03-16T10:44:51.856 INFO:tasks.workunit.client.0.smithi002.stdout:2/119: chown d7/dc/d1d/l1e 34 1 2024-03-16T10:44:51.871 INFO:tasks.workunit.client.0.smithi002.stdout:3/116: mkdir d1/dd/d28 0 2024-03-16T10:44:51.873 INFO:tasks.workunit.client.0.smithi002.stdout:5/69: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:51.874 INFO:tasks.workunit.client.0.smithi002.stdout:6/110: mknod d15/c23 0 2024-03-16T10:44:51.874 INFO:tasks.workunit.client.0.smithi002.stdout:6/111: chown f13 764880 1 2024-03-16T10:44:51.875 INFO:tasks.workunit.client.0.smithi002.stdout:6/112: chown d15/f18 2058 1 2024-03-16T10:44:51.876 INFO:tasks.workunit.client.0.smithi002.stdout:6/113: chown d15 29566698 1 2024-03-16T10:44:51.876 INFO:tasks.workunit.client.0.smithi002.stdout:2/120: dread d7/db/d12/f11 [0,4194304] 0 2024-03-16T10:44:51.878 INFO:tasks.workunit.client.0.smithi002.stdout:8/130: rename d7/l1a to d7/d9/d17/l2a 0 2024-03-16T10:44:51.878 INFO:tasks.workunit.client.0.smithi002.stdout:8/131: fsync d7/fc 0 2024-03-16T10:44:51.878 INFO:tasks.workunit.client.0.smithi002.stdout:8/132: readlink l6 0 2024-03-16T10:44:51.882 INFO:tasks.workunit.client.0.smithi002.stdout:1/128: mkdir d2/df/d12/d29 0 2024-03-16T10:44:51.883 INFO:tasks.workunit.client.0.smithi002.stdout:3/117: symlink d1/dd/d11/d15/d1e/l29 0 2024-03-16T10:44:51.885 INFO:tasks.workunit.client.0.smithi002.stdout:5/70: write d0/f6 [1165135,110556] 0 2024-03-16T10:44:51.886 INFO:tasks.workunit.client.0.smithi002.stdout:0/161: truncate d3/d28/d2a/f2c 2272377 0 2024-03-16T10:44:51.889 INFO:tasks.workunit.client.0.smithi002.stdout:6/114: chown d15/l1e 419 1 2024-03-16T10:44:51.890 INFO:tasks.workunit.client.0.smithi002.stdout:2/121: creat d7/dc/d1d/f1f x:0 0 0 2024-03-16T10:44:51.891 INFO:tasks.workunit.client.0.smithi002.stdout:1/129: dwrite d2/d3/f19 [0,4194304] 0 2024-03-16T10:44:51.892 INFO:tasks.workunit.client.0.smithi002.stdout:3/118: dwrite d1/dd/f10 [0,4194304] 0 2024-03-16T10:44:51.907 INFO:tasks.workunit.client.0.smithi002.stdout:0/162: stat d3/d5/f19 0 2024-03-16T10:44:51.907 INFO:tasks.workunit.client.0.smithi002.stdout:6/115: fsync f4 0 2024-03-16T10:44:51.907 INFO:tasks.workunit.client.0.smithi002.stdout:6/116: stat lb 0 2024-03-16T10:44:51.913 INFO:tasks.workunit.client.0.smithi002.stdout:9/101: write d5/d9/f1a [2247941,112565] 0 2024-03-16T10:44:51.913 INFO:tasks.workunit.client.0.smithi002.stdout:4/144: link d2/dd/c12 d2/c24 0 2024-03-16T10:44:51.915 INFO:tasks.workunit.client.0.smithi002.stdout:8/133: getdents d7/d9/d17/d23 0 2024-03-16T10:44:51.917 INFO:tasks.workunit.client.0.smithi002.stdout:0/163: creat d3/d28/f33 x:0 0 0 2024-03-16T10:44:51.918 INFO:tasks.workunit.client.0.smithi002.stdout:4/145: dread d2/dd/f1b [0,4194304] 0 2024-03-16T10:44:51.919 INFO:tasks.workunit.client.0.smithi002.stdout:4/146: chown d2/c18 349000321 1 2024-03-16T10:44:51.921 INFO:tasks.workunit.client.0.smithi002.stdout:0/164: dread d3/de/f1d [0,4194304] 0 2024-03-16T10:44:51.922 INFO:tasks.workunit.client.0.smithi002.stdout:0/165: write d3/d5/fb [863920,60399] 0 2024-03-16T10:44:51.922 INFO:tasks.workunit.client.0.smithi002.stdout:0/166: stat d3/de 0 2024-03-16T10:44:51.929 INFO:tasks.workunit.client.0.smithi002.stdout:9/102: rename d5/d9/d18/l1d to d5/d9/d18/l21 0 2024-03-16T10:44:51.929 INFO:tasks.workunit.client.0.smithi002.stdout:9/103: fdatasync d5/d9/f1a 0 2024-03-16T10:44:51.930 INFO:tasks.workunit.client.0.smithi002.stdout:9/104: stat d5/d9/c19 0 2024-03-16T10:44:51.931 INFO:tasks.workunit.client.0.smithi002.stdout:9/105: write d5/f7 [3967333,92655] 0 2024-03-16T10:44:51.933 INFO:tasks.workunit.client.0.smithi002.stdout:2/122: rmdir d7/dc/d1d 39 2024-03-16T10:44:51.936 INFO:tasks.workunit.client.0.smithi002.stdout:8/134: dread d7/f19 [0,4194304] 0 2024-03-16T10:44:51.939 INFO:tasks.workunit.client.0.smithi002.stdout:0/167: unlink d3/d5/fb 0 2024-03-16T10:44:51.940 INFO:tasks.workunit.client.0.smithi002.stdout:1/130: link d2/d3/d1c/c24 d2/df/d12/d29/c2a 0 2024-03-16T10:44:51.943 INFO:tasks.workunit.client.0.smithi002.stdout:1/131: dread d2/d3/f17 [0,4194304] 0 2024-03-16T10:44:51.945 INFO:tasks.workunit.client.0.smithi002.stdout:3/119: write d1/dd/d11/f14 [960748,111178] 0 2024-03-16T10:44:51.949 INFO:tasks.workunit.client.0.smithi002.stdout:2/123: rmdir d7/dc 39 2024-03-16T10:44:51.954 INFO:tasks.workunit.client.0.smithi002.stdout:6/117: creat d15/f24 x:0 0 0 2024-03-16T10:44:51.957 INFO:tasks.workunit.client.0.smithi002.stdout:0/168: fsync d3/de/f1d 0 2024-03-16T10:44:51.962 INFO:tasks.workunit.client.0.smithi002.stdout:7/108: sync 2024-03-16T10:44:51.963 INFO:tasks.workunit.client.0.smithi002.stdout:3/120: mknod d1/dd/c2a 0 2024-03-16T10:44:51.964 INFO:tasks.workunit.client.0.smithi002.stdout:3/121: dread - d1/dd/d11/d15/d1e/f24 zero size 2024-03-16T10:44:51.966 INFO:tasks.workunit.client.0.smithi002.stdout:2/124: write d7/dc/f1c [220922,97430] 0 2024-03-16T10:44:51.967 INFO:tasks.workunit.client.0.smithi002.stdout:4/147: rmdir d2/dd/da/d21 0 2024-03-16T10:44:51.968 INFO:tasks.workunit.client.0.smithi002.stdout:4/148: truncate d2/d6/f1e 1008337 0 2024-03-16T10:44:51.968 INFO:tasks.workunit.client.0.smithi002.stdout:4/149: readlink d2/dd/l11 0 2024-03-16T10:44:51.970 INFO:tasks.workunit.client.0.smithi002.stdout:0/169: creat d3/d28/f34 x:0 0 0 2024-03-16T10:44:51.973 INFO:tasks.workunit.client.0.smithi002.stdout:1/132: symlink d2/l2b 0 2024-03-16T10:44:52.021 INFO:tasks.workunit.client.0.smithi002.stdout:3/122: mkdir d1/dd/d11/d15/d2b 0 2024-03-16T10:44:52.021 INFO:tasks.workunit.client.0.smithi002.stdout:4/150: creat d2/dd/da/f25 x:0 0 0 2024-03-16T10:44:52.022 INFO:tasks.workunit.client.0.smithi002.stdout:1/133: rmdir d2/d3 39 2024-03-16T10:44:52.022 INFO:tasks.workunit.client.0.smithi002.stdout:9/106: getdents d5 0 2024-03-16T10:44:52.026 INFO:tasks.workunit.client.0.smithi002.stdout:9/107: mkdir d5/d9/d22 0 2024-03-16T10:44:52.028 INFO:tasks.workunit.client.0.smithi002.stdout:2/125: creat d7/dc/f20 x:0 0 0 2024-03-16T10:44:52.029 INFO:tasks.workunit.client.0.smithi002.stdout:1/134: mknod d2/c2c 0 2024-03-16T10:44:52.030 INFO:tasks.workunit.client.0.smithi002.stdout:2/126: chown d7/dc/f1c 421 1 2024-03-16T10:44:52.031 INFO:tasks.workunit.client.0.smithi002.stdout:7/109: dwrite d3/f11 [0,4194304] 0 2024-03-16T10:44:52.032 INFO:tasks.workunit.client.0.smithi002.stdout:3/123: dread d1/dd/d11/d15/f1b [0,4194304] 0 2024-03-16T10:44:52.033 INFO:tasks.workunit.client.0.smithi002.stdout:9/108: mkdir d5/d9/d18/d23 0 2024-03-16T10:44:52.037 INFO:tasks.workunit.client.0.smithi002.stdout:7/110: creat d3/f1b x:0 0 0 2024-03-16T10:44:52.038 INFO:tasks.workunit.client.0.smithi002.stdout:1/135: dwrite d2/f21 [0,4194304] 0 2024-03-16T10:44:52.039 INFO:tasks.workunit.client.0.smithi002.stdout:3/124: mkdir d1/dd/d11/d15/d1e/d2c 0 2024-03-16T10:44:52.040 INFO:tasks.workunit.client.0.smithi002.stdout:9/109: rmdir d5/d9 39 2024-03-16T10:44:52.042 INFO:tasks.workunit.client.0.smithi002.stdout:2/127: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:52.045 INFO:tasks.workunit.client.0.smithi002.stdout:7/111: mkdir d3/d4/d1c 0 2024-03-16T10:44:52.046 INFO:tasks.workunit.client.0.smithi002.stdout:7/112: write d3/f11 [897690,79465] 0 2024-03-16T10:44:52.048 INFO:tasks.workunit.client.0.smithi002.stdout:3/125: rmdir d1/dd 39 2024-03-16T10:44:52.053 INFO:tasks.workunit.client.0.smithi002.stdout:2/128: readlink d7/l9 0 2024-03-16T10:44:52.053 INFO:tasks.workunit.client.0.smithi002.stdout:1/136: symlink d2/d3/l2d 0 2024-03-16T10:44:52.056 INFO:tasks.workunit.client.0.smithi002.stdout:7/113: unlink d3/f16 0 2024-03-16T10:44:52.056 INFO:tasks.workunit.client.0.smithi002.stdout:7/114: fsync d3/f14 0 2024-03-16T10:44:52.065 INFO:tasks.workunit.client.0.smithi002.stdout:9/110: mkdir d5/d9/dc/d1e/d24 0 2024-03-16T10:44:52.066 INFO:tasks.workunit.client.0.smithi002.stdout:9/111: write d5/d9/dc/f11 [3759445,21467] 0 2024-03-16T10:44:52.066 INFO:tasks.workunit.client.0.smithi002.stdout:7/115: mkdir d3/d4/d1d 0 2024-03-16T10:44:52.067 INFO:tasks.workunit.client.0.smithi002.stdout:9/112: chown d5/d9/d22 3186911 1 2024-03-16T10:44:52.068 INFO:tasks.workunit.client.0.smithi002.stdout:0/170: write d3/d28/d2a/f2c [2459963,112224] 0 2024-03-16T10:44:52.071 INFO:tasks.workunit.client.0.smithi002.stdout:2/129: creat d7/dc/d1d/f21 x:0 0 0 2024-03-16T10:44:52.072 INFO:tasks.workunit.client.0.smithi002.stdout:4/151: truncate d2/d6/f10 2911841 0 2024-03-16T10:44:52.073 INFO:tasks.workunit.client.0.smithi002.stdout:1/137: dread d2/d3/f7 [0,4194304] 0 2024-03-16T10:44:52.074 INFO:tasks.workunit.client.0.smithi002.stdout:7/116: creat d3/d4/f1e x:0 0 0 2024-03-16T10:44:52.075 INFO:tasks.workunit.client.0.smithi002.stdout:9/113: dwrite d5/d9/fe [4194304,4194304] 0 2024-03-16T10:44:52.076 INFO:tasks.workunit.client.0.smithi002.stdout:7/117: read d3/fd [3416507,30293] 0 2024-03-16T10:44:52.077 INFO:tasks.workunit.client.0.smithi002.stdout:9/114: dread - d5/d9/dc/f16 zero size 2024-03-16T10:44:52.077 INFO:tasks.workunit.client.0.smithi002.stdout:8/135: sync 2024-03-16T10:44:52.077 INFO:tasks.workunit.client.0.smithi002.stdout:5/71: sync 2024-03-16T10:44:52.077 INFO:tasks.workunit.client.0.smithi002.stdout:6/118: sync 2024-03-16T10:44:52.080 INFO:tasks.workunit.client.0.smithi002.stdout:5/72: truncate d0/f6 1930695 0 2024-03-16T10:44:52.080 INFO:tasks.workunit.client.0.smithi002.stdout:9/115: readlink d5/l12 0 2024-03-16T10:44:52.080 INFO:tasks.workunit.client.0.smithi002.stdout:2/130: creat d7/db/d12/f22 x:0 0 0 2024-03-16T10:44:52.080 INFO:tasks.workunit.client.0.smithi002.stdout:1/138: symlink d2/d3/l2e 0 2024-03-16T10:44:52.082 INFO:tasks.workunit.client.0.smithi002.stdout:2/131: chown d7/dc/d1d 10 1 2024-03-16T10:44:52.084 INFO:tasks.workunit.client.0.smithi002.stdout:0/171: mknod d3/c35 0 2024-03-16T10:44:52.084 INFO:tasks.workunit.client.0.smithi002.stdout:1/139: write d2/d3/f1f [840211,108037] 0 2024-03-16T10:44:52.086 INFO:tasks.workunit.client.0.smithi002.stdout:4/152: rename d2/dd/da/l15 to d2/dd/d1c/l26 0 2024-03-16T10:44:52.087 INFO:tasks.workunit.client.0.smithi002.stdout:7/118: dwrite d3/f15 [0,4194304] 0 2024-03-16T10:44:52.088 INFO:tasks.workunit.client.0.smithi002.stdout:6/119: dread fc [0,4194304] 0 2024-03-16T10:44:52.089 INFO:tasks.workunit.client.0.smithi002.stdout:6/120: read - d15/d1b/f22 zero size 2024-03-16T10:44:52.091 INFO:tasks.workunit.client.0.smithi002.stdout:4/153: dread d2/dd/d1c/f1d [0,4194304] 0 2024-03-16T10:44:52.091 INFO:tasks.workunit.client.0.smithi002.stdout:6/121: chown d15/d1b/l1d 0 1 2024-03-16T10:44:52.091 INFO:tasks.workunit.client.0.smithi002.stdout:3/126: link d1/dd/d11/l18 d1/dd/d11/d15/d1e/d22/l2d 0 2024-03-16T10:44:52.092 INFO:tasks.workunit.client.0.smithi002.stdout:3/127: write d1/dd/d11/d15/f17 [412079,124291] 0 2024-03-16T10:44:52.096 INFO:tasks.workunit.client.0.smithi002.stdout:9/116: rename d5/d9/d18 to d5/d9/d25 0 2024-03-16T10:44:52.097 INFO:tasks.workunit.client.0.smithi002.stdout:9/117: truncate d5/f7 4827239 0 2024-03-16T10:44:52.098 INFO:tasks.workunit.client.0.smithi002.stdout:9/118: read d5/d9/dc/f11 [123714,94191] 0 2024-03-16T10:44:52.098 INFO:tasks.workunit.client.0.smithi002.stdout:7/119: mknod d3/d4/c1f 0 2024-03-16T10:44:52.099 INFO:tasks.workunit.client.0.smithi002.stdout:9/119: write d5/d9/dc/f20 [3490725,22393] 0 2024-03-16T10:44:52.102 INFO:tasks.workunit.client.0.smithi002.stdout:4/154: mknod d2/dd/d1c/c27 0 2024-03-16T10:44:52.103 INFO:tasks.workunit.client.0.smithi002.stdout:4/155: chown d2/dd/da/ff 3778657 1 2024-03-16T10:44:52.105 INFO:tasks.workunit.client.0.smithi002.stdout:5/73: mkdir d0/d16 0 2024-03-16T10:44:52.106 INFO:tasks.workunit.client.0.smithi002.stdout:3/128: creat d1/dd/d11/d15/f2e x:0 0 0 2024-03-16T10:44:52.107 INFO:tasks.workunit.client.0.smithi002.stdout:0/172: creat d3/d28/f36 x:0 0 0 2024-03-16T10:44:52.118 INFO:tasks.workunit.client.0.smithi002.stdout:4/156: creat d2/dd/da/f28 x:0 0 0 2024-03-16T10:44:52.119 INFO:tasks.workunit.client.0.smithi002.stdout:5/74: chown d0/ca 854 1 2024-03-16T10:44:52.121 INFO:tasks.workunit.client.0.smithi002.stdout:2/132: link d7/dc/d1d/l1e d7/dc/d1d/l23 0 2024-03-16T10:44:52.121 INFO:tasks.workunit.client.0.smithi002.stdout:0/173: read d3/f26 [2561894,78978] 0 2024-03-16T10:44:52.122 INFO:tasks.workunit.client.0.smithi002.stdout:7/120: rename d3/l12 to d3/l20 0 2024-03-16T10:44:52.123 INFO:tasks.workunit.client.0.smithi002.stdout:1/140: link d2/df/d12/c22 d2/d3/d1c/c2f 0 2024-03-16T10:44:52.124 INFO:tasks.workunit.client.0.smithi002.stdout:9/120: mknod d5/c26 0 2024-03-16T10:44:52.124 INFO:tasks.workunit.client.0.smithi002.stdout:3/129: dwrite d1/dd/d11/f14 [0,4194304] 0 2024-03-16T10:44:52.125 INFO:tasks.workunit.client.0.smithi002.stdout:3/130: fdatasync d1/dd/f13 0 2024-03-16T10:44:52.126 INFO:tasks.workunit.client.0.smithi002.stdout:3/131: chown d1/dd/d11/d15/d1e/d22 5390634 1 2024-03-16T10:44:52.127 INFO:tasks.workunit.client.0.smithi002.stdout:3/132: readlink d1/dd/d11/d15/d1e/l29 0 2024-03-16T10:44:52.129 INFO:tasks.workunit.client.0.smithi002.stdout:0/174: mknod d3/d28/c37 0 2024-03-16T10:44:52.131 INFO:tasks.workunit.client.0.smithi002.stdout:2/133: unlink d7/dc/d1d/f1f 0 2024-03-16T10:44:52.134 INFO:tasks.workunit.client.0.smithi002.stdout:7/121: mkdir d3/d4/d10/d21 0 2024-03-16T10:44:52.135 INFO:tasks.workunit.client.0.smithi002.stdout:1/141: write d2/f28 [5439124,84599] 0 2024-03-16T10:44:52.135 INFO:tasks.workunit.client.0.smithi002.stdout:6/122: link f8 d15/f25 0 2024-03-16T10:44:52.137 INFO:tasks.workunit.client.0.smithi002.stdout:9/121: mknod d5/d9/dc/c27 0 2024-03-16T10:44:52.140 INFO:tasks.workunit.client.0.smithi002.stdout:5/75: symlink d0/l17 0 2024-03-16T10:44:52.141 INFO:tasks.workunit.client.0.smithi002.stdout:5/76: readlink d0/l13 0 2024-03-16T10:44:52.141 INFO:tasks.workunit.client.0.smithi002.stdout:3/133: creat d1/dd/d11/d15/f2f x:0 0 0 2024-03-16T10:44:52.142 INFO:tasks.workunit.client.0.smithi002.stdout:5/77: read d0/f6 [44747,35794] 0 2024-03-16T10:44:52.143 INFO:tasks.workunit.client.0.smithi002.stdout:7/122: dwrite d3/f1a [0,4194304] 0 2024-03-16T10:44:52.144 INFO:tasks.workunit.client.0.smithi002.stdout:0/175: fsync d3/fc 0 2024-03-16T10:44:52.144 INFO:tasks.workunit.client.0.smithi002.stdout:7/123: stat d3/d4/d10/d21 0 2024-03-16T10:44:52.145 INFO:tasks.workunit.client.0.smithi002.stdout:4/157: rmdir d2 39 2024-03-16T10:44:52.147 INFO:tasks.workunit.client.0.smithi002.stdout:2/134: dread f4 [0,4194304] 0 2024-03-16T10:44:52.147 INFO:tasks.workunit.client.0.smithi002.stdout:1/142: mknod d2/df/d12/c30 0 2024-03-16T10:44:52.152 INFO:tasks.workunit.client.0.smithi002.stdout:3/134: creat d1/dd/d11/f30 x:0 0 0 2024-03-16T10:44:52.153 INFO:tasks.workunit.client.0.smithi002.stdout:5/78: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:52.154 INFO:tasks.workunit.client.0.smithi002.stdout:5/79: truncate d0/f6 4959675 0 2024-03-16T10:44:52.155 INFO:tasks.workunit.client.0.smithi002.stdout:0/176: fdatasync d3/f2d 0 2024-03-16T10:44:52.155 INFO:tasks.workunit.client.0.smithi002.stdout:5/80: read d0/f6 [2546302,34908] 0 2024-03-16T10:44:52.156 INFO:tasks.workunit.client.0.smithi002.stdout:5/81: chown d0/l15 246584853 1 2024-03-16T10:44:52.157 INFO:tasks.workunit.client.0.smithi002.stdout:4/158: write d2/dd/da/f25 [900523,9780] 0 2024-03-16T10:44:52.157 INFO:tasks.workunit.client.0.smithi002.stdout:5/82: stat d0/l11 0 2024-03-16T10:44:52.159 INFO:tasks.workunit.client.0.smithi002.stdout:9/122: getdents d5/d9/d25/d23 0 2024-03-16T10:44:52.162 INFO:tasks.workunit.client.0.smithi002.stdout:2/135: mknod d7/dc/d1d/c24 0 2024-03-16T10:44:52.162 INFO:tasks.workunit.client.0.smithi002.stdout:5/83: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:52.163 INFO:tasks.workunit.client.0.smithi002.stdout:1/143: mknod d2/d3/d1c/c31 0 2024-03-16T10:44:52.168 INFO:tasks.workunit.client.0.smithi002.stdout:9/123: dwrite d5/d9/dc/f1f [0,4194304] 0 2024-03-16T10:44:52.169 INFO:tasks.workunit.client.0.smithi002.stdout:7/124: symlink d3/d4/d1c/l22 0 2024-03-16T10:44:52.171 INFO:tasks.workunit.client.0.smithi002.stdout:4/159: chown d2/c24 0 1 2024-03-16T10:44:52.172 INFO:tasks.workunit.client.0.smithi002.stdout:6/123: rename d15/d1b/l1d to d15/l26 0 2024-03-16T10:44:52.178 INFO:tasks.workunit.client.0.smithi002.stdout:5/84: stat d0/c12 0 2024-03-16T10:44:52.178 INFO:tasks.workunit.client.0.smithi002.stdout:3/135: creat d1/dd/d28/f31 x:0 0 0 2024-03-16T10:44:52.182 INFO:tasks.workunit.client.0.smithi002.stdout:9/124: dwrite d5/d9/f1a [0,4194304] 0 2024-03-16T10:44:52.187 INFO:tasks.workunit.client.0.smithi002.stdout:4/160: creat d2/dd/f29 x:0 0 0 2024-03-16T10:44:52.188 INFO:tasks.workunit.client.0.smithi002.stdout:7/125: dwrite d3/f14 [4194304,4194304] 0 2024-03-16T10:44:52.192 INFO:tasks.workunit.client.0.smithi002.stdout:1/144: link d2/df/d12/l23 d2/df/d12/d29/l32 0 2024-03-16T10:44:52.192 INFO:tasks.workunit.client.0.smithi002.stdout:6/124: rmdir d15/d1b 39 2024-03-16T10:44:52.192 INFO:tasks.workunit.client.0.smithi002.stdout:3/136: mknod d1/dd/d11/d15/d2b/c32 0 2024-03-16T10:44:52.192 INFO:tasks.workunit.client.0.smithi002.stdout:1/145: dread - d2/df/f27 zero size 2024-03-16T10:44:52.194 INFO:tasks.workunit.client.0.smithi002.stdout:3/137: dread - d1/f16 zero size 2024-03-16T10:44:52.194 INFO:tasks.workunit.client.0.smithi002.stdout:9/125: mkdir d5/d9/dc/d28 0 2024-03-16T10:44:52.196 INFO:tasks.workunit.client.0.smithi002.stdout:2/136: dwrite f6 [4194304,4194304] 0 2024-03-16T10:44:52.196 INFO:tasks.workunit.client.0.smithi002.stdout:5/85: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:52.196 INFO:tasks.workunit.client.0.smithi002.stdout:4/161: symlink d2/dd/da/l2a 0 2024-03-16T10:44:52.197 INFO:tasks.workunit.client.0.smithi002.stdout:3/138: readlink d1/l7 0 2024-03-16T10:44:52.197 INFO:tasks.workunit.client.0.smithi002.stdout:0/177: write d3/de/f15 [1176520,114911] 0 2024-03-16T10:44:52.198 INFO:tasks.workunit.client.0.smithi002.stdout:2/137: dread - d7/dc/d1d/f21 zero size 2024-03-16T10:44:52.198 INFO:tasks.workunit.client.0.smithi002.stdout:1/146: unlink d2/f21 0 2024-03-16T10:44:52.203 INFO:tasks.workunit.client.0.smithi002.stdout:4/162: symlink d2/dd/l2b 0 2024-03-16T10:44:52.204 INFO:tasks.workunit.client.0.smithi002.stdout:4/163: fdatasync d2/d6/f1e 0 2024-03-16T10:44:52.205 INFO:tasks.workunit.client.0.smithi002.stdout:4/164: readlink d2/dd/l2b 0 2024-03-16T10:44:52.208 INFO:tasks.workunit.client.0.smithi002.stdout:7/126: creat d3/d4/d1d/f23 x:0 0 0 2024-03-16T10:44:52.210 INFO:tasks.workunit.client.0.smithi002.stdout:4/165: dread d2/dd/da/f25 [0,4194304] 0 2024-03-16T10:44:52.211 INFO:tasks.workunit.client.0.smithi002.stdout:8/136: sync 2024-03-16T10:44:52.212 INFO:tasks.workunit.client.0.smithi002.stdout:8/137: chown d7/fd 52 1 2024-03-16T10:44:52.214 INFO:tasks.workunit.client.0.smithi002.stdout:5/86: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:52.215 INFO:tasks.workunit.client.0.smithi002.stdout:3/139: rename d1/dd/d11/d15/f2f to d1/dd/d28/f33 0 2024-03-16T10:44:52.216 INFO:tasks.workunit.client.0.smithi002.stdout:4/166: rename d2/dd to d2/dd/da/d2c 22 2024-03-16T10:44:52.218 INFO:tasks.workunit.client.0.smithi002.stdout:6/125: dwrite d15/f25 [0,4194304] 0 2024-03-16T10:44:52.221 INFO:tasks.workunit.client.0.smithi002.stdout:0/178: unlink d3/f1a 0 2024-03-16T10:44:52.226 INFO:tasks.workunit.client.0.smithi002.stdout:9/126: creat d5/f29 x:0 0 0 2024-03-16T10:44:52.226 INFO:tasks.workunit.client.0.smithi002.stdout:0/179: dread d3/de/f15 [0,4194304] 0 2024-03-16T10:44:52.227 INFO:tasks.workunit.client.0.smithi002.stdout:8/138: unlink d7/c12 0 2024-03-16T10:44:52.230 INFO:tasks.workunit.client.0.smithi002.stdout:4/167: chown d2/dd/d1c/l26 21 1 2024-03-16T10:44:52.231 INFO:tasks.workunit.client.0.smithi002.stdout:6/126: rmdir d15 39 2024-03-16T10:44:52.232 INFO:tasks.workunit.client.0.smithi002.stdout:5/87: fdatasync d0/f6 0 2024-03-16T10:44:52.232 INFO:tasks.workunit.client.0.smithi002.stdout:6/127: fsync fd 0 2024-03-16T10:44:52.234 INFO:tasks.workunit.client.0.smithi002.stdout:5/88: dread d0/f6 [4194304,4194304] 0 2024-03-16T10:44:52.234 INFO:tasks.workunit.client.0.smithi002.stdout:2/138: creat d7/dc/f25 x:0 0 0 2024-03-16T10:44:52.237 INFO:tasks.workunit.client.0.smithi002.stdout:6/128: dread fc [0,4194304] 0 2024-03-16T10:44:52.238 INFO:tasks.workunit.client.0.smithi002.stdout:0/180: symlink d3/d28/l38 0 2024-03-16T10:44:52.239 INFO:tasks.workunit.client.0.smithi002.stdout:9/127: rename d5/d9/dc/l17 to d5/d9/dc/l2a 0 2024-03-16T10:44:52.240 INFO:tasks.workunit.client.0.smithi002.stdout:8/139: mknod d7/d9/d17/d20/c2b 0 2024-03-16T10:44:52.243 INFO:tasks.workunit.client.0.smithi002.stdout:4/168: creat d2/d6/f2d x:0 0 0 2024-03-16T10:44:52.246 INFO:tasks.workunit.client.0.smithi002.stdout:4/169: truncate d2/d6/f22 615959 0 2024-03-16T10:44:52.246 INFO:tasks.workunit.client.0.smithi002.stdout:9/128: dwrite d5/fd [0,4194304] 0 2024-03-16T10:44:52.249 INFO:tasks.workunit.client.0.smithi002.stdout:2/139: creat d7/dc/d1d/f26 x:0 0 0 2024-03-16T10:44:52.250 INFO:tasks.workunit.client.0.smithi002.stdout:1/147: getdents d2/d3/d1c 0 2024-03-16T10:44:52.253 INFO:tasks.workunit.client.0.smithi002.stdout:9/129: dread d5/d9/f13 [0,4194304] 0 2024-03-16T10:44:52.253 INFO:tasks.workunit.client.0.smithi002.stdout:3/140: rename d1/dd/d11/c1f to d1/c34 0 2024-03-16T10:44:52.254 INFO:tasks.workunit.client.0.smithi002.stdout:9/130: fdatasync d5/f7 0 2024-03-16T10:44:52.255 INFO:tasks.workunit.client.0.smithi002.stdout:7/127: link d3/l13 d3/l24 0 2024-03-16T10:44:52.256 INFO:tasks.workunit.client.0.smithi002.stdout:7/128: dread - d3/f1b zero size 2024-03-16T10:44:52.256 INFO:tasks.workunit.client.0.smithi002.stdout:4/170: mkdir d2/dd/da/d2e 0 2024-03-16T10:44:52.259 INFO:tasks.workunit.client.0.smithi002.stdout:6/129: chown d15/l26 1213 1 2024-03-16T10:44:52.260 INFO:tasks.workunit.client.0.smithi002.stdout:2/140: truncate f4 339212 0 2024-03-16T10:44:52.263 INFO:tasks.workunit.client.0.smithi002.stdout:0/181: rename d3/d5/cf to d3/d5/c39 0 2024-03-16T10:44:52.266 INFO:tasks.workunit.client.0.smithi002.stdout:1/148: dwrite d2/d3/f7 [4194304,4194304] 0 2024-03-16T10:44:52.267 INFO:tasks.workunit.client.0.smithi002.stdout:3/141: write d1/dd/d28/f33 [207246,56926] 0 2024-03-16T10:44:52.267 INFO:tasks.workunit.client.0.smithi002.stdout:9/131: mkdir d5/d9/dc/d2b 0 2024-03-16T10:44:52.268 INFO:tasks.workunit.client.0.smithi002.stdout:9/132: fsync d5/d9/dc/f1f 0 2024-03-16T10:44:52.269 INFO:tasks.workunit.client.0.smithi002.stdout:7/129: mknod d3/d4/d10/c25 0 2024-03-16T10:44:52.272 INFO:tasks.workunit.client.0.smithi002.stdout:3/142: chown d1/dd/d11/d15/d1e/d2c 2593 1 2024-03-16T10:44:52.273 INFO:tasks.workunit.client.0.smithi002.stdout:4/171: chown d2/d6/f10 246221 1 2024-03-16T10:44:52.273 INFO:tasks.workunit.client.0.smithi002.stdout:7/130: truncate d3/d4/d1d/f23 59825 0 2024-03-16T10:44:52.274 INFO:tasks.workunit.client.0.smithi002.stdout:5/89: creat d0/f18 x:0 0 0 2024-03-16T10:44:52.275 INFO:tasks.workunit.client.0.smithi002.stdout:4/172: stat d2/d6 0 2024-03-16T10:44:52.277 INFO:tasks.workunit.client.0.smithi002.stdout:4/173: fsync d2/d6/f2d 0 2024-03-16T10:44:52.281 INFO:tasks.workunit.client.0.smithi002.stdout:7/131: mknod d3/d4/d1c/c26 0 2024-03-16T10:44:52.283 INFO:tasks.workunit.client.0.smithi002.stdout:8/140: dwrite d7/f14 [0,4194304] 0 2024-03-16T10:44:52.286 INFO:tasks.workunit.client.0.smithi002.stdout:8/141: write d7/f26 [119898,36459] 0 2024-03-16T10:44:52.287 INFO:tasks.workunit.client.0.smithi002.stdout:4/174: creat d2/dd/d1c/f2f x:0 0 0 2024-03-16T10:44:52.289 INFO:tasks.workunit.client.0.smithi002.stdout:6/130: dwrite f11 [0,4194304] 0 2024-03-16T10:44:52.292 INFO:tasks.workunit.client.0.smithi002.stdout:5/90: creat d0/d16/f19 x:0 0 0 2024-03-16T10:44:52.295 INFO:tasks.workunit.client.0.smithi002.stdout:7/132: dwrite d3/d4/d1d/f23 [0,4194304] 0 2024-03-16T10:44:52.296 INFO:tasks.workunit.client.0.smithi002.stdout:7/133: chown d3/f1b 1805 1 2024-03-16T10:44:52.299 INFO:tasks.workunit.client.0.smithi002.stdout:0/182: link d3/d5/f19 d3/f3a 0 2024-03-16T10:44:52.302 INFO:tasks.workunit.client.0.smithi002.stdout:1/149: creat d2/df/f33 x:0 0 0 2024-03-16T10:44:52.302 INFO:tasks.workunit.client.0.smithi002.stdout:9/133: write d5/d9/dc/f11 [2839259,105424] 0 2024-03-16T10:44:52.302 INFO:tasks.workunit.client.0.smithi002.stdout:4/175: rmdir d2/dd/da 39 2024-03-16T10:44:52.303 INFO:tasks.workunit.client.0.smithi002.stdout:9/134: readlink d5/d9/d25/l1b 0 2024-03-16T10:44:52.305 INFO:tasks.workunit.client.0.smithi002.stdout:3/143: rename d1/dd/d11/f14 to d1/f35 0 2024-03-16T10:44:52.310 INFO:tasks.workunit.client.0.smithi002.stdout:2/141: getdents d7/db 0 2024-03-16T10:44:52.311 INFO:tasks.workunit.client.0.smithi002.stdout:3/144: read - d1/f16 zero size 2024-03-16T10:44:52.311 INFO:tasks.workunit.client.0.smithi002.stdout:0/183: creat d3/d28/f3b x:0 0 0 2024-03-16T10:44:52.316 INFO:tasks.workunit.client.0.smithi002.stdout:0/184: chown d3/d5/f2f 19000 1 2024-03-16T10:44:52.317 INFO:tasks.workunit.client.0.smithi002.stdout:8/142: creat d7/d9/d25/f2c x:0 0 0 2024-03-16T10:44:52.319 INFO:tasks.workunit.client.0.smithi002.stdout:4/176: dwrite d2/dd/d1c/f1d [0,4194304] 0 2024-03-16T10:44:52.320 INFO:tasks.workunit.client.0.smithi002.stdout:9/135: dwrite d5/fd [0,4194304] 0 2024-03-16T10:44:52.320 INFO:tasks.workunit.client.0.smithi002.stdout:8/143: chown d7/f14 8 1 2024-03-16T10:44:52.320 INFO:tasks.workunit.client.0.smithi002.stdout:5/91: mknod d0/c1a 0 2024-03-16T10:44:52.323 INFO:tasks.workunit.client.0.smithi002.stdout:4/177: stat d2/d6/c1f 0 2024-03-16T10:44:52.330 INFO:tasks.workunit.client.0.smithi002.stdout:1/150: link d2/df/f33 d2/df/d12/d29/f34 0 2024-03-16T10:44:52.332 INFO:tasks.workunit.client.0.smithi002.stdout:5/92: rename d0/l15 to d0/d16/l1b 0 2024-03-16T10:44:52.332 INFO:tasks.workunit.client.0.smithi002.stdout:6/131: link lf d15/l27 0 2024-03-16T10:44:52.332 INFO:tasks.workunit.client.0.smithi002.stdout:8/144: fdatasync d7/fd 0 2024-03-16T10:44:52.333 INFO:tasks.workunit.client.0.smithi002.stdout:1/151: rename d2/df/d12 to d2/df/d12/d35 22 2024-03-16T10:44:52.335 INFO:tasks.workunit.client.0.smithi002.stdout:9/136: symlink d5/l2c 0 2024-03-16T10:44:52.336 INFO:tasks.workunit.client.0.smithi002.stdout:3/145: dwrite d1/dd/d11/d15/f1b [4194304,4194304] 0 2024-03-16T10:44:52.336 INFO:tasks.workunit.client.0.smithi002.stdout:8/145: write d7/fc [4128524,35278] 0 2024-03-16T10:44:52.338 INFO:tasks.workunit.client.0.smithi002.stdout:9/137: chown d5/d9/dc/c15 36308613 1 2024-03-16T10:44:52.345 INFO:tasks.workunit.client.0.smithi002.stdout:9/138: chown d5/d9/fe 42756 1 2024-03-16T10:44:52.346 INFO:tasks.workunit.client.0.smithi002.stdout:7/134: truncate d3/f11 60974 0 2024-03-16T10:44:52.350 INFO:tasks.workunit.client.0.smithi002.stdout:9/139: write d5/d9/dc/f1f [3102740,127430] 0 2024-03-16T10:44:52.351 INFO:tasks.workunit.client.0.smithi002.stdout:4/178: dwrite d2/dd/da/ff [0,4194304] 0 2024-03-16T10:44:52.354 INFO:tasks.workunit.client.0.smithi002.stdout:9/140: write d5/fd [3517357,126599] 0 2024-03-16T10:44:52.354 INFO:tasks.workunit.client.0.smithi002.stdout:4/179: read - d2/f20 zero size 2024-03-16T10:44:52.355 INFO:tasks.workunit.client.0.smithi002.stdout:3/146: rename d1/dd/d11/f27 to d1/dd/d28/f36 0 2024-03-16T10:44:52.356 INFO:tasks.workunit.client.0.smithi002.stdout:6/132: creat d15/d1b/f28 x:0 0 0 2024-03-16T10:44:52.358 INFO:tasks.workunit.client.0.smithi002.stdout:7/135: creat d3/f27 x:0 0 0 2024-03-16T10:44:52.360 INFO:tasks.workunit.client.0.smithi002.stdout:8/146: dwrite f4 [0,4194304] 0 2024-03-16T10:44:52.363 INFO:tasks.workunit.client.0.smithi002.stdout:9/141: stat d5/d9/dc/f20 0 2024-03-16T10:44:52.370 INFO:tasks.workunit.client.0.smithi002.stdout:4/180: fsync d2/fe 0 2024-03-16T10:44:52.370 INFO:tasks.workunit.client.0.smithi002.stdout:6/133: readlink lf 0 2024-03-16T10:44:52.371 INFO:tasks.workunit.client.0.smithi002.stdout:8/147: dread d7/f14 [0,4194304] 0 2024-03-16T10:44:52.374 INFO:tasks.workunit.client.0.smithi002.stdout:4/181: fdatasync d2/d6/f17 0 2024-03-16T10:44:52.374 INFO:tasks.workunit.client.0.smithi002.stdout:8/148: chown d7/fd 1 1 2024-03-16T10:44:52.374 INFO:tasks.workunit.client.0.smithi002.stdout:5/93: link d0/ca d0/c1c 0 2024-03-16T10:44:52.375 INFO:tasks.workunit.client.0.smithi002.stdout:8/149: stat l3 0 2024-03-16T10:44:52.378 INFO:tasks.workunit.client.0.smithi002.stdout:2/142: write f4 [702435,103154] 0 2024-03-16T10:44:52.382 INFO:tasks.workunit.client.0.smithi002.stdout:4/182: write d2/dd/d1c/f2f [734307,35218] 0 2024-03-16T10:44:52.383 INFO:tasks.workunit.client.0.smithi002.stdout:1/152: dwrite d2/df/f33 [0,4194304] 0 2024-03-16T10:44:52.383 INFO:tasks.workunit.client.0.smithi002.stdout:9/142: dwrite d5/d9/f13 [0,4194304] 0 2024-03-16T10:44:52.384 INFO:tasks.workunit.client.0.smithi002.stdout:7/136: rmdir d3/d4/d10/d21 0 2024-03-16T10:44:52.385 INFO:tasks.workunit.client.0.smithi002.stdout:1/153: readlink d2/d3/l2d 0 2024-03-16T10:44:52.387 INFO:tasks.workunit.client.0.smithi002.stdout:6/134: rmdir d15/d1b 39 2024-03-16T10:44:52.389 INFO:tasks.workunit.client.0.smithi002.stdout:8/150: creat d7/d9/d17/f2d x:0 0 0 2024-03-16T10:44:52.389 INFO:tasks.workunit.client.0.smithi002.stdout:4/183: symlink d2/d6/l30 0 2024-03-16T10:44:52.391 INFO:tasks.workunit.client.0.smithi002.stdout:6/135: truncate fd 1140036 0 2024-03-16T10:44:52.391 INFO:tasks.workunit.client.0.smithi002.stdout:2/143: unlink d7/dc/d1d/l23 0 2024-03-16T10:44:52.395 INFO:tasks.workunit.client.0.smithi002.stdout:5/94: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:52.398 INFO:tasks.workunit.client.0.smithi002.stdout:6/136: read - d15/d1b/f1f zero size 2024-03-16T10:44:52.399 INFO:tasks.workunit.client.0.smithi002.stdout:9/143: creat d5/d9/dc/d28/f2d x:0 0 0 2024-03-16T10:44:52.400 INFO:tasks.workunit.client.0.smithi002.stdout:4/184: mknod d2/d6/c31 0 2024-03-16T10:44:52.401 INFO:tasks.workunit.client.0.smithi002.stdout:7/137: dwrite d3/f15 [0,4194304] 0 2024-03-16T10:44:52.401 INFO:tasks.workunit.client.0.smithi002.stdout:1/154: dwrite d2/f13 [4194304,4194304] 0 2024-03-16T10:44:52.402 INFO:tasks.workunit.client.0.smithi002.stdout:2/144: mkdir d7/d27 0 2024-03-16T10:44:52.403 INFO:tasks.workunit.client.0.smithi002.stdout:6/137: rmdir d15/d1b 39 2024-03-16T10:44:52.405 INFO:tasks.workunit.client.0.smithi002.stdout:5/95: mkdir d0/d16/d1d 0 2024-03-16T10:44:52.406 INFO:tasks.workunit.client.0.smithi002.stdout:8/151: mkdir d7/d9/d17/d23/d2e 0 2024-03-16T10:44:52.407 INFO:tasks.workunit.client.0.smithi002.stdout:5/96: fsync d0/f6 0 2024-03-16T10:44:52.407 INFO:tasks.workunit.client.0.smithi002.stdout:4/185: mkdir d2/d6/d32 0 2024-03-16T10:44:52.408 INFO:tasks.workunit.client.0.smithi002.stdout:5/97: dread - d0/f18 zero size 2024-03-16T10:44:52.412 INFO:tasks.workunit.client.0.smithi002.stdout:5/98: write d0/f18 [452288,95458] 0 2024-03-16T10:44:52.412 INFO:tasks.workunit.client.0.smithi002.stdout:1/155: symlink d2/d3/l36 0 2024-03-16T10:44:52.413 INFO:tasks.workunit.client.0.smithi002.stdout:1/156: readlink d2/d3/ld 0 2024-03-16T10:44:52.413 INFO:tasks.workunit.client.0.smithi002.stdout:6/138: rmdir d15 39 2024-03-16T10:44:52.413 INFO:tasks.workunit.client.0.smithi002.stdout:3/147: truncate d1/dd/d11/f20 6904121 0 2024-03-16T10:44:52.415 INFO:tasks.workunit.client.0.smithi002.stdout:5/99: chown d0/l13 161010 1 2024-03-16T10:44:52.416 INFO:tasks.workunit.client.0.smithi002.stdout:1/157: write d2/df/f27 [587007,29481] 0 2024-03-16T10:44:52.417 INFO:tasks.workunit.client.0.smithi002.stdout:8/152: creat d7/d9/f2f x:0 0 0 2024-03-16T10:44:52.418 INFO:tasks.workunit.client.0.smithi002.stdout:2/145: mkdir d7/dc/d28 0 2024-03-16T10:44:52.419 INFO:tasks.workunit.client.0.smithi002.stdout:1/158: chown d2/c2c 0 1 2024-03-16T10:44:52.420 INFO:tasks.workunit.client.0.smithi002.stdout:8/153: truncate d7/d9/d17/f1f 257391 0 2024-03-16T10:44:52.421 INFO:tasks.workunit.client.0.smithi002.stdout:1/159: chown d2/d3/d1c/c31 23 1 2024-03-16T10:44:52.421 INFO:tasks.workunit.client.0.smithi002.stdout:3/148: rename d1/dd/d11/d15/d2b to d1/dd/d28/d37 0 2024-03-16T10:44:52.424 INFO:tasks.workunit.client.0.smithi002.stdout:8/154: creat d7/d13/f30 x:0 0 0 2024-03-16T10:44:52.424 INFO:tasks.workunit.client.0.smithi002.stdout:1/160: chown d2/df/d12/c1a 28897170 1 2024-03-16T10:44:52.425 INFO:tasks.workunit.client.0.smithi002.stdout:4/186: rename d2/dd to d2/d33 0 2024-03-16T10:44:52.428 INFO:tasks.workunit.client.0.smithi002.stdout:2/146: symlink d7/l29 0 2024-03-16T10:44:52.431 INFO:tasks.workunit.client.0.smithi002.stdout:8/155: dwrite f4 [0,4194304] 0 2024-03-16T10:44:52.433 INFO:tasks.workunit.client.0.smithi002.stdout:1/161: creat d2/df/d12/f37 x:0 0 0 2024-03-16T10:44:52.436 INFO:tasks.workunit.client.0.smithi002.stdout:3/149: symlink d1/dd/d11/d15/d1e/d2c/l38 0 2024-03-16T10:44:52.436 INFO:tasks.workunit.client.0.smithi002.stdout:6/139: creat d15/f29 x:0 0 0 2024-03-16T10:44:52.438 INFO:tasks.workunit.client.0.smithi002.stdout:2/147: creat d7/db/f2a x:0 0 0 2024-03-16T10:44:52.443 INFO:tasks.workunit.client.0.smithi002.stdout:5/100: getdents d0/d16 0 2024-03-16T10:44:52.444 INFO:tasks.workunit.client.0.smithi002.stdout:1/162: symlink d2/d3/d1c/l38 0 2024-03-16T10:44:52.445 INFO:tasks.workunit.client.0.smithi002.stdout:1/163: readlink d2/d3/d1c/l38 0 2024-03-16T10:44:52.451 INFO:tasks.workunit.client.0.smithi002.stdout:4/187: creat d2/f34 x:0 0 0 2024-03-16T10:44:52.452 INFO:tasks.workunit.client.0.smithi002.stdout:4/188: chown d2/d33/da/ff 6687 1 2024-03-16T10:44:52.452 INFO:tasks.workunit.client.0.smithi002.stdout:4/189: chown d2/d6 367744 1 2024-03-16T10:44:52.454 INFO:tasks.workunit.client.0.smithi002.stdout:2/148: dread f6 [0,4194304] 0 2024-03-16T10:44:52.455 INFO:tasks.workunit.client.0.smithi002.stdout:6/140: creat d15/d1b/f2a x:0 0 0 2024-03-16T10:44:52.455 INFO:tasks.workunit.client.0.smithi002.stdout:2/149: write d7/f8 [2563786,6704] 0 2024-03-16T10:44:52.458 INFO:tasks.workunit.client.0.smithi002.stdout:2/150: dread d7/dc/f1a [0,4194304] 0 2024-03-16T10:44:52.458 INFO:tasks.workunit.client.0.smithi002.stdout:1/164: rename d2/d3/f19 to d2/d3/f39 0 2024-03-16T10:44:52.460 INFO:tasks.workunit.client.0.smithi002.stdout:8/156: creat d7/f31 x:0 0 0 2024-03-16T10:44:52.460 INFO:tasks.workunit.client.0.smithi002.stdout:8/157: fsync d7/f21 0 2024-03-16T10:44:52.465 INFO:tasks.workunit.client.0.smithi002.stdout:5/101: truncate d0/f6 657594 0 2024-03-16T10:44:52.466 INFO:tasks.workunit.client.0.smithi002.stdout:8/158: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:52.467 INFO:tasks.workunit.client.0.smithi002.stdout:8/159: dread - d7/d9/d17/f2d zero size 2024-03-16T10:44:52.470 INFO:tasks.workunit.client.0.smithi002.stdout:5/102: dread d0/f18 [0,4194304] 0 2024-03-16T10:44:52.471 INFO:tasks.workunit.client.0.smithi002.stdout:0/185: sync 2024-03-16T10:44:52.472 INFO:tasks.workunit.client.0.smithi002.stdout:2/151: rmdir d7/db 39 2024-03-16T10:44:52.473 INFO:tasks.workunit.client.0.smithi002.stdout:5/103: truncate d0/f18 1562490 0 2024-03-16T10:44:52.474 INFO:tasks.workunit.client.0.smithi002.stdout:1/165: symlink d2/df/d12/l3a 0 2024-03-16T10:44:52.474 INFO:tasks.workunit.client.0.smithi002.stdout:6/141: dwrite d15/f19 [0,4194304] 0 2024-03-16T10:44:52.475 INFO:tasks.workunit.client.0.smithi002.stdout:6/142: fsync d15/f18 0 2024-03-16T10:44:52.479 INFO:tasks.workunit.client.0.smithi002.stdout:4/190: mkdir d2/d33/d35 0 2024-03-16T10:44:52.481 INFO:tasks.workunit.client.0.smithi002.stdout:1/166: dwrite d2/f5 [0,4194304] 0 2024-03-16T10:44:52.539 INFO:tasks.workunit.client.0.smithi002.stdout:8/160: rmdir d7/d9/d25 39 2024-03-16T10:44:52.543 INFO:tasks.workunit.client.0.smithi002.stdout:0/186: rename d3/f26 to d3/d27/f3c 0 2024-03-16T10:44:52.546 INFO:tasks.workunit.client.0.smithi002.stdout:1/167: dread d2/d3/f39 [0,4194304] 0 2024-03-16T10:44:52.547 INFO:tasks.workunit.client.0.smithi002.stdout:8/161: dwrite d7/f26 [0,4194304] 0 2024-03-16T10:44:52.548 INFO:tasks.workunit.client.0.smithi002.stdout:8/162: truncate d7/f31 662733 0 2024-03-16T10:44:52.548 INFO:tasks.workunit.client.0.smithi002.stdout:2/152: creat d7/f2b x:0 0 0 2024-03-16T10:44:52.549 INFO:tasks.workunit.client.0.smithi002.stdout:2/153: chown d7/f2b 8 1 2024-03-16T10:44:52.549 INFO:tasks.workunit.client.0.smithi002.stdout:2/154: dread - d7/f2b zero size 2024-03-16T10:44:52.550 INFO:tasks.workunit.client.0.smithi002.stdout:8/163: dread d7/f31 [0,4194304] 0 2024-03-16T10:44:52.551 INFO:tasks.workunit.client.0.smithi002.stdout:0/187: mkdir d3/d28/d3d 0 2024-03-16T10:44:52.551 INFO:tasks.workunit.client.0.smithi002.stdout:8/164: dread - d7/d13/f30 zero size 2024-03-16T10:44:52.552 INFO:tasks.workunit.client.0.smithi002.stdout:8/165: stat f4 0 2024-03-16T10:44:52.556 INFO:tasks.workunit.client.0.smithi002.stdout:2/155: rename d7/l29 to d7/dc/d28/l2c 0 2024-03-16T10:44:52.557 INFO:tasks.workunit.client.0.smithi002.stdout:2/156: fdatasync d7/dc/f20 0 2024-03-16T10:44:52.558 INFO:tasks.workunit.client.0.smithi002.stdout:2/157: write d7/f2b [722790,35227] 0 2024-03-16T10:44:52.560 INFO:tasks.workunit.client.0.smithi002.stdout:0/188: dwrite d3/de/f13 [4194304,4194304] 0 2024-03-16T10:44:52.562 INFO:tasks.workunit.client.0.smithi002.stdout:2/158: dread f4 [0,4194304] 0 2024-03-16T10:44:52.564 INFO:tasks.workunit.client.0.smithi002.stdout:2/159: write d7/dc/f1c [229710,2512] 0 2024-03-16T10:44:52.565 INFO:tasks.workunit.client.0.smithi002.stdout:5/104: write d0/f6 [196114,58918] 0 2024-03-16T10:44:52.566 INFO:tasks.workunit.client.0.smithi002.stdout:6/143: truncate d15/f1c 192750 0 2024-03-16T10:44:52.566 INFO:tasks.workunit.client.0.smithi002.stdout:6/144: stat f11 0 2024-03-16T10:44:52.570 INFO:tasks.workunit.client.0.smithi002.stdout:2/160: readlink d7/db/d12/l14 0 2024-03-16T10:44:52.571 INFO:tasks.workunit.client.0.smithi002.stdout:2/161: write d7/dc/f20 [790636,102591] 0 2024-03-16T10:44:52.573 INFO:tasks.workunit.client.0.smithi002.stdout:5/105: chown d0/c1c 53128671 1 2024-03-16T10:44:52.573 INFO:tasks.workunit.client.0.smithi002.stdout:1/168: truncate d2/f28 5649367 0 2024-03-16T10:44:52.574 INFO:tasks.workunit.client.0.smithi002.stdout:1/169: readlink d2/d3/l36 0 2024-03-16T10:44:52.575 INFO:tasks.workunit.client.0.smithi002.stdout:8/166: creat d7/d9/d25/f32 x:0 0 0 2024-03-16T10:44:52.576 INFO:tasks.workunit.client.0.smithi002.stdout:6/145: mkdir d15/d1b/d2b 0 2024-03-16T10:44:52.576 INFO:tasks.workunit.client.0.smithi002.stdout:2/162: dread d7/db/d12/f11 [0,4194304] 0 2024-03-16T10:44:52.576 INFO:tasks.workunit.client.0.smithi002.stdout:6/146: read - d15/f24 zero size 2024-03-16T10:44:52.578 INFO:tasks.workunit.client.0.smithi002.stdout:5/106: symlink d0/d16/l1e 0 2024-03-16T10:44:52.579 INFO:tasks.workunit.client.0.smithi002.stdout:8/167: readlink d7/d9/d17/l2a 0 2024-03-16T10:44:52.581 INFO:tasks.workunit.client.0.smithi002.stdout:6/147: mkdir d15/d1b/d2c 0 2024-03-16T10:44:52.583 INFO:tasks.workunit.client.0.smithi002.stdout:2/163: dread f4 [0,4194304] 0 2024-03-16T10:44:52.584 INFO:tasks.workunit.client.0.smithi002.stdout:5/107: mkdir d0/d1f 0 2024-03-16T10:44:52.584 INFO:tasks.workunit.client.0.smithi002.stdout:1/170: link d2/df/c11 d2/d3/c3b 0 2024-03-16T10:44:52.591 INFO:tasks.workunit.client.0.smithi002.stdout:5/108: mkdir d0/d16/d1d/d20 0 2024-03-16T10:44:52.592 INFO:tasks.workunit.client.0.smithi002.stdout:2/164: write f6 [2486432,53284] 0 2024-03-16T10:44:52.594 INFO:tasks.workunit.client.0.smithi002.stdout:5/109: write d0/d16/f19 [887491,82133] 0 2024-03-16T10:44:52.597 INFO:tasks.workunit.client.0.smithi002.stdout:8/168: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:52.601 INFO:tasks.workunit.client.0.smithi002.stdout:6/148: dwrite f9 [4194304,4194304] 0 2024-03-16T10:44:52.603 INFO:tasks.workunit.client.0.smithi002.stdout:6/149: write f8 [711545,76670] 0 2024-03-16T10:44:52.605 INFO:tasks.workunit.client.0.smithi002.stdout:5/110: fsync d0/f18 0 2024-03-16T10:44:52.606 INFO:tasks.workunit.client.0.smithi002.stdout:0/189: dread d3/fc [0,4194304] 0 2024-03-16T10:44:52.606 INFO:tasks.workunit.client.0.smithi002.stdout:8/169: creat d7/d9/d17/d23/f33 x:0 0 0 2024-03-16T10:44:52.610 INFO:tasks.workunit.client.0.smithi002.stdout:8/170: read d7/d9/d17/f27 [3066719,104053] 0 2024-03-16T10:44:52.610 INFO:tasks.workunit.client.0.smithi002.stdout:2/165: creat d7/d27/f2d x:0 0 0 2024-03-16T10:44:52.612 INFO:tasks.workunit.client.0.smithi002.stdout:2/166: dread - d7/dc/d1d/f26 zero size 2024-03-16T10:44:52.616 INFO:tasks.workunit.client.0.smithi002.stdout:0/190: creat d3/d28/f3e x:0 0 0 2024-03-16T10:44:52.619 INFO:tasks.workunit.client.0.smithi002.stdout:0/191: chown d3/de/c32 1533 1 2024-03-16T10:44:52.621 INFO:tasks.workunit.client.0.smithi002.stdout:6/150: rename d15/d1b to d15/d2d 0 2024-03-16T10:44:52.622 INFO:tasks.workunit.client.0.smithi002.stdout:8/171: mkdir d7/d9/d17/d23/d2e/d34 0 2024-03-16T10:44:52.623 INFO:tasks.workunit.client.0.smithi002.stdout:1/171: dwrite d2/f5 [0,4194304] 0 2024-03-16T10:44:52.625 INFO:tasks.workunit.client.0.smithi002.stdout:6/151: write d15/f1a [43876,39485] 0 2024-03-16T10:44:52.626 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:52 smithi002 ceph-mon[29248]: pgmap v29: 65 pgs: 65 active+clean; 343 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 26 MiB/s wr, 431 op/s 2024-03-16T10:44:52.626 INFO:tasks.workunit.client.0.smithi002.stdout:5/111: getdents d0/d16/d1d 0 2024-03-16T10:44:52.627 INFO:tasks.workunit.client.0.smithi002.stdout:5/112: chown d0/d1f 115 1 2024-03-16T10:44:52.628 INFO:tasks.workunit.client.0.smithi002.stdout:6/152: write d15/d2d/f22 [332750,61364] 0 2024-03-16T10:44:52.634 INFO:tasks.workunit.client.0.smithi002.stdout:1/172: mknod d2/df/d12/d29/c3c 0 2024-03-16T10:44:52.637 INFO:tasks.workunit.client.0.smithi002.stdout:8/172: creat d7/f35 x:0 0 0 2024-03-16T10:44:52.638 INFO:tasks.workunit.client.0.smithi002.stdout:9/144: sync 2024-03-16T10:44:52.638 INFO:tasks.workunit.client.0.smithi002.stdout:7/138: sync 2024-03-16T10:44:52.640 INFO:tasks.workunit.client.0.smithi002.stdout:2/167: dread f3 [0,4194304] 0 2024-03-16T10:44:52.642 INFO:tasks.workunit.client.0.smithi002.stdout:6/153: dwrite d15/f25 [0,4194304] 0 2024-03-16T10:44:52.643 INFO:tasks.workunit.client.0.smithi002.stdout:1/173: mknod d2/c3d 0 2024-03-16T10:44:52.644 INFO:tasks.workunit.client.0.smithi002.stdout:7/139: mkdir d3/d4/d1d/d28 0 2024-03-16T10:44:52.645 INFO:tasks.workunit.client.0.smithi002.stdout:9/145: chown d5/d9/dc/c27 7 1 2024-03-16T10:44:52.646 INFO:tasks.workunit.client.0.smithi002.stdout:5/113: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:52.650 INFO:tasks.workunit.client.0.smithi002.stdout:1/174: write d2/d3/f1f [435244,130185] 0 2024-03-16T10:44:52.651 INFO:tasks.workunit.client.0.smithi002.stdout:6/154: creat d15/d2d/f2e x:0 0 0 2024-03-16T10:44:52.652 INFO:tasks.workunit.client.0.smithi002.stdout:0/192: dwrite d3/d27/f3c [0,4194304] 0 2024-03-16T10:44:52.657 INFO:tasks.workunit.client.0.smithi002.stdout:5/114: symlink d0/d16/d1d/l21 0 2024-03-16T10:44:52.661 INFO:tasks.workunit.client.0.smithi002.stdout:1/175: write d2/df/d12/f37 [520375,88480] 0 2024-03-16T10:44:52.661 INFO:tasks.workunit.client.0.smithi002.stdout:0/193: rmdir d3 39 2024-03-16T10:44:52.664 INFO:tasks.workunit.client.0.smithi002.stdout:6/155: unlink d15/l27 0 2024-03-16T10:44:52.666 INFO:tasks.workunit.client.0.smithi002.stdout:9/146: dwrite d5/d9/f1a [4194304,4194304] 0 2024-03-16T10:44:52.666 INFO:tasks.workunit.client.0.smithi002.stdout:2/168: write d7/dc/f1a [670472,46850] 0 2024-03-16T10:44:52.666 INFO:tasks.workunit.client.0.smithi002.stdout:6/156: chown d15/f18 4227301 1 2024-03-16T10:44:52.672 INFO:tasks.workunit.client.0.smithi002.stdout:7/140: truncate d3/fd 779722 0 2024-03-16T10:44:52.672 INFO:tasks.workunit.client.0.smithi002.stdout:5/115: unlink d0/d16/l1e 0 2024-03-16T10:44:52.675 INFO:tasks.workunit.client.0.smithi002.stdout:6/157: read f8 [1003620,108878] 0 2024-03-16T10:44:52.675 INFO:tasks.workunit.client.0.smithi002.stdout:5/116: readlink d0/d16/l1b 0 2024-03-16T10:44:52.675 INFO:tasks.workunit.client.0.smithi002.stdout:0/194: write d3/de/f13 [4746740,26915] 0 2024-03-16T10:44:52.676 INFO:tasks.workunit.client.0.smithi002.stdout:2/169: write f6 [6189829,25237] 0 2024-03-16T10:44:52.677 INFO:tasks.workunit.client.0.smithi002.stdout:1/176: mknod d2/df/d12/c3e 0 2024-03-16T10:44:52.678 INFO:tasks.workunit.client.0.smithi002.stdout:0/195: dread - d3/f2d zero size 2024-03-16T10:44:52.680 INFO:tasks.workunit.client.0.smithi002.stdout:9/147: mknod d5/d9/d25/d23/c2e 0 2024-03-16T10:44:52.680 INFO:tasks.workunit.client.0.smithi002.stdout:8/173: dwrite d7/f14 [4194304,4194304] 0 2024-03-16T10:44:52.687 INFO:tasks.workunit.client.0.smithi002.stdout:1/177: dread d2/df/d12/d29/f34 [0,4194304] 0 2024-03-16T10:44:52.690 INFO:tasks.workunit.client.0.smithi002.stdout:8/174: symlink d7/d9/d17/l36 0 2024-03-16T10:44:52.690 INFO:tasks.workunit.client.0.smithi002.stdout:9/148: dwrite d5/f29 [0,4194304] 0 2024-03-16T10:44:52.693 INFO:tasks.workunit.client.0.smithi002.stdout:5/117: getdents d0/d16/d1d/d20 0 2024-03-16T10:44:52.695 INFO:tasks.workunit.client.0.smithi002.stdout:5/118: truncate d0/f6 5166679 0 2024-03-16T10:44:52.697 INFO:tasks.workunit.client.0.smithi002.stdout:1/178: rmdir d2/df/d12/d29 39 2024-03-16T10:44:52.699 INFO:tasks.workunit.client.0.smithi002.stdout:6/158: creat d15/f2f x:0 0 0 2024-03-16T10:44:52.708 INFO:tasks.workunit.client.0.smithi002.stdout:2/170: rename d7/dc to d7/d2e 0 2024-03-16T10:44:52.708 INFO:tasks.workunit.client.0.smithi002.stdout:2/171: stat d7 0 2024-03-16T10:44:52.713 INFO:tasks.workunit.client.0.smithi002.stdout:8/175: link d7/f35 d7/d13/f37 0 2024-03-16T10:44:52.714 INFO:tasks.workunit.client.0.smithi002.stdout:8/176: read d7/f19 [411492,120782] 0 2024-03-16T10:44:52.715 INFO:tasks.workunit.client.0.smithi002.stdout:2/172: write f3 [3472210,99533] 0 2024-03-16T10:44:52.716 INFO:tasks.workunit.client.0.smithi002.stdout:2/173: stat d7/f2b 0 2024-03-16T10:44:52.716 INFO:tasks.workunit.client.0.smithi002.stdout:5/119: mknod d0/d1f/c22 0 2024-03-16T10:44:52.717 INFO:tasks.workunit.client.0.smithi002.stdout:2/174: write d7/d2e/f1a [227035,20961] 0 2024-03-16T10:44:52.719 INFO:tasks.workunit.client.0.smithi002.stdout:6/159: creat d15/d2d/d2c/f30 x:0 0 0 2024-03-16T10:44:52.722 INFO:tasks.workunit.client.0.smithi002.stdout:1/179: creat d2/df/d12/f3f x:0 0 0 2024-03-16T10:44:52.723 INFO:tasks.workunit.client.0.smithi002.stdout:5/120: creat d0/d16/d1d/f23 x:0 0 0 2024-03-16T10:44:52.724 INFO:tasks.workunit.client.0.smithi002.stdout:2/175: mkdir d7/d2e/d1d/d2f 0 2024-03-16T10:44:52.725 INFO:tasks.workunit.client.0.smithi002.stdout:6/160: creat d15/d2d/d2c/f31 x:0 0 0 2024-03-16T10:44:52.727 INFO:tasks.workunit.client.0.smithi002.stdout:1/180: mknod d2/d3/d1c/c40 0 2024-03-16T10:44:52.730 INFO:tasks.workunit.client.0.smithi002.stdout:2/176: unlink f4 0 2024-03-16T10:44:52.730 INFO:tasks.workunit.client.0.smithi002.stdout:2/177: write d7/f8 [3427033,100385] 0 2024-03-16T10:44:52.731 INFO:tasks.workunit.client.0.smithi002.stdout:8/177: link d7/d13/l1b d7/d9/d17/d23/l38 0 2024-03-16T10:44:52.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:52 smithi026 ceph-mon[37041]: pgmap v29: 65 pgs: 65 active+clean; 343 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 26 MiB/s wr, 431 op/s 2024-03-16T10:44:52.734 INFO:tasks.workunit.client.0.smithi002.stdout:5/121: mkdir d0/d24 0 2024-03-16T10:44:52.735 INFO:tasks.workunit.client.0.smithi002.stdout:6/161: mknod d15/d2d/d2b/c32 0 2024-03-16T10:44:52.736 INFO:tasks.workunit.client.0.smithi002.stdout:6/162: chown d15/f1a 140088146 1 2024-03-16T10:44:52.737 INFO:tasks.workunit.client.0.smithi002.stdout:8/178: dwrite d7/d9/f2f [0,4194304] 0 2024-03-16T10:44:52.737 INFO:tasks.workunit.client.0.smithi002.stdout:1/181: symlink d2/df/d12/d29/l41 0 2024-03-16T10:44:52.744 INFO:tasks.workunit.client.0.smithi002.stdout:8/179: mkdir d7/d9/d17/d23/d39 0 2024-03-16T10:44:52.746 INFO:tasks.workunit.client.0.smithi002.stdout:6/163: dwrite d15/f2f [0,4194304] 0 2024-03-16T10:44:52.753 INFO:tasks.workunit.client.0.smithi002.stdout:5/122: link d0/l11 d0/d1f/l25 0 2024-03-16T10:44:52.756 INFO:tasks.workunit.client.0.smithi002.stdout:8/180: mkdir d7/d9/d17/d23/d2e/d3a 0 2024-03-16T10:44:52.757 INFO:tasks.workunit.client.0.smithi002.stdout:6/164: rmdir d15/d2d/d2b 39 2024-03-16T10:44:52.759 INFO:tasks.workunit.client.0.smithi002.stdout:2/178: getdents d7/db/d12 0 2024-03-16T10:44:52.761 INFO:tasks.workunit.client.0.smithi002.stdout:1/182: rename d2/c1d to d2/df/c42 0 2024-03-16T10:44:52.763 INFO:tasks.workunit.client.0.smithi002.stdout:5/123: mkdir d0/d16/d26 0 2024-03-16T10:44:52.763 INFO:tasks.workunit.client.0.smithi002.stdout:8/181: dwrite d7/f21 [0,4194304] 0 2024-03-16T10:44:52.766 INFO:tasks.workunit.client.0.smithi002.stdout:2/179: rmdir d7 39 2024-03-16T10:44:52.767 INFO:tasks.workunit.client.0.smithi002.stdout:3/150: sync 2024-03-16T10:44:52.767 INFO:tasks.workunit.client.0.smithi002.stdout:4/191: sync 2024-03-16T10:44:52.786 INFO:tasks.workunit.client.0.smithi002.stdout:6/165: creat d15/f33 x:0 0 0 2024-03-16T10:44:52.787 INFO:tasks.workunit.client.0.smithi002.stdout:6/166: chown d15/d2d/d2c/f31 930 1 2024-03-16T10:44:52.787 INFO:tasks.workunit.client.0.smithi002.stdout:8/182: symlink d7/d9/d17/d23/d2e/d3a/l3b 0 2024-03-16T10:44:52.788 INFO:tasks.workunit.client.0.smithi002.stdout:8/183: dread - d7/d9/d17/f2d zero size 2024-03-16T10:44:52.789 INFO:tasks.workunit.client.0.smithi002.stdout:2/180: stat d7/d2e/d28/l2c 0 2024-03-16T10:44:52.789 INFO:tasks.workunit.client.0.smithi002.stdout:2/181: chown d7/d2e/f20 1977 1 2024-03-16T10:44:52.792 INFO:tasks.workunit.client.0.smithi002.stdout:3/151: creat d1/dd/d11/d15/d1e/d22/f39 x:0 0 0 2024-03-16T10:44:52.799 INFO:tasks.workunit.client.0.smithi002.stdout:2/182: rename d7/d2e/l13 to d7/db/l30 0 2024-03-16T10:44:52.799 INFO:tasks.workunit.client.0.smithi002.stdout:2/183: fdatasync d7/d2e/f25 0 2024-03-16T10:44:52.801 INFO:tasks.workunit.client.0.smithi002.stdout:5/124: getdents d0/d16/d1d 0 2024-03-16T10:44:52.803 INFO:tasks.workunit.client.0.smithi002.stdout:1/183: write d2/d3/fe [1524751,11589] 0 2024-03-16T10:44:52.805 INFO:tasks.workunit.client.0.smithi002.stdout:2/184: dwrite d7/d2e/f1a [0,4194304] 0 2024-03-16T10:44:52.806 INFO:tasks.workunit.client.0.smithi002.stdout:2/185: write d7/d2e/f20 [1104074,119592] 0 2024-03-16T10:44:52.807 INFO:tasks.workunit.client.0.smithi002.stdout:2/186: chown d7/d2e/f25 2 1 2024-03-16T10:44:52.807 INFO:tasks.workunit.client.0.smithi002.stdout:2/187: dread - d7/db/d12/f22 zero size 2024-03-16T10:44:52.812 INFO:tasks.workunit.client.0.smithi002.stdout:6/167: creat d15/d2d/f34 x:0 0 0 2024-03-16T10:44:52.814 INFO:tasks.workunit.client.0.smithi002.stdout:5/125: mknod d0/d16/d1d/c27 0 2024-03-16T10:44:52.814 INFO:tasks.workunit.client.0.smithi002.stdout:6/168: chown d15/l1e 44 1 2024-03-16T10:44:52.816 INFO:tasks.workunit.client.0.smithi002.stdout:4/192: dwrite d2/fe [0,4194304] 0 2024-03-16T10:44:52.817 INFO:tasks.workunit.client.0.smithi002.stdout:2/188: dwrite d7/db/d12/f22 [0,4194304] 0 2024-03-16T10:44:52.817 INFO:tasks.workunit.client.0.smithi002.stdout:1/184: rmdir d2 39 2024-03-16T10:44:52.818 INFO:tasks.workunit.client.0.smithi002.stdout:8/184: link d7/d9/d17/l2a d7/d9/d17/d20/l3c 0 2024-03-16T10:44:52.820 INFO:tasks.workunit.client.0.smithi002.stdout:4/193: chown d2/d33/f1b 13531 1 2024-03-16T10:44:52.823 INFO:tasks.workunit.client.0.smithi002.stdout:5/126: dwrite d0/d16/f19 [0,4194304] 0 2024-03-16T10:44:52.834 INFO:tasks.workunit.client.0.smithi002.stdout:2/189: chown d7/db/d12/le 8 1 2024-03-16T10:44:52.836 INFO:tasks.workunit.client.0.smithi002.stdout:8/185: creat d7/d9/d17/f3d x:0 0 0 2024-03-16T10:44:52.838 INFO:tasks.workunit.client.0.smithi002.stdout:4/194: rename d2/d33/da/c19 to d2/d33/da/d2e/c36 0 2024-03-16T10:44:52.841 INFO:tasks.workunit.client.0.smithi002.stdout:3/152: truncate d1/dd/f10 2993752 0 2024-03-16T10:44:52.852 INFO:tasks.workunit.client.0.smithi002.stdout:6/169: link f11 d15/d2d/d2b/f35 0 2024-03-16T10:44:52.853 INFO:tasks.workunit.client.0.smithi002.stdout:6/170: write d15/f33 [314713,87074] 0 2024-03-16T10:44:52.855 INFO:tasks.workunit.client.0.smithi002.stdout:7/141: sync 2024-03-16T10:44:52.855 INFO:tasks.workunit.client.0.smithi002.stdout:0/196: sync 2024-03-16T10:44:52.855 INFO:tasks.workunit.client.0.smithi002.stdout:9/149: sync 2024-03-16T10:44:52.855 INFO:tasks.workunit.client.0.smithi002.stdout:5/127: write d0/f6 [5384898,36717] 0 2024-03-16T10:44:52.856 INFO:tasks.workunit.client.0.smithi002.stdout:9/150: chown d5/d9/dc 446565120 1 2024-03-16T10:44:52.857 INFO:tasks.workunit.client.0.smithi002.stdout:5/128: truncate d0/d16/d1d/f23 84150 0 2024-03-16T10:44:52.858 INFO:tasks.workunit.client.0.smithi002.stdout:5/129: chown d0/d16/d1d/c27 240014 1 2024-03-16T10:44:52.859 INFO:tasks.workunit.client.0.smithi002.stdout:6/171: creat d15/d2d/d2c/f36 x:0 0 0 2024-03-16T10:44:52.862 INFO:tasks.workunit.client.0.smithi002.stdout:7/142: symlink d3/d4/d10/l29 0 2024-03-16T10:44:52.863 INFO:tasks.workunit.client.0.smithi002.stdout:4/195: creat d2/f37 x:0 0 0 2024-03-16T10:44:52.864 INFO:tasks.workunit.client.0.smithi002.stdout:4/196: fdatasync d2/f34 0 2024-03-16T10:44:52.865 INFO:tasks.workunit.client.0.smithi002.stdout:9/151: chown d5/d9/d25/l21 202 1 2024-03-16T10:44:52.866 INFO:tasks.workunit.client.0.smithi002.stdout:3/153: rename d1/dd/f13 to d1/f3a 0 2024-03-16T10:44:52.867 INFO:tasks.workunit.client.0.smithi002.stdout:1/185: getdents d2/df 0 2024-03-16T10:44:52.868 INFO:tasks.workunit.client.0.smithi002.stdout:7/143: dwrite d3/f14 [4194304,4194304] 0 2024-03-16T10:44:52.870 INFO:tasks.workunit.client.0.smithi002.stdout:2/190: getdents d7/db 0 2024-03-16T10:44:52.870 INFO:tasks.workunit.client.0.smithi002.stdout:7/144: write d3/f1b [849017,75416] 0 2024-03-16T10:44:52.874 INFO:tasks.workunit.client.0.smithi002.stdout:8/186: getdents d7/d9/d17/d23/d2e 0 2024-03-16T10:44:52.874 INFO:tasks.workunit.client.0.smithi002.stdout:0/197: creat d3/d28/d3d/f3f x:0 0 0 2024-03-16T10:44:52.875 INFO:tasks.workunit.client.0.smithi002.stdout:4/197: dwrite d2/fe [0,4194304] 0 2024-03-16T10:44:52.880 INFO:tasks.workunit.client.0.smithi002.stdout:9/152: rmdir d5 39 2024-03-16T10:44:52.891 INFO:tasks.workunit.client.0.smithi002.stdout:3/154: dread d1/f35 [4194304,4194304] 0 2024-03-16T10:44:52.898 INFO:tasks.workunit.client.0.smithi002.stdout:4/198: mknod d2/d6/c38 0 2024-03-16T10:44:52.898 INFO:tasks.workunit.client.0.smithi002.stdout:4/199: write d2/d6/f17 [1537177,87848] 0 2024-03-16T10:44:52.903 INFO:tasks.workunit.client.0.smithi002.stdout:6/172: rename d15/l16 to d15/d2d/d2c/l37 0 2024-03-16T10:44:52.903 INFO:tasks.workunit.client.0.smithi002.stdout:5/130: write d0/d16/f19 [4344081,104337] 0 2024-03-16T10:44:52.905 INFO:tasks.workunit.client.0.smithi002.stdout:6/173: dread d15/f1a [0,4194304] 0 2024-03-16T10:44:52.906 INFO:tasks.workunit.client.0.smithi002.stdout:8/187: mknod d7/d9/c3e 0 2024-03-16T10:44:52.909 INFO:tasks.workunit.client.0.smithi002.stdout:9/153: symlink d5/d9/l2f 0 2024-03-16T10:44:52.914 INFO:tasks.workunit.client.0.smithi002.stdout:2/191: write d7/d2e/f1a [4343045,103278] 0 2024-03-16T10:44:52.921 INFO:tasks.workunit.client.0.smithi002.stdout:3/155: rename d1/c34 to d1/dd/c3b 0 2024-03-16T10:44:52.922 INFO:tasks.workunit.client.0.smithi002.stdout:3/156: fsync d1/dd/d28/f33 0 2024-03-16T10:44:52.923 INFO:tasks.workunit.client.0.smithi002.stdout:0/198: dwrite d3/f3a [0,4194304] 0 2024-03-16T10:44:52.923 INFO:tasks.workunit.client.0.smithi002.stdout:5/131: mknod d0/c28 0 2024-03-16T10:44:52.925 INFO:tasks.workunit.client.0.smithi002.stdout:7/145: getdents d3/d4/d1c 0 2024-03-16T10:44:52.926 INFO:tasks.workunit.client.0.smithi002.stdout:8/188: symlink d7/d9/d17/d23/d39/l3f 0 2024-03-16T10:44:52.929 INFO:tasks.workunit.client.0.smithi002.stdout:5/132: dwrite d0/d16/f19 [0,4194304] 0 2024-03-16T10:44:52.932 INFO:tasks.workunit.client.0.smithi002.stdout:9/154: truncate d5/f7 2437457 0 2024-03-16T10:44:52.934 INFO:tasks.workunit.client.0.smithi002.stdout:6/174: rename lb to d15/d2d/d2b/l38 0 2024-03-16T10:44:52.934 INFO:tasks.workunit.client.0.smithi002.stdout:3/157: rename d1/dd to d1/dd/d28/d37/d3c 22 2024-03-16T10:44:52.937 INFO:tasks.workunit.client.0.smithi002.stdout:3/158: read d1/dd/d28/f36 [637987,110859] 0 2024-03-16T10:44:52.940 INFO:tasks.workunit.client.0.smithi002.stdout:9/155: dwrite d5/d9/dc/f11 [0,4194304] 0 2024-03-16T10:44:52.941 INFO:tasks.workunit.client.0.smithi002.stdout:9/156: chown d5/d9/dc/d1e/d24 2102 1 2024-03-16T10:44:52.944 INFO:tasks.workunit.client.0.smithi002.stdout:7/146: mkdir d3/d4/d2a 0 2024-03-16T10:44:52.947 INFO:tasks.workunit.client.0.smithi002.stdout:8/189: unlink d7/d13/f30 0 2024-03-16T10:44:52.948 INFO:tasks.workunit.client.0.smithi002.stdout:7/147: dread d3/f1a [0,4194304] 0 2024-03-16T10:44:52.949 INFO:tasks.workunit.client.0.smithi002.stdout:4/200: link d2/d33/da/l2a d2/d33/l39 0 2024-03-16T10:44:52.953 INFO:tasks.workunit.client.0.smithi002.stdout:5/133: creat d0/d16/f29 x:0 0 0 2024-03-16T10:44:52.953 INFO:tasks.workunit.client.0.smithi002.stdout:8/190: dwrite d7/d9/d25/f2c [0,4194304] 0 2024-03-16T10:44:52.957 INFO:tasks.workunit.client.0.smithi002.stdout:6/175: rename f9 to d15/d2d/d2c/f39 0 2024-03-16T10:44:52.958 INFO:tasks.workunit.client.0.smithi002.stdout:6/176: stat d15/d2d/f22 0 2024-03-16T10:44:52.982 INFO:tasks.workunit.client.0.smithi002.stdout:0/199: creat d3/de/f40 x:0 0 0 2024-03-16T10:44:52.984 INFO:tasks.workunit.client.0.smithi002.stdout:6/177: symlink d15/d2d/d2b/l3a 0 2024-03-16T10:44:52.988 INFO:tasks.workunit.client.0.smithi002.stdout:2/192: getdents d7/d2e 0 2024-03-16T10:44:52.990 INFO:tasks.workunit.client.0.smithi002.stdout:1/186: sync 2024-03-16T10:44:52.994 INFO:tasks.workunit.client.0.smithi002.stdout:2/193: dwrite d7/f2b [0,4194304] 0 2024-03-16T10:44:52.995 INFO:tasks.workunit.client.0.smithi002.stdout:2/194: write d7/d2e/f16 [934647,13076] 0 2024-03-16T10:44:52.996 INFO:tasks.workunit.client.0.smithi002.stdout:1/187: dread d2/df/d12/f37 [0,4194304] 0 2024-03-16T10:44:53.002 INFO:tasks.workunit.client.0.smithi002.stdout:5/134: symlink d0/d16/d1d/d20/l2a 0 2024-03-16T10:44:53.003 INFO:tasks.workunit.client.0.smithi002.stdout:3/159: dwrite d1/f1a [0,4194304] 0 2024-03-16T10:44:53.003 INFO:tasks.workunit.client.0.smithi002.stdout:2/195: dread d7/f2b [0,4194304] 0 2024-03-16T10:44:53.004 INFO:tasks.workunit.client.0.smithi002.stdout:3/160: readlink d1/dd/d11/d15/l21 0 2024-03-16T10:44:53.008 INFO:tasks.workunit.client.0.smithi002.stdout:2/196: dread d7/d2e/f20 [0,4194304] 0 2024-03-16T10:44:53.015 INFO:tasks.workunit.client.0.smithi002.stdout:2/197: dwrite d7/d2e/d1d/f21 [0,4194304] 0 2024-03-16T10:44:53.016 INFO:tasks.workunit.client.0.smithi002.stdout:2/198: readlink d7/db/d12/le 0 2024-03-16T10:44:53.017 INFO:tasks.workunit.client.0.smithi002.stdout:2/199: fsync d7/d2e/f20 0 2024-03-16T10:44:53.017 INFO:tasks.workunit.client.0.smithi002.stdout:2/200: chown d7/d2e/d1d/d2f 199917 1 2024-03-16T10:44:53.019 INFO:tasks.workunit.client.0.smithi002.stdout:9/157: truncate d5/fd 3384218 0 2024-03-16T10:44:53.025 INFO:tasks.workunit.client.0.smithi002.stdout:5/135: mkdir d0/d16/d1d/d2b 0 2024-03-16T10:44:53.025 INFO:tasks.workunit.client.0.smithi002.stdout:7/148: truncate d3/f1a 617890 0 2024-03-16T10:44:53.029 INFO:tasks.workunit.client.0.smithi002.stdout:3/161: mknod d1/dd/d11/c3d 0 2024-03-16T10:44:53.031 INFO:tasks.workunit.client.0.smithi002.stdout:8/191: truncate d7/fc 1279969 0 2024-03-16T10:44:53.034 INFO:tasks.workunit.client.0.smithi002.stdout:3/162: dwrite d1/f16 [0,4194304] 0 2024-03-16T10:44:53.039 INFO:tasks.workunit.client.0.smithi002.stdout:0/200: dwrite d3/de/f15 [0,4194304] 0 2024-03-16T10:44:53.072 INFO:tasks.workunit.client.0.smithi002.stdout:4/201: link d2/d33/l39 d2/d33/d1c/l3a 0 2024-03-16T10:44:53.072 INFO:tasks.workunit.client.0.smithi002.stdout:4/202: read - d2/f34 zero size 2024-03-16T10:44:53.075 INFO:tasks.workunit.client.0.smithi002.stdout:2/201: creat d7/db/d12/f31 x:0 0 0 2024-03-16T10:44:53.075 INFO:tasks.workunit.client.0.smithi002.stdout:2/202: truncate f6 8839578 0 2024-03-16T10:44:53.076 INFO:tasks.workunit.client.0.smithi002.stdout:2/203: readlink d7/db/d12/le 0 2024-03-16T10:44:53.077 INFO:tasks.workunit.client.0.smithi002.stdout:1/188: write d2/df/d12/d29/f34 [2586449,33315] 0 2024-03-16T10:44:53.077 INFO:tasks.workunit.client.0.smithi002.stdout:1/189: chown d2/c3d 182285 1 2024-03-16T10:44:53.081 INFO:tasks.workunit.client.0.smithi002.stdout:7/149: creat d3/d4/d1c/f2b x:0 0 0 2024-03-16T10:44:53.089 INFO:tasks.workunit.client.0.smithi002.stdout:8/192: rename d7/d13/l10 to d7/d9/d17/d23/l40 0 2024-03-16T10:44:53.091 INFO:tasks.workunit.client.0.smithi002.stdout:6/178: sync 2024-03-16T10:44:53.092 INFO:tasks.workunit.client.0.smithi002.stdout:6/179: truncate d15/d2d/d2c/f36 609643 0 2024-03-16T10:44:53.104 INFO:tasks.workunit.client.0.smithi002.stdout:1/190: mknod d2/d3/d1c/c43 0 2024-03-16T10:44:53.104 INFO:tasks.workunit.client.0.smithi002.stdout:1/191: chown d2/c16 6283626 1 2024-03-16T10:44:53.108 INFO:tasks.workunit.client.0.smithi002.stdout:5/136: link d0/d16/f29 d0/d1f/f2c 0 2024-03-16T10:44:53.111 INFO:tasks.workunit.client.0.smithi002.stdout:7/150: symlink d3/d4/l2c 0 2024-03-16T10:44:53.112 INFO:tasks.workunit.client.0.smithi002.stdout:5/137: dread d0/d16/f19 [0,4194304] 0 2024-03-16T10:44:53.120 INFO:tasks.workunit.client.0.smithi002.stdout:0/201: rename d3/de/c32 to d3/d31/c41 0 2024-03-16T10:44:53.121 INFO:tasks.workunit.client.0.smithi002.stdout:0/202: write d3/d5/f30 [1532070,126862] 0 2024-03-16T10:44:53.123 INFO:tasks.workunit.client.0.smithi002.stdout:6/180: creat d15/d2d/d2c/f3b x:0 0 0 2024-03-16T10:44:53.124 INFO:tasks.workunit.client.0.smithi002.stdout:7/151: dread d3/f15 [4194304,4194304] 0 2024-03-16T10:44:53.125 INFO:tasks.workunit.client.0.smithi002.stdout:4/203: mkdir d2/d33/d3b 0 2024-03-16T10:44:53.125 INFO:tasks.workunit.client.0.smithi002.stdout:4/204: chown d2/d33/da/d2e 2 1 2024-03-16T10:44:53.128 INFO:tasks.workunit.client.0.smithi002.stdout:2/204: link f6 d7/d2e/f32 0 2024-03-16T10:44:53.132 INFO:tasks.workunit.client.0.smithi002.stdout:1/192: unlink d2/d3/d1c/l38 0 2024-03-16T10:44:53.144 INFO:tasks.workunit.client.0.smithi002.stdout:5/138: dwrite d0/d16/f29 [0,4194304] 0 2024-03-16T10:44:53.148 INFO:tasks.workunit.client.0.smithi002.stdout:0/203: stat d3/c2b 0 2024-03-16T10:44:53.149 INFO:tasks.workunit.client.0.smithi002.stdout:0/204: fdatasync d3/d28/f34 0 2024-03-16T10:44:53.149 INFO:tasks.workunit.client.0.smithi002.stdout:6/181: creat d15/d2d/d2c/f3c x:0 0 0 2024-03-16T10:44:53.150 INFO:tasks.workunit.client.0.smithi002.stdout:7/152: mknod d3/d4/d1d/c2d 0 2024-03-16T10:44:53.151 INFO:tasks.workunit.client.0.smithi002.stdout:4/205: symlink d2/d33/da/d2e/l3c 0 2024-03-16T10:44:53.155 INFO:tasks.workunit.client.0.smithi002.stdout:1/193: rename d2/f5 to d2/d3/d1c/f44 0 2024-03-16T10:44:53.156 INFO:tasks.workunit.client.0.smithi002.stdout:6/182: dwrite d15/f33 [0,4194304] 0 2024-03-16T10:44:53.160 INFO:tasks.workunit.client.0.smithi002.stdout:5/139: chown d0/d1f/l25 0 1 2024-03-16T10:44:53.161 INFO:tasks.workunit.client.0.smithi002.stdout:3/163: getdents d1 0 2024-03-16T10:44:53.162 INFO:tasks.workunit.client.0.smithi002.stdout:0/205: unlink d3/d28/d2a/c2e 0 2024-03-16T10:44:53.167 INFO:tasks.workunit.client.0.smithi002.stdout:1/194: creat d2/df/f45 x:0 0 0 2024-03-16T10:44:53.167 INFO:tasks.workunit.client.0.smithi002.stdout:6/183: readlink d15/d2d/d2b/l38 0 2024-03-16T10:44:53.169 INFO:tasks.workunit.client.0.smithi002.stdout:5/140: rename d0/c12 to d0/d16/d1d/d20/c2d 0 2024-03-16T10:44:53.170 INFO:tasks.workunit.client.0.smithi002.stdout:3/164: unlink d1/dd/d11/d15/c23 0 2024-03-16T10:44:53.171 INFO:tasks.workunit.client.0.smithi002.stdout:3/165: write d1/dd/d11/d15/f2e [193436,84271] 0 2024-03-16T10:44:53.172 INFO:tasks.workunit.client.0.smithi002.stdout:0/206: unlink d3/d5/f14 0 2024-03-16T10:44:53.173 INFO:tasks.workunit.client.0.smithi002.stdout:0/207: fdatasync d3/d5/f18 0 2024-03-16T10:44:53.174 INFO:tasks.workunit.client.0.smithi002.stdout:7/153: link d3/d4/d1d/f23 d3/d4/d10/f2e 0 2024-03-16T10:44:53.177 INFO:tasks.workunit.client.0.smithi002.stdout:6/184: creat d15/d2d/d2c/f3d x:0 0 0 2024-03-16T10:44:53.181 INFO:tasks.workunit.client.0.smithi002.stdout:3/166: symlink d1/dd/d28/d37/l3e 0 2024-03-16T10:44:53.184 INFO:tasks.workunit.client.0.smithi002.stdout:0/208: mknod d3/d5/c42 0 2024-03-16T10:44:53.185 INFO:tasks.workunit.client.0.smithi002.stdout:7/154: rmdir d3/d4/d1c 39 2024-03-16T10:44:53.187 INFO:tasks.workunit.client.0.smithi002.stdout:7/155: readlink d3/d4/l2c 0 2024-03-16T10:44:53.189 INFO:tasks.workunit.client.0.smithi002.stdout:9/158: dwrite d5/f7 [0,4194304] 0 2024-03-16T10:44:53.190 INFO:tasks.workunit.client.0.smithi002.stdout:3/167: dwrite d1/dd/d11/f30 [0,4194304] 0 2024-03-16T10:44:53.191 INFO:tasks.workunit.client.0.smithi002.stdout:6/185: symlink d15/d2d/d2c/l3e 0 2024-03-16T10:44:53.194 INFO:tasks.workunit.client.0.smithi002.stdout:0/209: unlink d3/de/f1d 0 2024-03-16T10:44:53.200 INFO:tasks.workunit.client.0.smithi002.stdout:3/168: mkdir d1/dd/d11/d15/d1e/d22/d3f 0 2024-03-16T10:44:53.201 INFO:tasks.workunit.client.0.smithi002.stdout:6/186: mkdir d15/d2d/d2b/d3f 0 2024-03-16T10:44:53.202 INFO:tasks.workunit.client.0.smithi002.stdout:6/187: fdatasync d15/d2d/f2a 0 2024-03-16T10:44:53.203 INFO:tasks.workunit.client.0.smithi002.stdout:5/141: getdents d0/d16 0 2024-03-16T10:44:53.204 INFO:tasks.workunit.client.0.smithi002.stdout:7/156: creat d3/d4/d2a/f2f x:0 0 0 2024-03-16T10:44:53.205 INFO:tasks.workunit.client.0.smithi002.stdout:7/157: read - d3/d4/d2a/f2f zero size 2024-03-16T10:44:53.205 INFO:tasks.workunit.client.0.smithi002.stdout:9/159: creat d5/d9/dc/d1e/d24/f30 x:0 0 0 2024-03-16T10:44:53.205 INFO:tasks.workunit.client.0.smithi002.stdout:7/158: read - d3/f27 zero size 2024-03-16T10:44:53.209 INFO:tasks.workunit.client.0.smithi002.stdout:6/188: mknod d15/d2d/d2b/c40 0 2024-03-16T10:44:53.214 INFO:tasks.workunit.client.0.smithi002.stdout:9/160: mkdir d5/d9/dc/d1e/d24/d31 0 2024-03-16T10:44:53.216 INFO:tasks.workunit.client.0.smithi002.stdout:9/161: write d5/d9/dc/f20 [1730453,27012] 0 2024-03-16T10:44:53.217 INFO:tasks.workunit.client.0.smithi002.stdout:5/142: dwrite d0/d16/f19 [0,4194304] 0 2024-03-16T10:44:53.218 INFO:tasks.workunit.client.0.smithi002.stdout:3/169: link d1/dd/c2a d1/c40 0 2024-03-16T10:44:53.218 INFO:tasks.workunit.client.0.smithi002.stdout:5/143: stat d0/d1f/c22 0 2024-03-16T10:44:53.220 INFO:tasks.workunit.client.0.smithi002.stdout:6/189: fsync d15/f1c 0 2024-03-16T10:44:53.224 INFO:tasks.workunit.client.0.smithi002.stdout:9/162: rename d5/d9/f1a to d5/d9/dc/d1e/d24/d31/f32 0 2024-03-16T10:44:53.231 INFO:tasks.workunit.client.0.smithi002.stdout:9/163: creat d5/d9/d25/d23/f33 x:0 0 0 2024-03-16T10:44:53.232 INFO:tasks.workunit.client.0.smithi002.stdout:6/190: symlink d15/d2d/d2b/d3f/l41 0 2024-03-16T10:44:53.233 INFO:tasks.workunit.client.0.smithi002.stdout:6/191: read f13 [670098,106817] 0 2024-03-16T10:44:53.235 INFO:tasks.workunit.client.0.smithi002.stdout:9/164: mkdir d5/d9/d34 0 2024-03-16T10:44:53.236 INFO:tasks.workunit.client.0.smithi002.stdout:6/192: rmdir d15/d2d/d2b 39 2024-03-16T10:44:53.236 INFO:tasks.workunit.client.0.smithi002.stdout:9/165: chown d5/d9/dc/c27 209 1 2024-03-16T10:44:53.242 INFO:tasks.workunit.client.0.smithi002.stdout:2/205: rename d7/d2e/d1d/l1e to d7/d2e/l33 0 2024-03-16T10:44:53.243 INFO:tasks.workunit.client.0.smithi002.stdout:6/193: creat d15/d2d/f42 x:0 0 0 2024-03-16T10:44:53.247 INFO:tasks.workunit.client.0.smithi002.stdout:9/166: dwrite d5/d9/dc/d1e/d24/d31/f32 [4194304,4194304] 0 2024-03-16T10:44:53.248 INFO:tasks.workunit.client.0.smithi002.stdout:6/194: symlink d15/d2d/l43 0 2024-03-16T10:44:53.251 INFO:tasks.workunit.client.0.smithi002.stdout:9/167: mknod d5/d9/dc/c35 0 2024-03-16T10:44:53.260 INFO:tasks.workunit.client.0.smithi002.stdout:9/168: dwrite d5/f7 [4194304,4194304] 0 2024-03-16T10:44:53.261 INFO:tasks.workunit.client.0.smithi002.stdout:9/169: chown d5/d9/d25/d23/c2e 65395056 1 2024-03-16T10:44:53.262 INFO:tasks.workunit.client.0.smithi002.stdout:6/195: link d15/l17 d15/d2d/l44 0 2024-03-16T10:44:53.264 INFO:tasks.workunit.client.0.smithi002.stdout:4/206: truncate d2/d6/f17 2772480 0 2024-03-16T10:44:53.265 INFO:tasks.workunit.client.0.smithi002.stdout:9/170: creat d5/d9/dc/d1e/f36 x:0 0 0 2024-03-16T10:44:53.267 INFO:tasks.workunit.client.0.smithi002.stdout:7/159: fdatasync d3/f1a 0 2024-03-16T10:44:53.268 INFO:tasks.workunit.client.0.smithi002.stdout:8/193: creat d7/f41 x:0 0 0 2024-03-16T10:44:53.275 INFO:tasks.workunit.client.0.smithi002.stdout:9/171: creat d5/d9/d22/f37 x:0 0 0 2024-03-16T10:44:53.278 INFO:tasks.workunit.client.0.smithi002.stdout:8/194: symlink d7/d9/d17/d23/d2e/l42 0 2024-03-16T10:44:53.278 INFO:tasks.workunit.client.0.smithi002.stdout:9/172: chown d5/d9/dc/l1c 0 1 2024-03-16T10:44:53.279 INFO:tasks.workunit.client.0.smithi002.stdout:4/207: getdents d2/d33/da 0 2024-03-16T10:44:53.281 INFO:tasks.workunit.client.0.smithi002.stdout:6/196: dwrite d15/f25 [8388608,4194304] 0 2024-03-16T10:44:53.290 INFO:tasks.workunit.client.0.smithi002.stdout:7/160: dread d3/d4/d1d/f23 [0,4194304] 0 2024-03-16T10:44:53.293 INFO:tasks.workunit.client.0.smithi002.stdout:0/210: truncate d3/f22 389379 0 2024-03-16T10:44:53.300 INFO:tasks.workunit.client.0.smithi002.stdout:9/173: link d5/d9/l2f d5/d9/dc/d1e/d24/l38 0 2024-03-16T10:44:53.301 INFO:tasks.workunit.client.0.smithi002.stdout:6/197: truncate d15/d2d/d2b/f35 1769704 0 2024-03-16T10:44:53.303 INFO:tasks.workunit.client.0.smithi002.stdout:9/174: dread - d5/d9/dc/d1e/f36 zero size 2024-03-16T10:44:53.305 INFO:tasks.workunit.client.0.smithi002.stdout:2/206: truncate d7/db/d12/f22 2415901 0 2024-03-16T10:44:53.308 INFO:tasks.workunit.client.0.smithi002.stdout:6/198: creat d15/d2d/d2b/d3f/f45 x:0 0 0 2024-03-16T10:44:53.308 INFO:tasks.workunit.client.0.smithi002.stdout:7/161: link d3/d4/l2c d3/d4/d10/l30 0 2024-03-16T10:44:53.312 INFO:tasks.workunit.client.0.smithi002.stdout:4/208: dread d2/d33/f1b [0,4194304] 0 2024-03-16T10:44:53.313 INFO:tasks.workunit.client.0.smithi002.stdout:0/211: rename d3/de/f1e to d3/de/f43 0 2024-03-16T10:44:53.314 INFO:tasks.workunit.client.0.smithi002.stdout:9/175: dwrite d5/d9/dc/d1e/d24/f30 [0,4194304] 0 2024-03-16T10:44:53.315 INFO:tasks.workunit.client.0.smithi002.stdout:9/176: chown d5 180858211 1 2024-03-16T10:44:53.320 INFO:tasks.workunit.client.0.smithi002.stdout:2/207: mkdir d7/d2e/d34 0 2024-03-16T10:44:53.320 INFO:tasks.workunit.client.0.smithi002.stdout:8/195: write f4 [2268351,109217] 0 2024-03-16T10:44:53.323 INFO:tasks.workunit.client.0.smithi002.stdout:0/212: mkdir d3/d28/d3d/d44 0 2024-03-16T10:44:53.323 INFO:tasks.workunit.client.0.smithi002.stdout:1/195: sync 2024-03-16T10:44:53.323 INFO:tasks.workunit.client.0.smithi002.stdout:5/144: sync 2024-03-16T10:44:53.327 INFO:tasks.workunit.client.0.smithi002.stdout:7/162: mknod d3/d4/c31 0 2024-03-16T10:44:53.327 INFO:tasks.workunit.client.0.smithi002.stdout:5/145: truncate d0/d16/d1d/f23 312278 0 2024-03-16T10:44:53.327 INFO:tasks.workunit.client.0.smithi002.stdout:9/177: dread d5/f7 [0,4194304] 0 2024-03-16T10:44:53.329 INFO:tasks.workunit.client.0.smithi002.stdout:9/178: fsync d5/d9/dc/d1e/f36 0 2024-03-16T10:44:53.330 INFO:tasks.workunit.client.0.smithi002.stdout:6/199: dread fc [0,4194304] 0 2024-03-16T10:44:53.331 INFO:tasks.workunit.client.0.smithi002.stdout:2/208: rename f3 to d7/db/f35 0 2024-03-16T10:44:53.333 INFO:tasks.workunit.client.0.smithi002.stdout:2/209: write d7/db/d12/f11 [1703048,24668] 0 2024-03-16T10:44:53.335 INFO:tasks.workunit.client.0.smithi002.stdout:7/163: dread d3/d4/d1d/f23 [0,4194304] 0 2024-03-16T10:44:53.337 INFO:tasks.workunit.client.0.smithi002.stdout:9/179: dread d5/d9/dc/f11 [0,4194304] 0 2024-03-16T10:44:53.338 INFO:tasks.workunit.client.0.smithi002.stdout:0/213: creat d3/d28/d3d/f45 x:0 0 0 2024-03-16T10:44:53.345 INFO:tasks.workunit.client.0.smithi002.stdout:6/200: creat d15/d2d/d2c/f46 x:0 0 0 2024-03-16T10:44:53.346 INFO:tasks.workunit.client.0.smithi002.stdout:3/170: truncate d1/dd/d11/d15/f17 516600 0 2024-03-16T10:44:53.347 INFO:tasks.workunit.client.0.smithi002.stdout:1/196: chown d2/df/c42 4 1 2024-03-16T10:44:53.348 INFO:tasks.workunit.client.0.smithi002.stdout:1/197: stat d2/d3/f17 0 2024-03-16T10:44:53.351 INFO:tasks.workunit.client.0.smithi002.stdout:9/180: symlink d5/d9/dc/d1e/l39 0 2024-03-16T10:44:53.354 INFO:tasks.workunit.client.0.smithi002.stdout:5/146: mknod d0/d16/d26/c2e 0 2024-03-16T10:44:53.356 INFO:tasks.workunit.client.0.smithi002.stdout:5/147: readlink d0/l13 0 2024-03-16T10:44:53.358 INFO:tasks.workunit.client.0.smithi002.stdout:2/210: dwrite d7/d2e/f1a [4194304,4194304] 0 2024-03-16T10:44:53.360 INFO:tasks.workunit.client.0.smithi002.stdout:1/198: dwrite d2/df/f45 [0,4194304] 0 2024-03-16T10:44:53.362 INFO:tasks.workunit.client.0.smithi002.stdout:3/171: dread - d1/f3a zero size 2024-03-16T10:44:53.363 INFO:tasks.workunit.client.0.smithi002.stdout:3/172: readlink d1/dd/d11/d15/l19 0 2024-03-16T10:44:53.363 INFO:tasks.workunit.client.0.smithi002.stdout:6/201: symlink d15/d2d/d2b/d3f/l47 0 2024-03-16T10:44:53.368 INFO:tasks.workunit.client.0.smithi002.stdout:7/164: creat d3/d4/d1c/f32 x:0 0 0 2024-03-16T10:44:53.369 INFO:tasks.workunit.client.0.smithi002.stdout:7/165: dread - d3/d4/f1e zero size 2024-03-16T10:44:53.376 INFO:tasks.workunit.client.0.smithi002.stdout:1/199: dread - d2/df/d12/f3f zero size 2024-03-16T10:44:53.379 INFO:tasks.workunit.client.0.smithi002.stdout:7/166: creat d3/d4/d1c/f33 x:0 0 0 2024-03-16T10:44:53.381 INFO:tasks.workunit.client.0.smithi002.stdout:4/209: dwrite d2/d6/f17 [0,4194304] 0 2024-03-16T10:44:53.382 INFO:tasks.workunit.client.0.smithi002.stdout:4/210: dread - d2/d33/f29 zero size 2024-03-16T10:44:53.382 INFO:tasks.workunit.client.0.smithi002.stdout:5/148: mknod d0/d16/d1d/d2b/c2f 0 2024-03-16T10:44:53.383 INFO:tasks.workunit.client.0.smithi002.stdout:4/211: chown d2/d33/da/d2e/l3c 7 1 2024-03-16T10:44:53.383 INFO:tasks.workunit.client.0.smithi002.stdout:2/211: symlink d7/d2e/l36 0 2024-03-16T10:44:53.384 INFO:tasks.workunit.client.0.smithi002.stdout:3/173: fsync d1/f35 0 2024-03-16T10:44:53.389 INFO:tasks.workunit.client.0.smithi002.stdout:5/149: dwrite d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:44:53.391 INFO:tasks.workunit.client.0.smithi002.stdout:9/181: rmdir d5/d9/dc/d2b 0 2024-03-16T10:44:53.392 INFO:tasks.workunit.client.0.smithi002.stdout:7/167: mkdir d3/d4/d10/d34 0 2024-03-16T10:44:53.394 INFO:tasks.workunit.client.0.smithi002.stdout:7/168: truncate d3/d4/d1c/f32 536458 0 2024-03-16T10:44:53.396 INFO:tasks.workunit.client.0.smithi002.stdout:7/169: write d3/d4/d1c/f33 [510199,18290] 0 2024-03-16T10:44:53.396 INFO:tasks.workunit.client.0.smithi002.stdout:4/212: unlink d2/f20 0 2024-03-16T10:44:53.398 INFO:tasks.workunit.client.0.smithi002.stdout:3/174: write d1/f1a [2949210,4449] 0 2024-03-16T10:44:53.404 INFO:tasks.workunit.client.0.smithi002.stdout:9/182: creat d5/d9/d22/f3a x:0 0 0 2024-03-16T10:44:53.412 INFO:tasks.workunit.client.0.smithi002.stdout:4/213: unlink d2/d33/d1c/c27 0 2024-03-16T10:44:53.412 INFO:tasks.workunit.client.0.smithi002.stdout:4/214: dread - d2/d6/f2d zero size 2024-03-16T10:44:53.414 INFO:tasks.workunit.client.0.smithi002.stdout:0/214: truncate d3/d5/f30 463761 0 2024-03-16T10:44:53.416 INFO:tasks.workunit.client.0.smithi002.stdout:3/175: symlink d1/dd/d11/d15/d1e/d2c/l41 0 2024-03-16T10:44:53.417 INFO:tasks.workunit.client.0.smithi002.stdout:1/200: link d2/df/d12/l1b d2/d3/d1c/l46 0 2024-03-16T10:44:53.418 INFO:tasks.workunit.client.0.smithi002.stdout:1/201: chown d2/df/f45 214150411 1 2024-03-16T10:44:53.419 INFO:tasks.workunit.client.0.smithi002.stdout:9/183: rmdir d5/d9/d25/d23 39 2024-03-16T10:44:53.419 INFO:tasks.workunit.client.0.smithi002.stdout:4/215: dwrite d2/f37 [0,4194304] 0 2024-03-16T10:44:53.421 INFO:tasks.workunit.client.0.smithi002.stdout:4/216: dread - d2/d33/da/f28 zero size 2024-03-16T10:44:53.422 INFO:tasks.workunit.client.0.smithi002.stdout:2/212: symlink d7/d2e/d1d/d2f/l37 0 2024-03-16T10:44:53.426 INFO:tasks.workunit.client.0.smithi002.stdout:2/213: write d7/d2e/d1d/f21 [2943903,54850] 0 2024-03-16T10:44:53.426 INFO:tasks.workunit.client.0.smithi002.stdout:3/176: mknod d1/dd/d11/c42 0 2024-03-16T10:44:53.428 INFO:tasks.workunit.client.0.smithi002.stdout:6/202: dread f11 [0,4194304] 0 2024-03-16T10:44:53.428 INFO:tasks.workunit.client.0.smithi002.stdout:1/202: dwrite d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:53.428 INFO:tasks.workunit.client.0.smithi002.stdout:9/184: rmdir d5/d9/dc/d1e/d24/d31 39 2024-03-16T10:44:53.430 INFO:tasks.workunit.client.0.smithi002.stdout:9/185: write d5/d9/dc/d28/f2d [998574,51859] 0 2024-03-16T10:44:53.432 INFO:tasks.workunit.client.0.smithi002.stdout:4/217: unlink d2/d33/da/cc 0 2024-03-16T10:44:53.432 INFO:tasks.workunit.client.0.smithi002.stdout:7/170: rmdir d3 39 2024-03-16T10:44:53.432 INFO:tasks.workunit.client.0.smithi002.stdout:0/215: getdents d3/d28/d3d/d44 0 2024-03-16T10:44:53.433 INFO:tasks.workunit.client.0.smithi002.stdout:0/216: chown d3 1047973486 1 2024-03-16T10:44:53.434 INFO:tasks.workunit.client.0.smithi002.stdout:0/217: write d3/d28/f3b [1012513,64438] 0 2024-03-16T10:44:53.436 INFO:tasks.workunit.client.0.smithi002.stdout:0/218: fdatasync d3/f2d 0 2024-03-16T10:44:53.439 INFO:tasks.workunit.client.0.smithi002.stdout:4/218: dread d2/d6/f13 [0,4194304] 0 2024-03-16T10:44:53.441 INFO:tasks.workunit.client.0.smithi002.stdout:5/150: write d0/f18 [13368,59758] 0 2024-03-16T10:44:53.443 INFO:tasks.workunit.client.0.smithi002.stdout:1/203: write d2/df/f27 [1318340,94587] 0 2024-03-16T10:44:53.443 INFO:tasks.workunit.client.0.smithi002.stdout:3/177: rmdir d1/dd/d11/d15/d1e/d2c 39 2024-03-16T10:44:53.446 INFO:tasks.workunit.client.0.smithi002.stdout:0/219: dread d3/de/f15 [0,4194304] 0 2024-03-16T10:44:53.447 INFO:tasks.workunit.client.0.smithi002.stdout:5/151: truncate d0/d1f/f2c 4548684 0 2024-03-16T10:44:53.449 INFO:tasks.workunit.client.0.smithi002.stdout:2/214: dread d7/d2e/f1c [0,4194304] 0 2024-03-16T10:44:53.450 INFO:tasks.workunit.client.0.smithi002.stdout:6/203: symlink d15/l48 0 2024-03-16T10:44:53.450 INFO:tasks.workunit.client.0.smithi002.stdout:5/152: read d0/f18 [655795,126685] 0 2024-03-16T10:44:53.452 INFO:tasks.workunit.client.0.smithi002.stdout:4/219: mkdir d2/d33/d1c/d3d 0 2024-03-16T10:44:53.453 INFO:tasks.workunit.client.0.smithi002.stdout:7/171: symlink d3/d4/d10/l35 0 2024-03-16T10:44:53.453 INFO:tasks.workunit.client.0.smithi002.stdout:8/196: sync 2024-03-16T10:44:53.454 INFO:tasks.workunit.client.0.smithi002.stdout:8/197: fsync f4 0 2024-03-16T10:44:53.455 INFO:tasks.workunit.client.0.smithi002.stdout:9/186: dwrite d5/f7 [0,4194304] 0 2024-03-16T10:44:53.457 INFO:tasks.workunit.client.0.smithi002.stdout:3/178: rename d1/f3a to d1/dd/d11/d15/f43 0 2024-03-16T10:44:53.465 INFO:tasks.workunit.client.0.smithi002.stdout:0/220: creat d3/d28/d3d/f46 x:0 0 0 2024-03-16T10:44:53.465 INFO:tasks.workunit.client.0.smithi002.stdout:5/153: dread d0/d1f/f2c [0,4194304] 0 2024-03-16T10:44:53.465 INFO:tasks.workunit.client.0.smithi002.stdout:1/204: dwrite d2/f28 [4194304,4194304] 0 2024-03-16T10:44:53.465 INFO:tasks.workunit.client.0.smithi002.stdout:2/215: mkdir d7/d2e/d1d/d38 0 2024-03-16T10:44:53.466 INFO:tasks.workunit.client.0.smithi002.stdout:0/221: stat d3/d5/c42 0 2024-03-16T10:44:53.466 INFO:tasks.workunit.client.0.smithi002.stdout:6/204: mknod d15/d2d/d2b/d3f/c49 0 2024-03-16T10:44:53.467 INFO:tasks.workunit.client.0.smithi002.stdout:5/154: stat d0/d16/d1d/d2b 0 2024-03-16T10:44:53.468 INFO:tasks.workunit.client.0.smithi002.stdout:7/172: mknod d3/c36 0 2024-03-16T10:44:53.472 INFO:tasks.workunit.client.0.smithi002.stdout:8/198: unlink d7/d9/d25/f2c 0 2024-03-16T10:44:53.473 INFO:tasks.workunit.client.0.smithi002.stdout:7/173: chown d3/d4/d1c/f33 0 1 2024-03-16T10:44:53.473 INFO:tasks.workunit.client.0.smithi002.stdout:2/216: truncate d7/d2e/d1d/f26 929199 0 2024-03-16T10:44:53.475 INFO:tasks.workunit.client.0.smithi002.stdout:8/199: truncate d7/d9/d25/f32 375528 0 2024-03-16T10:44:53.476 INFO:tasks.workunit.client.0.smithi002.stdout:8/200: chown d7/f35 0 1 2024-03-16T10:44:53.476 INFO:tasks.workunit.client.0.smithi002.stdout:6/205: creat d15/d2d/d2b/f4a x:0 0 0 2024-03-16T10:44:53.477 INFO:tasks.workunit.client.0.smithi002.stdout:5/155: mknod d0/d16/d26/c30 0 2024-03-16T10:44:53.477 INFO:tasks.workunit.client.0.smithi002.stdout:0/222: dread d3/d5/f19 [0,4194304] 0 2024-03-16T10:44:53.479 INFO:tasks.workunit.client.0.smithi002.stdout:1/205: rename d2/df/d12/c3e to d2/d3/c47 0 2024-03-16T10:44:53.480 INFO:tasks.workunit.client.0.smithi002.stdout:1/206: chown d2/d3/l2e 4753226 1 2024-03-16T10:44:53.481 INFO:tasks.workunit.client.0.smithi002.stdout:7/174: mkdir d3/d37 0 2024-03-16T10:44:53.483 INFO:tasks.workunit.client.0.smithi002.stdout:9/187: creat d5/d9/d25/d23/f3b x:0 0 0 2024-03-16T10:44:53.485 INFO:tasks.workunit.client.0.smithi002.stdout:1/207: dread d2/df/d12/d29/f34 [0,4194304] 0 2024-03-16T10:44:53.485 INFO:tasks.workunit.client.0.smithi002.stdout:8/201: mknod d7/d9/d25/c43 0 2024-03-16T10:44:53.487 INFO:tasks.workunit.client.0.smithi002.stdout:1/208: read d2/d3/fc [879262,5786] 0 2024-03-16T10:44:53.487 INFO:tasks.workunit.client.0.smithi002.stdout:1/209: chown d2/df 417 1 2024-03-16T10:44:53.488 INFO:tasks.workunit.client.0.smithi002.stdout:6/206: mknod d15/d2d/d2c/c4b 0 2024-03-16T10:44:53.494 INFO:tasks.workunit.client.0.smithi002.stdout:9/188: mkdir d5/d9/d22/d3c 0 2024-03-16T10:44:53.497 INFO:tasks.workunit.client.0.smithi002.stdout:8/202: rename d7/f19 to d7/d9/d17/d23/d39/f44 0 2024-03-16T10:44:53.502 INFO:tasks.workunit.client.0.smithi002.stdout:3/179: fdatasync d1/dd/d11/d15/d1e/d25/f26 0 2024-03-16T10:44:53.502 INFO:tasks.workunit.client.0.smithi002.stdout:4/220: link d2/d33/da/f25 d2/d33/f3e 0 2024-03-16T10:44:53.510 INFO:tasks.workunit.client.0.smithi002.stdout:0/223: creat d3/f47 x:0 0 0 2024-03-16T10:44:53.512 INFO:tasks.workunit.client.0.smithi002.stdout:1/210: symlink d2/df/l48 0 2024-03-16T10:44:53.515 INFO:tasks.workunit.client.0.smithi002.stdout:6/207: symlink d15/d2d/l4c 0 2024-03-16T10:44:53.517 INFO:tasks.workunit.client.0.smithi002.stdout:4/221: fdatasync d2/d6/f10 0 2024-03-16T10:44:53.518 INFO:tasks.workunit.client.0.smithi002.stdout:3/180: creat d1/dd/d11/d15/d1e/d22/f44 x:0 0 0 2024-03-16T10:44:53.519 INFO:tasks.workunit.client.0.smithi002.stdout:5/156: link d0/d16/d1d/c27 d0/d16/d26/c31 0 2024-03-16T10:44:53.520 INFO:tasks.workunit.client.0.smithi002.stdout:6/208: dwrite fa [0,4194304] 0 2024-03-16T10:44:53.520 INFO:tasks.workunit.client.0.smithi002.stdout:7/175: link d3/cf d3/d4/d1c/c38 0 2024-03-16T10:44:53.520 INFO:tasks.workunit.client.0.smithi002.stdout:3/181: write d1/dd/d11/d15/d1e/d25/f26 [293135,26876] 0 2024-03-16T10:44:53.563 INFO:tasks.workunit.client.0.smithi002.stdout:2/217: getdents d7/db/d12 0 2024-03-16T10:44:53.566 INFO:tasks.workunit.client.0.smithi002.stdout:4/222: unlink d2/d6/cb 0 2024-03-16T10:44:53.566 INFO:tasks.workunit.client.0.smithi002.stdout:6/209: chown d15/d2d/d2b/c32 1001041 1 2024-03-16T10:44:53.569 INFO:tasks.workunit.client.0.smithi002.stdout:9/189: creat d5/d9/dc/d1e/d24/f3d x:0 0 0 2024-03-16T10:44:53.569 INFO:tasks.workunit.client.0.smithi002.stdout:7/176: rename d3/d4/d10/f2e to d3/d4/d2a/f39 0 2024-03-16T10:44:53.569 INFO:tasks.workunit.client.0.smithi002.stdout:8/203: link d7/fd d7/d9/d25/f45 0 2024-03-16T10:44:53.572 INFO:tasks.workunit.client.0.smithi002.stdout:3/182: read d1/dd/d11/d15/f17 [245526,101997] 0 2024-03-16T10:44:53.575 INFO:tasks.workunit.client.0.smithi002.stdout:6/210: rename d15/d2d/d2b/d3f/l47 to d15/d2d/d2b/l4d 0 2024-03-16T10:44:53.578 INFO:tasks.workunit.client.0.smithi002.stdout:9/190: dwrite d5/f7 [4194304,4194304] 0 2024-03-16T10:44:53.578 INFO:tasks.workunit.client.0.smithi002.stdout:7/177: dwrite d3/d4/f1e [0,4194304] 0 2024-03-16T10:44:53.581 INFO:tasks.workunit.client.0.smithi002.stdout:7/178: dread d3/f1a [0,4194304] 0 2024-03-16T10:44:53.585 INFO:tasks.workunit.client.0.smithi002.stdout:8/204: mknod d7/d9/d25/c46 0 2024-03-16T10:44:53.585 INFO:tasks.workunit.client.0.smithi002.stdout:6/211: dwrite d15/d2d/d2c/f3d [0,4194304] 0 2024-03-16T10:44:53.590 INFO:tasks.workunit.client.0.smithi002.stdout:2/218: link d7/db/d12/le d7/d2e/d1d/l39 0 2024-03-16T10:44:53.594 INFO:tasks.workunit.client.0.smithi002.stdout:1/211: write d2/df/d12/d29/f34 [222640,66729] 0 2024-03-16T10:44:53.595 INFO:tasks.workunit.client.0.smithi002.stdout:5/157: link d0/d16/d1d/d20/c2d d0/d16/d1d/d20/c32 0 2024-03-16T10:44:53.597 INFO:tasks.workunit.client.0.smithi002.stdout:4/223: write d2/d6/f1a [1310865,12355] 0 2024-03-16T10:44:53.597 INFO:tasks.workunit.client.0.smithi002.stdout:6/212: dwrite fa [4194304,4194304] 0 2024-03-16T10:44:53.597 INFO:tasks.workunit.client.0.smithi002.stdout:7/179: rename d3/f1b to d3/d4/d1d/d28/f3a 0 2024-03-16T10:44:53.600 INFO:tasks.workunit.client.0.smithi002.stdout:8/205: creat d7/d9/d17/d23/d2e/d3a/f47 x:0 0 0 2024-03-16T10:44:53.600 INFO:tasks.workunit.client.0.smithi002.stdout:6/213: stat d15/d2d/d2b/d3f/c49 0 2024-03-16T10:44:53.603 INFO:tasks.workunit.client.0.smithi002.stdout:6/214: dread - d15/f24 zero size 2024-03-16T10:44:53.603 INFO:tasks.workunit.client.0.smithi002.stdout:6/215: dread - d15/d2d/f28 zero size 2024-03-16T10:44:53.604 INFO:tasks.workunit.client.0.smithi002.stdout:9/191: mknod d5/d9/d22/d3c/c3e 0 2024-03-16T10:44:53.608 INFO:tasks.workunit.client.0.smithi002.stdout:1/212: unlink d2/d3/c9 0 2024-03-16T10:44:53.609 INFO:tasks.workunit.client.0.smithi002.stdout:5/158: mkdir d0/d16/d1d/d33 0 2024-03-16T10:44:53.611 INFO:tasks.workunit.client.0.smithi002.stdout:7/180: rename d3/f1a to d3/d4/d10/f3b 0 2024-03-16T10:44:53.613 INFO:tasks.workunit.client.0.smithi002.stdout:3/183: creat d1/f45 x:0 0 0 2024-03-16T10:44:53.617 INFO:tasks.workunit.client.0.smithi002.stdout:2/219: creat d7/d2e/d1d/d38/f3a x:0 0 0 2024-03-16T10:44:53.617 INFO:tasks.workunit.client.0.smithi002.stdout:4/224: dread d2/d33/da/ff [0,4194304] 0 2024-03-16T10:44:53.618 INFO:tasks.workunit.client.0.smithi002.stdout:1/213: creat d2/d3/f49 x:0 0 0 2024-03-16T10:44:53.621 INFO:tasks.workunit.client.0.smithi002.stdout:5/159: rename d0/d16/d26 to d0/d16/d34 0 2024-03-16T10:44:53.623 INFO:tasks.workunit.client.0.smithi002.stdout:7/181: creat d3/d4/d10/f3c x:0 0 0 2024-03-16T10:44:53.625 INFO:tasks.workunit.client.0.smithi002.stdout:6/216: symlink d15/d2d/l4e 0 2024-03-16T10:44:53.628 INFO:tasks.workunit.client.0.smithi002.stdout:9/192: mknod d5/d9/dc/d1e/d24/d31/c3f 0 2024-03-16T10:44:53.629 INFO:tasks.workunit.client.0.smithi002.stdout:7/182: dwrite d3/d4/d1c/f33 [0,4194304] 0 2024-03-16T10:44:53.630 INFO:tasks.workunit.client.0.smithi002.stdout:2/220: mkdir d7/d27/d3b 0 2024-03-16T10:44:53.630 INFO:tasks.workunit.client.0.smithi002.stdout:9/193: truncate d5/d9/dc/f16 255130 0 2024-03-16T10:44:53.630 INFO:tasks.workunit.client.0.smithi002.stdout:7/183: write d3/f27 [85117,95662] 0 2024-03-16T10:44:53.631 INFO:tasks.workunit.client.0.smithi002.stdout:4/225: readlink d2/d33/da/l2a 0 2024-03-16T10:44:53.635 INFO:tasks.workunit.client.0.smithi002.stdout:8/206: rename d7/f26 to d7/f48 0 2024-03-16T10:44:53.635 INFO:tasks.workunit.client.0.smithi002.stdout:8/207: stat d7/l24 0 2024-03-16T10:44:53.636 INFO:tasks.workunit.client.0.smithi002.stdout:5/160: symlink d0/d16/l35 0 2024-03-16T10:44:53.637 INFO:tasks.workunit.client.0.smithi002.stdout:5/161: chown d0/d16/d1d/l21 238634 1 2024-03-16T10:44:53.642 INFO:tasks.workunit.client.0.smithi002.stdout:3/184: symlink d1/dd/d11/d15/d1e/d22/l46 0 2024-03-16T10:44:53.643 INFO:tasks.workunit.client.0.smithi002.stdout:9/194: mknod d5/d9/d25/d23/c40 0 2024-03-16T10:44:53.646 INFO:tasks.workunit.client.0.smithi002.stdout:7/184: mkdir d3/d4/d1d/d3d 0 2024-03-16T10:44:53.646 INFO:tasks.workunit.client.0.smithi002.stdout:7/185: fsync d3/d4/d1c/f32 0 2024-03-16T10:44:53.653 INFO:tasks.workunit.client.0.smithi002.stdout:5/162: truncate d0/d16/f29 5276542 0 2024-03-16T10:44:53.656 INFO:tasks.workunit.client.0.smithi002.stdout:9/195: symlink d5/d9/l41 0 2024-03-16T10:44:53.658 INFO:tasks.workunit.client.0.smithi002.stdout:0/224: sync 2024-03-16T10:44:53.659 INFO:tasks.workunit.client.0.smithi002.stdout:2/221: mknod d7/d27/d3b/c3c 0 2024-03-16T10:44:53.660 INFO:tasks.workunit.client.0.smithi002.stdout:4/226: symlink d2/d33/d35/l3f 0 2024-03-16T10:44:53.661 INFO:tasks.workunit.client.0.smithi002.stdout:8/208: rename d7/d9/d17/d20/l3c to d7/d9/d17/d20/l49 0 2024-03-16T10:44:53.662 INFO:tasks.workunit.client.0.smithi002.stdout:1/214: write d2/d3/fc [603173,79953] 0 2024-03-16T10:44:53.662 INFO:tasks.workunit.client.0.smithi002.stdout:3/185: dwrite d1/f16 [0,4194304] 0 2024-03-16T10:44:53.665 INFO:tasks.workunit.client.0.smithi002.stdout:6/217: creat d15/f4f x:0 0 0 2024-03-16T10:44:53.670 INFO:tasks.workunit.client.0.smithi002.stdout:7/186: rename d3/d4/d1c/c38 to d3/d4/d10/c3e 0 2024-03-16T10:44:53.672 INFO:tasks.workunit.client.0.smithi002.stdout:0/225: mknod d3/d28/d3d/c48 0 2024-03-16T10:44:53.675 INFO:tasks.workunit.client.0.smithi002.stdout:5/163: stat d0/d16/f29 0 2024-03-16T10:44:53.678 INFO:tasks.workunit.client.0.smithi002.stdout:6/218: creat d15/d2d/d2c/f50 x:0 0 0 2024-03-16T10:44:53.680 INFO:tasks.workunit.client.0.smithi002.stdout:8/209: rename d7/f41 to d7/d9/d17/d23/d2e/f4a 0 2024-03-16T10:44:53.682 INFO:tasks.workunit.client.0.smithi002.stdout:1/215: dread d2/d3/fe [0,4194304] 0 2024-03-16T10:44:53.686 INFO:tasks.workunit.client.0.smithi002.stdout:5/164: truncate d0/f18 1856711 0 2024-03-16T10:44:53.690 INFO:tasks.workunit.client.0.smithi002.stdout:9/196: creat d5/d9/d25/f42 x:0 0 0 2024-03-16T10:44:53.695 INFO:tasks.workunit.client.0.smithi002.stdout:6/219: rename d15/d2d/l43 to d15/d2d/d2b/l51 0 2024-03-16T10:44:53.696 INFO:tasks.workunit.client.0.smithi002.stdout:9/197: dread d5/f7 [4194304,4194304] 0 2024-03-16T10:44:53.696 INFO:tasks.workunit.client.0.smithi002.stdout:3/186: truncate d1/dd/d11/d15/d1e/d25/f26 151574 0 2024-03-16T10:44:53.698 INFO:tasks.workunit.client.0.smithi002.stdout:9/198: fsync d5/d9/f13 0 2024-03-16T10:44:53.698 INFO:tasks.workunit.client.0.smithi002.stdout:4/227: dwrite d2/d33/da/f25 [0,4194304] 0 2024-03-16T10:44:53.703 INFO:tasks.workunit.client.0.smithi002.stdout:9/199: write d5/d9/dc/d28/f2d [957799,52465] 0 2024-03-16T10:44:53.704 INFO:tasks.workunit.client.0.smithi002.stdout:5/165: read d0/f18 [841540,106985] 0 2024-03-16T10:44:53.707 INFO:tasks.workunit.client.0.smithi002.stdout:6/220: dwrite d15/f24 [0,4194304] 0 2024-03-16T10:44:53.707 INFO:tasks.workunit.client.0.smithi002.stdout:0/226: getdents d3/d28 0 2024-03-16T10:44:53.714 INFO:tasks.workunit.client.0.smithi002.stdout:4/228: creat d2/d33/d35/f40 x:0 0 0 2024-03-16T10:44:53.721 INFO:tasks.workunit.client.0.smithi002.stdout:3/187: unlink d1/dd/c3b 0 2024-03-16T10:44:53.724 INFO:tasks.workunit.client.0.smithi002.stdout:9/200: creat d5/d9/dc/f43 x:0 0 0 2024-03-16T10:44:53.729 INFO:tasks.workunit.client.0.smithi002.stdout:6/221: dread d15/f1a [0,4194304] 0 2024-03-16T10:44:53.729 INFO:tasks.workunit.client.0.smithi002.stdout:6/222: dread - d15/d2d/f34 zero size 2024-03-16T10:44:53.731 INFO:tasks.workunit.client.0.smithi002.stdout:8/210: link l3 d7/l4b 0 2024-03-16T10:44:53.733 INFO:tasks.workunit.client.0.smithi002.stdout:4/229: creat d2/d33/da/f41 x:0 0 0 2024-03-16T10:44:53.735 INFO:tasks.workunit.client.0.smithi002.stdout:1/216: getdents d2/d3/d1c 0 2024-03-16T10:44:53.737 INFO:tasks.workunit.client.0.smithi002.stdout:8/211: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:53.739 INFO:tasks.workunit.client.0.smithi002.stdout:5/166: write d0/f18 [2272440,94066] 0 2024-03-16T10:44:53.742 INFO:tasks.workunit.client.0.smithi002.stdout:1/217: dread d2/d3/f1f [0,4194304] 0 2024-03-16T10:44:53.743 INFO:tasks.workunit.client.0.smithi002.stdout:6/223: creat d15/d2d/d2b/d3f/f52 x:0 0 0 2024-03-16T10:44:53.744 INFO:tasks.workunit.client.0.smithi002.stdout:1/218: read - d2/df/d12/f3f zero size 2024-03-16T10:44:53.745 INFO:tasks.workunit.client.0.smithi002.stdout:4/230: rmdir d2/d33/d1c 39 2024-03-16T10:44:53.747 INFO:tasks.workunit.client.0.smithi002.stdout:5/167: dread d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:44:53.747 INFO:tasks.workunit.client.0.smithi002.stdout:1/219: write d2/d3/d1c/f44 [729187,117069] 0 2024-03-16T10:44:53.748 INFO:tasks.workunit.client.0.smithi002.stdout:3/188: creat d1/dd/d11/d15/d1e/d22/d3f/f47 x:0 0 0 2024-03-16T10:44:53.749 INFO:tasks.workunit.client.0.smithi002.stdout:9/201: symlink d5/d9/d25/l44 0 2024-03-16T10:44:53.750 INFO:tasks.workunit.client.0.smithi002.stdout:5/168: read d0/f18 [1497433,66004] 0 2024-03-16T10:44:53.753 INFO:tasks.workunit.client.0.smithi002.stdout:8/212: rename d7/d9/d17/f1f to d7/d13/f4c 0 2024-03-16T10:44:53.755 INFO:tasks.workunit.client.0.smithi002.stdout:4/231: dwrite d2/d6/f22 [0,4194304] 0 2024-03-16T10:44:53.757 INFO:tasks.workunit.client.0.smithi002.stdout:5/169: dread d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:44:53.763 INFO:tasks.workunit.client.0.smithi002.stdout:5/170: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:53.765 INFO:tasks.workunit.client.0.smithi002.stdout:6/224: rename d15/d2d/d2b/f4a to d15/d2d/d2b/f53 0 2024-03-16T10:44:53.767 INFO:tasks.workunit.client.0.smithi002.stdout:6/225: truncate d15/d2d/d2b/d3f/f52 934588 0 2024-03-16T10:44:53.773 INFO:tasks.workunit.client.0.smithi002.stdout:3/189: unlink d1/dd/d11/f20 0 2024-03-16T10:44:53.773 INFO:tasks.workunit.client.0.smithi002.stdout:6/226: dwrite d15/d2d/d2c/f3c [0,4194304] 0 2024-03-16T10:44:53.782 INFO:tasks.workunit.client.0.smithi002.stdout:8/213: creat d7/d9/d17/d23/f4d x:0 0 0 2024-03-16T10:44:53.786 INFO:tasks.workunit.client.0.smithi002.stdout:4/232: creat d2/d33/da/f42 x:0 0 0 2024-03-16T10:44:53.788 INFO:tasks.workunit.client.0.smithi002.stdout:0/227: getdents d3/d5 0 2024-03-16T10:44:53.788 INFO:tasks.workunit.client.0.smithi002.stdout:0/228: dread - d3/d28/f33 zero size 2024-03-16T10:44:53.798 INFO:tasks.workunit.client.0.smithi002.stdout:6/227: creat d15/d2d/d2c/f54 x:0 0 0 2024-03-16T10:44:53.801 INFO:tasks.workunit.client.0.smithi002.stdout:1/220: link d2/d3/fc d2/d3/d1c/f4a 0 2024-03-16T10:44:53.801 INFO:tasks.workunit.client.0.smithi002.stdout:1/221: chown d2/d3/f8 160772 1 2024-03-16T10:44:53.807 INFO:tasks.workunit.client.0.smithi002.stdout:1/222: dwrite d2/d3/f49 [0,4194304] 0 2024-03-16T10:44:53.807 INFO:tasks.workunit.client.0.smithi002.stdout:8/214: read d7/fc [1279802,53158] 0 2024-03-16T10:44:53.808 INFO:tasks.workunit.client.0.smithi002.stdout:1/223: write d2/df/d12/f3f [887143,91644] 0 2024-03-16T10:44:53.813 INFO:tasks.workunit.client.0.smithi002.stdout:3/190: mknod d1/dd/d11/d15/d1e/d2c/c48 0 2024-03-16T10:44:53.814 INFO:tasks.workunit.client.0.smithi002.stdout:3/191: chown d1/dd/d11/d15/f17 1689886 1 2024-03-16T10:44:53.815 INFO:tasks.workunit.client.0.smithi002.stdout:6/228: rename f13 to d15/d2d/d2c/f55 0 2024-03-16T10:44:53.816 INFO:tasks.workunit.client.0.smithi002.stdout:9/202: link d5/c26 d5/d9/d25/d23/c45 0 2024-03-16T10:44:53.816 INFO:tasks.workunit.client.0.smithi002.stdout:7/187: sync 2024-03-16T10:44:53.817 INFO:tasks.workunit.client.0.smithi002.stdout:2/222: sync 2024-03-16T10:44:53.820 INFO:tasks.workunit.client.0.smithi002.stdout:4/233: mkdir d2/d33/d1c/d43 0 2024-03-16T10:44:53.826 INFO:tasks.workunit.client.0.smithi002.stdout:9/203: dwrite d5/d9/d25/d23/f3b [0,4194304] 0 2024-03-16T10:44:53.827 INFO:tasks.workunit.client.0.smithi002.stdout:8/215: dread d7/d9/fa [0,4194304] 0 2024-03-16T10:44:53.827 INFO:tasks.workunit.client.0.smithi002.stdout:3/192: chown d1/l7 62072563 1 2024-03-16T10:44:53.827 INFO:tasks.workunit.client.0.smithi002.stdout:9/204: chown d5/l2c 175128 1 2024-03-16T10:44:53.828 INFO:tasks.workunit.client.0.smithi002.stdout:8/216: fsync f4 0 2024-03-16T10:44:53.830 INFO:tasks.workunit.client.0.smithi002.stdout:8/217: read d7/d9/d25/f45 [2396599,37464] 0 2024-03-16T10:44:53.830 INFO:tasks.workunit.client.0.smithi002.stdout:8/218: chown d7/fc 12959743 1 2024-03-16T10:44:53.831 INFO:tasks.workunit.client.0.smithi002.stdout:7/188: unlink d3/d4/l17 0 2024-03-16T10:44:53.833 INFO:tasks.workunit.client.0.smithi002.stdout:2/223: mknod d7/d27/c3d 0 2024-03-16T10:44:53.840 INFO:tasks.workunit.client.0.smithi002.stdout:0/229: creat d3/f49 x:0 0 0 2024-03-16T10:44:53.845 INFO:tasks.workunit.client.0.smithi002.stdout:5/171: getdents d0 0 2024-03-16T10:44:53.847 INFO:tasks.workunit.client.0.smithi002.stdout:3/193: creat d1/dd/d28/d37/f49 x:0 0 0 2024-03-16T10:44:53.849 INFO:tasks.workunit.client.0.smithi002.stdout:6/229: chown f4 63743 1 2024-03-16T10:44:53.849 INFO:tasks.workunit.client.0.smithi002.stdout:1/224: dwrite d2/d3/f17 [0,4194304] 0 2024-03-16T10:44:53.851 INFO:tasks.workunit.client.0.smithi002.stdout:7/189: fsync d3/d4/d1d/d28/f3a 0 2024-03-16T10:44:53.852 INFO:tasks.workunit.client.0.smithi002.stdout:7/190: dread d3/d4/d1c/f32 [0,4194304] 0 2024-03-16T10:44:53.854 INFO:tasks.workunit.client.0.smithi002.stdout:4/234: getdents d2/d33/d1c/d3d 0 2024-03-16T10:44:53.856 INFO:tasks.workunit.client.0.smithi002.stdout:5/172: creat d0/d1f/f36 x:0 0 0 2024-03-16T10:44:53.858 INFO:tasks.workunit.client.0.smithi002.stdout:3/194: creat d1/dd/d11/f4a x:0 0 0 2024-03-16T10:44:53.859 INFO:tasks.workunit.client.0.smithi002.stdout:2/224: dread d7/d2e/f16 [0,4194304] 0 2024-03-16T10:44:53.860 INFO:tasks.workunit.client.0.smithi002.stdout:3/195: truncate d1/dd/d11/f4a 406349 0 2024-03-16T10:44:53.860 INFO:tasks.workunit.client.0.smithi002.stdout:6/230: stat lf 0 2024-03-16T10:44:53.861 INFO:tasks.workunit.client.0.smithi002.stdout:5/173: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:53.863 INFO:tasks.workunit.client.0.smithi002.stdout:3/196: truncate d1/dd/d28/d37/f49 940797 0 2024-03-16T10:44:53.864 INFO:tasks.workunit.client.0.smithi002.stdout:7/191: mkdir d3/d4/d10/d3f 0 2024-03-16T10:44:53.865 INFO:tasks.workunit.client.0.smithi002.stdout:0/230: symlink d3/de/l4a 0 2024-03-16T10:44:53.866 INFO:tasks.workunit.client.0.smithi002.stdout:0/231: write d3/d28/f34 [422557,47456] 0 2024-03-16T10:44:53.867 INFO:tasks.workunit.client.0.smithi002.stdout:0/232: dread - d3/d28/f33 zero size 2024-03-16T10:44:53.868 INFO:tasks.workunit.client.0.smithi002.stdout:9/205: creat d5/d9/dc/d1e/d24/f46 x:0 0 0 2024-03-16T10:44:53.871 INFO:tasks.workunit.client.0.smithi002.stdout:2/225: rename d7/db/d12/f31 to d7/d2e/d1d/f3e 0 2024-03-16T10:44:53.872 INFO:tasks.workunit.client.0.smithi002.stdout:0/233: dwrite d3/de/f10 [0,4194304] 0 2024-03-16T10:44:53.874 INFO:tasks.workunit.client.0.smithi002.stdout:0/234: read d3/f3a [3874515,124958] 0 2024-03-16T10:44:53.878 INFO:tasks.workunit.client.0.smithi002.stdout:2/226: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:53.883 INFO:tasks.workunit.client.0.smithi002.stdout:4/235: symlink d2/d33/d3b/l44 0 2024-03-16T10:44:53.884 INFO:tasks.workunit.client.0.smithi002.stdout:2/227: dwrite d7/d2e/f1a [0,4194304] 0 2024-03-16T10:44:53.891 INFO:tasks.workunit.client.0.smithi002.stdout:9/206: creat d5/d9/d22/f47 x:0 0 0 2024-03-16T10:44:53.892 INFO:tasks.workunit.client.0.smithi002.stdout:9/207: read - d5/d9/dc/d1e/d24/f46 zero size 2024-03-16T10:44:53.902 INFO:tasks.workunit.client.0.smithi002.stdout:2/228: dread d7/db/d12/f22 [0,4194304] 0 2024-03-16T10:44:53.902 INFO:tasks.workunit.client.0.smithi002.stdout:5/174: fdatasync d0/d1f/f2c 0 2024-03-16T10:44:53.905 INFO:tasks.workunit.client.0.smithi002.stdout:3/197: unlink d1/dd/d11/d15/d1e/d22/l2d 0 2024-03-16T10:44:53.906 INFO:tasks.workunit.client.0.smithi002.stdout:3/198: truncate d1/dd/d28/f31 330954 0 2024-03-16T10:44:53.909 INFO:tasks.workunit.client.0.smithi002.stdout:9/208: truncate d5/f7 681057 0 2024-03-16T10:44:53.910 INFO:tasks.workunit.client.0.smithi002.stdout:1/225: truncate d2/d3/fe 3019923 0 2024-03-16T10:44:53.910 INFO:tasks.workunit.client.0.smithi002.stdout:9/209: fdatasync d5/d9/d25/f42 0 2024-03-16T10:44:53.929 INFO:tasks.workunit.client.0.smithi002.stdout:6/231: getdents d15/d2d 0 2024-03-16T10:44:53.933 INFO:tasks.workunit.client.0.smithi002.stdout:6/232: dread f8 [4194304,4194304] 0 2024-03-16T10:44:53.939 INFO:tasks.workunit.client.0.smithi002.stdout:7/192: link d3/f11 d3/d4/d2a/f40 0 2024-03-16T10:44:53.943 INFO:tasks.workunit.client.0.smithi002.stdout:0/235: dwrite d3/de/f20 [0,4194304] 0 2024-03-16T10:44:53.949 INFO:tasks.workunit.client.0.smithi002.stdout:5/175: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:53.951 INFO:tasks.workunit.client.0.smithi002.stdout:2/229: rmdir d7/d2e/d34 0 2024-03-16T10:44:53.953 INFO:tasks.workunit.client.0.smithi002.stdout:4/236: truncate d2/d33/da/f25 916231 0 2024-03-16T10:44:53.968 INFO:tasks.workunit.client.0.smithi002.stdout:2/230: rmdir d7/d27 39 2024-03-16T10:44:53.969 INFO:tasks.workunit.client.0.smithi002.stdout:0/236: dwrite d3/d5/f23 [0,4194304] 0 2024-03-16T10:44:53.970 INFO:tasks.workunit.client.0.smithi002.stdout:7/193: creat d3/d4/d10/d3f/f41 x:0 0 0 2024-03-16T10:44:53.971 INFO:tasks.workunit.client.0.smithi002.stdout:2/231: write d7/d2e/d1d/f26 [1578556,85520] 0 2024-03-16T10:44:53.971 INFO:tasks.workunit.client.0.smithi002.stdout:6/233: write fc [326177,19499] 0 2024-03-16T10:44:53.972 INFO:tasks.workunit.client.0.smithi002.stdout:0/237: write d3/de/f20 [1999235,51996] 0 2024-03-16T10:44:53.973 INFO:tasks.workunit.client.0.smithi002.stdout:4/237: mkdir d2/d33/d1c/d45 0 2024-03-16T10:44:53.974 INFO:tasks.workunit.client.0.smithi002.stdout:6/234: chown d15/f2f 3814 1 2024-03-16T10:44:53.974 INFO:tasks.workunit.client.0.smithi002.stdout:3/199: link d1/lf d1/dd/d11/l4b 0 2024-03-16T10:44:53.976 INFO:tasks.workunit.client.0.smithi002.stdout:5/176: dwrite d0/f18 [0,4194304] 0 2024-03-16T10:44:53.978 INFO:tasks.workunit.client.0.smithi002.stdout:5/177: read d0/f18 [1841325,51775] 0 2024-03-16T10:44:53.979 INFO:tasks.workunit.client.0.smithi002.stdout:5/178: write d0/f18 [3627207,8205] 0 2024-03-16T10:44:53.980 INFO:tasks.workunit.client.0.smithi002.stdout:5/179: write d0/f18 [1140614,49325] 0 2024-03-16T10:44:53.988 INFO:tasks.workunit.client.0.smithi002.stdout:7/194: creat d3/d4/d1d/d28/f42 x:0 0 0 2024-03-16T10:44:53.995 INFO:tasks.workunit.client.0.smithi002.stdout:6/235: creat d15/d2d/d2c/f56 x:0 0 0 2024-03-16T10:44:53.995 INFO:tasks.workunit.client.0.smithi002.stdout:6/236: chown d15/l26 482452 1 2024-03-16T10:44:53.997 INFO:tasks.workunit.client.0.smithi002.stdout:3/200: symlink d1/dd/d11/d15/d1e/d2c/l4c 0 2024-03-16T10:44:54.000 INFO:tasks.workunit.client.0.smithi002.stdout:5/180: creat d0/d16/f37 x:0 0 0 2024-03-16T10:44:54.000 INFO:tasks.workunit.client.0.smithi002.stdout:2/232: mkdir d7/d27/d3f 0 2024-03-16T10:44:54.006 INFO:tasks.workunit.client.0.smithi002.stdout:6/237: symlink d15/d2d/d2b/l57 0 2024-03-16T10:44:54.007 INFO:tasks.workunit.client.0.smithi002.stdout:3/201: creat d1/dd/d11/d15/d1e/d22/d3f/f4d x:0 0 0 2024-03-16T10:44:54.008 INFO:tasks.workunit.client.0.smithi002.stdout:5/181: truncate d0/d16/f29 4456603 0 2024-03-16T10:44:54.010 INFO:tasks.workunit.client.0.smithi002.stdout:9/210: sync 2024-03-16T10:44:54.010 INFO:tasks.workunit.client.0.smithi002.stdout:8/219: sync 2024-03-16T10:44:54.012 INFO:tasks.workunit.client.0.smithi002.stdout:8/220: read d7/d9/f2f [1458760,22331] 0 2024-03-16T10:44:54.012 INFO:tasks.workunit.client.0.smithi002.stdout:4/238: rmdir d2/d33/d1c/d43 0 2024-03-16T10:44:54.013 INFO:tasks.workunit.client.0.smithi002.stdout:9/211: truncate d5/d9/dc/d1e/f36 201826 0 2024-03-16T10:44:54.013 INFO:tasks.workunit.client.0.smithi002.stdout:8/221: chown d7/f8 702 1 2024-03-16T10:44:54.013 INFO:tasks.workunit.client.0.smithi002.stdout:1/226: truncate d2/d3/fe 4038770 0 2024-03-16T10:44:54.015 INFO:tasks.workunit.client.0.smithi002.stdout:6/238: symlink d15/d2d/d2b/d3f/l58 0 2024-03-16T10:44:54.016 INFO:tasks.workunit.client.0.smithi002.stdout:3/202: creat d1/dd/d28/d37/f4e x:0 0 0 2024-03-16T10:44:54.017 INFO:tasks.workunit.client.0.smithi002.stdout:3/203: chown d1/dd/d28/d37/f49 539109 1 2024-03-16T10:44:54.017 INFO:tasks.workunit.client.0.smithi002.stdout:5/182: mknod d0/d16/d1d/d2b/c38 0 2024-03-16T10:44:54.019 INFO:tasks.workunit.client.0.smithi002.stdout:2/233: mkdir d7/d2e/d28/d40 0 2024-03-16T10:44:54.021 INFO:tasks.workunit.client.0.smithi002.stdout:0/238: dwrite d3/de/f15 [0,4194304] 0 2024-03-16T10:44:54.023 INFO:tasks.workunit.client.0.smithi002.stdout:0/239: truncate d3/d27/f3c 4995106 0 2024-03-16T10:44:54.023 INFO:tasks.workunit.client.0.smithi002.stdout:4/239: creat d2/d6/f46 x:0 0 0 2024-03-16T10:44:54.024 INFO:tasks.workunit.client.0.smithi002.stdout:9/212: creat d5/d9/d25/d23/f48 x:0 0 0 2024-03-16T10:44:54.025 INFO:tasks.workunit.client.0.smithi002.stdout:1/227: rename d2/d3/f49 to d2/d3/f4b 0 2024-03-16T10:44:54.026 INFO:tasks.workunit.client.0.smithi002.stdout:9/213: stat d5/d9/dc/f43 0 2024-03-16T10:44:54.027 INFO:tasks.workunit.client.0.smithi002.stdout:9/214: fdatasync d5/d9/dc/d1e/d24/f46 0 2024-03-16T10:44:54.028 INFO:tasks.workunit.client.0.smithi002.stdout:9/215: dread - d5/d9/dc/d1e/d24/f3d zero size 2024-03-16T10:44:54.029 INFO:tasks.workunit.client.0.smithi002.stdout:6/239: rename d15/d2d/d2b/c32 to d15/d2d/d2b/d3f/c59 0 2024-03-16T10:44:54.029 INFO:tasks.workunit.client.0.smithi002.stdout:1/228: rename d2/df/d12/d29 to d2/df/d12/d29/d4c 22 2024-03-16T10:44:54.030 INFO:tasks.workunit.client.0.smithi002.stdout:9/216: fsync d5/d9/dc/d1e/f36 0 2024-03-16T10:44:54.031 INFO:tasks.workunit.client.0.smithi002.stdout:5/183: mkdir d0/d16/d1d/d20/d39 0 2024-03-16T10:44:54.032 INFO:tasks.workunit.client.0.smithi002.stdout:7/195: getdents d3 0 2024-03-16T10:44:54.035 INFO:tasks.workunit.client.0.smithi002.stdout:2/234: creat d7/d2e/d1d/f41 x:0 0 0 2024-03-16T10:44:54.035 INFO:tasks.workunit.client.0.smithi002.stdout:9/217: dread d5/d9/dc/d1e/d24/d31/f32 [4194304,4194304] 0 2024-03-16T10:44:54.085 INFO:tasks.workunit.client.0.smithi002.stdout:2/235: chown d7/d2e/d1d/d38 0 1 2024-03-16T10:44:54.085 INFO:tasks.workunit.client.0.smithi002.stdout:0/240: creat d3/d31/f4b x:0 0 0 2024-03-16T10:44:54.085 INFO:tasks.workunit.client.0.smithi002.stdout:4/240: creat d2/d33/da/d2e/f47 x:0 0 0 2024-03-16T10:44:54.087 INFO:tasks.workunit.client.0.smithi002.stdout:4/241: readlink d2/d33/da/l2a 0 2024-03-16T10:44:54.087 INFO:tasks.workunit.client.0.smithi002.stdout:7/196: creat d3/d4/d10/d3f/f43 x:0 0 0 2024-03-16T10:44:54.089 INFO:tasks.workunit.client.0.smithi002.stdout:8/222: link d7/f31 d7/d9/d17/d23/d2e/f4e 0 2024-03-16T10:44:54.089 INFO:tasks.workunit.client.0.smithi002.stdout:9/218: stat d5/fd 0 2024-03-16T10:44:54.092 INFO:tasks.workunit.client.0.smithi002.stdout:2/236: dwrite d7/d2e/d1d/f26 [0,4194304] 0 2024-03-16T10:44:54.097 INFO:tasks.workunit.client.0.smithi002.stdout:7/197: mkdir d3/d4/d1c/d44 0 2024-03-16T10:44:54.097 INFO:tasks.workunit.client.0.smithi002.stdout:7/198: chown d3/d4/lc 1942 1 2024-03-16T10:44:54.098 INFO:tasks.workunit.client.0.smithi002.stdout:3/204: getdents d1 0 2024-03-16T10:44:54.099 INFO:tasks.workunit.client.0.smithi002.stdout:7/199: truncate d3/d4/d10/d3f/f41 701114 0 2024-03-16T10:44:54.102 INFO:tasks.workunit.client.0.smithi002.stdout:8/223: symlink d7/d9/d25/l4f 0 2024-03-16T10:44:54.104 INFO:tasks.workunit.client.0.smithi002.stdout:1/229: link d2/c3d d2/d3/c4d 0 2024-03-16T10:44:54.110 INFO:tasks.workunit.client.0.smithi002.stdout:4/242: symlink d2/l48 0 2024-03-16T10:44:54.111 INFO:tasks.workunit.client.0.smithi002.stdout:4/243: truncate d2/d33/da/f42 589552 0 2024-03-16T10:44:54.113 INFO:tasks.workunit.client.0.smithi002.stdout:9/219: truncate d5/d9/fe 5757895 0 2024-03-16T10:44:54.115 INFO:tasks.workunit.client.0.smithi002.stdout:2/237: getdents d7/d2e/d28/d40 0 2024-03-16T10:44:54.117 INFO:tasks.workunit.client.0.smithi002.stdout:0/241: getdents d3/d27 0 2024-03-16T10:44:54.118 INFO:tasks.workunit.client.0.smithi002.stdout:1/230: dread d2/df/f27 [0,4194304] 0 2024-03-16T10:44:54.118 INFO:tasks.workunit.client.0.smithi002.stdout:4/244: dwrite d2/d33/da/f42 [0,4194304] 0 2024-03-16T10:44:54.125 INFO:tasks.workunit.client.0.smithi002.stdout:9/220: read d5/f7 [304119,110113] 0 2024-03-16T10:44:54.129 INFO:tasks.workunit.client.0.smithi002.stdout:2/238: mknod d7/d2e/d1d/d2f/c42 0 2024-03-16T10:44:54.131 INFO:tasks.workunit.client.0.smithi002.stdout:0/242: chown d3/l16 25952300 1 2024-03-16T10:44:54.131 INFO:tasks.workunit.client.0.smithi002.stdout:8/224: link d7/d13/l28 d7/d9/d25/l50 0 2024-03-16T10:44:54.133 INFO:tasks.workunit.client.0.smithi002.stdout:0/243: chown d3/d5/fd 54921263 1 2024-03-16T10:44:54.133 INFO:tasks.workunit.client.0.smithi002.stdout:8/225: read f4 [1629109,79198] 0 2024-03-16T10:44:54.135 INFO:tasks.workunit.client.0.smithi002.stdout:5/184: dwrite d0/d16/f29 [0,4194304] 0 2024-03-16T10:44:54.135 INFO:tasks.workunit.client.0.smithi002.stdout:1/231: write d2/d3/f4b [1662708,86001] 0 2024-03-16T10:44:54.136 INFO:tasks.workunit.client.0.smithi002.stdout:1/232: fdatasync d2/f28 0 2024-03-16T10:44:54.137 INFO:tasks.workunit.client.0.smithi002.stdout:9/221: creat d5/d9/dc/d28/f49 x:0 0 0 2024-03-16T10:44:54.137 INFO:tasks.workunit.client.0.smithi002.stdout:5/185: write d0/d1f/f2c [3042856,12955] 0 2024-03-16T10:44:54.141 INFO:tasks.workunit.client.0.smithi002.stdout:8/226: fdatasync d7/fd 0 2024-03-16T10:44:54.142 INFO:tasks.workunit.client.0.smithi002.stdout:9/222: write d5/d9/dc/f16 [28448,38104] 0 2024-03-16T10:44:54.144 INFO:tasks.workunit.client.0.smithi002.stdout:0/244: unlink d3/c7 0 2024-03-16T10:44:54.146 INFO:tasks.workunit.client.0.smithi002.stdout:8/227: dread d7/d13/f4c [0,4194304] 0 2024-03-16T10:44:54.146 INFO:tasks.workunit.client.0.smithi002.stdout:0/245: chown d3/d31 1708166 1 2024-03-16T10:44:54.146 INFO:tasks.workunit.client.0.smithi002.stdout:4/245: dwrite d2/d33/da/ff [0,4194304] 0 2024-03-16T10:44:54.148 INFO:tasks.workunit.client.0.smithi002.stdout:8/228: write d7/d9/d25/f32 [1160192,8929] 0 2024-03-16T10:44:54.149 INFO:tasks.workunit.client.0.smithi002.stdout:2/239: rename d7/d2e/l36 to d7/l43 0 2024-03-16T10:44:54.150 INFO:tasks.workunit.client.0.smithi002.stdout:8/229: stat d7/d9/d17/d20/c22 0 2024-03-16T10:44:54.151 INFO:tasks.workunit.client.0.smithi002.stdout:2/240: chown d7/db/d12 307420276 1 2024-03-16T10:44:54.152 INFO:tasks.workunit.client.0.smithi002.stdout:9/223: creat d5/d9/dc/d1e/d24/d31/f4a x:0 0 0 2024-03-16T10:44:54.155 INFO:tasks.workunit.client.0.smithi002.stdout:9/224: write d5/d9/dc/d28/f2d [885759,92905] 0 2024-03-16T10:44:54.156 INFO:tasks.workunit.client.0.smithi002.stdout:5/186: rename d0/l11 to d0/d16/d1d/l3a 0 2024-03-16T10:44:54.157 INFO:tasks.workunit.client.0.smithi002.stdout:1/233: creat d2/df/d12/f4e x:0 0 0 2024-03-16T10:44:54.158 INFO:tasks.workunit.client.0.smithi002.stdout:2/241: mkdir d7/d2e/d28/d44 0 2024-03-16T10:44:54.160 INFO:tasks.workunit.client.0.smithi002.stdout:1/234: chown d2/df/l48 325522886 1 2024-03-16T10:44:54.160 INFO:tasks.workunit.client.0.smithi002.stdout:2/242: dread - d7/db/f2a zero size 2024-03-16T10:44:54.160 INFO:tasks.workunit.client.0.smithi002.stdout:4/246: rmdir d2/d33/d1c/d3d 0 2024-03-16T10:44:54.161 INFO:tasks.workunit.client.0.smithi002.stdout:0/246: dwrite d3/d5/f19 [0,4194304] 0 2024-03-16T10:44:54.162 INFO:tasks.workunit.client.0.smithi002.stdout:8/230: creat d7/d9/f51 x:0 0 0 2024-03-16T10:44:54.166 INFO:tasks.workunit.client.0.smithi002.stdout:9/225: truncate d5/d9/dc/d1e/d24/d31/f4a 459573 0 2024-03-16T10:44:54.168 INFO:tasks.workunit.client.0.smithi002.stdout:0/247: creat d3/d27/f4c x:0 0 0 2024-03-16T10:44:54.171 INFO:tasks.workunit.client.0.smithi002.stdout:9/226: dread d5/d9/dc/d1e/f36 [0,4194304] 0 2024-03-16T10:44:54.172 INFO:tasks.workunit.client.0.smithi002.stdout:1/235: dread d2/d3/f4b [0,4194304] 0 2024-03-16T10:44:54.172 INFO:tasks.workunit.client.0.smithi002.stdout:4/247: dread d2/d33/f1b [0,4194304] 0 2024-03-16T10:44:54.173 INFO:tasks.workunit.client.0.smithi002.stdout:2/243: dwrite d7/d2e/d1d/f26 [0,4194304] 0 2024-03-16T10:44:54.174 INFO:tasks.workunit.client.0.smithi002.stdout:0/248: creat d3/d28/f4d x:0 0 0 2024-03-16T10:44:54.175 INFO:tasks.workunit.client.0.smithi002.stdout:4/248: chown d2/d33/da/d2e 10 1 2024-03-16T10:44:54.179 INFO:tasks.workunit.client.0.smithi002.stdout:4/249: dread - d2/d33/da/f28 zero size 2024-03-16T10:44:54.186 INFO:tasks.workunit.client.0.smithi002.stdout:5/187: creat d0/d16/d1d/f3b x:0 0 0 2024-03-16T10:44:54.187 INFO:tasks.workunit.client.0.smithi002.stdout:2/244: write d7/d2e/f32 [4582852,39561] 0 2024-03-16T10:44:54.188 INFO:tasks.workunit.client.0.smithi002.stdout:8/231: getdents d7/d9/d17/d23/d2e 0 2024-03-16T10:44:54.188 INFO:tasks.workunit.client.0.smithi002.stdout:1/236: dwrite d2/df/d12/f4e [0,4194304] 0 2024-03-16T10:44:54.189 INFO:tasks.workunit.client.0.smithi002.stdout:4/250: creat d2/d33/da/f49 x:0 0 0 2024-03-16T10:44:54.191 INFO:tasks.workunit.client.0.smithi002.stdout:9/227: creat d5/d9/dc/d1e/d24/f4b x:0 0 0 2024-03-16T10:44:54.192 INFO:tasks.workunit.client.0.smithi002.stdout:6/240: sync 2024-03-16T10:44:54.197 INFO:tasks.workunit.client.0.smithi002.stdout:0/249: dwrite d3/d28/d3d/f3f [0,4194304] 0 2024-03-16T10:44:54.198 INFO:tasks.workunit.client.0.smithi002.stdout:4/251: symlink d2/d33/d3b/l4a 0 2024-03-16T10:44:54.200 INFO:tasks.workunit.client.0.smithi002.stdout:6/241: symlink d15/d2d/d2b/d3f/l5a 0 2024-03-16T10:44:54.200 INFO:tasks.workunit.client.0.smithi002.stdout:9/228: creat d5/d9/dc/d28/f4c x:0 0 0 2024-03-16T10:44:54.202 INFO:tasks.workunit.client.0.smithi002.stdout:5/188: dwrite d0/d16/f19 [0,4194304] 0 2024-03-16T10:44:54.204 INFO:tasks.workunit.client.0.smithi002.stdout:5/189: fdatasync d0/d16/f37 0 2024-03-16T10:44:54.205 INFO:tasks.workunit.client.0.smithi002.stdout:8/232: rename d7/d9/d17/d23/d2e/f4a to d7/d9/d17/d23/d2e/f52 0 2024-03-16T10:44:54.207 INFO:tasks.workunit.client.0.smithi002.stdout:0/250: stat d3/d5/f30 0 2024-03-16T10:44:54.207 INFO:tasks.workunit.client.0.smithi002.stdout:1/237: dread d2/d3/fc [0,4194304] 0 2024-03-16T10:44:54.208 INFO:tasks.workunit.client.0.smithi002.stdout:9/229: dread d5/d9/dc/f16 [0,4194304] 0 2024-03-16T10:44:54.208 INFO:tasks.workunit.client.0.smithi002.stdout:2/245: dread d7/d2e/f20 [0,4194304] 0 2024-03-16T10:44:54.209 INFO:tasks.workunit.client.0.smithi002.stdout:4/252: creat d2/d33/d3b/f4b x:0 0 0 2024-03-16T10:44:54.212 INFO:tasks.workunit.client.0.smithi002.stdout:9/230: readlink d5/l8 0 2024-03-16T10:44:54.216 INFO:tasks.workunit.client.0.smithi002.stdout:1/238: mknod d2/d3/d1c/c4f 0 2024-03-16T10:44:54.217 INFO:tasks.workunit.client.0.smithi002.stdout:9/231: stat d5/d9/dc/d1e/d24/d31/f4a 0 2024-03-16T10:44:54.218 INFO:tasks.workunit.client.0.smithi002.stdout:6/242: dwrite d15/d2d/d2c/f30 [0,4194304] 0 2024-03-16T10:44:54.219 INFO:tasks.workunit.client.0.smithi002.stdout:1/239: fdatasync d2/df/d12/f3f 0 2024-03-16T10:44:54.220 INFO:tasks.workunit.client.0.smithi002.stdout:9/232: stat d5/d9/l41 0 2024-03-16T10:44:54.223 INFO:tasks.workunit.client.0.smithi002.stdout:9/233: chown d5/d9/dc/d1e/d24 4685671 1 2024-03-16T10:44:54.224 INFO:tasks.workunit.client.0.smithi002.stdout:5/190: dread d0/f18 [0,4194304] 0 2024-03-16T10:44:54.226 INFO:tasks.workunit.client.0.smithi002.stdout:4/253: dwrite d2/d33/da/f41 [0,4194304] 0 2024-03-16T10:44:54.226 INFO:tasks.workunit.client.0.smithi002.stdout:9/234: dread - d5/d9/d25/d23/f48 zero size 2024-03-16T10:44:54.227 INFO:tasks.workunit.client.0.smithi002.stdout:6/243: mknod d15/d2d/d2b/d3f/c5b 0 2024-03-16T10:44:54.227 INFO:tasks.workunit.client.0.smithi002.stdout:6/244: stat le 0 2024-03-16T10:44:54.228 INFO:tasks.workunit.client.0.smithi002.stdout:4/254: dread - d2/d33/d3b/f4b zero size 2024-03-16T10:44:54.229 INFO:tasks.workunit.client.0.smithi002.stdout:9/235: dread d5/f7 [0,4194304] 0 2024-03-16T10:44:54.233 INFO:tasks.workunit.client.0.smithi002.stdout:6/245: write d15/d2d/d2b/d3f/f45 [1033951,1531] 0 2024-03-16T10:44:54.236 INFO:tasks.workunit.client.0.smithi002.stdout:4/255: symlink d2/d33/d1c/l4c 0 2024-03-16T10:44:54.237 INFO:tasks.workunit.client.0.smithi002.stdout:6/246: symlink d15/d2d/l5c 0 2024-03-16T10:44:54.237 INFO:tasks.workunit.client.0.smithi002.stdout:9/236: read d5/d9/dc/d1e/d24/d31/f4a [320437,62146] 0 2024-03-16T10:44:54.239 INFO:tasks.workunit.client.0.smithi002.stdout:5/191: dread d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:44:54.240 INFO:tasks.workunit.client.0.smithi002.stdout:8/233: dwrite d7/d9/d25/f45 [4194304,4194304] 0 2024-03-16T10:44:54.243 INFO:tasks.workunit.client.0.smithi002.stdout:9/237: write d5/d9/dc/d28/f4c [815923,85547] 0 2024-03-16T10:44:54.243 INFO:tasks.workunit.client.0.smithi002.stdout:2/246: dwrite d7/d2e/f20 [0,4194304] 0 2024-03-16T10:44:54.245 INFO:tasks.workunit.client.0.smithi002.stdout:9/238: write d5/f29 [4342316,16076] 0 2024-03-16T10:44:54.250 INFO:tasks.workunit.client.0.smithi002.stdout:5/192: dread d0/d1f/f2c [0,4194304] 0 2024-03-16T10:44:54.251 INFO:tasks.workunit.client.0.smithi002.stdout:8/234: unlink d7/f35 0 2024-03-16T10:44:54.252 INFO:tasks.workunit.client.0.smithi002.stdout:8/235: chown d7/l24 281 1 2024-03-16T10:44:54.254 INFO:tasks.workunit.client.0.smithi002.stdout:2/247: chown d7/d2e/l33 671331248 1 2024-03-16T10:44:54.255 INFO:tasks.workunit.client.0.smithi002.stdout:2/248: chown d7/d2e/d1d/d2f 67133 1 2024-03-16T10:44:54.255 INFO:tasks.workunit.client.0.smithi002.stdout:4/256: dread d2/d6/f1a [0,4194304] 0 2024-03-16T10:44:54.257 INFO:tasks.workunit.client.0.smithi002.stdout:4/257: write d2/d6/f1e [865702,38209] 0 2024-03-16T10:44:54.262 INFO:tasks.workunit.client.0.smithi002.stdout:5/193: truncate d0/f18 343723 0 2024-03-16T10:44:54.267 INFO:tasks.workunit.client.0.smithi002.stdout:6/247: mkdir d15/d2d/d2c/d5d 0 2024-03-16T10:44:54.274 INFO:tasks.workunit.client.0.smithi002.stdout:5/194: stat d0/d16/d1d/d20/c32 0 2024-03-16T10:44:54.275 INFO:tasks.workunit.client.0.smithi002.stdout:5/195: chown d0/d16/d1d/d33 729331095 1 2024-03-16T10:44:54.278 INFO:tasks.workunit.client.0.smithi002.stdout:9/239: dwrite d5/d9/dc/d1e/d24/d31/f32 [0,4194304] 0 2024-03-16T10:44:54.281 INFO:tasks.workunit.client.0.smithi002.stdout:3/205: sync 2024-03-16T10:44:54.281 INFO:tasks.workunit.client.0.smithi002.stdout:7/200: sync 2024-03-16T10:44:54.281 INFO:tasks.workunit.client.0.smithi002.stdout:4/258: symlink d2/d6/d32/l4d 0 2024-03-16T10:44:54.282 INFO:tasks.workunit.client.0.smithi002.stdout:8/236: getdents d7/d13 0 2024-03-16T10:44:54.285 INFO:tasks.workunit.client.0.smithi002.stdout:3/206: write d1/dd/d11/f30 [3495433,70] 0 2024-03-16T10:44:54.287 INFO:tasks.workunit.client.0.smithi002.stdout:3/207: chown d1/dd/d11/d15/d1e/d25 865161963 1 2024-03-16T10:44:54.287 INFO:tasks.workunit.client.0.smithi002.stdout:9/240: mkdir d5/d9/d22/d3c/d4d 0 2024-03-16T10:44:54.287 INFO:tasks.workunit.client.0.smithi002.stdout:2/249: link d7/d2e/d28/l2c d7/d27/d3f/l45 0 2024-03-16T10:44:54.289 INFO:tasks.workunit.client.0.smithi002.stdout:4/259: creat d2/d6/d32/f4e x:0 0 0 2024-03-16T10:44:54.289 INFO:tasks.workunit.client.0.smithi002.stdout:9/241: fsync d5/d9/dc/d1e/f36 0 2024-03-16T10:44:54.290 INFO:tasks.workunit.client.0.smithi002.stdout:4/260: chown d2/d6/f13 289604 1 2024-03-16T10:44:54.291 INFO:tasks.workunit.client.0.smithi002.stdout:7/201: rename d3/d4/d1d/d28 to d3/d4/d1d/d45 0 2024-03-16T10:44:54.293 INFO:tasks.workunit.client.0.smithi002.stdout:3/208: write d1/dd/d11/d15/d1e/d22/d3f/f47 [649033,119797] 0 2024-03-16T10:44:54.293 INFO:tasks.workunit.client.0.smithi002.stdout:5/196: creat d0/d24/f3c x:0 0 0 2024-03-16T10:44:54.295 INFO:tasks.workunit.client.0.smithi002.stdout:2/250: mkdir d7/d2e/d46 0 2024-03-16T10:44:54.296 INFO:tasks.workunit.client.0.smithi002.stdout:8/237: getdents d7/d9/d17/d23/d39 0 2024-03-16T10:44:54.296 INFO:tasks.workunit.client.0.smithi002.stdout:6/248: dwrite f8 [0,4194304] 0 2024-03-16T10:44:54.297 INFO:tasks.workunit.client.0.smithi002.stdout:8/238: fdatasync d7/f14 0 2024-03-16T10:44:54.298 INFO:tasks.workunit.client.0.smithi002.stdout:4/261: creat d2/d33/da/f4f x:0 0 0 2024-03-16T10:44:54.300 INFO:tasks.workunit.client.0.smithi002.stdout:8/239: chown l6 1 1 2024-03-16T10:44:54.302 INFO:tasks.workunit.client.0.smithi002.stdout:5/197: readlink d0/d1f/l25 0 2024-03-16T10:44:54.304 INFO:tasks.workunit.client.0.smithi002.stdout:2/251: rmdir d7/d2e 39 2024-03-16T10:44:54.305 INFO:tasks.workunit.client.0.smithi002.stdout:9/242: dwrite d5/d9/dc/d28/f2d [0,4194304] 0 2024-03-16T10:44:54.314 INFO:tasks.workunit.client.0.smithi002.stdout:8/240: creat d7/d9/d17/d23/d2e/d3a/f53 x:0 0 0 2024-03-16T10:44:54.315 INFO:tasks.workunit.client.0.smithi002.stdout:6/249: mknod d15/d2d/d2b/c5e 0 2024-03-16T10:44:54.317 INFO:tasks.workunit.client.0.smithi002.stdout:5/198: getdents d0/d16/d1d/d33 0 2024-03-16T10:44:54.317 INFO:tasks.workunit.client.0.smithi002.stdout:3/209: write d1/f35 [5203048,12955] 0 2024-03-16T10:44:54.318 INFO:tasks.workunit.client.0.smithi002.stdout:8/241: rmdir d7/d9/d17/d20 39 2024-03-16T10:44:54.319 INFO:tasks.workunit.client.0.smithi002.stdout:9/243: dwrite d5/d9/d25/d23/f3b [0,4194304] 0 2024-03-16T10:44:54.323 INFO:tasks.workunit.client.0.smithi002.stdout:2/252: symlink d7/d2e/d28/d40/l47 0 2024-03-16T10:44:54.324 INFO:tasks.workunit.client.0.smithi002.stdout:3/210: chown d1/lf 27606 1 2024-03-16T10:44:54.328 INFO:tasks.workunit.client.0.smithi002.stdout:7/202: dwrite d3/f11 [0,4194304] 0 2024-03-16T10:44:54.328 INFO:tasks.workunit.client.0.smithi002.stdout:8/242: rmdir d7/d9/d25 39 2024-03-16T10:44:54.331 INFO:tasks.workunit.client.0.smithi002.stdout:6/250: creat d15/f5f x:0 0 0 2024-03-16T10:44:54.333 INFO:tasks.workunit.client.0.smithi002.stdout:8/243: read - d7/d9/f51 zero size 2024-03-16T10:44:54.334 INFO:tasks.workunit.client.0.smithi002.stdout:9/244: mknod d5/d9/d25/c4e 0 2024-03-16T10:44:54.334 INFO:tasks.workunit.client.0.smithi002.stdout:4/262: dwrite d2/d6/f17 [4194304,4194304] 0 2024-03-16T10:44:54.335 INFO:tasks.workunit.client.0.smithi002.stdout:7/203: chown d3/d4/d10/c18 1453 1 2024-03-16T10:44:54.338 INFO:tasks.workunit.client.0.smithi002.stdout:9/245: dread - d5/d9/d22/f3a zero size 2024-03-16T10:44:54.338 INFO:tasks.workunit.client.0.smithi002.stdout:6/251: write d15/d2d/d2c/f36 [682379,65823] 0 2024-03-16T10:44:54.339 INFO:tasks.workunit.client.0.smithi002.stdout:3/211: dread d1/dd/d11/d15/f17 [0,4194304] 0 2024-03-16T10:44:54.342 INFO:tasks.workunit.client.0.smithi002.stdout:7/204: dread - d3/d4/d10/d3f/f43 zero size 2024-03-16T10:44:54.342 INFO:tasks.workunit.client.0.smithi002.stdout:5/199: dwrite d0/d16/f19 [0,4194304] 0 2024-03-16T10:44:54.343 INFO:tasks.workunit.client.0.smithi002.stdout:8/244: dread - d7/d9/d17/d23/d2e/d3a/f53 zero size 2024-03-16T10:44:54.345 INFO:tasks.workunit.client.0.smithi002.stdout:4/263: write d2/d33/da/f41 [4345642,127902] 0 2024-03-16T10:44:54.349 INFO:tasks.workunit.client.0.smithi002.stdout:2/253: dwrite f6 [0,4194304] 0 2024-03-16T10:44:54.349 INFO:tasks.workunit.client.0.smithi002.stdout:9/246: chown d5/d9/dc/d28/f4c 910 1 2024-03-16T10:44:54.349 INFO:tasks.workunit.client.0.smithi002.stdout:5/200: dread - d0/d24/f3c zero size 2024-03-16T10:44:54.351 INFO:tasks.workunit.client.0.smithi002.stdout:2/254: read d7/d2e/f1a [4161898,19180] 0 2024-03-16T10:44:54.351 INFO:tasks.workunit.client.0.smithi002.stdout:9/247: dread - d5/d9/d22/f3a zero size 2024-03-16T10:44:54.353 INFO:tasks.workunit.client.0.smithi002.stdout:4/264: write d2/d6/d32/f4e [966152,126612] 0 2024-03-16T10:44:54.353 INFO:tasks.workunit.client.0.smithi002.stdout:8/245: fsync d7/d9/d17/d23/d2e/d3a/f53 0 2024-03-16T10:44:54.354 INFO:tasks.workunit.client.0.smithi002.stdout:6/252: symlink d15/d2d/d2b/d3f/l60 0 2024-03-16T10:44:54.354 INFO:tasks.workunit.client.0.smithi002.stdout:0/251: sync 2024-03-16T10:44:54.354 INFO:tasks.workunit.client.0.smithi002.stdout:7/205: fsync d3/f11 0 2024-03-16T10:44:54.358 INFO:tasks.workunit.client.0.smithi002.stdout:8/246: write d7/fd [3785644,59689] 0 2024-03-16T10:44:54.359 INFO:tasks.workunit.client.0.smithi002.stdout:6/253: chown d15/d2d/d2b/d3f/c5b 793842 1 2024-03-16T10:44:54.359 INFO:tasks.workunit.client.0.smithi002.stdout:3/212: symlink d1/dd/d28/d37/l4f 0 2024-03-16T10:44:54.361 INFO:tasks.workunit.client.0.smithi002.stdout:5/201: truncate d0/d1f/f2c 5422734 0 2024-03-16T10:44:54.362 INFO:tasks.workunit.client.0.smithi002.stdout:2/255: creat d7/db/f48 x:0 0 0 2024-03-16T10:44:54.365 INFO:tasks.workunit.client.0.smithi002.stdout:0/252: symlink d3/d28/d2a/l4e 0 2024-03-16T10:44:54.369 INFO:tasks.workunit.client.0.smithi002.stdout:6/254: creat d15/d2d/d2b/f61 x:0 0 0 2024-03-16T10:44:54.370 INFO:tasks.workunit.client.0.smithi002.stdout:2/256: mkdir d7/d2e/d1d/d38/d49 0 2024-03-16T10:44:54.371 INFO:tasks.workunit.client.0.smithi002.stdout:0/253: truncate d3/f47 99368 0 2024-03-16T10:44:54.372 INFO:tasks.workunit.client.0.smithi002.stdout:5/202: mknod d0/d16/d1d/d20/c3d 0 2024-03-16T10:44:54.372 INFO:tasks.workunit.client.0.smithi002.stdout:4/265: dwrite d2/d6/f17 [4194304,4194304] 0 2024-03-16T10:44:54.375 INFO:tasks.workunit.client.0.smithi002.stdout:0/254: rmdir d3/de 39 2024-03-16T10:44:54.376 INFO:tasks.workunit.client.0.smithi002.stdout:2/257: fsync d7/d2e/d1d/f41 0 2024-03-16T10:44:54.376 INFO:tasks.workunit.client.0.smithi002.stdout:5/203: fdatasync d0/d16/f37 0 2024-03-16T10:44:54.377 INFO:tasks.workunit.client.0.smithi002.stdout:9/248: rename d5/d9/d22/f3a to d5/d9/d25/f4f 0 2024-03-16T10:44:54.378 INFO:tasks.workunit.client.0.smithi002.stdout:3/213: dwrite d1/dd/d11/d15/f2e [0,4194304] 0 2024-03-16T10:44:54.380 INFO:tasks.workunit.client.0.smithi002.stdout:2/258: stat d7/db/d12/f11 0 2024-03-16T10:44:54.384 INFO:tasks.workunit.client.0.smithi002.stdout:9/249: chown d5/d9/l41 33272728 1 2024-03-16T10:44:54.384 INFO:tasks.workunit.client.0.smithi002.stdout:0/255: creat d3/d28/d2a/f4f x:0 0 0 2024-03-16T10:44:54.386 INFO:tasks.workunit.client.0.smithi002.stdout:4/266: rename d2/d33/d1c/f2f to d2/d6/d32/f50 0 2024-03-16T10:44:54.387 INFO:tasks.workunit.client.0.smithi002.stdout:8/247: dwrite d7/d9/d25/f45 [0,4194304] 0 2024-03-16T10:44:54.390 INFO:tasks.workunit.client.0.smithi002.stdout:0/256: read - d3/d28/f29 zero size 2024-03-16T10:44:54.390 INFO:tasks.workunit.client.0.smithi002.stdout:6/255: dwrite d15/d2d/d2c/f56 [0,4194304] 0 2024-03-16T10:44:54.396 INFO:tasks.workunit.client.0.smithi002.stdout:0/257: write d3/f47 [218404,30021] 0 2024-03-16T10:44:54.400 INFO:tasks.workunit.client.0.smithi002.stdout:3/214: rename d1/dd/d11/d15/l1c to d1/dd/d11/d15/d1e/d22/l50 0 2024-03-16T10:44:54.402 INFO:tasks.workunit.client.0.smithi002.stdout:2/259: dwrite d7/d2e/f20 [4194304,4194304] 0 2024-03-16T10:44:54.403 INFO:tasks.workunit.client.0.smithi002.stdout:9/250: dwrite d5/d9/dc/d1e/d24/f46 [0,4194304] 0 2024-03-16T10:44:54.403 INFO:tasks.workunit.client.0.smithi002.stdout:2/260: chown d7/f2b 193821560 1 2024-03-16T10:44:54.404 INFO:tasks.workunit.client.0.smithi002.stdout:9/251: fsync d5/d9/dc/f20 0 2024-03-16T10:44:54.404 INFO:tasks.workunit.client.0.smithi002.stdout:2/261: readlink d7/l9 0 2024-03-16T10:44:54.404 INFO:tasks.workunit.client.0.smithi002.stdout:3/215: dread - d1/dd/d28/d37/f4e zero size 2024-03-16T10:44:54.407 INFO:tasks.workunit.client.0.smithi002.stdout:8/248: unlink d7/d13/c29 0 2024-03-16T10:44:54.407 INFO:tasks.workunit.client.0.smithi002.stdout:3/216: chown d1/dd/d28/f33 130440528 1 2024-03-16T10:44:54.408 INFO:tasks.workunit.client.0.smithi002.stdout:9/252: dread - d5/d9/dc/d1e/d24/f4b zero size 2024-03-16T10:44:54.412 INFO:tasks.workunit.client.0.smithi002.stdout:4/267: mknod d2/d33/c51 0 2024-03-16T10:44:54.413 INFO:tasks.workunit.client.0.smithi002.stdout:3/217: truncate d1/dd/d28/d37/f49 1307062 0 2024-03-16T10:44:54.415 INFO:tasks.workunit.client.0.smithi002.stdout:0/258: creat d3/d28/d3d/f50 x:0 0 0 2024-03-16T10:44:54.416 INFO:tasks.workunit.client.0.smithi002.stdout:0/259: fdatasync d3/d28/f29 0 2024-03-16T10:44:54.416 INFO:tasks.workunit.client.0.smithi002.stdout:4/268: dread d2/d33/f1b [0,4194304] 0 2024-03-16T10:44:54.417 INFO:tasks.workunit.client.0.smithi002.stdout:4/269: truncate d2/d6/f46 173483 0 2024-03-16T10:44:54.419 INFO:tasks.workunit.client.0.smithi002.stdout:1/240: sync 2024-03-16T10:44:54.421 INFO:tasks.workunit.client.0.smithi002.stdout:9/253: rmdir d5/d9/d25/d23 39 2024-03-16T10:44:54.422 INFO:tasks.workunit.client.0.smithi002.stdout:6/256: mknod d15/d2d/d2c/d5d/c62 0 2024-03-16T10:44:54.424 INFO:tasks.workunit.client.0.smithi002.stdout:3/218: creat d1/dd/d11/d15/d1e/d22/d3f/f51 x:0 0 0 2024-03-16T10:44:54.427 INFO:tasks.workunit.client.0.smithi002.stdout:6/257: dread d15/d2d/d2c/f36 [0,4194304] 0 2024-03-16T10:44:54.428 INFO:tasks.workunit.client.0.smithi002.stdout:5/204: truncate d0/f6 3762755 0 2024-03-16T10:44:54.429 INFO:tasks.workunit.client.0.smithi002.stdout:9/254: dwrite d5/d9/dc/d1e/d24/f4b [0,4194304] 0 2024-03-16T10:44:54.433 INFO:tasks.workunit.client.0.smithi002.stdout:1/241: creat d2/d3/f50 x:0 0 0 2024-03-16T10:44:54.438 INFO:tasks.workunit.client.0.smithi002.stdout:8/249: write d7/d9/f2f [134587,128731] 0 2024-03-16T10:44:54.441 INFO:tasks.workunit.client.0.smithi002.stdout:3/219: dread d1/dd/d11/d15/d1e/d25/f26 [0,4194304] 0 2024-03-16T10:44:54.442 INFO:tasks.workunit.client.0.smithi002.stdout:6/258: rmdir d15/d2d/d2b 39 2024-03-16T10:44:54.444 INFO:tasks.workunit.client.0.smithi002.stdout:2/262: truncate d7/d2e/f20 1894265 0 2024-03-16T10:44:54.447 INFO:tasks.workunit.client.0.smithi002.stdout:9/255: fdatasync d5/d9/d25/d23/f48 0 2024-03-16T10:44:54.448 INFO:tasks.workunit.client.0.smithi002.stdout:9/256: read - d5/d9/d22/f47 zero size 2024-03-16T10:44:54.449 INFO:tasks.workunit.client.0.smithi002.stdout:1/242: mkdir d2/d3/d51 0 2024-03-16T10:44:54.449 INFO:tasks.workunit.client.0.smithi002.stdout:6/259: dwrite d15/f25 [8388608,4194304] 0 2024-03-16T10:44:54.450 INFO:tasks.workunit.client.0.smithi002.stdout:9/257: chown d5/d9/lb 1640 1 2024-03-16T10:44:54.453 INFO:tasks.workunit.client.0.smithi002.stdout:4/270: dwrite d2/d33/f1b [0,4194304] 0 2024-03-16T10:44:54.459 INFO:tasks.workunit.client.0.smithi002.stdout:0/260: creat d3/de/f51 x:0 0 0 2024-03-16T10:44:54.459 INFO:tasks.workunit.client.0.smithi002.stdout:8/250: mknod d7/d9/d17/d23/d2e/d34/c54 0 2024-03-16T10:44:54.460 INFO:tasks.workunit.client.0.smithi002.stdout:8/251: write d7/f14 [8039810,73393] 0 2024-03-16T10:44:54.460 INFO:tasks.workunit.client.0.smithi002.stdout:9/258: rmdir d5/d9/dc/d1e/d24/d31 39 2024-03-16T10:44:54.464 INFO:tasks.workunit.client.0.smithi002.stdout:4/271: creat d2/d33/d3b/f52 x:0 0 0 2024-03-16T10:44:54.466 INFO:tasks.workunit.client.0.smithi002.stdout:5/205: getdents d0 0 2024-03-16T10:44:54.467 INFO:tasks.workunit.client.0.smithi002.stdout:2/263: mknod d7/d2e/d28/d44/c4a 0 2024-03-16T10:44:54.470 INFO:tasks.workunit.client.0.smithi002.stdout:9/259: dwrite d5/d9/d22/f47 [0,4194304] 0 2024-03-16T10:44:54.472 INFO:tasks.workunit.client.0.smithi002.stdout:2/264: truncate d7/db/f48 501389 0 2024-03-16T10:44:54.473 INFO:tasks.workunit.client.0.smithi002.stdout:9/260: fsync d5/d9/dc/d28/f2d 0 2024-03-16T10:44:54.475 INFO:tasks.workunit.client.0.smithi002.stdout:9/261: stat d5/d9/d25 0 2024-03-16T10:44:54.475 INFO:tasks.workunit.client.0.smithi002.stdout:5/206: chown d0/d16/f29 29368947 1 2024-03-16T10:44:54.476 INFO:tasks.workunit.client.0.smithi002.stdout:4/272: dwrite d2/d33/da/d2e/f47 [0,4194304] 0 2024-03-16T10:44:54.477 INFO:tasks.workunit.client.0.smithi002.stdout:5/207: write d0/d16/f19 [836923,123233] 0 2024-03-16T10:44:54.478 INFO:tasks.workunit.client.0.smithi002.stdout:5/208: stat d0/d16 0 2024-03-16T10:44:54.479 INFO:tasks.workunit.client.0.smithi002.stdout:1/243: rename d2/f28 to d2/f52 0 2024-03-16T10:44:54.483 INFO:tasks.workunit.client.0.smithi002.stdout:0/261: creat d3/d28/d3d/d44/f52 x:0 0 0 2024-03-16T10:44:54.488 INFO:tasks.workunit.client.0.smithi002.stdout:4/273: dwrite d2/d33/d3b/f4b [0,4194304] 0 2024-03-16T10:44:54.488 INFO:tasks.workunit.client.0.smithi002.stdout:8/252: mkdir d7/d55 0 2024-03-16T10:44:54.490 INFO:tasks.workunit.client.0.smithi002.stdout:1/244: dwrite d2/d3/f17 [0,4194304] 0 2024-03-16T10:44:54.490 INFO:tasks.workunit.client.0.smithi002.stdout:2/265: mkdir d7/d2e/d1d/d2f/d4b 0 2024-03-16T10:44:54.499 INFO:tasks.workunit.client.0.smithi002.stdout:6/260: rename d15/d2d/d2c/f39 to d15/d2d/d2b/f63 0 2024-03-16T10:44:54.504 INFO:tasks.workunit.client.0.smithi002.stdout:2/266: fsync d7/f2b 0 2024-03-16T10:44:54.505 INFO:tasks.workunit.client.0.smithi002.stdout:4/274: rename d2/d6/f22 to d2/d6/f53 0 2024-03-16T10:44:54.507 INFO:tasks.workunit.client.0.smithi002.stdout:5/209: creat d0/d16/d1d/d33/f3e x:0 0 0 2024-03-16T10:44:54.507 INFO:tasks.workunit.client.0.smithi002.stdout:6/261: fsync f11 0 2024-03-16T10:44:54.509 INFO:tasks.workunit.client.0.smithi002.stdout:5/210: write d0/d24/f3c [215211,36732] 0 2024-03-16T10:44:54.510 INFO:tasks.workunit.client.0.smithi002.stdout:5/211: dread - d0/d16/d1d/f3b zero size 2024-03-16T10:44:54.511 INFO:tasks.workunit.client.0.smithi002.stdout:4/275: dread d2/d6/f53 [0,4194304] 0 2024-03-16T10:44:54.512 INFO:tasks.workunit.client.0.smithi002.stdout:4/276: fdatasync d2/d33/da/f41 0 2024-03-16T10:44:54.513 INFO:tasks.workunit.client.0.smithi002.stdout:4/277: write d2/d6/f17 [6822879,118612] 0 2024-03-16T10:44:54.518 INFO:tasks.workunit.client.0.smithi002.stdout:9/262: getdents d5/d9/d22 0 2024-03-16T10:44:54.519 INFO:tasks.workunit.client.0.smithi002.stdout:0/262: write d3/d28/d2a/f2c [1141631,70254] 0 2024-03-16T10:44:54.520 INFO:tasks.workunit.client.0.smithi002.stdout:0/263: stat d3/d28/f3b 0 2024-03-16T10:44:54.521 INFO:tasks.workunit.client.0.smithi002.stdout:4/278: dwrite d2/d33/d3b/f4b [0,4194304] 0 2024-03-16T10:44:54.523 INFO:tasks.workunit.client.0.smithi002.stdout:4/279: dread - d2/f34 zero size 2024-03-16T10:44:54.523 INFO:tasks.workunit.client.0.smithi002.stdout:6/262: dread f4 [0,4194304] 0 2024-03-16T10:44:54.524 INFO:tasks.workunit.client.0.smithi002.stdout:8/253: creat d7/d13/f56 x:0 0 0 2024-03-16T10:44:54.524 INFO:tasks.workunit.client.0.smithi002.stdout:6/263: write d15/d2d/d2c/f3d [3296437,64680] 0 2024-03-16T10:44:54.525 INFO:tasks.workunit.client.0.smithi002.stdout:2/267: creat d7/f4c x:0 0 0 2024-03-16T10:44:54.526 INFO:tasks.workunit.client.0.smithi002.stdout:8/254: dread - d7/d9/d17/d23/d2e/d3a/f53 zero size 2024-03-16T10:44:54.528 INFO:tasks.workunit.client.0.smithi002.stdout:9/263: stat d5/d9/dc/d1e/d24/l38 0 2024-03-16T10:44:54.533 INFO:tasks.workunit.client.0.smithi002.stdout:5/212: dwrite d0/d1f/f2c [0,4194304] 0 2024-03-16T10:44:54.536 INFO:tasks.workunit.client.0.smithi002.stdout:4/280: creat d2/d33/d1c/d45/f54 x:0 0 0 2024-03-16T10:44:54.538 INFO:tasks.workunit.client.0.smithi002.stdout:6/264: symlink d15/d2d/l64 0 2024-03-16T10:44:54.539 INFO:tasks.workunit.client.0.smithi002.stdout:6/265: truncate d15/f18 345646 0 2024-03-16T10:44:54.540 INFO:tasks.workunit.client.0.smithi002.stdout:5/213: rename d0/l13 to d0/d16/d1d/l3f 0 2024-03-16T10:44:54.542 INFO:tasks.workunit.client.0.smithi002.stdout:4/281: write d2/d6/d32/f50 [531087,113437] 0 2024-03-16T10:44:54.543 INFO:tasks.workunit.client.0.smithi002.stdout:8/255: creat d7/d9/d17/f57 x:0 0 0 2024-03-16T10:44:54.544 INFO:tasks.workunit.client.0.smithi002.stdout:8/256: chown d7/d9/d25 12 1 2024-03-16T10:44:54.546 INFO:tasks.workunit.client.0.smithi002.stdout:0/264: truncate d3/d5/f19 1469187 0 2024-03-16T10:44:54.546 INFO:tasks.workunit.client.0.smithi002.stdout:0/265: fsync d3/d5/f2f 0 2024-03-16T10:44:54.549 INFO:tasks.workunit.client.0.smithi002.stdout:2/268: truncate f6 3859151 0 2024-03-16T10:44:54.555 INFO:tasks.workunit.client.0.smithi002.stdout:6/266: dread d15/f19 [0,4194304] 0 2024-03-16T10:44:54.556 INFO:tasks.workunit.client.0.smithi002.stdout:6/267: write d15/d2d/d2c/f3c [4106329,81526] 0 2024-03-16T10:44:54.558 INFO:tasks.workunit.client.0.smithi002.stdout:0/266: rename d3/de/f10 to d3/d28/d3d/d44/f53 0 2024-03-16T10:44:54.559 INFO:tasks.workunit.client.0.smithi002.stdout:0/267: write d3/fc [4969409,121852] 0 2024-03-16T10:44:54.560 INFO:tasks.workunit.client.0.smithi002.stdout:9/264: getdents d5/d9/dc/d1e/d24 0 2024-03-16T10:44:54.561 INFO:tasks.workunit.client.0.smithi002.stdout:9/265: write d5/d9/f13 [1805123,11729] 0 2024-03-16T10:44:54.561 INFO:tasks.workunit.client.0.smithi002.stdout:4/282: mknod d2/d33/c55 0 2024-03-16T10:44:54.564 INFO:tasks.workunit.client.0.smithi002.stdout:0/268: dread d3/f47 [0,4194304] 0 2024-03-16T10:44:54.569 INFO:tasks.workunit.client.0.smithi002.stdout:9/266: dwrite d5/d9/dc/d1e/d24/d31/f32 [0,4194304] 0 2024-03-16T10:44:54.772 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:54 smithi026 ceph-mon[37041]: pgmap v30: 65 pgs: 65 active+clean; 343 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 24 MiB/s wr, 264 op/s 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:4/283: creat d2/d33/da/d2e/f56 x:0 0 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:4/284: readlink d2/d6/l16 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:4/285: fsync d2/d33/d35/f40 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:8/257: link d7/d9/d17/f57 d7/d9/f58 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:8/258: rmdir d7/d9/d17/d23/d2e 39 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:9/267: symlink d5/d9/d22/d3c/d4d/l50 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:6/268: rename d15/d2d/d2b/d3f/l5a to d15/d2d/l65 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:2/269: getdents d7/d2e 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:5/214: rename d0/d16/d1d/d20/l2a to d0/d24/l40 0 2024-03-16T10:44:54.783 INFO:tasks.workunit.client.0.smithi002.stdout:9/268: write d5/d9/fe [521872,88217] 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:6/269: read - d15/d2d/d2c/f54 zero size 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:8/259: symlink d7/l59 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:4/286: dread d2/d33/f1b [0,4194304] 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:2/270: creat d7/d2e/d28/d44/f4d x:0 0 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:0/269: dwrite d3/d28/d3d/d44/f53 [0,4194304] 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:8/260: symlink d7/d9/d17/d23/d2e/d3a/l5a 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:8/261: mknod d7/d9/d17/d23/d39/c5b 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:2/271: link d7/l9 d7/d2e/d1d/d38/d49/l4e 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:6/270: dread d15/d2d/d2c/f3c [0,4194304] 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:4/287: dwrite d2/d33/da/d2e/f56 [0,4194304] 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:0/270: dwrite d3/d28/f4d [0,4194304] 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:8/262: chown d7/d13/f4c 43267799 1 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:4/288: mknod d2/d33/d1c/c57 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:2/272: creat d7/d2e/d1d/f4f x:0 0 0 2024-03-16T10:44:54.784 INFO:tasks.workunit.client.0.smithi002.stdout:8/263: mkdir d7/d9/d17/d23/d2e/d5c 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:8/264: read d7/f31 [487833,25894] 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:4/289: symlink d2/d33/da/l58 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:5/215: getdents d0/d16/d1d/d20 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:8/265: write d7/d9/fa [275699,32145] 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:5/216: unlink d0/d16/d1d/l3a 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:6/271: link d15/d2d/d2c/d5d/c62 d15/d2d/d2b/d3f/c66 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:0/271: getdents d3/d28 0 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:0/272: chown d3/f2d 16 1 2024-03-16T10:44:54.785 INFO:tasks.workunit.client.0.smithi002.stdout:0/273: truncate d3/d28/d2a/f2c 1376113 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:0/274: dread d3/d27/f3c [4194304,4194304] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:2/273: getdents d7/d2e/d28/d44 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:8/266: dread d7/f21 [0,4194304] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:2/274: write d7/db/f2a [1032532,44418] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:6/272: unlink d15/d2d/l4e 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:5/217: dwrite d0/d1f/f36 [0,4194304] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:0/275: dwrite d3/d28/d3d/d44/f53 [0,4194304] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:8/267: creat d7/d9/d17/d23/d39/f5d x:0 0 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:2/275: dread d7/d2e/d1d/f26 [0,4194304] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:5/218: dwrite d0/d16/f19 [0,4194304] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:5/219: fsync d0/d24/f3c 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:8/268: fsync d7/d9/d17/d23/d2e/f52 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:6/273: dwrite d15/d2d/f42 [0,4194304] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:8/269: read d7/d13/f4c [57801,747] 0 2024-03-16T10:44:54.786 INFO:tasks.workunit.client.0.smithi002.stdout:2/276: mkdir d7/d2e/d28/d40/d50 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:5/220: symlink d0/d16/d1d/l41 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:5/221: read - d0/d16/f37 zero size 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:5/222: write d0/d16/f37 [182489,61805] 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:8/270: dwrite d7/d9/d17/d23/f33 [0,4194304] 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:0/276: truncate d3/d5/f30 346849 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:0/277: creat d3/d28/f54 x:0 0 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:0/278: write d3/d28/f34 [773547,24208] 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:8/271: write d7/d9/d17/d23/d2e/f4e [253981,67070] 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:5/223: truncate d0/d1f/f36 1929992 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:5/224: truncate d0/d16/f37 597599 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:2/277: dwrite d7/d2e/f1a [0,4194304] 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:6/274: dread d15/f24 [0,4194304] 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:0/279: unlink d3/d28/d2a/f2c 0 2024-03-16T10:44:54.787 INFO:tasks.workunit.client.0.smithi002.stdout:8/272: dwrite d7/d13/f56 [0,4194304] 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:8/273: write d7/d9/d17/d23/d2e/d3a/f53 [560640,118265] 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:8/274: stat d7/l24 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:0/280: creat d3/d27/f55 x:0 0 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:0/281: write d3/d28/d3d/f3f [2534587,99173] 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:0/282: dwrite d3/d31/f4b [0,4194304] 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:0/283: chown d3/d28/d3d/d44/f52 0 1 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:5/225: truncate d0/d16/f37 132911 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:6/275: creat d15/f67 x:0 0 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:5/226: write d0/d16/f29 [2710085,57973] 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:2/278: truncate d7/db/d12/f11 4031866 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:5/227: write d0/d16/d1d/f3b [185604,24702] 0 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:7/206: sync 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:7/207: chown d3/f14 246 1 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:7/208: read - d3/d4/d10/d3f/f43 zero size 2024-03-16T10:44:54.788 INFO:tasks.workunit.client.0.smithi002.stdout:7/209: truncate d3/d4/d2a/f2f 413426 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:8/275: rename d7/d9/d17/d23/d2e/f52 to d7/d9/d17/d20/f5e 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:6/276: creat d15/d2d/d2b/f68 x:0 0 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:5/228: rmdir d0/d16/d1d 39 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:2/279: rename d7/d2e/f1c to d7/d27/d3b/f51 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:8/276: creat d7/d9/d17/d23/d2e/d3a/f5f x:0 0 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:0/284: write d3/d5/f18 [414922,103435] 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:7/210: unlink d3/d4/l2c 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:0/285: creat d3/d27/f56 x:0 0 0 2024-03-16T10:44:54.789 INFO:tasks.workunit.client.0.smithi002.stdout:5/229: chown d0/d16/d1d/c27 5454556 1 2024-03-16T10:44:54.790 INFO:tasks.workunit.client.0.smithi002.stdout:5/230: truncate d0/d16/d1d/d33/f3e 911450 0 2024-03-16T10:44:54.790 INFO:tasks.workunit.client.0.smithi002.stdout:6/277: creat d15/f69 x:0 0 0 2024-03-16T10:44:54.790 INFO:tasks.workunit.client.0.smithi002.stdout:0/286: dread d3/fc [0,4194304] 0 2024-03-16T10:44:54.790 INFO:tasks.workunit.client.0.smithi002.stdout:7/211: dread d3/d4/d1d/f23 [0,4194304] 0 2024-03-16T10:44:54.790 INFO:tasks.workunit.client.0.smithi002.stdout:6/278: dwrite d15/d2d/d2c/f31 [0,4194304] 0 2024-03-16T10:44:54.790 INFO:tasks.workunit.client.0.smithi002.stdout:7/212: write d3/d4/d2a/f2f [339620,82152] 0 2024-03-16T10:44:54.790 INFO:tasks.workunit.client.0.smithi002.stdout:8/277: dwrite d7/d9/d17/f57 [0,4194304] 0 2024-03-16T10:44:54.792 INFO:tasks.workunit.client.0.smithi002.stdout:7/213: chown d3/d4/d1d/d45/f42 3 1 2024-03-16T10:44:54.792 INFO:tasks.workunit.client.0.smithi002.stdout:2/280: creat d7/d2e/d28/d40/f52 x:0 0 0 2024-03-16T10:44:54.792 INFO:tasks.workunit.client.0.smithi002.stdout:6/279: dread d15/d2d/d2c/f31 [0,4194304] 0 2024-03-16T10:44:54.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:54 smithi002 ceph-mon[29248]: pgmap v30: 65 pgs: 65 active+clean; 343 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 24 MiB/s wr, 264 op/s 2024-03-16T10:44:54.801 INFO:tasks.workunit.client.0.smithi002.stdout:8/278: readlink d7/d13/l28 0 2024-03-16T10:44:54.803 INFO:tasks.workunit.client.0.smithi002.stdout:2/281: mknod d7/d27/d3b/c53 0 2024-03-16T10:44:54.809 INFO:tasks.workunit.client.0.smithi002.stdout:0/287: creat d3/f57 x:0 0 0 2024-03-16T10:44:54.809 INFO:tasks.workunit.client.0.smithi002.stdout:5/231: truncate d0/d1f/f2c 3339345 0 2024-03-16T10:44:54.811 INFO:tasks.workunit.client.0.smithi002.stdout:6/280: dwrite d15/d2d/d2b/f53 [0,4194304] 0 2024-03-16T10:44:54.813 INFO:tasks.workunit.client.0.smithi002.stdout:0/288: symlink d3/d31/l58 0 2024-03-16T10:44:54.816 INFO:tasks.workunit.client.0.smithi002.stdout:5/232: unlink d0/d16/d1d/d2b/c38 0 2024-03-16T10:44:54.818 INFO:tasks.workunit.client.0.smithi002.stdout:7/214: creat d3/d4/d10/d3f/f46 x:0 0 0 2024-03-16T10:44:54.820 INFO:tasks.workunit.client.0.smithi002.stdout:6/281: symlink d15/d2d/d2c/d5d/l6a 0 2024-03-16T10:44:54.820 INFO:tasks.workunit.client.0.smithi002.stdout:6/282: fsync d15/f29 0 2024-03-16T10:44:54.826 INFO:tasks.workunit.client.0.smithi002.stdout:5/233: creat d0/d16/d1d/d33/f42 x:0 0 0 2024-03-16T10:44:54.827 INFO:tasks.workunit.client.0.smithi002.stdout:8/279: creat d7/f60 x:0 0 0 2024-03-16T10:44:54.829 INFO:tasks.workunit.client.0.smithi002.stdout:7/215: rename d3/d4/d1c/l22 to d3/d4/d10/d3f/l47 0 2024-03-16T10:44:54.834 INFO:tasks.workunit.client.0.smithi002.stdout:0/289: fsync d3/d5/fd 0 2024-03-16T10:44:54.837 INFO:tasks.workunit.client.0.smithi002.stdout:7/216: dwrite d3/d4/d1d/d45/f42 [0,4194304] 0 2024-03-16T10:44:54.841 INFO:tasks.workunit.client.0.smithi002.stdout:7/217: dread d3/d4/d2a/f2f [0,4194304] 0 2024-03-16T10:44:54.842 INFO:tasks.workunit.client.0.smithi002.stdout:2/282: getdents d7/d27/d3b 0 2024-03-16T10:44:54.845 INFO:tasks.workunit.client.0.smithi002.stdout:3/220: sync 2024-03-16T10:44:54.846 INFO:tasks.workunit.client.0.smithi002.stdout:1/245: sync 2024-03-16T10:44:54.847 INFO:tasks.workunit.client.0.smithi002.stdout:7/218: dwrite d3/d4/d1d/d45/f3a [0,4194304] 0 2024-03-16T10:44:54.847 INFO:tasks.workunit.client.0.smithi002.stdout:3/221: read d1/dd/d11/d15/d1e/d25/f26 [141062,53577] 0 2024-03-16T10:44:54.851 INFO:tasks.workunit.client.0.smithi002.stdout:8/280: creat d7/d55/f61 x:0 0 0 2024-03-16T10:44:54.854 INFO:tasks.workunit.client.0.smithi002.stdout:6/283: creat d15/f6b x:0 0 0 2024-03-16T10:44:54.858 INFO:tasks.workunit.client.0.smithi002.stdout:8/281: dwrite d7/d9/d17/d23/f4d [0,4194304] 0 2024-03-16T10:44:54.861 INFO:tasks.workunit.client.0.smithi002.stdout:5/234: write d0/f6 [356290,2474] 0 2024-03-16T10:44:54.862 INFO:tasks.workunit.client.0.smithi002.stdout:1/246: write d2/f52 [530029,126779] 0 2024-03-16T10:44:54.865 INFO:tasks.workunit.client.0.smithi002.stdout:5/235: write d0/d24/f3c [770955,82903] 0 2024-03-16T10:44:54.865 INFO:tasks.workunit.client.0.smithi002.stdout:6/284: dwrite d15/d2d/d2c/f46 [0,4194304] 0 2024-03-16T10:44:54.868 INFO:tasks.workunit.client.0.smithi002.stdout:2/283: symlink d7/l54 0 2024-03-16T10:44:54.875 INFO:tasks.workunit.client.0.smithi002.stdout:2/284: dwrite d7/f4c [0,4194304] 0 2024-03-16T10:44:54.877 INFO:tasks.workunit.client.0.smithi002.stdout:2/285: write d7/d2e/d28/d44/f4d [453721,22594] 0 2024-03-16T10:44:54.877 INFO:tasks.workunit.client.0.smithi002.stdout:5/236: mkdir d0/d16/d1d/d43 0 2024-03-16T10:44:54.879 INFO:tasks.workunit.client.0.smithi002.stdout:1/247: rmdir d2/df/d12/d29 39 2024-03-16T10:44:54.883 INFO:tasks.workunit.client.0.smithi002.stdout:6/285: rmdir d15/d2d 39 2024-03-16T10:44:54.884 INFO:tasks.workunit.client.0.smithi002.stdout:3/222: link d1/dd/d11/d15/f17 d1/f52 0 2024-03-16T10:44:54.891 INFO:tasks.workunit.client.0.smithi002.stdout:2/286: read d7/d2e/f1a [8077166,68640] 0 2024-03-16T10:44:54.892 INFO:tasks.workunit.client.0.smithi002.stdout:2/287: readlink d7/d2e/d28/l2c 0 2024-03-16T10:44:54.893 INFO:tasks.workunit.client.0.smithi002.stdout:2/288: truncate d7/d27/f2d 589979 0 2024-03-16T10:44:54.895 INFO:tasks.workunit.client.0.smithi002.stdout:6/286: stat d15/d2d/d2c/d5d 0 2024-03-16T10:44:54.896 INFO:tasks.workunit.client.0.smithi002.stdout:3/223: creat d1/dd/d11/d15/f53 x:0 0 0 2024-03-16T10:44:54.896 INFO:tasks.workunit.client.0.smithi002.stdout:5/237: dread d0/d1f/f2c [0,4194304] 0 2024-03-16T10:44:54.896 INFO:tasks.workunit.client.0.smithi002.stdout:8/282: link d7/d13/l28 d7/d9/d17/d23/d2e/l62 0 2024-03-16T10:44:54.898 INFO:tasks.workunit.client.0.smithi002.stdout:1/248: symlink d2/df/d12/l53 0 2024-03-16T10:44:54.899 INFO:tasks.workunit.client.0.smithi002.stdout:3/224: write d1/dd/d11/f4a [1018135,17877] 0 2024-03-16T10:44:54.901 INFO:tasks.workunit.client.0.smithi002.stdout:3/225: fdatasync d1/dd/d11/d15/d1e/d22/f39 0 2024-03-16T10:44:54.902 INFO:tasks.workunit.client.0.smithi002.stdout:2/289: dwrite d7/db/f48 [0,4194304] 0 2024-03-16T10:44:54.904 INFO:tasks.workunit.client.0.smithi002.stdout:8/283: creat d7/d9/d17/d20/f63 x:0 0 0 2024-03-16T10:44:54.910 INFO:tasks.workunit.client.0.smithi002.stdout:5/238: dread d0/f18 [0,4194304] 0 2024-03-16T10:44:54.910 INFO:tasks.workunit.client.0.smithi002.stdout:1/249: symlink d2/d3/d1c/l54 0 2024-03-16T10:44:54.912 INFO:tasks.workunit.client.0.smithi002.stdout:2/290: chown d7/d2e/d1d/f21 949 1 2024-03-16T10:44:54.913 INFO:tasks.workunit.client.0.smithi002.stdout:2/291: write d7/db/f2a [331609,25556] 0 2024-03-16T10:44:54.913 INFO:tasks.workunit.client.0.smithi002.stdout:3/226: creat d1/dd/d28/f54 x:0 0 0 2024-03-16T10:44:54.917 INFO:tasks.workunit.client.0.smithi002.stdout:6/287: creat d15/d2d/f6c x:0 0 0 2024-03-16T10:44:54.920 INFO:tasks.workunit.client.0.smithi002.stdout:1/250: creat d2/d3/f55 x:0 0 0 2024-03-16T10:44:54.921 INFO:tasks.workunit.client.0.smithi002.stdout:1/251: chown d2/df/d12/f37 5 1 2024-03-16T10:44:54.926 INFO:tasks.workunit.client.0.smithi002.stdout:3/227: rename d1/dd/d28/f54 to d1/dd/d28/f55 0 2024-03-16T10:44:54.927 INFO:tasks.workunit.client.0.smithi002.stdout:8/284: unlink d7/d9/d17/d23/l38 0 2024-03-16T10:44:54.928 INFO:tasks.workunit.client.0.smithi002.stdout:2/292: dwrite d7/d2e/d1d/f3e [0,4194304] 0 2024-03-16T10:44:54.929 INFO:tasks.workunit.client.0.smithi002.stdout:2/293: write d7/f4c [3065512,53156] 0 2024-03-16T10:44:54.931 INFO:tasks.workunit.client.0.smithi002.stdout:6/288: creat d15/d2d/d2b/d3f/f6d x:0 0 0 2024-03-16T10:44:54.933 INFO:tasks.workunit.client.0.smithi002.stdout:5/239: mknod d0/d1f/c44 0 2024-03-16T10:44:54.948 INFO:tasks.workunit.client.0.smithi002.stdout:1/252: mkdir d2/d3/d51/d56 0 2024-03-16T10:44:54.948 INFO:tasks.workunit.client.0.smithi002.stdout:8/285: dwrite d7/d13/f4c [0,4194304] 0 2024-03-16T10:44:54.949 INFO:tasks.workunit.client.0.smithi002.stdout:2/294: dread d7/d2e/f16 [0,4194304] 0 2024-03-16T10:44:54.949 INFO:tasks.workunit.client.0.smithi002.stdout:8/286: chown d7/d13/f56 23 1 2024-03-16T10:44:54.952 INFO:tasks.workunit.client.0.smithi002.stdout:9/269: sync 2024-03-16T10:44:54.953 INFO:tasks.workunit.client.0.smithi002.stdout:4/290: sync 2024-03-16T10:44:54.954 INFO:tasks.workunit.client.0.smithi002.stdout:8/287: dread d7/d13/f4c [0,4194304] 0 2024-03-16T10:44:54.955 INFO:tasks.workunit.client.0.smithi002.stdout:8/288: truncate d7/d9/d25/f32 1670882 0 2024-03-16T10:44:54.960 INFO:tasks.workunit.client.0.smithi002.stdout:1/253: mknod d2/d3/d51/c57 0 2024-03-16T10:44:54.961 INFO:tasks.workunit.client.0.smithi002.stdout:3/228: link d1/dd/d28/d37/c32 d1/dd/d11/d15/d1e/d22/d3f/c56 0 2024-03-16T10:44:54.962 INFO:tasks.workunit.client.0.smithi002.stdout:3/229: write d1/f35 [6752201,84241] 0 2024-03-16T10:44:54.963 INFO:tasks.workunit.client.0.smithi002.stdout:3/230: chown d1/dd/d28 31 1 2024-03-16T10:44:54.967 INFO:tasks.workunit.client.0.smithi002.stdout:4/291: mkdir d2/d33/d35/d59 0 2024-03-16T10:44:54.968 INFO:tasks.workunit.client.0.smithi002.stdout:4/292: write d2/d33/da/d2e/f47 [4231964,76702] 0 2024-03-16T10:44:54.971 INFO:tasks.workunit.client.0.smithi002.stdout:8/289: mkdir d7/d9/d17/d20/d64 0 2024-03-16T10:44:54.977 INFO:tasks.workunit.client.0.smithi002.stdout:8/290: dwrite d7/d9/d17/d20/f63 [0,4194304] 0 2024-03-16T10:44:54.989 INFO:tasks.workunit.client.0.smithi002.stdout:6/289: getdents d15/d2d/d2b 0 2024-03-16T10:44:54.991 INFO:tasks.workunit.client.0.smithi002.stdout:5/240: getdents d0/d16/d1d/d33 0 2024-03-16T10:44:54.991 INFO:tasks.workunit.client.0.smithi002.stdout:5/241: fdatasync d0/d16/d1d/d33/f3e 0 2024-03-16T10:44:54.996 INFO:tasks.workunit.client.0.smithi002.stdout:8/291: creat d7/d9/d17/d23/d2e/d3a/f65 x:0 0 0 2024-03-16T10:44:54.997 INFO:tasks.workunit.client.0.smithi002.stdout:1/254: dwrite d2/df/f27 [0,4194304] 0 2024-03-16T10:44:55.000 INFO:tasks.workunit.client.0.smithi002.stdout:4/293: mknod d2/c5a 0 2024-03-16T10:44:55.003 INFO:tasks.workunit.client.0.smithi002.stdout:2/295: dwrite d7/db/d12/f11 [4194304,4194304] 0 2024-03-16T10:44:55.006 INFO:tasks.workunit.client.0.smithi002.stdout:1/255: chown d2/df/d12/d29/f34 5 1 2024-03-16T10:44:55.008 INFO:tasks.workunit.client.0.smithi002.stdout:9/270: getdents d5/d9/dc/d1e/d24/d31 0 2024-03-16T10:44:55.009 INFO:tasks.workunit.client.0.smithi002.stdout:4/294: creat d2/d33/d3b/f5b x:0 0 0 2024-03-16T10:44:55.009 INFO:tasks.workunit.client.0.smithi002.stdout:9/271: fdatasync d5/d9/dc/d28/f2d 0 2024-03-16T10:44:55.011 INFO:tasks.workunit.client.0.smithi002.stdout:5/242: mknod d0/d16/d1d/d20/d39/c45 0 2024-03-16T10:44:55.013 INFO:tasks.workunit.client.0.smithi002.stdout:4/295: write d2/d33/da/ff [3730540,111284] 0 2024-03-16T10:44:55.015 INFO:tasks.workunit.client.0.smithi002.stdout:2/296: dwrite d7/db/d12/f11 [4194304,4194304] 0 2024-03-16T10:44:55.023 INFO:tasks.workunit.client.0.smithi002.stdout:9/272: dwrite d5/d9/dc/d28/f4c [0,4194304] 0 2024-03-16T10:44:55.031 INFO:tasks.workunit.client.0.smithi002.stdout:2/297: fsync d7/db/d12/f11 0 2024-03-16T10:44:55.032 INFO:tasks.workunit.client.0.smithi002.stdout:4/296: rename d2/d33/da/l58 to d2/d33/d35/d59/l5c 0 2024-03-16T10:44:55.033 INFO:tasks.workunit.client.0.smithi002.stdout:4/297: write d2/d33/d1c/d45/f54 [47074,1883] 0 2024-03-16T10:44:55.033 INFO:tasks.workunit.client.0.smithi002.stdout:6/290: link d15/d2d/d2b/d3f/l41 d15/l6e 0 2024-03-16T10:44:55.038 INFO:tasks.workunit.client.0.smithi002.stdout:1/256: dread d2/df/f33 [0,4194304] 0 2024-03-16T10:44:55.039 INFO:tasks.workunit.client.0.smithi002.stdout:5/243: link d0/d24/f3c d0/d16/d1d/f46 0 2024-03-16T10:44:55.040 INFO:tasks.workunit.client.0.smithi002.stdout:4/298: dwrite d2/d33/da/f28 [0,4194304] 0 2024-03-16T10:44:55.041 INFO:tasks.workunit.client.0.smithi002.stdout:5/244: chown d0/d16/d1d/f46 6309 1 2024-03-16T10:44:55.042 INFO:tasks.workunit.client.0.smithi002.stdout:5/245: stat d0/d16/d1d/f3b 0 2024-03-16T10:44:55.042 INFO:tasks.workunit.client.0.smithi002.stdout:2/298: mknod d7/d2e/d1d/c55 0 2024-03-16T10:44:55.042 INFO:tasks.workunit.client.0.smithi002.stdout:5/246: chown d0/d16/d1d/d20/c2d 27551850 1 2024-03-16T10:44:55.043 INFO:tasks.workunit.client.0.smithi002.stdout:6/291: creat d15/d2d/d2c/d5d/f6f x:0 0 0 2024-03-16T10:44:55.044 INFO:tasks.workunit.client.0.smithi002.stdout:6/292: read d15/d2d/d2c/f55 [856330,1054] 0 2024-03-16T10:44:55.048 INFO:tasks.workunit.client.0.smithi002.stdout:4/299: symlink d2/d33/d35/d59/l5d 0 2024-03-16T10:44:55.049 INFO:tasks.workunit.client.0.smithi002.stdout:4/300: write d2/d33/f29 [88955,8022] 0 2024-03-16T10:44:55.049 INFO:tasks.workunit.client.0.smithi002.stdout:2/299: creat d7/d27/f56 x:0 0 0 2024-03-16T10:44:55.050 INFO:tasks.workunit.client.0.smithi002.stdout:4/301: fdatasync d2/d33/d3b/f4b 0 2024-03-16T10:44:55.052 INFO:tasks.workunit.client.0.smithi002.stdout:9/273: link d5/d9/dc/c14 d5/d9/dc/d28/c51 0 2024-03-16T10:44:55.054 INFO:tasks.workunit.client.0.smithi002.stdout:1/257: mkdir d2/d58 0 2024-03-16T10:44:55.056 INFO:tasks.workunit.client.0.smithi002.stdout:1/258: read d2/df/f27 [3411121,127334] 0 2024-03-16T10:44:55.058 INFO:tasks.workunit.client.0.smithi002.stdout:1/259: read d2/d3/f7 [642257,116075] 0 2024-03-16T10:44:55.058 INFO:tasks.workunit.client.0.smithi002.stdout:4/302: dwrite d2/d6/f1e [0,4194304] 0 2024-03-16T10:44:55.063 INFO:tasks.workunit.client.0.smithi002.stdout:2/300: dread d7/db/d12/f11 [0,4194304] 0 2024-03-16T10:44:55.070 INFO:tasks.workunit.client.0.smithi002.stdout:6/293: creat d15/d2d/f70 x:0 0 0 2024-03-16T10:44:55.072 INFO:tasks.workunit.client.0.smithi002.stdout:4/303: unlink d2/d33/c51 0 2024-03-16T10:44:55.077 INFO:tasks.workunit.client.0.smithi002.stdout:2/301: creat d7/db/f57 x:0 0 0 2024-03-16T10:44:55.077 INFO:tasks.workunit.client.0.smithi002.stdout:2/302: readlink d7/db/l1b 0 2024-03-16T10:44:55.080 INFO:tasks.workunit.client.0.smithi002.stdout:1/260: mknod d2/c59 0 2024-03-16T10:44:55.085 INFO:tasks.workunit.client.0.smithi002.stdout:6/294: dread d15/d2d/f20 [0,4194304] 0 2024-03-16T10:44:55.086 INFO:tasks.workunit.client.0.smithi002.stdout:2/303: link d7/d2e/d28/d40/f52 d7/db/f58 0 2024-03-16T10:44:55.087 INFO:tasks.workunit.client.0.smithi002.stdout:6/295: read - d15/f4f zero size 2024-03-16T10:44:55.088 INFO:tasks.workunit.client.0.smithi002.stdout:1/261: dread d2/d3/d1c/f4a [0,4194304] 0 2024-03-16T10:44:55.089 INFO:tasks.workunit.client.0.smithi002.stdout:2/304: dread - d7/d2e/d1d/d38/f3a zero size 2024-03-16T10:44:55.089 INFO:tasks.workunit.client.0.smithi002.stdout:5/247: dwrite d0/f18 [0,4194304] 0 2024-03-16T10:44:55.090 INFO:tasks.workunit.client.0.smithi002.stdout:5/248: chown d0/d24/f3c 957615224 1 2024-03-16T10:44:55.093 INFO:tasks.workunit.client.0.smithi002.stdout:6/296: mknod d15/d2d/d2c/c71 0 2024-03-16T10:44:55.093 INFO:tasks.workunit.client.0.smithi002.stdout:9/274: write d5/d9/dc/f11 [3134244,98460] 0 2024-03-16T10:44:55.093 INFO:tasks.workunit.client.0.smithi002.stdout:6/297: dread - d15/f5f zero size 2024-03-16T10:44:55.094 INFO:tasks.workunit.client.0.smithi002.stdout:6/298: readlink d15/d2d/d2b/l38 0 2024-03-16T10:44:55.095 INFO:tasks.workunit.client.0.smithi002.stdout:1/262: fsync d2/d3/f7 0 2024-03-16T10:44:55.100 INFO:tasks.workunit.client.0.smithi002.stdout:5/249: fdatasync d0/d16/d1d/f23 0 2024-03-16T10:44:55.100 INFO:tasks.workunit.client.0.smithi002.stdout:5/250: readlink d0/d16/d1d/l41 0 2024-03-16T10:44:55.101 INFO:tasks.workunit.client.0.smithi002.stdout:9/275: symlink d5/d9/d25/d23/l52 0 2024-03-16T10:44:55.103 INFO:tasks.workunit.client.0.smithi002.stdout:6/299: creat d15/d2d/d2c/d5d/f72 x:0 0 0 2024-03-16T10:44:55.106 INFO:tasks.workunit.client.0.smithi002.stdout:1/263: rename d2/d3/d1c/f44 to d2/d3/d51/f5a 0 2024-03-16T10:44:55.108 INFO:tasks.workunit.client.0.smithi002.stdout:2/305: link d7/d2e/d1d/f41 d7/d2e/d1d/d2f/f59 0 2024-03-16T10:44:55.112 INFO:tasks.workunit.client.0.smithi002.stdout:1/264: dwrite d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:55.112 INFO:tasks.workunit.client.0.smithi002.stdout:5/251: dread d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:44:55.113 INFO:tasks.workunit.client.0.smithi002.stdout:1/265: chown d2/d3/d1c/c43 5149062 1 2024-03-16T10:44:55.116 INFO:tasks.workunit.client.0.smithi002.stdout:2/306: creat d7/d2e/d28/d40/f5a x:0 0 0 2024-03-16T10:44:55.119 INFO:tasks.workunit.client.0.smithi002.stdout:6/300: mkdir d15/d2d/d73 0 2024-03-16T10:44:55.127 INFO:tasks.workunit.client.0.smithi002.stdout:6/301: mknod d15/d2d/d2c/d5d/c74 0 2024-03-16T10:44:55.128 INFO:tasks.workunit.client.0.smithi002.stdout:6/302: stat d15/d2d/d2b/l51 0 2024-03-16T10:44:55.129 INFO:tasks.workunit.client.0.smithi002.stdout:9/276: rename d5/d9/dc/d1e/d24/f46 to d5/f53 0 2024-03-16T10:44:55.130 INFO:tasks.workunit.client.0.smithi002.stdout:5/252: symlink d0/d16/d1d/d43/l47 0 2024-03-16T10:44:55.130 INFO:tasks.workunit.client.0.smithi002.stdout:1/266: mknod d2/d3/d51/d56/c5b 0 2024-03-16T10:44:55.133 INFO:tasks.workunit.client.0.smithi002.stdout:6/303: mkdir d15/d2d/d2b/d75 0 2024-03-16T10:44:55.135 INFO:tasks.workunit.client.0.smithi002.stdout:9/277: creat d5/d9/d22/d3c/d4d/f54 x:0 0 0 2024-03-16T10:44:55.136 INFO:tasks.workunit.client.0.smithi002.stdout:2/307: dread d7/d2e/f32 [0,4194304] 0 2024-03-16T10:44:55.136 INFO:tasks.workunit.client.0.smithi002.stdout:9/278: chown d5/d9/d22/d3c/d4d/f54 775 1 2024-03-16T10:44:55.137 INFO:tasks.workunit.client.0.smithi002.stdout:5/253: mkdir d0/d16/d1d/d43/d48 0 2024-03-16T10:44:55.138 INFO:tasks.workunit.client.0.smithi002.stdout:9/279: fdatasync d5/d9/dc/d1e/d24/f4b 0 2024-03-16T10:44:55.139 INFO:tasks.workunit.client.0.smithi002.stdout:1/267: fsync d2/d3/d1c/f4a 0 2024-03-16T10:44:55.139 INFO:tasks.workunit.client.0.smithi002.stdout:5/254: rename d0/d16/d1d to d0/d16/d1d/d43/d48/d49 22 2024-03-16T10:44:55.140 INFO:tasks.workunit.client.0.smithi002.stdout:2/308: dread f6 [0,4194304] 0 2024-03-16T10:44:55.141 INFO:tasks.workunit.client.0.smithi002.stdout:6/304: chown d15/d2d/d2b/d3f/c59 112911 1 2024-03-16T10:44:55.142 INFO:tasks.workunit.client.0.smithi002.stdout:6/305: chown f11 11 1 2024-03-16T10:44:55.142 INFO:tasks.workunit.client.0.smithi002.stdout:6/306: chown d15/d2d/d2b/f35 1 1 2024-03-16T10:44:55.145 INFO:tasks.workunit.client.0.smithi002.stdout:9/280: mkdir d5/d9/d55 0 2024-03-16T10:44:55.147 INFO:tasks.workunit.client.0.smithi002.stdout:9/281: read d5/d9/d25/d23/f3b [1365043,11130] 0 2024-03-16T10:44:55.147 INFO:tasks.workunit.client.0.smithi002.stdout:1/268: mkdir d2/df/d12/d29/d5c 0 2024-03-16T10:44:55.152 INFO:tasks.workunit.client.0.smithi002.stdout:6/307: creat d15/d2d/d2c/d5d/f76 x:0 0 0 2024-03-16T10:44:55.155 INFO:tasks.workunit.client.0.smithi002.stdout:9/282: creat d5/d9/d25/d23/f56 x:0 0 0 2024-03-16T10:44:55.156 INFO:tasks.workunit.client.0.smithi002.stdout:1/269: mkdir d2/d3/d1c/d5d 0 2024-03-16T10:44:55.159 INFO:tasks.workunit.client.0.smithi002.stdout:2/309: getdents d7/d2e/d28/d40/d50 0 2024-03-16T10:44:55.171 INFO:tasks.workunit.client.0.smithi002.stdout:1/270: dwrite d2/d3/f39 [0,4194304] 0 2024-03-16T10:44:55.171 INFO:tasks.workunit.client.0.smithi002.stdout:7/219: sync 2024-03-16T10:44:55.171 INFO:tasks.workunit.client.0.smithi002.stdout:0/290: sync 2024-03-16T10:44:55.173 INFO:tasks.workunit.client.0.smithi002.stdout:1/271: truncate d2/d3/f55 847816 0 2024-03-16T10:44:55.175 INFO:tasks.workunit.client.0.smithi002.stdout:2/310: link d7/d27/d3b/c53 d7/d2e/d1d/d2f/d4b/c5b 0 2024-03-16T10:44:55.177 INFO:tasks.workunit.client.0.smithi002.stdout:0/291: write d3/de/f43 [1802049,93564] 0 2024-03-16T10:44:55.178 INFO:tasks.workunit.client.0.smithi002.stdout:0/292: chown d3/de/f15 5 1 2024-03-16T10:44:55.178 INFO:tasks.workunit.client.0.smithi002.stdout:2/311: dread d7/db/f48 [0,4194304] 0 2024-03-16T10:44:55.178 INFO:tasks.workunit.client.0.smithi002.stdout:5/255: write d0/d16/d1d/f46 [306577,123251] 0 2024-03-16T10:44:55.180 INFO:tasks.workunit.client.0.smithi002.stdout:5/256: write d0/d16/f19 [1891957,65264] 0 2024-03-16T10:44:55.181 INFO:tasks.workunit.client.0.smithi002.stdout:2/312: truncate d7/d2e/d1d/d38/f3a 719564 0 2024-03-16T10:44:55.187 INFO:tasks.workunit.client.0.smithi002.stdout:6/308: rmdir d15 39 2024-03-16T10:44:55.189 INFO:tasks.workunit.client.0.smithi002.stdout:1/272: dread d2/df/d12/f37 [0,4194304] 0 2024-03-16T10:44:55.193 INFO:tasks.workunit.client.0.smithi002.stdout:9/283: write d5/d9/dc/d1e/d24/d31/f4a [212842,15703] 0 2024-03-16T10:44:55.194 INFO:tasks.workunit.client.0.smithi002.stdout:2/313: dread d7/db/d12/f22 [0,4194304] 0 2024-03-16T10:44:55.197 INFO:tasks.workunit.client.0.smithi002.stdout:0/293: mkdir d3/d59 0 2024-03-16T10:44:55.199 INFO:tasks.workunit.client.0.smithi002.stdout:3/231: sync 2024-03-16T10:44:55.199 INFO:tasks.workunit.client.0.smithi002.stdout:8/292: sync 2024-03-16T10:44:55.199 INFO:tasks.workunit.client.0.smithi002.stdout:4/304: sync 2024-03-16T10:44:55.200 INFO:tasks.workunit.client.0.smithi002.stdout:5/257: fdatasync d0/d1f/f36 0 2024-03-16T10:44:55.203 INFO:tasks.workunit.client.0.smithi002.stdout:0/294: dread d3/d28/d3d/d44/f53 [0,4194304] 0 2024-03-16T10:44:55.203 INFO:tasks.workunit.client.0.smithi002.stdout:1/273: rename d2/df/f27 to d2/d3/d51/d56/f5e 0 2024-03-16T10:44:55.203 INFO:tasks.workunit.client.0.smithi002.stdout:7/220: write d3/d4/d2a/f39 [3680693,115637] 0 2024-03-16T10:44:55.204 INFO:tasks.workunit.client.0.smithi002.stdout:9/284: fdatasync d5/d9/dc/f16 0 2024-03-16T10:44:55.206 INFO:tasks.workunit.client.0.smithi002.stdout:8/293: dread d7/d9/d17/d23/d39/f44 [0,4194304] 0 2024-03-16T10:44:55.207 INFO:tasks.workunit.client.0.smithi002.stdout:8/294: fdatasync d7/d9/d17/d23/d2e/d3a/f47 0 2024-03-16T10:44:55.209 INFO:tasks.workunit.client.0.smithi002.stdout:2/314: read d7/d2e/f20 [243949,9829] 0 2024-03-16T10:44:55.209 INFO:tasks.workunit.client.0.smithi002.stdout:6/309: mkdir d15/d2d/d2b/d3f/d77 0 2024-03-16T10:44:55.212 INFO:tasks.workunit.client.0.smithi002.stdout:2/315: dread d7/d2e/d28/d44/f4d [0,4194304] 0 2024-03-16T10:44:55.213 INFO:tasks.workunit.client.0.smithi002.stdout:2/316: readlink d7/db/d12/l14 0 2024-03-16T10:44:55.214 INFO:tasks.workunit.client.0.smithi002.stdout:2/317: readlink d7/d2e/d28/d40/l47 0 2024-03-16T10:44:55.216 INFO:tasks.workunit.client.0.smithi002.stdout:3/232: creat d1/dd/d11/d15/d1e/d22/d3f/f57 x:0 0 0 2024-03-16T10:44:55.218 INFO:tasks.workunit.client.0.smithi002.stdout:9/285: dread d5/d9/dc/f11 [0,4194304] 0 2024-03-16T10:44:55.218 INFO:tasks.workunit.client.0.smithi002.stdout:0/295: rmdir d3/d28/d2a 39 2024-03-16T10:44:55.219 INFO:tasks.workunit.client.0.smithi002.stdout:0/296: read - d3/d28/d3d/f45 zero size 2024-03-16T10:44:55.220 INFO:tasks.workunit.client.0.smithi002.stdout:0/297: fdatasync d3/de/f20 0 2024-03-16T10:44:55.223 INFO:tasks.workunit.client.0.smithi002.stdout:3/233: dwrite d1/dd/d11/d15/d1e/d22/f39 [0,4194304] 0 2024-03-16T10:44:55.224 INFO:tasks.workunit.client.0.smithi002.stdout:3/234: chown d1 5249 1 2024-03-16T10:44:55.226 INFO:tasks.workunit.client.0.smithi002.stdout:8/295: symlink d7/d55/l66 0 2024-03-16T10:44:55.233 INFO:tasks.workunit.client.0.smithi002.stdout:9/286: mknod d5/d9/d25/d23/c57 0 2024-03-16T10:44:55.235 INFO:tasks.workunit.client.0.smithi002.stdout:7/221: creat d3/d37/f48 x:0 0 0 2024-03-16T10:44:55.235 INFO:tasks.workunit.client.0.smithi002.stdout:0/298: symlink d3/d31/l5a 0 2024-03-16T10:44:55.236 INFO:tasks.workunit.client.0.smithi002.stdout:7/222: stat d3/d4/f1e 0 2024-03-16T10:44:55.236 INFO:tasks.workunit.client.0.smithi002.stdout:1/274: mknod d2/d3/d1c/d5d/c5f 0 2024-03-16T10:44:55.241 INFO:tasks.workunit.client.0.smithi002.stdout:3/235: symlink d1/dd/d28/d37/l58 0 2024-03-16T10:44:55.242 INFO:tasks.workunit.client.0.smithi002.stdout:6/310: symlink d15/l78 0 2024-03-16T10:44:55.244 INFO:tasks.workunit.client.0.smithi002.stdout:8/296: unlink d7/d13/f56 0 2024-03-16T10:44:55.249 INFO:tasks.workunit.client.0.smithi002.stdout:5/258: rename d0/d16/d1d/d20 to d0/d16/d4a 0 2024-03-16T10:44:55.251 INFO:tasks.workunit.client.0.smithi002.stdout:2/318: dwrite d7/d2e/d28/d40/f52 [0,4194304] 0 2024-03-16T10:44:55.252 INFO:tasks.workunit.client.0.smithi002.stdout:0/299: chown d3/d5/l21 1 1 2024-03-16T10:44:55.253 INFO:tasks.workunit.client.0.smithi002.stdout:8/297: dwrite d7/f8 [0,4194304] 0 2024-03-16T10:44:55.253 INFO:tasks.workunit.client.0.smithi002.stdout:7/223: unlink d3/d4/d2a/f2f 0 2024-03-16T10:44:55.256 INFO:tasks.workunit.client.0.smithi002.stdout:3/236: creat d1/dd/d28/d37/f59 x:0 0 0 2024-03-16T10:44:55.262 INFO:tasks.workunit.client.0.smithi002.stdout:4/305: getdents d2 0 2024-03-16T10:44:55.267 INFO:tasks.workunit.client.0.smithi002.stdout:9/287: rename d5/d9/l41 to d5/d9/d22/d3c/d4d/l58 0 2024-03-16T10:44:55.269 INFO:tasks.workunit.client.0.smithi002.stdout:9/288: dread d5/d9/dc/d1e/f36 [0,4194304] 0 2024-03-16T10:44:55.271 INFO:tasks.workunit.client.0.smithi002.stdout:6/311: write d15/f1c [839402,46123] 0 2024-03-16T10:44:55.273 INFO:tasks.workunit.client.0.smithi002.stdout:8/298: mkdir d7/d9/d25/d67 0 2024-03-16T10:44:55.274 INFO:tasks.workunit.client.0.smithi002.stdout:7/224: symlink d3/d4/d1c/l49 0 2024-03-16T10:44:55.274 INFO:tasks.workunit.client.0.smithi002.stdout:9/289: dread d5/f7 [0,4194304] 0 2024-03-16T10:44:55.275 INFO:tasks.workunit.client.0.smithi002.stdout:3/237: mkdir d1/d5a 0 2024-03-16T10:44:55.276 INFO:tasks.workunit.client.0.smithi002.stdout:4/306: mkdir d2/d33/da/d5e 0 2024-03-16T10:44:55.277 INFO:tasks.workunit.client.0.smithi002.stdout:0/300: mknod d3/c5b 0 2024-03-16T10:44:55.277 INFO:tasks.workunit.client.0.smithi002.stdout:5/259: getdents d0/d16/d1d/d43/d48 0 2024-03-16T10:44:55.278 INFO:tasks.workunit.client.0.smithi002.stdout:0/301: write d3/d28/f3e [687480,12089] 0 2024-03-16T10:44:55.279 INFO:tasks.workunit.client.0.smithi002.stdout:1/275: creat d2/df/d12/f60 x:0 0 0 2024-03-16T10:44:55.280 INFO:tasks.workunit.client.0.smithi002.stdout:6/312: mkdir d15/d2d/d2b/d79 0 2024-03-16T10:44:55.281 INFO:tasks.workunit.client.0.smithi002.stdout:1/276: dread d2/d3/f55 [0,4194304] 0 2024-03-16T10:44:55.284 INFO:tasks.workunit.client.0.smithi002.stdout:3/238: symlink d1/dd/d11/d15/l5b 0 2024-03-16T10:44:55.284 INFO:tasks.workunit.client.0.smithi002.stdout:3/239: chown d1/l7 6646 1 2024-03-16T10:44:55.285 INFO:tasks.workunit.client.0.smithi002.stdout:4/307: creat d2/d33/da/f5f x:0 0 0 2024-03-16T10:44:55.291 INFO:tasks.workunit.client.0.smithi002.stdout:6/313: write d15/f24 [3482283,53206] 0 2024-03-16T10:44:55.293 INFO:tasks.workunit.client.0.smithi002.stdout:6/314: dread - d15/d2d/d2c/f50 zero size 2024-03-16T10:44:55.294 INFO:tasks.workunit.client.0.smithi002.stdout:1/277: symlink d2/d3/d51/d56/l61 0 2024-03-16T10:44:55.296 INFO:tasks.workunit.client.0.smithi002.stdout:4/308: dread d2/d6/f10 [0,4194304] 0 2024-03-16T10:44:55.297 INFO:tasks.workunit.client.0.smithi002.stdout:3/240: rmdir d1/dd/d11/d15/d1e 39 2024-03-16T10:44:55.298 INFO:tasks.workunit.client.0.smithi002.stdout:2/319: getdents d7/d2e/d1d/d2f/d4b 0 2024-03-16T10:44:55.301 INFO:tasks.workunit.client.0.smithi002.stdout:9/290: dwrite d5/fd [0,4194304] 0 2024-03-16T10:44:55.301 INFO:tasks.workunit.client.0.smithi002.stdout:1/278: dread d2/df/d12/d29/f34 [0,4194304] 0 2024-03-16T10:44:55.302 INFO:tasks.workunit.client.0.smithi002.stdout:8/299: truncate d7/f48 2044642 0 2024-03-16T10:44:55.304 INFO:tasks.workunit.client.0.smithi002.stdout:4/309: dread d2/d33/f1b [0,4194304] 0 2024-03-16T10:44:55.306 INFO:tasks.workunit.client.0.smithi002.stdout:7/225: creat d3/d4/f4a x:0 0 0 2024-03-16T10:44:55.307 INFO:tasks.workunit.client.0.smithi002.stdout:6/315: dread - d15/d2d/f28 zero size 2024-03-16T10:44:55.307 INFO:tasks.workunit.client.0.smithi002.stdout:5/260: truncate d0/d16/d1d/f46 187504 0 2024-03-16T10:44:55.307 INFO:tasks.workunit.client.0.smithi002.stdout:0/302: creat d3/de/f5c x:0 0 0 2024-03-16T10:44:55.308 INFO:tasks.workunit.client.0.smithi002.stdout:1/279: dread d2/d3/d1c/f4a [0,4194304] 0 2024-03-16T10:44:55.310 INFO:tasks.workunit.client.0.smithi002.stdout:2/320: write d7/d27/d3b/f51 [593067,38726] 0 2024-03-16T10:44:55.312 INFO:tasks.workunit.client.0.smithi002.stdout:0/303: write d3/fc [4865698,51850] 0 2024-03-16T10:44:55.319 INFO:tasks.workunit.client.0.smithi002.stdout:2/321: dwrite d7/d2e/d28/d40/f52 [0,4194304] 0 2024-03-16T10:44:55.323 INFO:tasks.workunit.client.0.smithi002.stdout:0/304: dwrite d3/d28/d3d/d44/f52 [0,4194304] 0 2024-03-16T10:44:55.324 INFO:tasks.workunit.client.0.smithi002.stdout:0/305: fdatasync d3/d27/f55 0 2024-03-16T10:44:55.326 INFO:tasks.workunit.client.0.smithi002.stdout:4/310: link d2/d6/f13 d2/d33/d35/d59/f60 0 2024-03-16T10:44:55.326 INFO:tasks.workunit.client.0.smithi002.stdout:5/261: getdents d0/d16/d1d/d43/d48 0 2024-03-16T10:44:55.330 INFO:tasks.workunit.client.0.smithi002.stdout:8/300: creat d7/d13/f68 x:0 0 0 2024-03-16T10:44:55.330 INFO:tasks.workunit.client.0.smithi002.stdout:9/291: link d5/d9/d25/l1b d5/d9/dc/d1e/d24/l59 0 2024-03-16T10:44:55.332 INFO:tasks.workunit.client.0.smithi002.stdout:7/226: mknod d3/d4/d10/d34/c4b 0 2024-03-16T10:44:55.333 INFO:tasks.workunit.client.0.smithi002.stdout:5/262: dread d0/d1f/f36 [0,4194304] 0 2024-03-16T10:44:55.333 INFO:tasks.workunit.client.0.smithi002.stdout:9/292: fdatasync d5/d9/d22/f47 0 2024-03-16T10:44:55.335 INFO:tasks.workunit.client.0.smithi002.stdout:2/322: creat d7/d27/d3b/f5c x:0 0 0 2024-03-16T10:44:55.335 INFO:tasks.workunit.client.0.smithi002.stdout:0/306: dread d3/fc [0,4194304] 0 2024-03-16T10:44:55.337 INFO:tasks.workunit.client.0.smithi002.stdout:6/316: dwrite d15/d2d/d2b/f35 [0,4194304] 0 2024-03-16T10:44:55.338 INFO:tasks.workunit.client.0.smithi002.stdout:5/263: read - d0/d16/d1d/d33/f42 zero size 2024-03-16T10:44:55.338 INFO:tasks.workunit.client.0.smithi002.stdout:2/323: stat d7/d2e/d28/d40/f5a 0 2024-03-16T10:44:55.338 INFO:tasks.workunit.client.0.smithi002.stdout:9/293: chown d5/d9/d22/d3c/c3e 107 1 2024-03-16T10:44:55.338 INFO:tasks.workunit.client.0.smithi002.stdout:0/307: chown d3/d5/f18 10315497 1 2024-03-16T10:44:55.340 INFO:tasks.workunit.client.0.smithi002.stdout:0/308: write d3/f57 [855273,96299] 0 2024-03-16T10:44:55.340 INFO:tasks.workunit.client.0.smithi002.stdout:9/294: read d5/d9/dc/d1e/f36 [93734,63446] 0 2024-03-16T10:44:55.341 INFO:tasks.workunit.client.0.smithi002.stdout:5/264: truncate d0/d16/d1d/f3b 339826 0 2024-03-16T10:44:55.341 INFO:tasks.workunit.client.0.smithi002.stdout:4/311: rename d2/c24 to d2/d33/d3b/c61 0 2024-03-16T10:44:55.348 INFO:tasks.workunit.client.0.smithi002.stdout:2/324: dwrite d7/d2e/d1d/f21 [4194304,4194304] 0 2024-03-16T10:44:55.349 INFO:tasks.workunit.client.0.smithi002.stdout:8/301: chown d7/d9/d17/d20/l49 31758 1 2024-03-16T10:44:55.359 INFO:tasks.workunit.client.0.smithi002.stdout:6/317: creat d15/d2d/d2c/f7a x:0 0 0 2024-03-16T10:44:55.360 INFO:tasks.workunit.client.0.smithi002.stdout:7/227: dread d3/d4/d2a/f40 [0,4194304] 0 2024-03-16T10:44:55.366 INFO:tasks.workunit.client.0.smithi002.stdout:9/295: creat d5/d9/dc/d28/f5a x:0 0 0 2024-03-16T10:44:55.369 INFO:tasks.workunit.client.0.smithi002.stdout:8/302: mknod d7/d9/d17/d20/c69 0 2024-03-16T10:44:55.370 INFO:tasks.workunit.client.0.smithi002.stdout:1/280: link d2/df/d12/d29/c3c d2/df/d12/c62 0 2024-03-16T10:44:55.373 INFO:tasks.workunit.client.0.smithi002.stdout:9/296: dwrite d5/d9/dc/d28/f4c [0,4194304] 0 2024-03-16T10:44:55.375 INFO:tasks.workunit.client.0.smithi002.stdout:6/318: rmdir d15/d2d 39 2024-03-16T10:44:55.383 INFO:tasks.workunit.client.0.smithi002.stdout:9/297: dwrite d5/d9/dc/f43 [0,4194304] 0 2024-03-16T10:44:55.385 INFO:tasks.workunit.client.0.smithi002.stdout:2/325: link d7/f8 d7/db/d12/f5d 0 2024-03-16T10:44:55.390 INFO:tasks.workunit.client.0.smithi002.stdout:4/312: symlink d2/d33/d35/l62 0 2024-03-16T10:44:55.391 INFO:tasks.workunit.client.0.smithi002.stdout:0/309: rename d3/f57 to d3/de/f5d 0 2024-03-16T10:44:55.394 INFO:tasks.workunit.client.0.smithi002.stdout:9/298: rmdir d5/d9/d22 39 2024-03-16T10:44:55.395 INFO:tasks.workunit.client.0.smithi002.stdout:9/299: write d5/d9/dc/d1e/d24/d31/f32 [1732670,32384] 0 2024-03-16T10:44:55.399 INFO:tasks.workunit.client.0.smithi002.stdout:7/228: creat d3/d4/d10/f4c x:0 0 0 2024-03-16T10:44:55.401 INFO:tasks.workunit.client.0.smithi002.stdout:8/303: rename d7/d9/d25/l4f to d7/d9/d17/d23/d2e/l6a 0 2024-03-16T10:44:55.404 INFO:tasks.workunit.client.0.smithi002.stdout:9/300: dwrite d5/d9/dc/d1e/d24/f30 [4194304,4194304] 0 2024-03-16T10:44:55.407 INFO:tasks.workunit.client.0.smithi002.stdout:6/319: symlink d15/d2d/l7b 0 2024-03-16T10:44:55.408 INFO:tasks.workunit.client.0.smithi002.stdout:7/229: creat d3/d4/d2a/f4d x:0 0 0 2024-03-16T10:44:55.408 INFO:tasks.workunit.client.0.smithi002.stdout:5/265: dwrite d0/d16/f37 [0,4194304] 0 2024-03-16T10:44:55.415 INFO:tasks.workunit.client.0.smithi002.stdout:1/281: truncate d2/df/d12/f4e 1037394 0 2024-03-16T10:44:55.416 INFO:tasks.workunit.client.0.smithi002.stdout:2/326: creat d7/f5e x:0 0 0 2024-03-16T10:44:55.418 INFO:tasks.workunit.client.0.smithi002.stdout:2/327: dread - d7/d2e/d28/d40/f5a zero size 2024-03-16T10:44:55.419 INFO:tasks.workunit.client.0.smithi002.stdout:1/282: stat d2/df/f33 0 2024-03-16T10:44:55.419 INFO:tasks.workunit.client.0.smithi002.stdout:0/310: link d3/d28/l38 d3/d59/l5e 0 2024-03-16T10:44:55.420 INFO:tasks.workunit.client.0.smithi002.stdout:9/301: dwrite d5/d9/dc/f20 [0,4194304] 0 2024-03-16T10:44:55.425 INFO:tasks.workunit.client.0.smithi002.stdout:1/283: truncate d2/df/d12/f60 315592 0 2024-03-16T10:44:55.428 INFO:tasks.workunit.client.0.smithi002.stdout:6/320: mkdir d15/d2d/d2b/d79/d7c 0 2024-03-16T10:44:55.428 INFO:tasks.workunit.client.0.smithi002.stdout:5/266: chown d0/l17 676120495 1 2024-03-16T10:44:55.428 INFO:tasks.workunit.client.0.smithi002.stdout:7/230: dwrite d3/d4/d1d/d45/f3a [0,4194304] 0 2024-03-16T10:44:55.430 INFO:tasks.workunit.client.0.smithi002.stdout:8/304: rmdir d7/d9/d17/d20/d64 0 2024-03-16T10:44:55.430 INFO:tasks.workunit.client.0.smithi002.stdout:0/311: mknod d3/d28/d3d/d44/c5f 0 2024-03-16T10:44:55.433 INFO:tasks.workunit.client.0.smithi002.stdout:2/328: mkdir d7/d2e/d1d/d2f/d5f 0 2024-03-16T10:44:55.437 INFO:tasks.workunit.client.0.smithi002.stdout:5/267: mkdir d0/d16/d1d/d33/d4b 0 2024-03-16T10:44:55.438 INFO:tasks.workunit.client.0.smithi002.stdout:7/231: mknod d3/d4/d10/d3f/c4e 0 2024-03-16T10:44:55.439 INFO:tasks.workunit.client.0.smithi002.stdout:5/268: dread - d0/d16/d1d/d33/f42 zero size 2024-03-16T10:44:55.442 INFO:tasks.workunit.client.0.smithi002.stdout:1/284: dwrite d2/df/d12/d29/f34 [0,4194304] 0 2024-03-16T10:44:55.444 INFO:tasks.workunit.client.0.smithi002.stdout:6/321: symlink d15/l7d 0 2024-03-16T10:44:55.446 INFO:tasks.workunit.client.0.smithi002.stdout:6/322: write d15/d2d/d2b/f68 [782931,117824] 0 2024-03-16T10:44:55.450 INFO:tasks.workunit.client.0.smithi002.stdout:9/302: getdents d5/d9/d25 0 2024-03-16T10:44:55.452 INFO:tasks.workunit.client.0.smithi002.stdout:8/305: dread d7/d9/d25/f45 [4194304,4194304] 0 2024-03-16T10:44:55.453 INFO:tasks.workunit.client.0.smithi002.stdout:0/312: creat d3/f60 x:0 0 0 2024-03-16T10:44:55.459 INFO:tasks.workunit.client.0.smithi002.stdout:7/232: link d3/f11 d3/d4/d2a/f4f 0 2024-03-16T10:44:55.459 INFO:tasks.workunit.client.0.smithi002.stdout:5/269: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:55.459 INFO:tasks.workunit.client.0.smithi002.stdout:2/329: write d7/db/d12/f22 [2036759,31554] 0 2024-03-16T10:44:55.461 INFO:tasks.workunit.client.0.smithi002.stdout:2/330: stat d7/d2e/d1d/f4f 0 2024-03-16T10:44:55.463 INFO:tasks.workunit.client.0.smithi002.stdout:9/303: symlink d5/d9/dc/d1e/d24/d31/l5b 0 2024-03-16T10:44:55.465 INFO:tasks.workunit.client.0.smithi002.stdout:9/304: truncate d5/d9/d25/f4f 848551 0 2024-03-16T10:44:55.466 INFO:tasks.workunit.client.0.smithi002.stdout:7/233: creat d3/d4/d2a/f50 x:0 0 0 2024-03-16T10:44:55.469 INFO:tasks.workunit.client.0.smithi002.stdout:9/305: dread d5/d9/dc/f16 [0,4194304] 0 2024-03-16T10:44:55.472 INFO:tasks.workunit.client.0.smithi002.stdout:0/313: creat d3/d28/d2a/f61 x:0 0 0 2024-03-16T10:44:55.473 INFO:tasks.workunit.client.0.smithi002.stdout:1/285: write d2/df/d12/f4e [498730,20091] 0 2024-03-16T10:44:55.474 INFO:tasks.workunit.client.0.smithi002.stdout:0/314: chown d3/d28/f36 4 1 2024-03-16T10:44:55.475 INFO:tasks.workunit.client.0.smithi002.stdout:5/270: symlink d0/d16/l4c 0 2024-03-16T10:44:55.480 INFO:tasks.workunit.client.0.smithi002.stdout:8/306: getdents d7/d55 0 2024-03-16T10:44:55.487 INFO:tasks.workunit.client.0.smithi002.stdout:2/331: dwrite d7/d2e/f25 [0,4194304] 0 2024-03-16T10:44:55.490 INFO:tasks.workunit.client.0.smithi002.stdout:6/323: dwrite d15/f33 [4194304,4194304] 0 2024-03-16T10:44:55.491 INFO:tasks.workunit.client.0.smithi002.stdout:8/307: rmdir d7/d9/d17/d23/d39 39 2024-03-16T10:44:55.494 INFO:tasks.workunit.client.0.smithi002.stdout:6/324: write f8 [10258902,32501] 0 2024-03-16T10:44:55.494 INFO:tasks.workunit.client.0.smithi002.stdout:7/234: link d3/d4/d10/l29 d3/d4/d1c/l51 0 2024-03-16T10:44:55.496 INFO:tasks.workunit.client.0.smithi002.stdout:9/306: dwrite d5/d9/dc/d1e/d24/f30 [8388608,4194304] 0 2024-03-16T10:44:55.500 INFO:tasks.workunit.client.0.smithi002.stdout:3/241: sync 2024-03-16T10:44:55.500 INFO:tasks.workunit.client.0.smithi002.stdout:0/315: dwrite d3/d28/d3d/d44/f52 [0,4194304] 0 2024-03-16T10:44:55.501 INFO:tasks.workunit.client.0.smithi002.stdout:8/308: write d7/f14 [7013256,105007] 0 2024-03-16T10:44:55.504 INFO:tasks.workunit.client.0.smithi002.stdout:7/235: write d3/d4/d10/d3f/f43 [126354,66104] 0 2024-03-16T10:44:55.506 INFO:tasks.workunit.client.0.smithi002.stdout:5/271: write d0/d16/d1d/f46 [546636,120593] 0 2024-03-16T10:44:55.506 INFO:tasks.workunit.client.0.smithi002.stdout:1/286: dread d2/d3/d51/d56/f5e [0,4194304] 0 2024-03-16T10:44:55.506 INFO:tasks.workunit.client.0.smithi002.stdout:9/307: chown d5/d9/d25/d23/l52 0 1 2024-03-16T10:44:55.510 INFO:tasks.workunit.client.0.smithi002.stdout:3/242: fdatasync d1/dd/d11/d15/d1e/d22/d3f/f47 0 2024-03-16T10:44:55.511 INFO:tasks.workunit.client.0.smithi002.stdout:0/316: mknod d3/d28/d2a/c62 0 2024-03-16T10:44:55.512 INFO:tasks.workunit.client.0.smithi002.stdout:0/317: stat d3/c5b 0 2024-03-16T10:44:55.515 INFO:tasks.workunit.client.0.smithi002.stdout:9/308: symlink d5/d9/d25/l5c 0 2024-03-16T10:44:55.516 INFO:tasks.workunit.client.0.smithi002.stdout:6/325: mkdir d15/d2d/d2b/d3f/d77/d7e 0 2024-03-16T10:44:55.517 INFO:tasks.workunit.client.0.smithi002.stdout:3/243: write d1/dd/d28/f31 [1118884,3501] 0 2024-03-16T10:44:55.518 INFO:tasks.workunit.client.0.smithi002.stdout:1/287: mknod d2/df/d12/c63 0 2024-03-16T10:44:55.520 INFO:tasks.workunit.client.0.smithi002.stdout:6/326: chown d15/d2d/d73 74822644 1 2024-03-16T10:44:55.521 INFO:tasks.workunit.client.0.smithi002.stdout:8/309: dwrite d7/d9/f51 [0,4194304] 0 2024-03-16T10:44:55.526 INFO:tasks.workunit.client.0.smithi002.stdout:8/310: chown d7/d9/d25/f32 2069343 1 2024-03-16T10:44:55.526 INFO:tasks.workunit.client.0.smithi002.stdout:5/272: creat d0/d16/f4d x:0 0 0 2024-03-16T10:44:55.527 INFO:tasks.workunit.client.0.smithi002.stdout:2/332: dwrite f6 [0,4194304] 0 2024-03-16T10:44:55.528 INFO:tasks.workunit.client.0.smithi002.stdout:8/311: fdatasync d7/d9/d17/d20/f63 0 2024-03-16T10:44:55.534 INFO:tasks.workunit.client.0.smithi002.stdout:7/236: link d3/d4/d2a/f40 d3/f52 0 2024-03-16T10:44:55.534 INFO:tasks.workunit.client.0.smithi002.stdout:3/244: dread d1/dd/d11/d15/d1e/d22/f39 [0,4194304] 0 2024-03-16T10:44:55.535 INFO:tasks.workunit.client.0.smithi002.stdout:7/237: chown d3/d4/lc 2768 1 2024-03-16T10:44:55.535 INFO:tasks.workunit.client.0.smithi002.stdout:0/318: creat d3/d28/f63 x:0 0 0 2024-03-16T10:44:55.536 INFO:tasks.workunit.client.0.smithi002.stdout:5/273: unlink d0/d16/d1d/d2b/c2f 0 2024-03-16T10:44:55.541 INFO:tasks.workunit.client.0.smithi002.stdout:3/245: write d1/dd/d11/d15/d1e/d22/d3f/f51 [87729,1083] 0 2024-03-16T10:44:55.541 INFO:tasks.workunit.client.0.smithi002.stdout:4/313: sync 2024-03-16T10:44:55.543 INFO:tasks.workunit.client.0.smithi002.stdout:4/314: write d2/d33/da/f4f [462246,98570] 0 2024-03-16T10:44:55.546 INFO:tasks.workunit.client.0.smithi002.stdout:0/319: unlink d3/de/f15 0 2024-03-16T10:44:55.547 INFO:tasks.workunit.client.0.smithi002.stdout:0/320: chown d3/de/f13 5213 1 2024-03-16T10:44:55.548 INFO:tasks.workunit.client.0.smithi002.stdout:0/321: fsync d3/d28/f4d 0 2024-03-16T10:44:55.549 INFO:tasks.workunit.client.0.smithi002.stdout:7/238: creat d3/d4/d1d/d45/f53 x:0 0 0 2024-03-16T10:44:55.550 INFO:tasks.workunit.client.0.smithi002.stdout:0/322: chown d3/d28/d2a/f4f 116497233 1 2024-03-16T10:44:55.551 INFO:tasks.workunit.client.0.smithi002.stdout:7/239: read - d3/d4/d1c/f2b zero size 2024-03-16T10:44:55.552 INFO:tasks.workunit.client.0.smithi002.stdout:9/309: truncate d5/f53 3880856 0 2024-03-16T10:44:55.552 INFO:tasks.workunit.client.0.smithi002.stdout:6/327: dread d15/f24 [0,4194304] 0 2024-03-16T10:44:55.552 INFO:tasks.workunit.client.0.smithi002.stdout:5/274: truncate d0/d16/f29 22170 0 2024-03-16T10:44:55.554 INFO:tasks.workunit.client.0.smithi002.stdout:2/333: write d7/d2e/d1d/d2f/f59 [798937,128880] 0 2024-03-16T10:44:55.555 INFO:tasks.workunit.client.0.smithi002.stdout:1/288: dread d2/df/f45 [0,4194304] 0 2024-03-16T10:44:55.556 INFO:tasks.workunit.client.0.smithi002.stdout:3/246: mknod d1/dd/d11/d15/d1e/d22/d3f/c5c 0 2024-03-16T10:44:55.557 INFO:tasks.workunit.client.0.smithi002.stdout:4/315: creat d2/d33/d35/f63 x:0 0 0 2024-03-16T10:44:55.559 INFO:tasks.workunit.client.0.smithi002.stdout:4/316: read d2/d33/da/f41 [2009585,54018] 0 2024-03-16T10:44:55.559 INFO:tasks.workunit.client.0.smithi002.stdout:5/275: dread d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:44:55.560 INFO:tasks.workunit.client.0.smithi002.stdout:4/317: fsync d2/d33/da/f5f 0 2024-03-16T10:44:55.561 INFO:tasks.workunit.client.0.smithi002.stdout:6/328: symlink d15/d2d/d2b/d79/l7f 0 2024-03-16T10:44:55.564 INFO:tasks.workunit.client.0.smithi002.stdout:1/289: readlink d2/df/d12/l23 0 2024-03-16T10:44:55.603 INFO:tasks.workunit.client.0.smithi002.stdout:8/312: getdents d7/d9/d17/d23 0 2024-03-16T10:44:55.604 INFO:tasks.workunit.client.0.smithi002.stdout:7/240: symlink d3/d4/d1c/d44/l54 0 2024-03-16T10:44:55.605 INFO:tasks.workunit.client.0.smithi002.stdout:6/329: readlink d15/l1e 0 2024-03-16T10:44:55.605 INFO:tasks.workunit.client.0.smithi002.stdout:0/323: write d3/f47 [837628,65654] 0 2024-03-16T10:44:55.606 INFO:tasks.workunit.client.0.smithi002.stdout:8/313: read d7/d9/d17/f57 [305525,59569] 0 2024-03-16T10:44:55.606 INFO:tasks.workunit.client.0.smithi002.stdout:9/310: link d5/d9/dc/d28/f4c d5/d9/dc/d1e/f5d 0 2024-03-16T10:44:55.606 INFO:tasks.workunit.client.0.smithi002.stdout:5/276: mknod d0/c4e 0 2024-03-16T10:44:55.607 INFO:tasks.workunit.client.0.smithi002.stdout:3/247: truncate d1/dd/f10 3441044 0 2024-03-16T10:44:55.611 INFO:tasks.workunit.client.0.smithi002.stdout:3/248: chown d1/dd/d11/d15/d1e/d22/d3f 488950474 1 2024-03-16T10:44:55.611 INFO:tasks.workunit.client.0.smithi002.stdout:9/311: dread - d5/d9/dc/d28/f5a zero size 2024-03-16T10:44:55.613 INFO:tasks.workunit.client.0.smithi002.stdout:8/314: truncate d7/d9/d17/d23/d2e/f4e 1458767 0 2024-03-16T10:44:55.613 INFO:tasks.workunit.client.0.smithi002.stdout:1/290: mknod d2/df/d12/d29/d5c/c64 0 2024-03-16T10:44:55.615 INFO:tasks.workunit.client.0.smithi002.stdout:3/249: chown d1/dd/d11/d15/d1e/d22/d3f/f4d 170 1 2024-03-16T10:44:55.616 INFO:tasks.workunit.client.0.smithi002.stdout:5/277: truncate d0/d24/f3c 781025 0 2024-03-16T10:44:55.616 INFO:tasks.workunit.client.0.smithi002.stdout:0/324: dwrite d3/de/f13 [4194304,4194304] 0 2024-03-16T10:44:55.619 INFO:tasks.workunit.client.0.smithi002.stdout:4/318: creat d2/d33/f64 x:0 0 0 2024-03-16T10:44:55.619 INFO:tasks.workunit.client.0.smithi002.stdout:6/330: mkdir d15/d80 0 2024-03-16T10:44:55.625 INFO:tasks.workunit.client.0.smithi002.stdout:0/325: dwrite d3/d31/f4b [0,4194304] 0 2024-03-16T10:44:55.630 INFO:tasks.workunit.client.0.smithi002.stdout:8/315: rename d7/d9/d17/d23/d2e/d3a/f65 to d7/d9/d17/d20/f6b 0 2024-03-16T10:44:55.637 INFO:tasks.workunit.client.0.smithi002.stdout:9/312: symlink d5/d9/d22/d3c/d4d/l5e 0 2024-03-16T10:44:55.639 INFO:tasks.workunit.client.0.smithi002.stdout:8/316: dwrite f4 [4194304,4194304] 0 2024-03-16T10:44:55.641 INFO:tasks.workunit.client.0.smithi002.stdout:0/326: symlink d3/d31/l64 0 2024-03-16T10:44:55.642 INFO:tasks.workunit.client.0.smithi002.stdout:0/327: write d3/d28/d3d/f50 [631912,35046] 0 2024-03-16T10:44:55.644 INFO:tasks.workunit.client.0.smithi002.stdout:0/328: chown d3/d28/d3d/d44/f52 3244 1 2024-03-16T10:44:55.644 INFO:tasks.workunit.client.0.smithi002.stdout:0/329: readlink d3/d5/l21 0 2024-03-16T10:44:55.649 INFO:tasks.workunit.client.0.smithi002.stdout:7/241: write d3/d4/d2a/f40 [4086080,92009] 0 2024-03-16T10:44:55.655 INFO:tasks.workunit.client.0.smithi002.stdout:6/331: link d15/d2d/l64 d15/d2d/d2b/d79/l81 0 2024-03-16T10:44:55.662 INFO:tasks.workunit.client.0.smithi002.stdout:9/313: write d5/f7 [421740,122481] 0 2024-03-16T10:44:55.665 INFO:tasks.workunit.client.0.smithi002.stdout:3/250: link d1/dd/d11/d15/d1e/d22/l50 d1/dd/d11/d15/d1e/l5d 0 2024-03-16T10:44:55.667 INFO:tasks.workunit.client.0.smithi002.stdout:7/242: mkdir d3/d4/d10/d55 0 2024-03-16T10:44:55.669 INFO:tasks.workunit.client.0.smithi002.stdout:4/319: dwrite d2/d6/f10 [0,4194304] 0 2024-03-16T10:44:55.669 INFO:tasks.workunit.client.0.smithi002.stdout:5/278: getdents d0/d16/d1d/d43 0 2024-03-16T10:44:55.669 INFO:tasks.workunit.client.0.smithi002.stdout:8/317: truncate d7/d9/d17/d23/d2e/f4e 749965 0 2024-03-16T10:44:55.669 INFO:tasks.workunit.client.0.smithi002.stdout:6/332: mkdir d15/d82 0 2024-03-16T10:44:55.673 INFO:tasks.workunit.client.0.smithi002.stdout:4/320: fdatasync d2/d6/f2d 0 2024-03-16T10:44:55.674 INFO:tasks.workunit.client.0.smithi002.stdout:6/333: rmdir d15/d2d/d2c 39 2024-03-16T10:44:55.676 INFO:tasks.workunit.client.0.smithi002.stdout:3/251: dread d1/dd/d28/d37/f49 [0,4194304] 0 2024-03-16T10:44:55.679 INFO:tasks.workunit.client.0.smithi002.stdout:7/243: rename d3/d4/d2a/f39 to d3/d4/d1d/d3d/f56 0 2024-03-16T10:44:55.679 INFO:tasks.workunit.client.0.smithi002.stdout:0/330: dwrite d3/d28/d3d/d44/f53 [4194304,4194304] 0 2024-03-16T10:44:55.681 INFO:tasks.workunit.client.0.smithi002.stdout:0/331: stat d3/c2b 0 2024-03-16T10:44:55.685 INFO:tasks.workunit.client.0.smithi002.stdout:9/314: dwrite d5/d9/dc/d1e/d24/d31/f4a [0,4194304] 0 2024-03-16T10:44:55.688 INFO:tasks.workunit.client.0.smithi002.stdout:4/321: symlink d2/d33/d3b/l65 0 2024-03-16T10:44:55.692 INFO:tasks.workunit.client.0.smithi002.stdout:8/318: dwrite d7/d9/d17/d23/d39/f5d [0,4194304] 0 2024-03-16T10:44:55.694 INFO:tasks.workunit.client.0.smithi002.stdout:7/244: rmdir d3/d4 39 2024-03-16T10:44:55.699 INFO:tasks.workunit.client.0.smithi002.stdout:3/252: rename d1/dd/d28/f36 to d1/dd/f5e 0 2024-03-16T10:44:55.700 INFO:tasks.workunit.client.0.smithi002.stdout:9/315: rmdir d5 39 2024-03-16T10:44:55.703 INFO:tasks.workunit.client.0.smithi002.stdout:6/334: dread fa [0,4194304] 0 2024-03-16T10:44:55.705 INFO:tasks.workunit.client.0.smithi002.stdout:4/322: mkdir d2/d6/d32/d66 0 2024-03-16T10:44:55.707 INFO:tasks.workunit.client.0.smithi002.stdout:6/335: dread d15/f33 [4194304,4194304] 0 2024-03-16T10:44:55.709 INFO:tasks.workunit.client.0.smithi002.stdout:0/332: mkdir d3/d28/d65 0 2024-03-16T10:44:55.713 INFO:tasks.workunit.client.0.smithi002.stdout:3/253: symlink d1/dd/d28/d37/l5f 0 2024-03-16T10:44:55.713 INFO:tasks.workunit.client.0.smithi002.stdout:4/323: dwrite d2/d33/d35/f63 [0,4194304] 0 2024-03-16T10:44:55.715 INFO:tasks.workunit.client.0.smithi002.stdout:5/279: getdents d0/d16/d1d/d43 0 2024-03-16T10:44:55.717 INFO:tasks.workunit.client.0.smithi002.stdout:4/324: write d2/d33/d35/f40 [422238,129587] 0 2024-03-16T10:44:55.718 INFO:tasks.workunit.client.0.smithi002.stdout:6/336: write d15/d2d/f2e [763372,56007] 0 2024-03-16T10:44:55.722 INFO:tasks.workunit.client.0.smithi002.stdout:7/245: creat d3/f57 x:0 0 0 2024-03-16T10:44:55.723 INFO:tasks.workunit.client.0.smithi002.stdout:3/254: creat d1/dd/d11/d15/d1e/d22/d3f/f60 x:0 0 0 2024-03-16T10:44:55.725 INFO:tasks.workunit.client.0.smithi002.stdout:8/319: dread d7/d9/f58 [0,4194304] 0 2024-03-16T10:44:55.725 INFO:tasks.workunit.client.0.smithi002.stdout:5/280: creat d0/d16/d1d/d2b/f4f x:0 0 0 2024-03-16T10:44:55.727 INFO:tasks.workunit.client.0.smithi002.stdout:3/255: fdatasync d1/dd/d11/d15/d1e/d22/d3f/f47 0 2024-03-16T10:44:55.727 INFO:tasks.workunit.client.0.smithi002.stdout:0/333: dwrite d3/d5/f2f [0,4194304] 0 2024-03-16T10:44:55.732 INFO:tasks.workunit.client.0.smithi002.stdout:7/246: mknod d3/d37/c58 0 2024-03-16T10:44:55.733 INFO:tasks.workunit.client.0.smithi002.stdout:6/337: creat d15/d2d/d2b/d3f/d77/d7e/f83 x:0 0 0 2024-03-16T10:44:55.734 INFO:tasks.workunit.client.0.smithi002.stdout:9/316: dwrite d5/d9/d25/f42 [0,4194304] 0 2024-03-16T10:44:55.738 INFO:tasks.workunit.client.0.smithi002.stdout:8/320: mknod d7/d9/d17/d23/d2e/d34/c6c 0 2024-03-16T10:44:55.743 INFO:tasks.workunit.client.0.smithi002.stdout:7/247: dwrite d3/d37/f48 [0,4194304] 0 2024-03-16T10:44:55.744 INFO:tasks.workunit.client.0.smithi002.stdout:5/281: rename d0/d16/d34/c2e to d0/d16/d1d/d33/c50 0 2024-03-16T10:44:55.745 INFO:tasks.workunit.client.0.smithi002.stdout:9/317: dread d5/d9/dc/f20 [0,4194304] 0 2024-03-16T10:44:55.745 INFO:tasks.workunit.client.0.smithi002.stdout:5/282: fsync d0/d16/f4d 0 2024-03-16T10:44:55.754 INFO:tasks.workunit.client.0.smithi002.stdout:9/318: dwrite d5/d9/fe [4194304,4194304] 0 2024-03-16T10:44:55.761 INFO:tasks.workunit.client.0.smithi002.stdout:7/248: dread d3/f15 [4194304,4194304] 0 2024-03-16T10:44:55.840 INFO:tasks.workunit.client.0.smithi002.stdout:3/256: mkdir d1/dd/d28/d61 0 2024-03-16T10:44:55.842 INFO:tasks.workunit.client.0.smithi002.stdout:1/291: sync 2024-03-16T10:44:55.842 INFO:tasks.workunit.client.0.smithi002.stdout:2/334: sync 2024-03-16T10:44:55.846 INFO:tasks.workunit.client.0.smithi002.stdout:8/321: readlink d7/d9/d17/d23/d2e/l62 0 2024-03-16T10:44:55.846 INFO:tasks.workunit.client.0.smithi002.stdout:8/322: fsync d7/d9/d25/f32 0 2024-03-16T10:44:55.849 INFO:tasks.workunit.client.0.smithi002.stdout:5/283: creat d0/d16/d4a/d39/f51 x:0 0 0 2024-03-16T10:44:55.854 INFO:tasks.workunit.client.0.smithi002.stdout:3/257: fdatasync d1/dd/d11/d15/d1e/d22/f39 0 2024-03-16T10:44:55.855 INFO:tasks.workunit.client.0.smithi002.stdout:6/338: creat d15/d80/f84 x:0 0 0 2024-03-16T10:44:55.857 INFO:tasks.workunit.client.0.smithi002.stdout:2/335: unlink d7/d27/f2d 0 2024-03-16T10:44:55.861 INFO:tasks.workunit.client.0.smithi002.stdout:9/319: dread d5/f53 [0,4194304] 0 2024-03-16T10:44:55.861 INFO:tasks.workunit.client.0.smithi002.stdout:9/320: read d5/f7 [644740,76448] 0 2024-03-16T10:44:55.862 INFO:tasks.workunit.client.0.smithi002.stdout:8/323: unlink d7/d9/f2f 0 2024-03-16T10:44:55.863 INFO:tasks.workunit.client.0.smithi002.stdout:2/336: dwrite d7/db/f58 [0,4194304] 0 2024-03-16T10:44:55.865 INFO:tasks.workunit.client.0.smithi002.stdout:5/284: mknod d0/d16/d1d/c52 0 2024-03-16T10:44:55.867 INFO:tasks.workunit.client.0.smithi002.stdout:7/249: symlink d3/d4/d1c/d44/l59 0 2024-03-16T10:44:55.871 INFO:tasks.workunit.client.0.smithi002.stdout:7/250: dread d3/f27 [0,4194304] 0 2024-03-16T10:44:55.881 INFO:tasks.workunit.client.0.smithi002.stdout:9/321: fdatasync d5/d9/dc/f11 0 2024-03-16T10:44:55.884 INFO:tasks.workunit.client.0.smithi002.stdout:0/334: getdents d3/d59 0 2024-03-16T10:44:55.888 INFO:tasks.workunit.client.0.smithi002.stdout:5/285: rename d0/d16/d1d/d33/f3e to d0/d16/d4a/d39/f53 0 2024-03-16T10:44:55.895 INFO:tasks.workunit.client.0.smithi002.stdout:9/322: mknod d5/d9/d22/c5f 0 2024-03-16T10:44:55.896 INFO:tasks.workunit.client.0.smithi002.stdout:8/324: fdatasync d7/d9/d17/d23/d2e/f4e 0 2024-03-16T10:44:55.897 INFO:tasks.workunit.client.0.smithi002.stdout:8/325: read d7/d9/f51 [3404928,124543] 0 2024-03-16T10:44:55.898 INFO:tasks.workunit.client.0.smithi002.stdout:0/335: dread d3/d28/d3d/f3f [0,4194304] 0 2024-03-16T10:44:55.899 INFO:tasks.workunit.client.0.smithi002.stdout:8/326: read d7/d9/d17/d20/f63 [301349,73028] 0 2024-03-16T10:44:55.903 INFO:tasks.workunit.client.0.smithi002.stdout:5/286: rename d0/d16/d4a/c3d to d0/d16/d1d/d2b/c54 0 2024-03-16T10:44:55.907 INFO:tasks.workunit.client.0.smithi002.stdout:3/258: creat d1/dd/d11/d15/f62 x:0 0 0 2024-03-16T10:44:55.907 INFO:tasks.workunit.client.0.smithi002.stdout:3/259: chown d1/dd/d28/f33 922190 1 2024-03-16T10:44:55.908 INFO:tasks.workunit.client.0.smithi002.stdout:3/260: fdatasync d1/dd/d11/d15/d1e/d22/d3f/f51 0 2024-03-16T10:44:55.909 INFO:tasks.workunit.client.0.smithi002.stdout:7/251: symlink d3/d4/d10/l5a 0 2024-03-16T10:44:55.910 INFO:tasks.workunit.client.0.smithi002.stdout:1/292: getdents d2/d3/d51 0 2024-03-16T10:44:55.919 INFO:tasks.workunit.client.0.smithi002.stdout:2/337: link d7/d2e/d1d/d2f/c42 d7/d2e/d1d/d2f/d4b/c60 0 2024-03-16T10:44:55.919 INFO:tasks.workunit.client.0.smithi002.stdout:2/338: chown d7/db/d12/f11 178223998 1 2024-03-16T10:44:55.919 INFO:tasks.workunit.client.0.smithi002.stdout:2/339: chown d7/f2b 663257185 1 2024-03-16T10:44:55.922 INFO:tasks.workunit.client.0.smithi002.stdout:5/287: mkdir d0/d16/d4a/d55 0 2024-03-16T10:44:55.925 INFO:tasks.workunit.client.0.smithi002.stdout:7/252: mkdir d3/d4/d1c/d5b 0 2024-03-16T10:44:55.925 INFO:tasks.workunit.client.0.smithi002.stdout:7/253: write d3/d4/d10/f4c [92072,81898] 0 2024-03-16T10:44:55.927 INFO:tasks.workunit.client.0.smithi002.stdout:3/261: rename d1/dd/d28/d61 to d1/dd/d11/d15/d1e/d2c/d63 0 2024-03-16T10:44:55.933 INFO:tasks.workunit.client.0.smithi002.stdout:0/336: mknod d3/d28/d65/c66 0 2024-03-16T10:44:55.935 INFO:tasks.workunit.client.0.smithi002.stdout:8/327: mknod d7/d9/d17/c6d 0 2024-03-16T10:44:55.938 INFO:tasks.workunit.client.0.smithi002.stdout:2/340: symlink d7/d2e/d28/l61 0 2024-03-16T10:44:55.942 INFO:tasks.workunit.client.0.smithi002.stdout:5/288: chown d0/c28 0 1 2024-03-16T10:44:55.949 INFO:tasks.workunit.client.0.smithi002.stdout:0/337: mkdir d3/d28/d2a/d67 0 2024-03-16T10:44:55.952 INFO:tasks.workunit.client.0.smithi002.stdout:8/328: creat d7/d9/d17/d23/d2e/d34/f6e x:0 0 0 2024-03-16T10:44:55.953 INFO:tasks.workunit.client.0.smithi002.stdout:9/323: truncate d5/d9/fe 3615971 0 2024-03-16T10:44:55.954 INFO:tasks.workunit.client.0.smithi002.stdout:9/324: chown d5/d9/lb 75173429 1 2024-03-16T10:44:55.955 INFO:tasks.workunit.client.0.smithi002.stdout:2/341: creat d7/d2e/d1d/d2f/d4b/f62 x:0 0 0 2024-03-16T10:44:55.965 INFO:tasks.workunit.client.0.smithi002.stdout:4/325: sync 2024-03-16T10:44:55.969 INFO:tasks.workunit.client.0.smithi002.stdout:9/325: rmdir d5/d9/d25 39 2024-03-16T10:44:55.973 INFO:tasks.workunit.client.0.smithi002.stdout:5/289: mknod d0/d16/c56 0 2024-03-16T10:44:55.975 INFO:tasks.workunit.client.0.smithi002.stdout:3/262: link d1/dd/d11/c3d d1/c64 0 2024-03-16T10:44:55.978 INFO:tasks.workunit.client.0.smithi002.stdout:1/293: getdents d2/df 0 2024-03-16T10:44:55.980 INFO:tasks.workunit.client.0.smithi002.stdout:0/338: mknod d3/c68 0 2024-03-16T10:44:55.980 INFO:tasks.workunit.client.0.smithi002.stdout:5/290: dwrite d0/d16/d1d/f46 [0,4194304] 0 2024-03-16T10:44:55.982 INFO:tasks.workunit.client.0.smithi002.stdout:7/254: truncate d3/d4/f1e 2866221 0 2024-03-16T10:44:55.983 INFO:tasks.workunit.client.0.smithi002.stdout:3/263: dread d1/dd/f5e [0,4194304] 0 2024-03-16T10:44:55.989 INFO:tasks.workunit.client.0.smithi002.stdout:2/342: link d7/d27/f56 d7/d2e/d28/d40/f63 0 2024-03-16T10:44:55.989 INFO:tasks.workunit.client.0.smithi002.stdout:8/329: write d7/fc [976553,94337] 0 2024-03-16T10:44:55.993 INFO:tasks.workunit.client.0.smithi002.stdout:1/294: rename d2/d3/f17 to d2/d3/f65 0 2024-03-16T10:44:55.994 INFO:tasks.workunit.client.0.smithi002.stdout:1/295: chown d2/d3/f1f 2 1 2024-03-16T10:44:55.994 INFO:tasks.workunit.client.0.smithi002.stdout:1/296: fdatasync d2/d3/f39 0 2024-03-16T10:44:55.995 INFO:tasks.workunit.client.0.smithi002.stdout:0/339: fdatasync d3/f3a 0 2024-03-16T10:44:55.996 INFO:tasks.workunit.client.0.smithi002.stdout:0/340: stat d3/d28/d3d 0 2024-03-16T10:44:55.997 INFO:tasks.workunit.client.0.smithi002.stdout:5/291: rmdir d0/d16/d34 39 2024-03-16T10:44:55.998 INFO:tasks.workunit.client.0.smithi002.stdout:6/339: sync 2024-03-16T10:44:56.000 INFO:tasks.workunit.client.0.smithi002.stdout:7/255: mkdir d3/d4/d1c/d44/d5c 0 2024-03-16T10:44:56.000 INFO:tasks.workunit.client.0.smithi002.stdout:6/340: write d15/d2d/f2e [1269970,118137] 0 2024-03-16T10:44:56.003 INFO:tasks.workunit.client.0.smithi002.stdout:0/341: dwrite d3/d28/d2a/f61 [0,4194304] 0 2024-03-16T10:44:56.003 INFO:tasks.workunit.client.0.smithi002.stdout:0/342: chown d3/d5/fd 8366 1 2024-03-16T10:44:56.004 INFO:tasks.workunit.client.0.smithi002.stdout:4/326: rename d2/f37 to d2/d33/d1c/f67 0 2024-03-16T10:44:56.028 INFO:tasks.workunit.client.0.smithi002.stdout:0/343: mknod d3/d28/d3d/c69 0 2024-03-16T10:44:56.029 INFO:tasks.workunit.client.0.smithi002.stdout:0/344: chown d3/d28/c37 12 1 2024-03-16T10:44:56.032 INFO:tasks.workunit.client.0.smithi002.stdout:9/326: creat d5/d9/d25/d23/f60 x:0 0 0 2024-03-16T10:44:56.035 INFO:tasks.workunit.client.0.smithi002.stdout:1/297: dwrite d2/df/d12/f37 [0,4194304] 0 2024-03-16T10:44:56.040 INFO:tasks.workunit.client.0.smithi002.stdout:1/298: dread d2/d3/d51/d56/f5e [0,4194304] 0 2024-03-16T10:44:56.045 INFO:tasks.workunit.client.0.smithi002.stdout:8/330: truncate d7/d9/d17/d23/d39/f5d 2592140 0 2024-03-16T10:44:56.050 INFO:tasks.workunit.client.0.smithi002.stdout:6/341: creat d15/d2d/f85 x:0 0 0 2024-03-16T10:44:56.051 INFO:tasks.workunit.client.0.smithi002.stdout:5/292: rename d0/d16/d1d/f46 to d0/f57 0 2024-03-16T10:44:56.054 INFO:tasks.workunit.client.0.smithi002.stdout:0/345: link d3/d27/f56 d3/d28/d65/f6a 0 2024-03-16T10:44:56.055 INFO:tasks.workunit.client.0.smithi002.stdout:3/264: link d1/dd/d28/d37/c32 d1/c65 0 2024-03-16T10:44:56.055 INFO:tasks.workunit.client.0.smithi002.stdout:1/299: dread d2/d3/d51/f5a [4194304,4194304] 0 2024-03-16T10:44:56.056 INFO:tasks.workunit.client.0.smithi002.stdout:1/300: dread - d2/d3/f50 zero size 2024-03-16T10:44:56.056 INFO:tasks.workunit.client.0.smithi002.stdout:3/265: read d1/dd/d11/f4a [722735,80100] 0 2024-03-16T10:44:56.057 INFO:tasks.workunit.client.0.smithi002.stdout:1/301: read d2/df/d12/f37 [168650,121730] 0 2024-03-16T10:44:56.060 INFO:tasks.workunit.client.0.smithi002.stdout:7/256: rename d3/d4/f4a to d3/d4/d10/d55/f5d 0 2024-03-16T10:44:56.063 INFO:tasks.workunit.client.0.smithi002.stdout:1/302: dwrite d2/d3/fc [0,4194304] 0 2024-03-16T10:44:56.071 INFO:tasks.workunit.client.0.smithi002.stdout:9/327: dwrite d5/f7 [0,4194304] 0 2024-03-16T10:44:56.073 INFO:tasks.workunit.client.0.smithi002.stdout:0/346: mknod d3/d28/d2a/d67/c6b 0 2024-03-16T10:44:56.074 INFO:tasks.workunit.client.0.smithi002.stdout:0/347: write d3/d27/f55 [309916,121858] 0 2024-03-16T10:44:56.076 INFO:tasks.workunit.client.0.smithi002.stdout:0/348: readlink d3/d28/d2a/l4e 0 2024-03-16T10:44:56.077 INFO:tasks.workunit.client.0.smithi002.stdout:0/349: write d3/f47 [1439461,7827] 0 2024-03-16T10:44:56.078 INFO:tasks.workunit.client.0.smithi002.stdout:3/266: mknod d1/d5a/c66 0 2024-03-16T10:44:56.082 INFO:tasks.workunit.client.0.smithi002.stdout:7/257: write d3/fd [1444057,84511] 0 2024-03-16T10:44:56.084 INFO:tasks.workunit.client.0.smithi002.stdout:1/303: rename d2/d3/c47 to d2/d3/c66 0 2024-03-16T10:44:56.088 INFO:tasks.workunit.client.0.smithi002.stdout:5/293: creat d0/d1f/f58 x:0 0 0 2024-03-16T10:44:56.089 INFO:tasks.workunit.client.0.smithi002.stdout:7/258: dwrite d3/d4/d2a/f50 [0,4194304] 0 2024-03-16T10:44:56.089 INFO:tasks.workunit.client.0.smithi002.stdout:9/328: creat d5/d9/d34/f61 x:0 0 0 2024-03-16T10:44:56.091 INFO:tasks.workunit.client.0.smithi002.stdout:5/294: fdatasync d0/d16/f4d 0 2024-03-16T10:44:56.092 INFO:tasks.workunit.client.0.smithi002.stdout:0/350: mkdir d3/de/d6c 0 2024-03-16T10:44:56.094 INFO:tasks.workunit.client.0.smithi002.stdout:7/259: mkdir d3/d4/d1d/d3d/d5e 0 2024-03-16T10:44:56.094 INFO:tasks.workunit.client.0.smithi002.stdout:0/351: chown d3/d28/f33 42997 1 2024-03-16T10:44:56.099 INFO:tasks.workunit.client.0.smithi002.stdout:0/352: rmdir d3/d27 39 2024-03-16T10:44:56.100 INFO:tasks.workunit.client.0.smithi002.stdout:3/267: dwrite d1/dd/f5e [0,4194304] 0 2024-03-16T10:44:56.102 INFO:tasks.workunit.client.0.smithi002.stdout:1/304: dread d2/d3/d51/f5a [0,4194304] 0 2024-03-16T10:44:56.104 INFO:tasks.workunit.client.0.smithi002.stdout:7/260: dread d3/d4/d2a/f40 [0,4194304] 0 2024-03-16T10:44:56.105 INFO:tasks.workunit.client.0.smithi002.stdout:9/329: creat d5/d9/d25/f62 x:0 0 0 2024-03-16T10:44:56.107 INFO:tasks.workunit.client.0.smithi002.stdout:5/295: link d0/d1f/f58 d0/d16/d4a/d39/f59 0 2024-03-16T10:44:56.115 INFO:tasks.workunit.client.0.smithi002.stdout:3/268: fsync d1/dd/f10 0 2024-03-16T10:44:56.115 INFO:tasks.workunit.client.0.smithi002.stdout:3/269: stat d1/dd/d11/d15/d1e/l29 0 2024-03-16T10:44:56.116 INFO:tasks.workunit.client.0.smithi002.stdout:1/305: mknod d2/c67 0 2024-03-16T10:44:56.121 INFO:tasks.workunit.client.0.smithi002.stdout:5/296: creat d0/d16/d34/f5a x:0 0 0 2024-03-16T10:44:56.121 INFO:tasks.workunit.client.0.smithi002.stdout:5/297: chown d0/c28 3139 1 2024-03-16T10:44:56.130 INFO:tasks.workunit.client.0.smithi002.stdout:7/261: getdents d3/d4/d1c 0 2024-03-16T10:44:56.131 INFO:tasks.workunit.client.0.smithi002.stdout:9/330: link d5/d9/dc/d1e/d24/d31/f32 d5/d9/d55/f63 0 2024-03-16T10:44:56.132 INFO:tasks.workunit.client.0.smithi002.stdout:3/270: creat d1/dd/d11/d15/f67 x:0 0 0 2024-03-16T10:44:56.135 INFO:tasks.workunit.client.0.smithi002.stdout:1/306: creat d2/f68 x:0 0 0 2024-03-16T10:44:56.138 INFO:tasks.workunit.client.0.smithi002.stdout:9/331: rename d5/d9/d22/f37 to d5/d9/dc/d28/f64 0 2024-03-16T10:44:56.139 INFO:tasks.workunit.client.0.smithi002.stdout:3/271: creat d1/dd/f68 x:0 0 0 2024-03-16T10:44:56.139 INFO:tasks.workunit.client.0.smithi002.stdout:1/307: symlink d2/df/l69 0 2024-03-16T10:44:56.141 INFO:tasks.workunit.client.0.smithi002.stdout:9/332: chown d5/d9/dc/c35 10 1 2024-03-16T10:44:56.141 INFO:tasks.workunit.client.0.smithi002.stdout:7/262: mkdir d3/d5f 0 2024-03-16T10:44:56.141 INFO:tasks.workunit.client.0.smithi002.stdout:5/298: dwrite d0/d16/d4a/d39/f59 [0,4194304] 0 2024-03-16T10:44:56.142 INFO:tasks.workunit.client.0.smithi002.stdout:9/333: read d5/f7 [1035595,56107] 0 2024-03-16T10:44:56.143 INFO:tasks.workunit.client.0.smithi002.stdout:1/308: rmdir d2/d3/d51 39 2024-03-16T10:44:56.144 INFO:tasks.workunit.client.0.smithi002.stdout:1/309: write d2/d3/f39 [4785891,88109] 0 2024-03-16T10:44:56.147 INFO:tasks.workunit.client.0.smithi002.stdout:3/272: mknod d1/dd/d11/d15/d1e/d2c/d63/c69 0 2024-03-16T10:44:56.149 INFO:tasks.workunit.client.0.smithi002.stdout:7/263: unlink d3/d4/c1f 0 2024-03-16T10:44:56.149 INFO:tasks.workunit.client.0.smithi002.stdout:3/273: stat d1/dd/d28/f33 0 2024-03-16T10:44:56.151 INFO:tasks.workunit.client.0.smithi002.stdout:7/264: dread - d3/d4/d1c/f2b zero size 2024-03-16T10:44:56.153 INFO:tasks.workunit.client.0.smithi002.stdout:5/299: dwrite d0/d16/d1d/d2b/f4f [0,4194304] 0 2024-03-16T10:44:56.153 INFO:tasks.workunit.client.0.smithi002.stdout:9/334: dwrite d5/d9/d25/f42 [0,4194304] 0 2024-03-16T10:44:56.154 INFO:tasks.workunit.client.0.smithi002.stdout:7/265: write d3/d37/f48 [1080148,81236] 0 2024-03-16T10:44:56.164 INFO:tasks.workunit.client.0.smithi002.stdout:3/274: creat d1/dd/d28/f6a x:0 0 0 2024-03-16T10:44:56.165 INFO:tasks.workunit.client.0.smithi002.stdout:9/335: fsync d5/d9/dc/d1e/d24/d31/f4a 0 2024-03-16T10:44:56.165 INFO:tasks.workunit.client.0.smithi002.stdout:7/266: dwrite d3/d37/f48 [0,4194304] 0 2024-03-16T10:44:56.166 INFO:tasks.workunit.client.0.smithi002.stdout:3/275: stat d1/dd/d11/d15/d1e/d22/f44 0 2024-03-16T10:44:56.168 INFO:tasks.workunit.client.0.smithi002.stdout:5/300: mkdir d0/d16/d5b 0 2024-03-16T10:44:56.172 INFO:tasks.workunit.client.0.smithi002.stdout:7/267: dread - d3/d4/d10/f3c zero size 2024-03-16T10:44:56.172 INFO:tasks.workunit.client.0.smithi002.stdout:1/310: rmdir d2/df/d12/d29 39 2024-03-16T10:44:56.175 INFO:tasks.workunit.client.0.smithi002.stdout:5/301: stat d0/d16/d1d/d2b/c54 0 2024-03-16T10:44:56.176 INFO:tasks.workunit.client.0.smithi002.stdout:9/336: dwrite d5/d9/d22/f47 [0,4194304] 0 2024-03-16T10:44:56.180 INFO:tasks.workunit.client.0.smithi002.stdout:7/268: creat d3/d4/d1d/d3d/d5e/f60 x:0 0 0 2024-03-16T10:44:56.181 INFO:tasks.workunit.client.0.smithi002.stdout:5/302: symlink d0/d16/d34/l5c 0 2024-03-16T10:44:56.182 INFO:tasks.workunit.client.0.smithi002.stdout:9/337: creat d5/d9/dc/d28/f65 x:0 0 0 2024-03-16T10:44:56.184 INFO:tasks.workunit.client.0.smithi002.stdout:1/311: mknod d2/df/d12/d29/d5c/c6a 0 2024-03-16T10:44:56.190 INFO:tasks.workunit.client.0.smithi002.stdout:3/276: dwrite d1/f52 [0,4194304] 0 2024-03-16T10:44:56.190 INFO:tasks.workunit.client.0.smithi002.stdout:7/269: mkdir d3/d61 0 2024-03-16T10:44:56.191 INFO:tasks.workunit.client.0.smithi002.stdout:9/338: symlink d5/d9/dc/d1e/d24/d31/l66 0 2024-03-16T10:44:56.192 INFO:tasks.workunit.client.0.smithi002.stdout:1/312: creat d2/d3/d1c/d5d/f6b x:0 0 0 2024-03-16T10:44:56.194 INFO:tasks.workunit.client.0.smithi002.stdout:9/339: dread - d5/d9/dc/d28/f5a zero size 2024-03-16T10:44:56.194 INFO:tasks.workunit.client.0.smithi002.stdout:3/277: chown d1/c65 64531664 1 2024-03-16T10:44:56.196 INFO:tasks.workunit.client.0.smithi002.stdout:5/303: dwrite d0/d16/d1d/f3b [0,4194304] 0 2024-03-16T10:44:56.198 INFO:tasks.workunit.client.0.smithi002.stdout:1/313: symlink d2/df/d12/d29/l6c 0 2024-03-16T10:44:56.200 INFO:tasks.workunit.client.0.smithi002.stdout:7/270: rename d3/d4/d10/l35 to d3/d37/l62 0 2024-03-16T10:44:56.202 INFO:tasks.workunit.client.0.smithi002.stdout:3/278: mknod d1/dd/d11/d15/d1e/d2c/c6b 0 2024-03-16T10:44:56.203 INFO:tasks.workunit.client.0.smithi002.stdout:5/304: chown d0/d16/d1d/l3f 1084415 1 2024-03-16T10:44:56.204 INFO:tasks.workunit.client.0.smithi002.stdout:1/314: creat d2/df/d12/d29/d5c/f6d x:0 0 0 2024-03-16T10:44:56.206 INFO:tasks.workunit.client.0.smithi002.stdout:3/279: mknod d1/dd/c6c 0 2024-03-16T10:44:56.207 INFO:tasks.workunit.client.0.smithi002.stdout:3/280: write d1/f16 [2577098,24185] 0 2024-03-16T10:44:56.209 INFO:tasks.workunit.client.0.smithi002.stdout:2/343: sync 2024-03-16T10:44:56.209 INFO:tasks.workunit.client.0.smithi002.stdout:4/327: sync 2024-03-16T10:44:56.211 INFO:tasks.workunit.client.0.smithi002.stdout:1/315: dread d2/df/d12/f3f [0,4194304] 0 2024-03-16T10:44:56.211 INFO:tasks.workunit.client.0.smithi002.stdout:3/281: chown d1/dd/d28/d37/c32 20 1 2024-03-16T10:44:56.212 INFO:tasks.workunit.client.0.smithi002.stdout:5/305: getdents d0/d16/d4a/d39 0 2024-03-16T10:44:56.214 INFO:tasks.workunit.client.0.smithi002.stdout:4/328: read d2/d33/d35/f63 [834660,124241] 0 2024-03-16T10:44:56.216 INFO:tasks.workunit.client.0.smithi002.stdout:9/340: dread d5/d9/dc/d28/f2d [0,4194304] 0 2024-03-16T10:44:56.216 INFO:tasks.workunit.client.0.smithi002.stdout:9/341: chown d5/d9/dc/f43 1 1 2024-03-16T10:44:56.217 INFO:tasks.workunit.client.0.smithi002.stdout:3/282: dread d1/dd/d11/d15/d1e/d22/d3f/f47 [0,4194304] 0 2024-03-16T10:44:56.218 INFO:tasks.workunit.client.0.smithi002.stdout:7/271: rename d3/d4/d10 to d3/d4/d1c/d44/d5c/d63 0 2024-03-16T10:44:56.219 INFO:tasks.workunit.client.0.smithi002.stdout:7/272: chown d3/d4/d1c/d44/d5c/d63/l5a 16960693 1 2024-03-16T10:44:56.219 INFO:tasks.workunit.client.0.smithi002.stdout:7/273: chown d3/d4/d1d/d3d/f56 3 1 2024-03-16T10:44:56.221 INFO:tasks.workunit.client.0.smithi002.stdout:1/316: unlink d2/d3/d1c/c24 0 2024-03-16T10:44:56.222 INFO:tasks.workunit.client.0.smithi002.stdout:4/329: chown d2/d6/d32/f4e 1975896 1 2024-03-16T10:44:56.226 INFO:tasks.workunit.client.0.smithi002.stdout:2/344: rename d7/d27/d3b/f51 to d7/d2e/d28/d44/f64 0 2024-03-16T10:44:56.230 INFO:tasks.workunit.client.0.smithi002.stdout:7/274: stat d3/d4/d1c/l51 0 2024-03-16T10:44:56.231 INFO:tasks.workunit.client.0.smithi002.stdout:1/317: creat d2/df/d12/d29/d5c/f6e x:0 0 0 2024-03-16T10:44:56.234 INFO:tasks.workunit.client.0.smithi002.stdout:5/306: rename d0/d16/d34 to d0/d16/d1d/d2b/d5d 0 2024-03-16T10:44:56.236 INFO:tasks.workunit.client.0.smithi002.stdout:9/342: dread d5/d9/d25/d23/f3b [0,4194304] 0 2024-03-16T10:44:56.236 INFO:tasks.workunit.client.0.smithi002.stdout:2/345: mknod d7/d2e/d1d/d2f/c65 0 2024-03-16T10:44:56.239 INFO:tasks.workunit.client.0.smithi002.stdout:2/346: dread d7/d2e/f25 [0,4194304] 0 2024-03-16T10:44:56.240 INFO:tasks.workunit.client.0.smithi002.stdout:2/347: dread - d7/d2e/d28/d40/f5a zero size 2024-03-16T10:44:56.241 INFO:tasks.workunit.client.0.smithi002.stdout:2/348: write d7/db/d12/f22 [2434866,4338] 0 2024-03-16T10:44:56.242 INFO:tasks.workunit.client.0.smithi002.stdout:3/283: rename d1/f45 to d1/dd/d11/d15/d1e/d2c/f6d 0 2024-03-16T10:44:56.246 INFO:tasks.workunit.client.0.smithi002.stdout:9/343: mkdir d5/d9/dc/d28/d67 0 2024-03-16T10:44:56.250 INFO:tasks.workunit.client.0.smithi002.stdout:1/318: unlink d2/d3/d51/d56/f5e 0 2024-03-16T10:44:56.251 INFO:tasks.workunit.client.0.smithi002.stdout:1/319: stat d2/df/d12/c1e 0 2024-03-16T10:44:56.251 INFO:tasks.workunit.client.0.smithi002.stdout:8/331: sync 2024-03-16T10:44:56.251 INFO:tasks.workunit.client.0.smithi002.stdout:6/342: sync 2024-03-16T10:44:56.251 INFO:tasks.workunit.client.0.smithi002.stdout:0/353: sync 2024-03-16T10:44:56.251 INFO:tasks.workunit.client.0.smithi002.stdout:1/320: chown d2/df 124 1 2024-03-16T10:44:56.256 INFO:tasks.workunit.client.0.smithi002.stdout:1/321: dwrite d2/d3/d1c/f4a [0,4194304] 0 2024-03-16T10:44:56.258 INFO:tasks.workunit.client.0.smithi002.stdout:5/307: write d0/d16/f29 [585660,65188] 0 2024-03-16T10:44:56.259 INFO:tasks.workunit.client.0.smithi002.stdout:2/349: symlink d7/d2e/d1d/d2f/l66 0 2024-03-16T10:44:56.261 INFO:tasks.workunit.client.0.smithi002.stdout:4/330: rename d2/d33/d3b to d2/d6/d32/d68 0 2024-03-16T10:44:56.265 INFO:tasks.workunit.client.0.smithi002.stdout:3/284: mknod d1/dd/d28/c6e 0 2024-03-16T10:44:56.267 INFO:tasks.workunit.client.0.smithi002.stdout:9/344: creat d5/d9/d34/f68 x:0 0 0 2024-03-16T10:44:56.268 INFO:tasks.workunit.client.0.smithi002.stdout:6/343: fdatasync d15/f24 0 2024-03-16T10:44:56.270 INFO:tasks.workunit.client.0.smithi002.stdout:0/354: truncate d3/f3a 247169 0 2024-03-16T10:44:56.271 INFO:tasks.workunit.client.0.smithi002.stdout:0/355: write d3/d28/f33 [408485,106744] 0 2024-03-16T10:44:56.273 INFO:tasks.workunit.client.0.smithi002.stdout:1/322: write d2/df/d12/f37 [2006611,29498] 0 2024-03-16T10:44:56.274 INFO:tasks.workunit.client.0.smithi002.stdout:5/308: symlink d0/d16/d1d/d2b/d5d/l5e 0 2024-03-16T10:44:56.278 INFO:tasks.workunit.client.0.smithi002.stdout:7/275: getdents d3/d4/d1c/d44 0 2024-03-16T10:44:56.279 INFO:tasks.workunit.client.0.smithi002.stdout:4/331: dread d2/d6/d32/d68/f4b [0,4194304] 0 2024-03-16T10:44:56.281 INFO:tasks.workunit.client.0.smithi002.stdout:4/332: stat d2/d6/f17 0 2024-03-16T10:44:56.281 INFO:tasks.workunit.client.0.smithi002.stdout:2/350: dwrite d7/d2e/f25 [0,4194304] 0 2024-03-16T10:44:56.283 INFO:tasks.workunit.client.0.smithi002.stdout:6/344: dread d15/d2d/d2b/f53 [0,4194304] 0 2024-03-16T10:44:56.283 INFO:tasks.workunit.client.0.smithi002.stdout:4/333: dread - d2/d6/d32/d68/f5b zero size 2024-03-16T10:44:56.289 INFO:tasks.workunit.client.0.smithi002.stdout:2/351: fsync d7/db/f35 0 2024-03-16T10:44:56.292 INFO:tasks.workunit.client.0.smithi002.stdout:4/334: mkdir d2/d6/d69 0 2024-03-16T10:44:56.293 INFO:tasks.workunit.client.0.smithi002.stdout:9/345: link d5/d9/d25/d23/f3b d5/d9/d55/f69 0 2024-03-16T10:44:56.293 INFO:tasks.workunit.client.0.smithi002.stdout:4/335: chown d2/d6/d32 464 1 2024-03-16T10:44:56.293 INFO:tasks.workunit.client.0.smithi002.stdout:7/276: mkdir d3/d4/d1c/d5b/d64 0 2024-03-16T10:44:56.296 INFO:tasks.workunit.client.0.smithi002.stdout:5/309: creat d0/d16/f5f x:0 0 0 2024-03-16T10:44:56.297 INFO:tasks.workunit.client.0.smithi002.stdout:5/310: stat d0/d16/d1d/d2b/d5d/l5c 0 2024-03-16T10:44:56.299 INFO:tasks.workunit.client.0.smithi002.stdout:8/332: rename d7/d13/f37 to d7/d9/f6f 0 2024-03-16T10:44:56.299 INFO:tasks.workunit.client.0.smithi002.stdout:3/285: rename d1/dd/d11/d15/d1e to d1/dd/d11/d15/d1e/d25/d6f 22 2024-03-16T10:44:56.299 INFO:tasks.workunit.client.0.smithi002.stdout:6/345: mkdir d15/d2d/d2c/d86 0 2024-03-16T10:44:56.300 INFO:tasks.workunit.client.0.smithi002.stdout:8/333: chown d7/d55/l66 13544184 1 2024-03-16T10:44:56.300 INFO:tasks.workunit.client.0.smithi002.stdout:3/286: chown d1/fb 31 1 2024-03-16T10:44:56.302 INFO:tasks.workunit.client.0.smithi002.stdout:8/334: truncate d7/d9/d17/d20/f6b 71132 0 2024-03-16T10:44:56.302 INFO:tasks.workunit.client.0.smithi002.stdout:7/277: symlink d3/d4/d1c/d44/d5c/d63/d34/l65 0 2024-03-16T10:44:56.302 INFO:tasks.workunit.client.0.smithi002.stdout:1/323: link d2/c14 d2/df/c6f 0 2024-03-16T10:44:56.305 INFO:tasks.workunit.client.0.smithi002.stdout:5/311: symlink d0/d16/d1d/d2b/l60 0 2024-03-16T10:44:56.307 INFO:tasks.workunit.client.0.smithi002.stdout:5/312: truncate d0/d16/f4d 708908 0 2024-03-16T10:44:56.307 INFO:tasks.workunit.client.0.smithi002.stdout:6/346: rmdir d15/d2d/d2b/d3f 39 2024-03-16T10:44:56.309 INFO:tasks.workunit.client.0.smithi002.stdout:3/287: unlink d1/dd/d11/d15/d1e/d22/d3f/c56 0 2024-03-16T10:44:56.310 INFO:tasks.workunit.client.0.smithi002.stdout:5/313: dread d0/f57 [0,4194304] 0 2024-03-16T10:44:56.311 INFO:tasks.workunit.client.0.smithi002.stdout:8/335: creat d7/d9/d25/f70 x:0 0 0 2024-03-16T10:44:56.312 INFO:tasks.workunit.client.0.smithi002.stdout:5/314: truncate d0/d1f/f2c 1392324 0 2024-03-16T10:44:56.312 INFO:tasks.workunit.client.0.smithi002.stdout:1/324: write d2/f52 [9246480,39406] 0 2024-03-16T10:44:56.313 INFO:tasks.workunit.client.0.smithi002.stdout:7/278: rename d3/d4/d2a to d3/d61/d66 0 2024-03-16T10:44:56.315 INFO:tasks.workunit.client.0.smithi002.stdout:7/279: dread - d3/d4/d1c/d44/d5c/d63/d55/f5d zero size 2024-03-16T10:44:56.316 INFO:tasks.workunit.client.0.smithi002.stdout:2/352: creat d7/d2e/f67 x:0 0 0 2024-03-16T10:44:56.321 INFO:tasks.workunit.client.0.smithi002.stdout:3/288: symlink d1/dd/d11/d15/d1e/d22/l70 0 2024-03-16T10:44:56.322 INFO:tasks.workunit.client.0.smithi002.stdout:4/336: dwrite d2/d33/d35/f63 [0,4194304] 0 2024-03-16T10:44:56.325 INFO:tasks.workunit.client.0.smithi002.stdout:8/336: rename d7/d9/d17/d23/d39/f44 to d7/d9/d17/d23/d2e/d34/f71 0 2024-03-16T10:44:56.328 INFO:tasks.workunit.client.0.smithi002.stdout:9/346: getdents d5/d9/d34 0 2024-03-16T10:44:56.332 INFO:tasks.workunit.client.0.smithi002.stdout:4/337: dwrite d2/d33/da/f42 [0,4194304] 0 2024-03-16T10:44:56.333 INFO:tasks.workunit.client.0.smithi002.stdout:8/337: dread d7/d9/d25/f32 [0,4194304] 0 2024-03-16T10:44:56.333 INFO:tasks.workunit.client.0.smithi002.stdout:3/289: symlink d1/dd/l71 0 2024-03-16T10:44:56.335 INFO:tasks.workunit.client.0.smithi002.stdout:1/325: creat d2/d58/f70 x:0 0 0 2024-03-16T10:44:56.336 INFO:tasks.workunit.client.0.smithi002.stdout:3/290: truncate d1/f1a 5228047 0 2024-03-16T10:44:56.337 INFO:tasks.workunit.client.0.smithi002.stdout:2/353: write d7/db/d12/f5d [1575040,64141] 0 2024-03-16T10:44:56.342 INFO:tasks.workunit.client.0.smithi002.stdout:8/338: creat d7/d9/d17/d23/d39/f72 x:0 0 0 2024-03-16T10:44:56.342 INFO:tasks.workunit.client.0.smithi002.stdout:4/338: dwrite d2/d6/f46 [0,4194304] 0 2024-03-16T10:44:56.343 INFO:tasks.workunit.client.0.smithi002.stdout:5/315: getdents d0/d16/d1d/d43 0 2024-03-16T10:44:56.343 INFO:tasks.workunit.client.0.smithi002.stdout:1/326: mkdir d2/d3/d1c/d5d/d71 0 2024-03-16T10:44:56.344 INFO:tasks.workunit.client.0.smithi002.stdout:5/316: fdatasync d0/d16/d1d/f3b 0 2024-03-16T10:44:56.346 INFO:tasks.workunit.client.0.smithi002.stdout:7/280: rename d3/l24 to d3/d37/l67 0 2024-03-16T10:44:56.348 INFO:tasks.workunit.client.0.smithi002.stdout:4/339: write d2/d33/da/d2e/f56 [1606713,41997] 0 2024-03-16T10:44:56.348 INFO:tasks.workunit.client.0.smithi002.stdout:7/281: chown d3/c36 8886973 1 2024-03-16T10:44:56.348 INFO:tasks.workunit.client.0.smithi002.stdout:3/291: symlink d1/dd/d11/d15/d1e/d2c/d63/l72 0 2024-03-16T10:44:56.348 INFO:tasks.workunit.client.0.smithi002.stdout:5/317: write d0/d16/d1d/d2b/d5d/f5a [563878,24670] 0 2024-03-16T10:44:56.350 INFO:tasks.workunit.client.0.smithi002.stdout:4/340: write d2/d33/da/f28 [2179347,104942] 0 2024-03-16T10:44:56.352 INFO:tasks.workunit.client.0.smithi002.stdout:3/292: chown d1/dd/d11/d15/l21 1 1 2024-03-16T10:44:56.353 INFO:tasks.workunit.client.0.smithi002.stdout:6/347: getdents d15/d2d/d2b/d79 0 2024-03-16T10:44:56.354 INFO:tasks.workunit.client.0.smithi002.stdout:8/339: symlink d7/d9/d17/d23/d2e/l73 0 2024-03-16T10:44:56.354 INFO:tasks.workunit.client.0.smithi002.stdout:1/327: readlink d2/df/d12/l3a 0 2024-03-16T10:44:56.355 INFO:tasks.workunit.client.0.smithi002.stdout:2/354: dwrite d7/d2e/f25 [0,4194304] 0 2024-03-16T10:44:56.357 INFO:tasks.workunit.client.0.smithi002.stdout:8/340: write d7/fc [82475,17727] 0 2024-03-16T10:44:56.358 INFO:tasks.workunit.client.0.smithi002.stdout:6/348: truncate d15/d2d/d2c/f50 702618 0 2024-03-16T10:44:56.358 INFO:tasks.workunit.client.0.smithi002.stdout:9/347: rmdir d5/d9/dc/d28/d67 0 2024-03-16T10:44:56.362 INFO:tasks.workunit.client.0.smithi002.stdout:5/318: write d0/f57 [3226795,13390] 0 2024-03-16T10:44:56.365 INFO:tasks.workunit.client.0.smithi002.stdout:1/328: unlink d2/df/d12/l3a 0 2024-03-16T10:44:56.367 INFO:tasks.workunit.client.0.smithi002.stdout:2/355: write d7/d2e/d1d/d2f/d4b/f62 [506730,23431] 0 2024-03-16T10:44:56.370 INFO:tasks.workunit.client.0.smithi002.stdout:7/282: rename l2 to d3/d4/d1d/l68 0 2024-03-16T10:44:56.372 INFO:tasks.workunit.client.0.smithi002.stdout:3/293: mkdir d1/dd/d11/d15/d1e/d73 0 2024-03-16T10:44:56.376 INFO:tasks.workunit.client.0.smithi002.stdout:6/349: mkdir d15/d2d/d2b/d79/d7c/d87 0 2024-03-16T10:44:56.377 INFO:tasks.workunit.client.0.smithi002.stdout:4/341: creat d2/d33/f6a x:0 0 0 2024-03-16T10:44:56.377 INFO:tasks.workunit.client.0.smithi002.stdout:7/283: dwrite d3/d4/d1c/d44/d5c/d63/d3f/f43 [0,4194304] 0 2024-03-16T10:44:56.378 INFO:tasks.workunit.client.0.smithi002.stdout:7/284: fdatasync d3/d4/d1c/f32 0 2024-03-16T10:44:56.382 INFO:tasks.workunit.client.0.smithi002.stdout:6/350: truncate d15/d2d/f1f 194292 0 2024-03-16T10:44:56.382 INFO:tasks.workunit.client.0.smithi002.stdout:8/341: getdents d7/d9/d17/d23 0 2024-03-16T10:44:56.383 INFO:tasks.workunit.client.0.smithi002.stdout:6/351: chown d15/l78 57855 1 2024-03-16T10:44:56.383 INFO:tasks.workunit.client.0.smithi002.stdout:7/285: creat d3/d4/d1d/d3d/d5e/f69 x:0 0 0 2024-03-16T10:44:56.385 INFO:tasks.workunit.client.0.smithi002.stdout:8/342: read d7/f8 [665987,79607] 0 2024-03-16T10:44:56.386 INFO:tasks.workunit.client.0.smithi002.stdout:7/286: mkdir d3/d37/d6a 0 2024-03-16T10:44:56.389 INFO:tasks.workunit.client.0.smithi002.stdout:7/287: creat d3/d4/d1c/d44/d5c/d63/d34/f6b x:0 0 0 2024-03-16T10:44:56.389 INFO:tasks.workunit.client.0.smithi002.stdout:6/352: dread d15/f33 [4194304,4194304] 0 2024-03-16T10:44:56.390 INFO:tasks.workunit.client.0.smithi002.stdout:6/353: chown d15/f69 455829 1 2024-03-16T10:44:56.396 INFO:tasks.workunit.client.0.smithi002.stdout:7/288: dwrite d3/d4/d1c/d44/d5c/d63/f3c [0,4194304] 0 2024-03-16T10:44:56.397 INFO:tasks.workunit.client.0.smithi002.stdout:6/354: dwrite d15/d2d/d2c/d5d/f76 [0,4194304] 0 2024-03-16T10:44:56.400 INFO:tasks.workunit.client.0.smithi002.stdout:7/289: symlink d3/d61/l6c 0 2024-03-16T10:44:56.401 INFO:tasks.workunit.client.0.smithi002.stdout:6/355: mknod d15/d2d/d2b/d79/c88 0 2024-03-16T10:44:56.401 INFO:tasks.workunit.client.0.smithi002.stdout:7/290: fsync d3/d4/d1d/d45/f53 0 2024-03-16T10:44:56.402 INFO:tasks.workunit.client.0.smithi002.stdout:7/291: stat d3/d4/d1c/d44/d5c/d63/d55/f5d 0 2024-03-16T10:44:56.406 INFO:tasks.workunit.client.0.smithi002.stdout:6/356: symlink d15/d2d/d2b/d3f/d77/d7e/l89 0 2024-03-16T10:44:56.406 INFO:tasks.workunit.client.0.smithi002.stdout:6/357: fsync d15/f2f 0 2024-03-16T10:44:56.411 INFO:tasks.workunit.client.0.smithi002.stdout:6/358: truncate d15/d2d/f2a 462513 0 2024-03-16T10:44:56.412 INFO:tasks.workunit.client.0.smithi002.stdout:6/359: write d15/f69 [714867,93580] 0 2024-03-16T10:44:56.416 INFO:tasks.workunit.client.0.smithi002.stdout:6/360: getdents d15/d80 0 2024-03-16T10:44:56.471 INFO:tasks.workunit.client.0.smithi002.stdout:0/356: sync 2024-03-16T10:44:56.476 INFO:tasks.workunit.client.0.smithi002.stdout:0/357: dwrite d3/d28/f33 [0,4194304] 0 2024-03-16T10:44:56.479 INFO:tasks.workunit.client.0.smithi002.stdout:0/358: symlink d3/d28/d2a/d67/l6d 0 2024-03-16T10:44:56.481 INFO:tasks.workunit.client.0.smithi002.stdout:0/359: unlink d3/d28/d3d/f3f 0 2024-03-16T10:44:56.483 INFO:tasks.workunit.client.0.smithi002.stdout:0/360: stat d3/d59/l5e 0 2024-03-16T10:44:56.484 INFO:tasks.workunit.client.0.smithi002.stdout:0/361: chown d3/d27/f55 130207081 1 2024-03-16T10:44:56.544 INFO:tasks.workunit.client.0.smithi002.stdout:8/343: sync 2024-03-16T10:44:56.544 INFO:tasks.workunit.client.0.smithi002.stdout:1/329: sync 2024-03-16T10:44:56.544 INFO:tasks.workunit.client.0.smithi002.stdout:4/342: sync 2024-03-16T10:44:56.545 INFO:tasks.workunit.client.0.smithi002.stdout:2/356: write d7/db/f48 [1778988,120267] 0 2024-03-16T10:44:56.545 INFO:tasks.workunit.client.0.smithi002.stdout:9/348: write d5/d9/d25/d23/f3b [216021,32861] 0 2024-03-16T10:44:56.549 INFO:tasks.workunit.client.0.smithi002.stdout:5/319: truncate d0/d16/f19 1685942 0 2024-03-16T10:44:56.553 INFO:tasks.workunit.client.0.smithi002.stdout:4/343: getdents d2/d6/d32/d66 0 2024-03-16T10:44:56.556 INFO:tasks.workunit.client.0.smithi002.stdout:3/294: dwrite d1/dd/d28/d37/f49 [0,4194304] 0 2024-03-16T10:44:56.558 INFO:tasks.workunit.client.0.smithi002.stdout:5/320: dwrite d0/d16/d4a/d39/f59 [0,4194304] 0 2024-03-16T10:44:56.558 INFO:tasks.workunit.client.0.smithi002.stdout:9/349: mknod d5/d9/c6a 0 2024-03-16T10:44:56.562 INFO:tasks.workunit.client.0.smithi002.stdout:8/344: creat d7/d9/d17/f74 x:0 0 0 2024-03-16T10:44:56.566 INFO:tasks.workunit.client.0.smithi002.stdout:6/361: rename d15/d2d/d2b/d79 to d15/d2d/d8a 0 2024-03-16T10:44:56.570 INFO:tasks.workunit.client.0.smithi002.stdout:3/295: mkdir d1/dd/d74 0 2024-03-16T10:44:56.570 INFO:tasks.workunit.client.0.smithi002.stdout:4/344: dread d2/d33/f3e [0,4194304] 0 2024-03-16T10:44:56.571 INFO:tasks.workunit.client.0.smithi002.stdout:3/296: dread - d1/dd/d28/d37/f59 zero size 2024-03-16T10:44:56.572 INFO:tasks.workunit.client.0.smithi002.stdout:7/292: truncate d3/d37/f48 995764 0 2024-03-16T10:44:56.573 INFO:tasks.workunit.client.0.smithi002.stdout:9/350: rename d5/d9/dc/d28/f64 to d5/d9/dc/d1e/f6b 0 2024-03-16T10:44:56.573 INFO:tasks.workunit.client.0.smithi002.stdout:3/297: write d1/dd/d11/d15/d1e/d22/f39 [3153068,85762] 0 2024-03-16T10:44:56.574 INFO:tasks.workunit.client.0.smithi002.stdout:3/298: readlink d1/lc 0 2024-03-16T10:44:56.575 INFO:tasks.workunit.client.0.smithi002.stdout:9/351: write d5/d9/dc/d1e/f36 [342830,108827] 0 2024-03-16T10:44:56.575 INFO:tasks.workunit.client.0.smithi002.stdout:7/293: dread d3/d4/d1c/d44/d5c/d63/f4c [0,4194304] 0 2024-03-16T10:44:56.576 INFO:tasks.workunit.client.0.smithi002.stdout:5/321: symlink d0/l61 0 2024-03-16T10:44:56.581 INFO:tasks.workunit.client.0.smithi002.stdout:6/362: symlink d15/d2d/d2b/d75/l8b 0 2024-03-16T10:44:56.581 INFO:tasks.workunit.client.0.smithi002.stdout:4/345: symlink d2/d6/d32/d66/l6b 0 2024-03-16T10:44:56.583 INFO:tasks.workunit.client.0.smithi002.stdout:5/322: rmdir d0/d16/d1d/d2b/d5d 39 2024-03-16T10:44:56.597 INFO:tasks.workunit.client.0.smithi002.stdout:4/346: creat d2/d33/d1c/f6c x:0 0 0 2024-03-16T10:44:56.597 INFO:tasks.workunit.client.0.smithi002.stdout:8/345: rename d7/f14 to d7/d9/f75 0 2024-03-16T10:44:56.598 INFO:tasks.workunit.client.0.smithi002.stdout:1/330: truncate d2/d3/f8 261995 0 2024-03-16T10:44:56.599 INFO:tasks.workunit.client.0.smithi002.stdout:2/357: truncate d7/db/f58 1759726 0 2024-03-16T10:44:56.600 INFO:tasks.workunit.client.0.smithi002.stdout:5/323: symlink d0/d16/d5b/l62 0 2024-03-16T10:44:56.602 INFO:tasks.workunit.client.0.smithi002.stdout:8/346: dread - d7/d9/d17/d23/d2e/d3a/f47 zero size 2024-03-16T10:44:56.603 INFO:tasks.workunit.client.0.smithi002.stdout:0/362: dwrite d3/d5/f19 [0,4194304] 0 2024-03-16T10:44:56.605 INFO:tasks.workunit.client.0.smithi002.stdout:9/352: rename d5/d9/d25/d23/f60 to d5/d9/dc/d1e/f6c 0 2024-03-16T10:44:56.608 INFO:tasks.workunit.client.0.smithi002.stdout:9/353: write d5/d9/d25/d23/f56 [921761,106807] 0 2024-03-16T10:44:56.609 INFO:tasks.workunit.client.0.smithi002.stdout:0/363: dread d3/d27/f55 [0,4194304] 0 2024-03-16T10:44:56.609 INFO:tasks.workunit.client.0.smithi002.stdout:5/324: dread d0/d16/d1d/f3b [0,4194304] 0 2024-03-16T10:44:56.611 INFO:tasks.workunit.client.0.smithi002.stdout:2/358: dread - d7/d2e/d28/d40/f5a zero size 2024-03-16T10:44:56.612 INFO:tasks.workunit.client.0.smithi002.stdout:4/347: mknod d2/c6d 0 2024-03-16T10:44:56.613 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:56 smithi002 ceph-mon[29248]: pgmap v31: 65 pgs: 65 active+clean; 343 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 24 MiB/s wr, 264 op/s 2024-03-16T10:44:56.614 INFO:tasks.workunit.client.0.smithi002.stdout:4/348: chown d2/d6/c1f 220680 1 2024-03-16T10:44:56.617 INFO:tasks.workunit.client.0.smithi002.stdout:0/364: dwrite d3/d28/f34 [0,4194304] 0 2024-03-16T10:44:56.619 INFO:tasks.workunit.client.0.smithi002.stdout:7/294: rename d3/d4/d1d/d3d/d5e/f69 to d3/d4/d1c/d5b/f6d 0 2024-03-16T10:44:56.622 INFO:tasks.workunit.client.0.smithi002.stdout:4/349: dwrite d2/d6/f1e [0,4194304] 0 2024-03-16T10:44:56.623 INFO:tasks.workunit.client.0.smithi002.stdout:9/354: unlink d5/d9/dc/d1e/f5d 0 2024-03-16T10:44:56.626 INFO:tasks.workunit.client.0.smithi002.stdout:5/325: rmdir d0/d16/d4a/d39 39 2024-03-16T10:44:56.629 INFO:tasks.workunit.client.0.smithi002.stdout:1/331: dwrite d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:44:56.631 INFO:tasks.workunit.client.0.smithi002.stdout:1/332: chown d2/d3/d1c/f4a 126 1 2024-03-16T10:44:56.634 INFO:tasks.workunit.client.0.smithi002.stdout:9/355: dwrite d5/d9/d22/f47 [0,4194304] 0 2024-03-16T10:44:56.640 INFO:tasks.workunit.client.0.smithi002.stdout:0/365: mknod d3/d31/c6e 0 2024-03-16T10:44:56.644 INFO:tasks.workunit.client.0.smithi002.stdout:9/356: dread d5/d9/dc/d1e/d24/d31/f4a [0,4194304] 0 2024-03-16T10:44:56.644 INFO:tasks.workunit.client.0.smithi002.stdout:1/333: dwrite d2/d3/fc [0,4194304] 0 2024-03-16T10:44:56.645 INFO:tasks.workunit.client.0.smithi002.stdout:1/334: stat d2/d3/d1c 0 2024-03-16T10:44:56.646 INFO:tasks.workunit.client.0.smithi002.stdout:1/335: fdatasync d2/df/d12/f60 0 2024-03-16T10:44:56.650 INFO:tasks.workunit.client.0.smithi002.stdout:2/359: creat d7/d2e/d28/d40/d50/f68 x:0 0 0 2024-03-16T10:44:56.651 INFO:tasks.workunit.client.0.smithi002.stdout:2/360: chown f6 107202978 1 2024-03-16T10:44:56.651 INFO:tasks.workunit.client.0.smithi002.stdout:2/361: fdatasync d7/f8 0 2024-03-16T10:44:56.652 INFO:tasks.workunit.client.0.smithi002.stdout:5/326: rmdir d0/d16/d1d/d43 39 2024-03-16T10:44:56.653 INFO:tasks.workunit.client.0.smithi002.stdout:2/362: write d7/d2e/d1d/d38/f3a [919248,120233] 0 2024-03-16T10:44:56.653 INFO:tasks.workunit.client.0.smithi002.stdout:8/347: link d7/d9/d17/l2a d7/d9/d17/d23/d39/l76 0 2024-03-16T10:44:56.657 INFO:tasks.workunit.client.0.smithi002.stdout:6/363: rename d15/d2d/d2b/d3f/l41 to d15/d82/l8c 0 2024-03-16T10:44:56.659 INFO:tasks.workunit.client.0.smithi002.stdout:0/366: symlink d3/d28/d3d/l6f 0 2024-03-16T10:44:56.659 INFO:tasks.workunit.client.0.smithi002.stdout:0/367: dread - d3/de/f51 zero size 2024-03-16T10:44:56.661 INFO:tasks.workunit.client.0.smithi002.stdout:7/295: link d3/d4/d1c/d44/d5c/d63/d3f/f41 d3/d37/d6a/f6e 0 2024-03-16T10:44:56.661 INFO:tasks.workunit.client.0.smithi002.stdout:0/368: read - d3/d28/d3d/f46 zero size 2024-03-16T10:44:56.661 INFO:tasks.workunit.client.0.smithi002.stdout:6/364: dread d15/f69 [0,4194304] 0 2024-03-16T10:44:56.662 INFO:tasks.workunit.client.0.smithi002.stdout:0/369: write d3/d5/fd [782569,45192] 0 2024-03-16T10:44:56.664 INFO:tasks.workunit.client.0.smithi002.stdout:0/370: fsync d3/d5/fd 0 2024-03-16T10:44:56.666 INFO:tasks.workunit.client.0.smithi002.stdout:0/371: write d3/d28/d3d/f46 [891014,22478] 0 2024-03-16T10:44:56.666 INFO:tasks.workunit.client.0.smithi002.stdout:5/327: chown d0/d1f/c22 401027 1 2024-03-16T10:44:56.671 INFO:tasks.workunit.client.0.smithi002.stdout:5/328: dread d0/d1f/f58 [0,4194304] 0 2024-03-16T10:44:56.673 INFO:tasks.workunit.client.0.smithi002.stdout:9/357: rename d5/d9/l2f to d5/d9/d22/d3c/d4d/l6d 0 2024-03-16T10:44:56.673 INFO:tasks.workunit.client.0.smithi002.stdout:6/365: dread d15/d2d/f42 [0,4194304] 0 2024-03-16T10:44:56.674 INFO:tasks.workunit.client.0.smithi002.stdout:6/366: chown lf 28 1 2024-03-16T10:44:56.675 INFO:tasks.workunit.client.0.smithi002.stdout:7/296: write d3/d4/d1d/f23 [4051594,48824] 0 2024-03-16T10:44:56.678 INFO:tasks.workunit.client.0.smithi002.stdout:7/297: write d3/d61/d66/f4d [839757,11425] 0 2024-03-16T10:44:56.678 INFO:tasks.workunit.client.0.smithi002.stdout:4/350: link d2/d33/c14 d2/d33/d1c/c6e 0 2024-03-16T10:44:56.681 INFO:tasks.workunit.client.0.smithi002.stdout:4/351: dread d2/d33/f29 [0,4194304] 0 2024-03-16T10:44:56.683 INFO:tasks.workunit.client.0.smithi002.stdout:0/372: rmdir d3/d28/d3d/d44 39 2024-03-16T10:44:56.684 INFO:tasks.workunit.client.0.smithi002.stdout:9/358: dwrite d5/d9/d22/d3c/d4d/f54 [0,4194304] 0 2024-03-16T10:44:56.687 INFO:tasks.workunit.client.0.smithi002.stdout:8/348: write d7/d9/d17/f57 [4635091,5258] 0 2024-03-16T10:44:56.692 INFO:tasks.workunit.client.0.smithi002.stdout:0/373: dwrite d3/d5/f19 [0,4194304] 0 2024-03-16T10:44:56.694 INFO:tasks.workunit.client.0.smithi002.stdout:5/329: dread d0/d16/d1d/d2b/d5d/f5a [0,4194304] 0 2024-03-16T10:44:56.696 INFO:tasks.workunit.client.0.smithi002.stdout:3/299: sync 2024-03-16T10:44:56.706 INFO:tasks.workunit.client.0.smithi002.stdout:2/363: rmdir d7/d2e/d1d/d2f/d5f 0 2024-03-16T10:44:56.709 INFO:tasks.workunit.client.0.smithi002.stdout:0/374: creat d3/d28/d2a/d67/f70 x:0 0 0 2024-03-16T10:44:56.710 INFO:tasks.workunit.client.0.smithi002.stdout:6/367: mknod d15/d2d/d8a/d7c/d87/c8d 0 2024-03-16T10:44:56.711 INFO:tasks.workunit.client.0.smithi002.stdout:0/375: truncate d3/d28/f29 843196 0 2024-03-16T10:44:56.715 INFO:tasks.workunit.client.0.smithi002.stdout:3/300: creat d1/d5a/f75 x:0 0 0 2024-03-16T10:44:56.719 INFO:tasks.workunit.client.0.smithi002.stdout:4/352: dwrite d2/d6/f1a [0,4194304] 0 2024-03-16T10:44:56.729 INFO:tasks.workunit.client.0.smithi002.stdout:2/364: link d7/db/d12/f5d d7/d27/d3f/f69 0 2024-03-16T10:44:56.730 INFO:tasks.workunit.client.0.smithi002.stdout:1/336: rename d2/df/f33 to d2/df/d12/f72 0 2024-03-16T10:44:56.730 INFO:tasks.workunit.client.0.smithi002.stdout:5/330: creat d0/d16/f63 x:0 0 0 2024-03-16T10:44:56.731 INFO:tasks.workunit.client.0.smithi002.stdout:4/353: fsync d2/d33/f1b 0 2024-03-16T10:44:56.732 INFO:tasks.workunit.client.0.smithi002.stdout:3/301: fsync d1/dd/d11/d15/d1e/d22/d3f/f51 0 2024-03-16T10:44:56.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:56 smithi026 ceph-mon[37041]: pgmap v31: 65 pgs: 65 active+clean; 343 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 24 MiB/s wr, 264 op/s 2024-03-16T10:44:56.734 INFO:tasks.workunit.client.0.smithi002.stdout:7/298: getdents d3/d4/d1d/d3d/d5e 0 2024-03-16T10:44:56.734 INFO:tasks.workunit.client.0.smithi002.stdout:6/368: dwrite d15/f6b [0,4194304] 0 2024-03-16T10:44:56.740 INFO:tasks.workunit.client.0.smithi002.stdout:5/331: chown d0/d16/d1d/d2b/c54 19973667 1 2024-03-16T10:44:56.742 INFO:tasks.workunit.client.0.smithi002.stdout:4/354: creat d2/d33/d35/f6f x:0 0 0 2024-03-16T10:44:56.743 INFO:tasks.workunit.client.0.smithi002.stdout:9/359: dwrite d5/d9/fe [0,4194304] 0 2024-03-16T10:44:56.744 INFO:tasks.workunit.client.0.smithi002.stdout:3/302: dread - d1/dd/d11/d15/f43 zero size 2024-03-16T10:44:56.745 INFO:tasks.workunit.client.0.smithi002.stdout:0/376: dwrite d3/d28/d3d/f50 [0,4194304] 0 2024-03-16T10:44:56.750 INFO:tasks.workunit.client.0.smithi002.stdout:3/303: chown d1/dd/d11/d15/l5b 331 1 2024-03-16T10:44:56.750 INFO:tasks.workunit.client.0.smithi002.stdout:6/369: symlink d15/d2d/d2b/l8e 0 2024-03-16T10:44:56.751 INFO:tasks.workunit.client.0.smithi002.stdout:0/377: stat d3/d28/d2a/d67/l6d 0 2024-03-16T10:44:56.753 INFO:tasks.workunit.client.0.smithi002.stdout:4/355: dread d2/d33/f3e [0,4194304] 0 2024-03-16T10:44:56.753 INFO:tasks.workunit.client.0.smithi002.stdout:1/337: write d2/d3/f8 [155466,27913] 0 2024-03-16T10:44:56.753 INFO:tasks.workunit.client.0.smithi002.stdout:7/299: symlink d3/d4/d1c/d5b/d64/l6f 0 2024-03-16T10:44:56.753 INFO:tasks.workunit.client.0.smithi002.stdout:2/365: dwrite d7/d2e/d1d/d38/f3a [0,4194304] 0 2024-03-16T10:44:56.754 INFO:tasks.workunit.client.0.smithi002.stdout:0/378: write d3/f47 [489053,103373] 0 2024-03-16T10:44:56.755 INFO:tasks.workunit.client.0.smithi002.stdout:9/360: creat d5/d9/d22/f6e x:0 0 0 2024-03-16T10:44:56.758 INFO:tasks.workunit.client.0.smithi002.stdout:0/379: stat d3/c35 0 2024-03-16T10:44:56.759 INFO:tasks.workunit.client.0.smithi002.stdout:3/304: mknod d1/dd/d28/c76 0 2024-03-16T10:44:56.762 INFO:tasks.workunit.client.0.smithi002.stdout:6/370: fdatasync d15/d2d/f22 0 2024-03-16T10:44:56.767 INFO:tasks.workunit.client.0.smithi002.stdout:4/356: dwrite d2/d6/d32/f50 [0,4194304] 0 2024-03-16T10:44:56.769 INFO:tasks.workunit.client.0.smithi002.stdout:4/357: chown d2/d33/da/f25 487261 1 2024-03-16T10:44:56.771 INFO:tasks.workunit.client.0.smithi002.stdout:9/361: dread d5/d9/d25/f4f [0,4194304] 0 2024-03-16T10:44:56.780 INFO:tasks.workunit.client.0.smithi002.stdout:0/380: symlink d3/d28/d3d/l71 0 2024-03-16T10:44:56.782 INFO:tasks.workunit.client.0.smithi002.stdout:2/366: truncate d7/db/d12/f5d 3367054 0 2024-03-16T10:44:56.782 INFO:tasks.workunit.client.0.smithi002.stdout:0/381: write d3/d28/d2a/f61 [1224300,120880] 0 2024-03-16T10:44:56.783 INFO:tasks.workunit.client.0.smithi002.stdout:7/300: write d3/d4/d1c/d44/d5c/d63/f3b [339964,83458] 0 2024-03-16T10:44:56.784 INFO:tasks.workunit.client.0.smithi002.stdout:1/338: dwrite d2/d3/f1f [0,4194304] 0 2024-03-16T10:44:56.786 INFO:tasks.workunit.client.0.smithi002.stdout:0/382: write d3/de/f43 [642837,3532] 0 2024-03-16T10:44:56.787 INFO:tasks.workunit.client.0.smithi002.stdout:1/339: fdatasync d2/df/d12/d29/d5c/f6e 0 2024-03-16T10:44:56.793 INFO:tasks.workunit.client.0.smithi002.stdout:2/367: rmdir d7/db 39 2024-03-16T10:44:56.798 INFO:tasks.workunit.client.0.smithi002.stdout:7/301: fdatasync d3/f27 0 2024-03-16T10:44:56.801 INFO:tasks.workunit.client.0.smithi002.stdout:6/371: truncate fd 587810 0 2024-03-16T10:44:56.802 INFO:tasks.workunit.client.0.smithi002.stdout:7/302: dread d3/f11 [0,4194304] 0 2024-03-16T10:44:56.802 INFO:tasks.workunit.client.0.smithi002.stdout:0/383: unlink d3/de/f40 0 2024-03-16T10:44:56.803 INFO:tasks.workunit.client.0.smithi002.stdout:7/303: truncate d3/d4/d1c/d5b/f6d 940995 0 2024-03-16T10:44:56.804 INFO:tasks.workunit.client.0.smithi002.stdout:0/384: chown d3/d28/d2a/c62 328 1 2024-03-16T10:44:56.805 INFO:tasks.workunit.client.0.smithi002.stdout:7/304: stat d3/d4/d1d/d45/f53 0 2024-03-16T10:44:56.805 INFO:tasks.workunit.client.0.smithi002.stdout:0/385: truncate d3/f60 184559 0 2024-03-16T10:44:56.805 INFO:tasks.workunit.client.0.smithi002.stdout:3/305: link d1/dd/d11/d15/f53 d1/dd/d11/d15/d1e/f77 0 2024-03-16T10:44:56.810 INFO:tasks.workunit.client.0.smithi002.stdout:9/362: creat d5/f6f x:0 0 0 2024-03-16T10:44:56.811 INFO:tasks.workunit.client.0.smithi002.stdout:9/363: dread - d5/d9/dc/d1e/f6b zero size 2024-03-16T10:44:56.813 INFO:tasks.workunit.client.0.smithi002.stdout:9/364: fdatasync d5/d9/dc/d1e/d24/f4b 0 2024-03-16T10:44:56.814 INFO:tasks.workunit.client.0.smithi002.stdout:6/372: chown d15/d2d/f34 5763 1 2024-03-16T10:44:56.815 INFO:tasks.workunit.client.0.smithi002.stdout:9/365: chown d5/d9/dc/d1e/f36 5191 1 2024-03-16T10:44:56.815 INFO:tasks.workunit.client.0.smithi002.stdout:9/366: dread - d5/d9/d25/d23/f48 zero size 2024-03-16T10:44:56.816 INFO:tasks.workunit.client.0.smithi002.stdout:9/367: dread - d5/d9/d25/f62 zero size 2024-03-16T10:44:56.818 INFO:tasks.workunit.client.0.smithi002.stdout:0/386: dread d3/de/f13 [4194304,4194304] 0 2024-03-16T10:44:56.819 INFO:tasks.workunit.client.0.smithi002.stdout:0/387: readlink d3/d28/d3d/l71 0 2024-03-16T10:44:56.820 INFO:tasks.workunit.client.0.smithi002.stdout:0/388: stat d3/d28/f34 0 2024-03-16T10:44:56.820 INFO:tasks.workunit.client.0.smithi002.stdout:0/389: readlink d3/l16 0 2024-03-16T10:44:56.821 INFO:tasks.workunit.client.0.smithi002.stdout:3/306: creat d1/dd/d28/d37/f78 x:0 0 0 2024-03-16T10:44:56.823 INFO:tasks.workunit.client.0.smithi002.stdout:1/340: symlink d2/df/l73 0 2024-03-16T10:44:56.830 INFO:tasks.workunit.client.0.smithi002.stdout:9/368: mknod d5/d9/d25/d23/c70 0 2024-03-16T10:44:56.835 INFO:tasks.workunit.client.0.smithi002.stdout:3/307: rename d1/dd/d28/d37/f4e to d1/dd/d11/d15/d1e/d2c/d63/f79 0 2024-03-16T10:44:56.836 INFO:tasks.workunit.client.0.smithi002.stdout:3/308: write d1/dd/d11/d15/f62 [243863,53255] 0 2024-03-16T10:44:56.838 INFO:tasks.workunit.client.0.smithi002.stdout:3/309: chown d1/dd/c6c 1354981500 1 2024-03-16T10:44:56.840 INFO:tasks.workunit.client.0.smithi002.stdout:1/341: mkdir d2/df/d12/d74 0 2024-03-16T10:44:56.841 INFO:tasks.workunit.client.0.smithi002.stdout:7/305: dwrite d3/f15 [4194304,4194304] 0 2024-03-16T10:44:56.843 INFO:tasks.workunit.client.0.smithi002.stdout:3/310: symlink d1/d5a/l7a 0 2024-03-16T10:44:56.843 INFO:tasks.workunit.client.0.smithi002.stdout:2/368: truncate d7/d2e/d1d/d38/f3a 752589 0 2024-03-16T10:44:56.844 INFO:tasks.workunit.client.0.smithi002.stdout:2/369: chown d7/d2e/d28/d40 1391 1 2024-03-16T10:44:56.846 INFO:tasks.workunit.client.0.smithi002.stdout:9/369: link d5/d9/fe d5/d9/d25/f71 0 2024-03-16T10:44:56.848 INFO:tasks.workunit.client.0.smithi002.stdout:7/306: write d3/d61/d66/f40 [1577927,43799] 0 2024-03-16T10:44:56.850 INFO:tasks.workunit.client.0.smithi002.stdout:2/370: creat d7/d2e/d28/f6a x:0 0 0 2024-03-16T10:44:56.851 INFO:tasks.workunit.client.0.smithi002.stdout:3/311: dwrite d1/dd/d28/d37/f78 [0,4194304] 0 2024-03-16T10:44:56.853 INFO:tasks.workunit.client.0.smithi002.stdout:9/370: mknod d5/d9/c72 0 2024-03-16T10:44:56.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.855+0000 7f007c891700 1 -- 172.21.15.2:0/543389669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 msgr2=0x7f0078103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.855+0000 7f007c891700 1 --2- 172.21.15.2:0/543389669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 0x7f0078103950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0068009870 tx=0x7f0068009b80 comp rx=0 tx=0).stop 2024-03-16T10:44:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.855+0000 7f007c891700 1 -- 172.21.15.2:0/543389669 shutdown_connections 2024-03-16T10:44:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.855+0000 7f007c891700 1 --2- 172.21.15.2:0/543389669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 0x7f0078106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.855+0000 7f007c891700 1 --2- 172.21.15.2:0/543389669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 0x7f0078103950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.855+0000 7f007c891700 1 -- 172.21.15.2:0/543389669 >> 172.21.15.2:0/543389669 conn(0x7f00780faf70 msgr2=0x7f00780fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.856+0000 7f007c891700 1 -- 172.21.15.2:0/543389669 shutdown_connections 2024-03-16T10:44:56.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.856+0000 7f007c891700 1 -- 172.21.15.2:0/543389669 wait complete. 2024-03-16T10:44:56.858 INFO:tasks.workunit.client.0.smithi002.stdout:9/371: dread d5/d9/dc/d1e/f36 [0,4194304] 0 2024-03-16T10:44:56.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.857+0000 7f007c891700 1 Processor -- start 2024-03-16T10:44:56.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.857+0000 7f007c891700 1 -- start start 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.857+0000 7f007c891700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 0x7f0078100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.857+0000 7f007c891700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 0x7f00780ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.857+0000 7f007c891700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007819a120 con 0x7f0078103e90 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.857+0000 7f007c891700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007819a260 con 0x7f0078101560 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.858+0000 7f0075d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 0x7f00780ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.858+0000 7f0075d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 0x7f00780ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57472/0 (socket says 172.21.15.2:57472) 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.858+0000 7f0075d9b700 1 -- 172.21.15.2:0/3554904688 learned_addr learned my addr 172.21.15.2:0/3554904688 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:56.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.858+0000 7f007659c700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 0x7f0078100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:56.860 INFO:tasks.workunit.client.0.smithi002.stdout:9/372: dread - d5/d9/dc/d28/f5a zero size 2024-03-16T10:44:56.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.858+0000 7f0075d9b700 1 -- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 msgr2=0x7f0078100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:56.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.858+0000 7f0075d9b700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 0x7f0078100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:56.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.858+0000 7f0075d9b700 1 -- 172.21.15.2:0/3554904688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068009520 con 0x7f0078103e90 2024-03-16T10:44:56.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.859+0000 7f0075d9b700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 0x7f00780ff510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f006c00e940 tx=0x7f006c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:56.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.859+0000 7f00677fe700 1 -- 172.21.15.2:0/3554904688 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006c00eee0 con 0x7f0078103e90 2024-03-16T10:44:56.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.859+0000 7f00677fe700 1 -- 172.21.15.2:0/3554904688 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006c004cf0 con 0x7f0078103e90 2024-03-16T10:44:56.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.859+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007819a540 con 0x7f0078103e90 2024-03-16T10:44:56.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.859+0000 7f00677fe700 1 -- 172.21.15.2:0/3554904688 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006c005650 con 0x7f0078103e90 2024-03-16T10:44:56.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.859+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007819ab10 con 0x7f0078103e90 2024-03-16T10:44:56.863 INFO:tasks.workunit.client.0.smithi002.stdout:2/371: dwrite d7/d2e/d28/d40/d50/f68 [0,4194304] 0 2024-03-16T10:44:56.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.861+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007804ea80 con 0x7f0078103e90 2024-03-16T10:44:56.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.862+0000 7f00677fe700 1 -- 172.21.15.2:0/3554904688 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f006c021440 con 0x7f0078103e90 2024-03-16T10:44:56.865 INFO:tasks.workunit.client.0.smithi002.stdout:7/307: link d3/d4/d1c/d44/d5c/d63/d34/f6b d3/d4/d1d/f70 0 2024-03-16T10:44:56.866 INFO:tasks.workunit.client.0.smithi002.stdout:3/312: rename d1/dd/d11/d15/d1e to d1/dd/d28/d7b 0 2024-03-16T10:44:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.863+0000 7f00677fe700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f006006d3d0 0x7f006006f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:56.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.863+0000 7f00677fe700 1 -- 172.21.15.2:0/3554904688 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f006c014070 con 0x7f0078103e90 2024-03-16T10:44:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.863+0000 7f007659c700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f006006d3d0 0x7f006006f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:56.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.864+0000 7f007659c700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f006006d3d0 0x7f006006f890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f006800b820 tx=0x7f0068019040 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:56.871 INFO:tasks.workunit.client.0.smithi002.stdout:9/373: chown d5/d9/d22/d3c/d4d/l6d 1154250 1 2024-03-16T10:44:56.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:56.871+0000 7f00677fe700 1 -- 172.21.15.2:0/3554904688 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f006c05a460 con 0x7f0078103e90 2024-03-16T10:44:56.873 INFO:tasks.workunit.client.0.smithi002.stdout:2/372: mknod d7/d27/d3b/c6b 0 2024-03-16T10:44:56.873 INFO:tasks.workunit.client.0.smithi002.stdout:9/374: write d5/d9/dc/d28/f65 [934399,130663] 0 2024-03-16T10:44:56.875 INFO:tasks.workunit.client.0.smithi002.stdout:7/308: fsync d3/d4/d1c/d44/d5c/d63/d55/f5d 0 2024-03-16T10:44:56.876 INFO:tasks.workunit.client.0.smithi002.stdout:3/313: dwrite d1/dd/d28/d7b/d22/f44 [0,4194304] 0 2024-03-16T10:44:56.877 INFO:tasks.workunit.client.0.smithi002.stdout:9/375: write d5/d9/d22/d3c/d4d/f54 [1067791,65534] 0 2024-03-16T10:44:56.879 INFO:tasks.workunit.client.0.smithi002.stdout:7/309: dread - d3/d4/d1c/d44/d5c/d63/d34/f6b zero size 2024-03-16T10:44:56.881 INFO:tasks.workunit.client.0.smithi002.stdout:2/373: mkdir d7/d2e/d28/d40/d6c 0 2024-03-16T10:44:56.881 INFO:tasks.workunit.client.0.smithi002.stdout:7/310: readlink d3/d4/d1c/d44/d5c/d63/l5a 0 2024-03-16T10:44:56.882 INFO:tasks.workunit.client.0.smithi002.stdout:7/311: write d3/d61/d66/f4f [4520834,26679] 0 2024-03-16T10:44:56.900 INFO:tasks.workunit.client.0.smithi002.stdout:2/374: rmdir d7/db/d12 39 2024-03-16T10:44:56.903 INFO:tasks.workunit.client.0.smithi002.stdout:2/375: readlink d7/l43 0 2024-03-16T10:44:56.905 INFO:tasks.workunit.client.0.smithi002.stdout:2/376: mkdir d7/d2e/d28/d44/d6d 0 2024-03-16T10:44:56.908 INFO:tasks.workunit.client.0.smithi002.stdout:7/312: getdents d3/d4/d1c/d5b/d64 0 2024-03-16T10:44:56.913 INFO:tasks.workunit.client.0.smithi002.stdout:9/376: dwrite d5/f7 [0,4194304] 0 2024-03-16T10:44:56.916 INFO:tasks.workunit.client.0.smithi002.stdout:3/314: truncate d1/dd/d28/d37/f49 577260 0 2024-03-16T10:44:56.917 INFO:tasks.workunit.client.0.smithi002.stdout:3/315: read - d1/dd/d28/d7b/f24 zero size 2024-03-16T10:44:56.922 INFO:tasks.workunit.client.0.smithi002.stdout:2/377: write d7/f8 [517268,69894] 0 2024-03-16T10:44:56.923 INFO:tasks.workunit.client.0.smithi002.stdout:3/316: creat d1/dd/d28/d7b/f7c x:0 0 0 2024-03-16T10:44:56.924 INFO:tasks.workunit.client.0.smithi002.stdout:3/317: dread - d1/dd/d28/d37/f59 zero size 2024-03-16T10:44:56.924 INFO:tasks.workunit.client.0.smithi002.stdout:2/378: chown d7/d2e/d28/d40/f5a 522720 1 2024-03-16T10:44:56.925 INFO:tasks.workunit.client.0.smithi002.stdout:2/379: write d7/db/f57 [447595,13504] 0 2024-03-16T10:44:56.932 INFO:tasks.workunit.client.0.smithi002.stdout:3/318: dwrite d1/dd/d28/d7b/d22/d3f/f47 [0,4194304] 0 2024-03-16T10:44:56.935 INFO:tasks.workunit.client.0.smithi002.stdout:3/319: truncate d1/dd/d28/d7b/d22/d3f/f4d 732252 0 2024-03-16T10:44:56.935 INFO:tasks.workunit.client.0.smithi002.stdout:3/320: chown d1/dd/c6c 872947 1 2024-03-16T10:44:56.936 INFO:tasks.workunit.client.0.smithi002.stdout:2/380: write d7/d2e/d28/d40/f63 [561015,60693] 0 2024-03-16T10:44:56.936 INFO:tasks.workunit.client.0.smithi002.stdout:3/321: readlink d1/dd/d28/d37/l4f 0 2024-03-16T10:44:56.941 INFO:tasks.workunit.client.0.smithi002.stdout:2/381: read d7/db/d12/f5d [1452279,51476] 0 2024-03-16T10:44:56.944 INFO:tasks.workunit.client.0.smithi002.stdout:2/382: mkdir d7/d2e/d1d/d2f/d4b/d6e 0 2024-03-16T10:44:56.945 INFO:tasks.workunit.client.0.smithi002.stdout:3/322: creat d1/dd/d11/d15/f7d x:0 0 0 2024-03-16T10:44:56.946 INFO:tasks.workunit.client.0.smithi002.stdout:2/383: truncate d7/d2e/d1d/f41 1410263 0 2024-03-16T10:44:56.948 INFO:tasks.workunit.client.0.smithi002.stdout:3/323: rename d1/dd/d11/l4b to d1/dd/l7e 0 2024-03-16T10:44:56.950 INFO:tasks.workunit.client.0.smithi002.stdout:3/324: write d1/dd/d28/d37/f78 [2844641,26868] 0 2024-03-16T10:44:56.952 INFO:tasks.workunit.client.0.smithi002.stdout:3/325: write d1/dd/d28/d7b/d22/f39 [5039065,106493] 0 2024-03-16T10:44:56.954 INFO:tasks.workunit.client.0.smithi002.stdout:3/326: chown d1/dd/d28/d37/l58 293735534 1 2024-03-16T10:44:56.954 INFO:tasks.workunit.client.0.smithi002.stdout:2/384: dwrite d7/d2e/d1d/f26 [0,4194304] 0 2024-03-16T10:44:56.958 INFO:tasks.workunit.client.0.smithi002.stdout:3/327: creat d1/dd/d11/f7f x:0 0 0 2024-03-16T10:44:56.958 INFO:tasks.workunit.client.0.smithi002.stdout:2/385: mknod d7/d27/d3b/c6f 0 2024-03-16T10:44:56.959 INFO:tasks.workunit.client.0.smithi002.stdout:3/328: dread - d1/dd/d28/d7b/d2c/d63/f79 zero size 2024-03-16T10:44:56.960 INFO:tasks.workunit.client.0.smithi002.stdout:3/329: chown d1/dd/d11/c42 1225582 1 2024-03-16T10:44:56.964 INFO:tasks.workunit.client.0.smithi002.stdout:3/330: dread d1/dd/d28/d37/f78 [0,4194304] 0 2024-03-16T10:44:56.967 INFO:tasks.workunit.client.0.smithi002.stdout:3/331: fsync d1/dd/d11/f4a 0 2024-03-16T10:44:56.968 INFO:tasks.workunit.client.0.smithi002.stdout:3/332: fdatasync d1/dd/f5e 0 2024-03-16T10:44:56.971 INFO:tasks.workunit.client.0.smithi002.stdout:2/386: dwrite d7/d27/d3f/f69 [0,4194304] 0 2024-03-16T10:44:56.975 INFO:tasks.workunit.client.0.smithi002.stdout:2/387: mkdir d7/db/d70 0 2024-03-16T10:44:56.976 INFO:tasks.workunit.client.0.smithi002.stdout:2/388: stat d7/db/f57 0 2024-03-16T10:44:56.976 INFO:tasks.workunit.client.0.smithi002.stdout:3/333: dwrite d1/dd/d11/d15/f2e [0,4194304] 0 2024-03-16T10:44:56.979 INFO:tasks.workunit.client.0.smithi002.stdout:2/389: creat d7/d2e/d28/d44/f71 x:0 0 0 2024-03-16T10:44:56.992 INFO:tasks.workunit.client.0.smithi002.stdout:3/334: dwrite d1/dd/d11/d15/f53 [0,4194304] 0 2024-03-16T10:44:56.995 INFO:tasks.workunit.client.0.smithi002.stdout:3/335: write d1/dd/d11/d15/f17 [5000850,43820] 0 2024-03-16T10:44:57.003 INFO:tasks.workunit.client.0.smithi002.stdout:3/336: dwrite d1/dd/f68 [0,4194304] 0 2024-03-16T10:44:57.010 INFO:tasks.workunit.client.0.smithi002.stdout:3/337: link d1/dd/d28/d7b/d25/f26 d1/dd/d74/f80 0 2024-03-16T10:44:57.012 INFO:tasks.workunit.client.0.smithi002.stdout:3/338: creat d1/dd/d28/d7b/d2c/f81 x:0 0 0 2024-03-16T10:44:57.023 INFO:tasks.workunit.client.0.smithi002.stdout:3/339: dread d1/f16 [0,4194304] 0 2024-03-16T10:44:57.024 INFO:tasks.workunit.client.0.smithi002.stdout:3/340: write d1/d5a/f75 [404529,85409] 0 2024-03-16T10:44:57.031 INFO:tasks.workunit.client.0.smithi002.stdout:3/341: dwrite d1/dd/d11/d15/f62 [0,4194304] 0 2024-03-16T10:44:57.037 INFO:tasks.workunit.client.0.smithi002.stdout:3/342: read d1/f1d [63690,103368] 0 2024-03-16T10:44:57.043 INFO:tasks.workunit.client.0.smithi002.stdout:3/343: link d1/dd/d28/d37/c32 d1/dd/c82 0 2024-03-16T10:44:57.045 INFO:tasks.workunit.client.0.smithi002.stdout:3/344: symlink d1/dd/d28/l83 0 2024-03-16T10:44:57.047 INFO:tasks.workunit.client.0.smithi002.stdout:3/345: dread - d1/dd/d28/d7b/f7c zero size 2024-03-16T10:44:57.054 INFO:tasks.workunit.client.0.smithi002.stdout:3/346: dwrite d1/dd/d28/d7b/f24 [0,4194304] 0 2024-03-16T10:44:57.056 INFO:tasks.workunit.client.0.smithi002.stdout:3/347: write d1/dd/d28/f6a [138327,23007] 0 2024-03-16T10:44:57.065 INFO:tasks.workunit.client.0.smithi002.stdout:3/348: dwrite d1/f16 [4194304,4194304] 0 2024-03-16T10:44:57.071 INFO:tasks.workunit.client.0.smithi002.stdout:3/349: dwrite d1/dd/d11/f30 [0,4194304] 0 2024-03-16T10:44:57.074 INFO:tasks.workunit.client.0.smithi002.stdout:3/350: symlink d1/dd/d28/d7b/d2c/l84 0 2024-03-16T10:44:57.077 INFO:tasks.workunit.client.0.smithi002.stdout:8/349: sync 2024-03-16T10:44:57.077 INFO:tasks.workunit.client.0.smithi002.stdout:5/332: sync 2024-03-16T10:44:57.078 INFO:tasks.workunit.client.0.smithi002.stdout:3/351: dread d1/dd/f68 [0,4194304] 0 2024-03-16T10:44:57.084 INFO:tasks.workunit.client.0.smithi002.stdout:5/333: chown d0/d16/d4a/d39/f59 13220 1 2024-03-16T10:44:57.084 INFO:tasks.workunit.client.0.smithi002.stdout:8/350: mknod d7/d9/d17/d20/c77 0 2024-03-16T10:44:57.094 INFO:tasks.workunit.client.0.smithi002.stdout:3/352: link d1/lf d1/dd/d28/d7b/d73/l85 0 2024-03-16T10:44:57.097 INFO:tasks.workunit.client.0.smithi002.stdout:8/351: dread d7/d9/d17/d23/d39/f5d [0,4194304] 0 2024-03-16T10:44:57.101 INFO:tasks.workunit.client.0.smithi002.stdout:8/352: creat d7/d9/f78 x:0 0 0 2024-03-16T10:44:57.102 INFO:tasks.workunit.client.0.smithi002.stdout:5/334: dwrite d0/d16/d1d/d2b/d5d/f5a [0,4194304] 0 2024-03-16T10:44:57.103 INFO:tasks.workunit.client.0.smithi002.stdout:8/353: truncate d7/d9/d25/f32 2229807 0 2024-03-16T10:44:57.106 INFO:tasks.workunit.client.0.smithi002.stdout:5/335: unlink d0/d16/f29 0 2024-03-16T10:44:57.107 INFO:tasks.workunit.client.0.smithi002.stdout:5/336: readlink d0/d16/d1d/d2b/d5d/l5c 0 2024-03-16T10:44:57.112 INFO:tasks.workunit.client.0.smithi002.stdout:5/337: creat d0/d16/d1d/d2b/d5d/f64 x:0 0 0 2024-03-16T10:44:57.121 INFO:tasks.workunit.client.0.smithi002.stdout:5/338: dread d0/d1f/f36 [0,4194304] 0 2024-03-16T10:44:57.123 INFO:tasks.workunit.client.0.smithi002.stdout:8/354: dwrite d7/f48 [0,4194304] 0 2024-03-16T10:44:57.127 INFO:tasks.workunit.client.0.smithi002.stdout:8/355: rmdir d7/d9/d17/d23/d2e/d3a 39 2024-03-16T10:44:57.128 INFO:tasks.workunit.client.0.smithi002.stdout:8/356: truncate d7/d9/f6f 631378 0 2024-03-16T10:44:57.129 INFO:tasks.workunit.client.0.smithi002.stdout:5/339: dread d0/d16/d1d/f3b [0,4194304] 0 2024-03-16T10:44:57.132 INFO:tasks.workunit.client.0.smithi002.stdout:8/357: creat d7/d9/d17/d23/d2e/d34/f79 x:0 0 0 2024-03-16T10:44:57.137 INFO:tasks.workunit.client.0.smithi002.stdout:8/358: link d7/f8 d7/d9/d17/d23/d39/f7a 0 2024-03-16T10:44:57.138 INFO:tasks.workunit.client.0.smithi002.stdout:8/359: write d7/d9/d17/d23/d39/f72 [593831,61462] 0 2024-03-16T10:44:57.142 INFO:tasks.workunit.client.0.smithi002.stdout:5/340: write d0/d1f/f36 [2931763,40526] 0 2024-03-16T10:44:57.144 INFO:tasks.workunit.client.0.smithi002.stdout:8/360: dread d7/fc [0,4194304] 0 2024-03-16T10:44:57.145 INFO:tasks.workunit.client.0.smithi002.stdout:8/361: write d7/f60 [507105,63860] 0 2024-03-16T10:44:57.149 INFO:tasks.workunit.client.0.smithi002.stdout:8/362: rename d7/d9/d17/f57 to d7/d55/f7b 0 2024-03-16T10:44:57.157 INFO:tasks.workunit.client.0.smithi002.stdout:8/363: dwrite d7/d9/d17/d20/f6b [0,4194304] 0 2024-03-16T10:44:57.162 INFO:tasks.workunit.client.0.smithi002.stdout:8/364: creat d7/d55/f7c x:0 0 0 2024-03-16T10:44:57.164 INFO:tasks.workunit.client.0.smithi002.stdout:8/365: symlink d7/d9/d25/l7d 0 2024-03-16T10:44:57.165 INFO:tasks.workunit.client.0.smithi002.stdout:8/366: dread - d7/d55/f7c zero size 2024-03-16T10:44:57.168 INFO:tasks.workunit.client.0.smithi002.stdout:8/367: symlink d7/d9/d25/l7e 0 2024-03-16T10:44:57.170 INFO:tasks.workunit.client.0.smithi002.stdout:8/368: write d7/f21 [135953,15050] 0 2024-03-16T10:44:57.172 INFO:tasks.workunit.client.0.smithi002.stdout:8/369: rename d7/d9/d17/d23 to d7/d9/d17/d23/d2e/d5c/d7f 22 2024-03-16T10:44:57.173 INFO:tasks.workunit.client.0.smithi002.stdout:8/370: readlink d7/d9/d17/d23/d2e/l42 0 2024-03-16T10:44:57.175 INFO:tasks.workunit.client.0.smithi002.stdout:8/371: symlink d7/d9/l80 0 2024-03-16T10:44:57.177 INFO:tasks.workunit.client.0.smithi002.stdout:8/372: creat d7/d9/d25/d67/f81 x:0 0 0 2024-03-16T10:44:57.178 INFO:tasks.workunit.client.0.smithi002.stdout:8/373: mkdir d7/d9/d17/d20/d82 0 2024-03-16T10:44:57.179 INFO:tasks.workunit.client.0.smithi002.stdout:8/374: chown d7/d9/d17/f2d 120437 1 2024-03-16T10:44:57.180 INFO:tasks.workunit.client.0.smithi002.stdout:8/375: write d7/f21 [376505,38594] 0 2024-03-16T10:44:57.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.185+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0078100190 con 0x7f006006d3d0 2024-03-16T10:44:57.187 INFO:tasks.workunit.client.0.smithi002.stdout:8/376: dwrite d7/d9/d17/f2d [0,4194304] 0 2024-03-16T10:44:57.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.186+0000 7f00677fe700 1 -- 172.21.15.2:0/3554904688 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f0078100190 con 0x7f006006d3d0 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.190+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f006006d3d0 msgr2=0x7f006006f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.190+0000 7f007c891700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f006006d3d0 0x7f006006f890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f006800b820 tx=0x7f0068019040 comp rx=0 tx=0).stop 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.190+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 msgr2=0x7f00780ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.190+0000 7f007c891700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 0x7f00780ff510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f006c00e940 tx=0x7f006c00ed00 comp rx=0 tx=0).stop 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.191+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 shutdown_connections 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.191+0000 7f007c891700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f006006d3d0 0x7f006006f890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.191+0000 7f007c891700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0078101560 0x7f0078100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.191+0000 7f007c891700 1 --2- 172.21.15.2:0/3554904688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0078103e90 0x7f00780ff510 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.191+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 >> 172.21.15.2:0/3554904688 conn(0x7f00780faf70 msgr2=0x7f00780fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.191+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 shutdown_connections 2024-03-16T10:44:57.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.191+0000 7f007c891700 1 -- 172.21.15.2:0/3554904688 wait complete. 2024-03-16T10:44:57.194 INFO:tasks.workunit.client.0.smithi002.stdout:8/377: mkdir d7/d9/d17/d83 0 2024-03-16T10:44:57.194 INFO:tasks.workunit.client.0.smithi002.stdout:8/378: fsync d7/d13/f68 0 2024-03-16T10:44:57.200 INFO:tasks.workunit.client.0.smithi002.stdout:8/379: rename d7/d9/d25/f32 to d7/d13/f84 0 2024-03-16T10:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:44:57.211 INFO:tasks.workunit.client.0.smithi002.stdout:4/358: sync 2024-03-16T10:44:57.211 INFO:tasks.workunit.client.0.smithi002.stdout:6/373: sync 2024-03-16T10:44:57.211 INFO:tasks.workunit.client.0.smithi002.stdout:0/390: sync 2024-03-16T10:44:57.212 INFO:tasks.workunit.client.0.smithi002.stdout:6/374: dread - d15/d80/f84 zero size 2024-03-16T10:44:57.212 INFO:tasks.workunit.client.0.smithi002.stdout:4/359: dread - d2/d33/da/f49 zero size 2024-03-16T10:44:57.214 INFO:tasks.workunit.client.0.smithi002.stdout:6/375: unlink d15/l26 0 2024-03-16T10:44:57.216 INFO:tasks.workunit.client.0.smithi002.stdout:6/376: mkdir d15/d2d/d2c/d8f 0 2024-03-16T10:44:57.217 INFO:tasks.workunit.client.0.smithi002.stdout:0/391: rename d3/d27/f56 to d3/d5/f72 0 2024-03-16T10:44:57.219 INFO:tasks.workunit.client.0.smithi002.stdout:6/377: mknod d15/d2d/d2b/d3f/d77/d7e/c90 0 2024-03-16T10:44:57.220 INFO:tasks.workunit.client.0.smithi002.stdout:4/360: link d2/d33/d35/d59/l5c d2/d33/da/d5e/l70 0 2024-03-16T10:44:57.225 INFO:tasks.workunit.client.0.smithi002.stdout:0/392: dwrite d3/f49 [0,4194304] 0 2024-03-16T10:44:57.226 INFO:tasks.workunit.client.0.smithi002.stdout:4/361: rename d2/d33/c9 to d2/d33/da/d5e/c71 0 2024-03-16T10:44:57.227 INFO:tasks.workunit.client.0.smithi002.stdout:4/362: chown d2/d33/f1b 776272 1 2024-03-16T10:44:57.229 INFO:tasks.workunit.client.0.smithi002.stdout:4/363: readlink d2/d33/l39 0 2024-03-16T10:44:57.230 INFO:tasks.workunit.client.0.smithi002.stdout:6/378: dwrite d15/d2d/f6c [0,4194304] 0 2024-03-16T10:44:57.238 INFO:tasks.workunit.client.0.smithi002.stdout:0/393: dwrite d3/d28/d2a/d67/f70 [0,4194304] 0 2024-03-16T10:44:57.240 INFO:tasks.workunit.client.0.smithi002.stdout:4/364: dread d2/d6/f1a [0,4194304] 0 2024-03-16T10:44:57.243 INFO:tasks.workunit.client.0.smithi002.stdout:0/394: dread d3/d28/d3d/f50 [0,4194304] 0 2024-03-16T10:44:57.250 INFO:tasks.workunit.client.0.smithi002.stdout:0/395: unlink d3/c2b 0 2024-03-16T10:44:57.251 INFO:tasks.workunit.client.0.smithi002.stdout:0/396: write d3/d28/f3b [295233,11348] 0 2024-03-16T10:44:57.252 INFO:tasks.workunit.client.0.smithi002.stdout:0/397: fsync d3/f3a 0 2024-03-16T10:44:57.257 INFO:tasks.workunit.client.0.smithi002.stdout:6/379: truncate d15/d2d/d2c/f3c 1075354 0 2024-03-16T10:44:57.260 INFO:tasks.workunit.client.0.smithi002.stdout:6/380: creat d15/d82/f91 x:0 0 0 2024-03-16T10:44:57.261 INFO:tasks.workunit.client.0.smithi002.stdout:6/381: stat d15/d2d/d2c 0 2024-03-16T10:44:57.262 INFO:tasks.workunit.client.0.smithi002.stdout:4/365: dread d2/d6/f53 [0,4194304] 0 2024-03-16T10:44:57.263 INFO:tasks.workunit.client.0.smithi002.stdout:0/398: write d3/d27/f3c [5089570,97068] 0 2024-03-16T10:44:57.264 INFO:tasks.workunit.client.0.smithi002.stdout:0/399: chown d3/d28 919 1 2024-03-16T10:44:57.268 INFO:tasks.workunit.client.0.smithi002.stdout:6/382: link d15/l78 d15/d2d/d8a/d7c/d87/l92 0 2024-03-16T10:44:57.269 INFO:tasks.workunit.client.0.smithi002.stdout:6/383: fdatasync d15/d2d/d2b/d3f/d77/d7e/f83 0 2024-03-16T10:44:57.271 INFO:tasks.workunit.client.0.smithi002.stdout:6/384: read f4 [3199287,103763] 0 2024-03-16T10:44:57.272 INFO:tasks.workunit.client.0.smithi002.stdout:0/400: rename d3/d27/f4c to d3/f73 0 2024-03-16T10:44:57.275 INFO:tasks.workunit.client.0.smithi002.stdout:6/385: dread d15/d2d/f42 [0,4194304] 0 2024-03-16T10:44:57.276 INFO:tasks.workunit.client.0.smithi002.stdout:4/366: dread d2/d33/da/d2e/f56 [0,4194304] 0 2024-03-16T10:44:57.278 INFO:tasks.workunit.client.0.smithi002.stdout:0/401: rename d3/de/f5d to d3/d28/d3d/f74 0 2024-03-16T10:44:57.282 INFO:tasks.workunit.client.0.smithi002.stdout:4/367: rename d2/d33/d35/d59/l5d to d2/d33/d35/l72 0 2024-03-16T10:44:57.286 INFO:tasks.workunit.client.0.smithi002.stdout:0/402: unlink d3/d27/f3c 0 2024-03-16T10:44:57.292 INFO:tasks.workunit.client.0.smithi002.stdout:4/368: link d2/d33/da/f5f d2/d6/f73 0 2024-03-16T10:44:57.293 INFO:tasks.workunit.client.0.smithi002.stdout:4/369: chown d2/d33/f29 10 1 2024-03-16T10:44:57.295 INFO:tasks.workunit.client.0.smithi002.stdout:0/403: fsync d3/d28/d3d/f74 0 2024-03-16T10:44:57.295 INFO:tasks.workunit.client.0.smithi002.stdout:0/404: dread - d3/de/f51 zero size 2024-03-16T10:44:57.296 INFO:tasks.workunit.client.0.smithi002.stdout:4/370: chown d2/d33/da/d5e/c71 3389877 1 2024-03-16T10:44:57.297 INFO:tasks.workunit.client.0.smithi002.stdout:6/386: write d15/d2d/d2c/f55 [3123357,91461] 0 2024-03-16T10:44:57.298 INFO:tasks.workunit.client.0.smithi002.stdout:0/405: write d3/d5/f18 [1483323,69481] 0 2024-03-16T10:44:57.298 INFO:tasks.workunit.client.0.smithi002.stdout:6/387: stat d15/d2d/d2b/d75 0 2024-03-16T10:44:57.298 INFO:tasks.workunit.client.0.smithi002.stdout:4/371: rmdir d2/d6/d32/d68 39 2024-03-16T10:44:57.300 INFO:tasks.workunit.client.0.smithi002.stdout:6/388: truncate d15/d2d/f70 300572 0 2024-03-16T10:44:57.302 INFO:tasks.workunit.client.0.smithi002.stdout:0/406: mkdir d3/d31/d75 0 2024-03-16T10:44:57.303 INFO:tasks.workunit.client.0.smithi002.stdout:4/372: creat d2/d33/da/f74 x:0 0 0 2024-03-16T10:44:57.305 INFO:tasks.workunit.client.0.smithi002.stdout:6/389: read d15/f69 [34121,108767] 0 2024-03-16T10:44:57.306 INFO:tasks.workunit.client.0.smithi002.stdout:0/407: symlink d3/d28/d65/l76 0 2024-03-16T10:44:57.308 INFO:tasks.workunit.client.0.smithi002.stdout:4/373: creat d2/d33/da/d5e/f75 x:0 0 0 2024-03-16T10:44:57.310 INFO:tasks.workunit.client.0.smithi002.stdout:4/374: fdatasync d2/d6/f53 0 2024-03-16T10:44:57.310 INFO:tasks.workunit.client.0.smithi002.stdout:6/390: getdents d15/d80 0 2024-03-16T10:44:57.313 INFO:tasks.workunit.client.0.smithi002.stdout:6/391: rmdir d15/d2d 39 2024-03-16T10:44:57.314 INFO:tasks.workunit.client.0.smithi002.stdout:0/408: getdents d3/d5 0 2024-03-16T10:44:57.314 INFO:tasks.workunit.client.0.smithi002.stdout:9/377: sync 2024-03-16T10:44:57.314 INFO:tasks.workunit.client.0.smithi002.stdout:7/313: sync 2024-03-16T10:44:57.314 INFO:tasks.workunit.client.0.smithi002.stdout:1/342: sync 2024-03-16T10:44:57.315 INFO:tasks.workunit.client.0.smithi002.stdout:4/375: dread d2/d33/da/d2e/f56 [0,4194304] 0 2024-03-16T10:44:57.316 INFO:tasks.workunit.client.0.smithi002.stdout:6/392: fdatasync d15/d2d/d2c/d5d/f6f 0 2024-03-16T10:44:57.317 INFO:tasks.workunit.client.0.smithi002.stdout:7/314: truncate d3/d4/d1c/f2b 513225 0 2024-03-16T10:44:57.318 INFO:tasks.workunit.client.0.smithi002.stdout:0/409: creat d3/d28/d65/f77 x:0 0 0 2024-03-16T10:44:57.318 INFO:tasks.workunit.client.0.smithi002.stdout:1/343: chown d2/df/l48 38 1 2024-03-16T10:44:57.321 INFO:tasks.workunit.client.0.smithi002.stdout:7/315: symlink d3/d4/d1d/d45/l71 0 2024-03-16T10:44:57.325 INFO:tasks.workunit.client.0.smithi002.stdout:1/344: creat d2/d3/d51/f75 x:0 0 0 2024-03-16T10:44:57.325 INFO:tasks.workunit.client.0.smithi002.stdout:9/378: dwrite d5/d9/d25/f42 [0,4194304] 0 2024-03-16T10:44:57.325 INFO:tasks.workunit.client.0.smithi002.stdout:4/376: getdents d2/d6/d69 0 2024-03-16T10:44:57.327 INFO:tasks.workunit.client.0.smithi002.stdout:7/316: unlink d3/d4/d1d/c2d 0 2024-03-16T10:44:57.332 INFO:tasks.workunit.client.0.smithi002.stdout:6/393: dwrite d15/d2d/d2b/f61 [0,4194304] 0 2024-03-16T10:44:57.332 INFO:tasks.workunit.client.0.smithi002.stdout:9/379: rmdir d5/d9/d55 39 2024-03-16T10:44:57.334 INFO:tasks.workunit.client.0.smithi002.stdout:7/317: chown d3/d4/d1c/f33 71233 1 2024-03-16T10:44:57.334 INFO:tasks.workunit.client.0.smithi002.stdout:0/410: dwrite d3/d28/d3d/f46 [0,4194304] 0 2024-03-16T10:44:57.335 INFO:tasks.workunit.client.0.smithi002.stdout:4/377: truncate d2/d33/d35/f40 724832 0 2024-03-16T10:44:57.337 INFO:tasks.workunit.client.0.smithi002.stdout:4/378: read - d2/d33/d1c/f6c zero size 2024-03-16T10:44:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.335+0000 7f9904bc6700 1 -- 172.21.15.2:0/2852491313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99000ffa10 msgr2=0x7f99000ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.335+0000 7f9904bc6700 1 --2- 172.21.15.2:0/2852491313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99000ffa10 0x7f99000ffe30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009870 tx=0x7f98f4009b80 comp rx=0 tx=0).stop 2024-03-16T10:44:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.336+0000 7f9904bc6700 1 -- 172.21.15.2:0/2852491313 shutdown_connections 2024-03-16T10:44:57.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.336+0000 7f9904bc6700 1 --2- 172.21.15.2:0/2852491313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9900100c10 0x7f9900101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.336+0000 7f9904bc6700 1 --2- 172.21.15.2:0/2852491313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99000ffa10 0x7f99000ffe30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.336+0000 7f9904bc6700 1 -- 172.21.15.2:0/2852491313 >> 172.21.15.2:0/2852491313 conn(0x7f99000fafb0 msgr2=0x7f99000fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.336+0000 7f9904bc6700 1 -- 172.21.15.2:0/2852491313 shutdown_connections 2024-03-16T10:44:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.336+0000 7f9904bc6700 1 -- 172.21.15.2:0/2852491313 wait complete. 2024-03-16T10:44:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.337+0000 7f9904bc6700 1 Processor -- start 2024-03-16T10:44:57.339 INFO:tasks.workunit.client.0.smithi002.stdout:0/411: write d3/d28/d2a/f61 [4157845,101561] 0 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f9904bc6700 1 -- start start 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f9904bc6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99000ffa10 0x7f990019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f9904bc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9900100c10 0x7f990019d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f9904bc6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990019d6e0 con 0x7f99000ffa10 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f9904bc6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990019d820 con 0x7f9900100c10 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f98fe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99000ffa10 0x7f990019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f98fe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99000ffa10 0x7f990019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57480/0 (socket says 172.21.15.2:57480) 2024-03-16T10:44:57.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.338+0000 7f98fe59c700 1 -- 172.21.15.2:0/800190801 learned_addr learned my addr 172.21.15.2:0/800190801 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.339+0000 7f98fdd9b700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9900100c10 0x7f990019d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.339+0000 7f98fe59c700 1 -- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9900100c10 msgr2=0x7f990019d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.339+0000 7f98fe59c700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9900100c10 0x7f990019d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.339+0000 7f98fe59c700 1 -- 172.21.15.2:0/800190801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f4009520 con 0x7f99000ffa10 2024-03-16T10:44:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.339+0000 7f98fdd9b700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9900100c10 0x7f990019d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:44:57.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.339+0000 7f98fe59c700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99000ffa10 0x7f990019cb80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98f4004700 tx=0x7f98f4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.339+0000 7f98ef7fe700 1 -- 172.21.15.2:0/800190801 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98f401c070 con 0x7f99000ffa10 2024-03-16T10:44:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.340+0000 7f98ef7fe700 1 -- 172.21.15.2:0/800190801 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98f400fbf0 con 0x7f99000ffa10 2024-03-16T10:44:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.340+0000 7f98ef7fe700 1 -- 172.21.15.2:0/800190801 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98f4022de0 con 0x7f99000ffa10 2024-03-16T10:44:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.340+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990010d0c0 con 0x7f99000ffa10 2024-03-16T10:44:57.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.340+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990010d630 con 0x7f99000ffa10 2024-03-16T10:44:57.342 INFO:tasks.workunit.client.0.smithi002.stdout:4/379: write d2/d33/da/d2e/f47 [2054104,91872] 0 2024-03-16T10:44:57.343 INFO:tasks.workunit.client.0.smithi002.stdout:4/380: stat d2/d6/d32/l4d 0 2024-03-16T10:44:57.343 INFO:tasks.workunit.client.0.smithi002.stdout:6/394: creat d15/d2d/d8a/d7c/f93 x:0 0 0 2024-03-16T10:44:57.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.341+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e0005320 con 0x7f99000ffa10 2024-03-16T10:44:57.344 INFO:tasks.workunit.client.0.smithi002.stdout:1/345: truncate d2/d3/f7 5516153 0 2024-03-16T10:44:57.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.343+0000 7f98ef7fe700 1 -- 172.21.15.2:0/800190801 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f98f4020070 con 0x7f99000ffa10 2024-03-16T10:44:57.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.343+0000 7f98ef7fe700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f98e806d3d0 0x7f98e806f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:57.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.343+0000 7f98fdd9b700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f98e806d3d0 0x7f98e806f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:57.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.344+0000 7f98ef7fe700 1 -- 172.21.15.2:0/800190801 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f98f408dd30 con 0x7f99000ffa10 2024-03-16T10:44:57.345 INFO:tasks.workunit.client.0.smithi002.stdout:4/381: truncate d2/d33/da/d2e/f56 4200106 0 2024-03-16T10:44:57.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.344+0000 7f98fdd9b700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f98e806d3d0 0x7f98e806f890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98f0005fd0 tx=0x7f98f0009310 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:57.346 INFO:tasks.workunit.client.0.smithi002.stdout:7/318: creat d3/d37/d6a/f72 x:0 0 0 2024-03-16T10:44:57.348 INFO:tasks.workunit.client.0.smithi002.stdout:9/380: unlink d5/d9/dc/f20 0 2024-03-16T10:44:57.350 INFO:tasks.workunit.client.0.smithi002.stdout:0/412: mknod d3/d28/d65/c78 0 2024-03-16T10:44:57.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.351+0000 7f98ef7fe700 1 -- 172.21.15.2:0/800190801 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f98f405b240 con 0x7f99000ffa10 2024-03-16T10:44:57.353 INFO:tasks.workunit.client.0.smithi002.stdout:1/346: unlink d2/c67 0 2024-03-16T10:44:57.355 INFO:tasks.workunit.client.0.smithi002.stdout:4/382: unlink d2/d6/c1f 0 2024-03-16T10:44:57.357 INFO:tasks.workunit.client.0.smithi002.stdout:7/319: mknod d3/d4/d1c/c73 0 2024-03-16T10:44:57.357 INFO:tasks.workunit.client.0.smithi002.stdout:9/381: rmdir d5/d9 39 2024-03-16T10:44:57.359 INFO:tasks.workunit.client.0.smithi002.stdout:1/347: dread d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:44:57.360 INFO:tasks.workunit.client.0.smithi002.stdout:6/395: mknod d15/d2d/d2b/c94 0 2024-03-16T10:44:57.361 INFO:tasks.workunit.client.0.smithi002.stdout:1/348: chown d2/d3/d1c/d5d/d71 468 1 2024-03-16T10:44:57.361 INFO:tasks.workunit.client.0.smithi002.stdout:6/396: stat d15/d2d/f22 0 2024-03-16T10:44:57.363 INFO:tasks.workunit.client.0.smithi002.stdout:1/349: fsync d2/d3/fc 0 2024-03-16T10:44:57.365 INFO:tasks.workunit.client.0.smithi002.stdout:7/320: dwrite d3/d37/d6a/f72 [0,4194304] 0 2024-03-16T10:44:57.366 INFO:tasks.workunit.client.0.smithi002.stdout:9/382: fdatasync d5/d9/d34/f68 0 2024-03-16T10:44:57.371 INFO:tasks.workunit.client.0.smithi002.stdout:4/383: mknod d2/d6/d32/d68/c76 0 2024-03-16T10:44:57.372 INFO:tasks.workunit.client.0.smithi002.stdout:4/384: dread - d2/d6/d32/d68/f5b zero size 2024-03-16T10:44:57.374 INFO:tasks.workunit.client.0.smithi002.stdout:9/383: stat d5/d9/dc/d28/c51 0 2024-03-16T10:44:57.375 INFO:tasks.workunit.client.0.smithi002.stdout:9/384: fdatasync d5/d9/dc/d1e/d24/f4b 0 2024-03-16T10:44:57.377 INFO:tasks.workunit.client.0.smithi002.stdout:7/321: creat d3/d61/d66/f74 x:0 0 0 2024-03-16T10:44:57.400 INFO:tasks.workunit.client.0.smithi002.stdout:0/413: rename d3/de/f43 to d3/d28/f79 0 2024-03-16T10:44:57.400 INFO:tasks.workunit.client.0.smithi002.stdout:7/322: dread - d3/d4/d1d/f70 zero size 2024-03-16T10:44:57.402 INFO:tasks.workunit.client.0.smithi002.stdout:0/414: write d3/de/f20 [3941332,37968] 0 2024-03-16T10:44:57.402 INFO:tasks.workunit.client.0.smithi002.stdout:6/397: rename d15/d2d/d8a/d7c/f93 to d15/d2d/d73/f95 0 2024-03-16T10:44:57.406 INFO:tasks.workunit.client.0.smithi002.stdout:4/385: rmdir d2/d6/d69 0 2024-03-16T10:44:57.408 INFO:tasks.workunit.client.0.smithi002.stdout:0/415: creat d3/d28/d65/f7a x:0 0 0 2024-03-16T10:44:57.409 INFO:tasks.workunit.client.0.smithi002.stdout:6/398: fdatasync d15/d2d/d2b/f63 0 2024-03-16T10:44:57.410 INFO:tasks.workunit.client.0.smithi002.stdout:9/385: link d5/f29 d5/d9/d25/f73 0 2024-03-16T10:44:57.411 INFO:tasks.workunit.client.0.smithi002.stdout:9/386: stat d5/d9/lb 0 2024-03-16T10:44:57.411 INFO:tasks.workunit.client.0.smithi002.stdout:4/386: dread d2/d33/da/d2e/f56 [0,4194304] 0 2024-03-16T10:44:57.413 INFO:tasks.workunit.client.0.smithi002.stdout:9/387: read d5/d9/dc/d1e/d24/f4b [576133,90319] 0 2024-03-16T10:44:57.414 INFO:tasks.workunit.client.0.smithi002.stdout:9/388: readlink d5/d9/d22/d3c/d4d/l50 0 2024-03-16T10:44:57.416 INFO:tasks.workunit.client.0.smithi002.stdout:4/387: stat d2/fe 0 2024-03-16T10:44:57.418 INFO:tasks.workunit.client.0.smithi002.stdout:0/416: truncate d3/d28/f4d 3768073 0 2024-03-16T10:44:57.419 INFO:tasks.workunit.client.0.smithi002.stdout:6/399: dread d15/d2d/d2c/f3c [0,4194304] 0 2024-03-16T10:44:57.421 INFO:tasks.workunit.client.0.smithi002.stdout:9/389: creat d5/d9/dc/f74 x:0 0 0 2024-03-16T10:44:57.423 INFO:tasks.workunit.client.0.smithi002.stdout:0/417: unlink d3/c5b 0 2024-03-16T10:44:57.427 INFO:tasks.workunit.client.0.smithi002.stdout:9/390: creat d5/d9/d22/d3c/d4d/f75 x:0 0 0 2024-03-16T10:44:57.431 INFO:tasks.workunit.client.0.smithi002.stdout:6/400: mkdir d15/d2d/d2c/d8f/d96 0 2024-03-16T10:44:57.433 INFO:tasks.workunit.client.0.smithi002.stdout:7/323: dwrite d3/d37/d6a/f6e [0,4194304] 0 2024-03-16T10:44:57.434 INFO:tasks.workunit.client.0.smithi002.stdout:4/388: creat d2/f77 x:0 0 0 2024-03-16T10:44:57.437 INFO:tasks.workunit.client.0.smithi002.stdout:4/389: rename d2/d33 to d2/d33/da/d2e/d78 22 2024-03-16T10:44:57.438 INFO:tasks.workunit.client.0.smithi002.stdout:2/390: sync 2024-03-16T10:44:57.438 INFO:tasks.workunit.client.0.smithi002.stdout:3/353: sync 2024-03-16T10:44:57.442 INFO:tasks.workunit.client.0.smithi002.stdout:7/324: mkdir d3/d4/d1c/d44/d5c/d63/d3f/d75 0 2024-03-16T10:44:57.444 INFO:tasks.workunit.client.0.smithi002.stdout:6/401: dwrite d15/d80/f84 [0,4194304] 0 2024-03-16T10:44:57.446 INFO:tasks.workunit.client.0.smithi002.stdout:2/391: truncate d7/d2e/d28/d44/f71 748726 0 2024-03-16T10:44:57.449 INFO:tasks.workunit.client.0.smithi002.stdout:7/325: readlink d3/l20 0 2024-03-16T10:44:57.452 INFO:tasks.workunit.client.0.smithi002.stdout:6/402: dread d15/d2d/f22 [0,4194304] 0 2024-03-16T10:44:57.454 INFO:tasks.workunit.client.0.smithi002.stdout:3/354: dwrite d1/f16 [4194304,4194304] 0 2024-03-16T10:44:57.454 INFO:tasks.workunit.client.0.smithi002.stdout:4/390: dwrite d2/d33/da/f4f [0,4194304] 0 2024-03-16T10:44:57.455 INFO:tasks.workunit.client.0.smithi002.stdout:9/391: write d5/d9/d55/f63 [268305,5764] 0 2024-03-16T10:44:57.456 INFO:tasks.workunit.client.0.smithi002.stdout:2/392: mkdir d7/db/d70/d72 0 2024-03-16T10:44:57.457 INFO:tasks.workunit.client.0.smithi002.stdout:7/326: mknod d3/d4/d1c/c76 0 2024-03-16T10:44:57.458 INFO:tasks.workunit.client.0.smithi002.stdout:6/403: chown d15/d82/f91 4616531 1 2024-03-16T10:44:57.460 INFO:tasks.workunit.client.0.smithi002.stdout:4/391: fdatasync d2/d6/f53 0 2024-03-16T10:44:57.461 INFO:tasks.workunit.client.0.smithi002.stdout:3/355: read d1/dd/d28/d37/f78 [152394,128878] 0 2024-03-16T10:44:57.462 INFO:tasks.workunit.client.0.smithi002.stdout:0/418: dwrite d3/fc [0,4194304] 0 2024-03-16T10:44:57.464 INFO:tasks.workunit.client.0.smithi002.stdout:7/327: unlink d3/d4/d1c/c73 0 2024-03-16T10:44:57.464 INFO:tasks.workunit.client.0.smithi002.stdout:3/356: dread - d1/dd/d28/d7b/d2c/f6d zero size 2024-03-16T10:44:57.466 INFO:tasks.workunit.client.0.smithi002.stdout:6/404: write d15/d2d/f42 [376813,73781] 0 2024-03-16T10:44:57.466 INFO:tasks.workunit.client.0.smithi002.stdout:3/357: dread - d1/dd/d28/f55 zero size 2024-03-16T10:44:57.469 INFO:tasks.workunit.client.0.smithi002.stdout:6/405: dread d15/d2d/f2a [0,4194304] 0 2024-03-16T10:44:57.470 INFO:tasks.workunit.client.0.smithi002.stdout:9/392: truncate d5/d9/dc/f11 1384031 0 2024-03-16T10:44:57.470 INFO:tasks.workunit.client.0.smithi002.stdout:0/419: dread - d3/d28/f36 zero size 2024-03-16T10:44:57.471 INFO:tasks.workunit.client.0.smithi002.stdout:9/393: fdatasync d5/d9/f13 0 2024-03-16T10:44:57.474 INFO:tasks.workunit.client.0.smithi002.stdout:0/420: write d3/d31/f4b [557300,82492] 0 2024-03-16T10:44:57.474 INFO:tasks.workunit.client.0.smithi002.stdout:8/380: sync 2024-03-16T10:44:57.475 INFO:tasks.workunit.client.0.smithi002.stdout:5/341: sync 2024-03-16T10:44:57.475 INFO:tasks.workunit.client.0.smithi002.stdout:0/421: stat d3/d31/d75 0 2024-03-16T10:44:57.476 INFO:tasks.workunit.client.0.smithi002.stdout:7/328: creat d3/d4/d1c/d44/d5c/d63/d55/f77 x:0 0 0 2024-03-16T10:44:57.478 INFO:tasks.workunit.client.0.smithi002.stdout:7/329: chown d3/d4 0 1 2024-03-16T10:44:57.481 INFO:tasks.workunit.client.0.smithi002.stdout:5/342: chown d0/d16/d1d/d2b/d5d/c30 40130 1 2024-03-16T10:44:57.481 INFO:tasks.workunit.client.0.smithi002.stdout:4/392: creat d2/f79 x:0 0 0 2024-03-16T10:44:57.483 INFO:tasks.workunit.client.0.smithi002.stdout:3/358: link d1/dd/d28/d7b/l29 d1/dd/d28/d7b/d22/l86 0 2024-03-16T10:44:57.485 INFO:tasks.workunit.client.0.smithi002.stdout:8/381: write d7/d9/d17/d23/d2e/d3a/f53 [1175966,92200] 0 2024-03-16T10:44:57.487 INFO:tasks.workunit.client.0.smithi002.stdout:2/393: write d7/d2e/d1d/d38/f3a [1008229,56777] 0 2024-03-16T10:44:57.487 INFO:tasks.workunit.client.0.smithi002.stdout:4/393: truncate d2/d33/f6a 2189 0 2024-03-16T10:44:57.489 INFO:tasks.workunit.client.0.smithi002.stdout:0/422: dread d3/fc [0,4194304] 0 2024-03-16T10:44:57.489 INFO:tasks.workunit.client.0.smithi002.stdout:7/330: creat d3/d4/d1c/d44/d5c/d63/d34/f78 x:0 0 0 2024-03-16T10:44:57.489 INFO:tasks.workunit.client.0.smithi002.stdout:6/406: dwrite f8 [0,4194304] 0 2024-03-16T10:44:57.489 INFO:tasks.workunit.client.0.smithi002.stdout:5/343: mkdir d0/d16/d1d/d33/d65 0 2024-03-16T10:44:57.492 INFO:tasks.workunit.client.0.smithi002.stdout:8/382: fdatasync d7/d9/d17/d23/d2e/d3a/f47 0 2024-03-16T10:44:57.492 INFO:tasks.workunit.client.0.smithi002.stdout:5/344: write d0/f57 [2535705,103626] 0 2024-03-16T10:44:57.492 INFO:tasks.workunit.client.0.smithi002.stdout:3/359: mkdir d1/dd/d11/d15/d87 0 2024-03-16T10:44:57.493 INFO:tasks.workunit.client.0.smithi002.stdout:9/394: dread d5/d9/dc/d1e/d24/f4b [0,4194304] 0 2024-03-16T10:44:57.540 INFO:tasks.workunit.client.0.smithi002.stdout:5/345: dread - d0/d16/d1d/d33/f42 zero size 2024-03-16T10:44:57.540 INFO:tasks.workunit.client.0.smithi002.stdout:4/394: rename d2/d33/d35/l3f to d2/d33/d35/d59/l7a 0 2024-03-16T10:44:57.544 INFO:tasks.workunit.client.0.smithi002.stdout:8/383: rename f4 to d7/d9/d17/d20/d82/f85 0 2024-03-16T10:44:57.545 INFO:tasks.workunit.client.0.smithi002.stdout:6/407: dread d15/d80/f84 [0,4194304] 0 2024-03-16T10:44:57.546 INFO:tasks.workunit.client.0.smithi002.stdout:8/384: chown d7/d9/d25 217434641 1 2024-03-16T10:44:57.548 INFO:tasks.workunit.client.0.smithi002.stdout:7/331: dwrite d3/d4/d1c/d44/d5c/d63/f4c [0,4194304] 0 2024-03-16T10:44:57.548 INFO:tasks.workunit.client.0.smithi002.stdout:0/423: creat d3/de/f7b x:0 0 0 2024-03-16T10:44:57.550 INFO:tasks.workunit.client.0.smithi002.stdout:7/332: write d3/d4/d1d/d3d/f56 [2710949,109816] 0 2024-03-16T10:44:57.552 INFO:tasks.workunit.client.0.smithi002.stdout:4/395: mknod d2/c7b 0 2024-03-16T10:44:57.558 INFO:tasks.workunit.client.0.smithi002.stdout:6/408: mkdir d15/d2d/d73/d97 0 2024-03-16T10:44:57.559 INFO:tasks.workunit.client.0.smithi002.stdout:8/385: creat d7/d55/f86 x:0 0 0 2024-03-16T10:44:57.561 INFO:tasks.workunit.client.0.smithi002.stdout:0/424: mkdir d3/d5/d7c 0 2024-03-16T10:44:57.563 INFO:tasks.workunit.client.0.smithi002.stdout:4/396: rmdir d2/d33/d35 39 2024-03-16T10:44:57.564 INFO:tasks.workunit.client.0.smithi002.stdout:6/409: dwrite d15/d2d/d2b/f61 [0,4194304] 0 2024-03-16T10:44:57.572 INFO:tasks.workunit.client.0.smithi002.stdout:5/346: rename d0/d24/f3c to d0/d16/f66 0 2024-03-16T10:44:57.574 INFO:tasks.workunit.client.0.smithi002.stdout:4/397: dread d2/d33/da/d2e/f56 [0,4194304] 0 2024-03-16T10:44:57.577 INFO:tasks.workunit.client.0.smithi002.stdout:9/395: dwrite d5/f53 [0,4194304] 0 2024-03-16T10:44:57.580 INFO:tasks.workunit.client.0.smithi002.stdout:3/360: dwrite d1/dd/d74/f80 [0,4194304] 0 2024-03-16T10:44:57.581 INFO:tasks.workunit.client.0.smithi002.stdout:9/396: readlink d5/l8 0 2024-03-16T10:44:57.583 INFO:tasks.workunit.client.0.smithi002.stdout:9/397: dread - d5/d9/dc/d28/f5a zero size 2024-03-16T10:44:57.585 INFO:tasks.workunit.client.0.smithi002.stdout:7/333: rename d3/d37/l62 to d3/d4/d1d/l79 0 2024-03-16T10:44:57.585 INFO:tasks.workunit.client.0.smithi002.stdout:8/386: rename d7/d9/d25/d67 to d7/d9/d25/d67/d87 22 2024-03-16T10:44:57.591 INFO:tasks.workunit.client.0.smithi002.stdout:4/398: dwrite d2/d33/f64 [0,4194304] 0 2024-03-16T10:44:57.596 INFO:tasks.workunit.client.0.smithi002.stdout:8/387: dread d7/d9/f6f [0,4194304] 0 2024-03-16T10:44:57.597 INFO:tasks.workunit.client.0.smithi002.stdout:0/425: write d3/d5/f23 [5623714,22007] 0 2024-03-16T10:44:57.599 INFO:tasks.workunit.client.0.smithi002.stdout:0/426: fdatasync d3/f49 0 2024-03-16T10:44:57.601 INFO:tasks.workunit.client.0.smithi002.stdout:1/350: sync 2024-03-16T10:44:57.606 INFO:tasks.workunit.client.0.smithi002.stdout:6/410: dwrite d15/d2d/d2c/f36 [0,4194304] 0 2024-03-16T10:44:57.608 INFO:tasks.workunit.client.0.smithi002.stdout:6/411: fdatasync d15/f5f 0 2024-03-16T10:44:57.610 INFO:tasks.workunit.client.0.smithi002.stdout:5/347: link d0/c10 d0/d16/d1d/d2b/d5d/c67 0 2024-03-16T10:44:57.610 INFO:tasks.workunit.client.0.smithi002.stdout:3/361: dread d1/dd/d28/d7b/d22/d3f/f47 [0,4194304] 0 2024-03-16T10:44:57.613 INFO:tasks.workunit.client.0.smithi002.stdout:9/398: dwrite d5/f29 [0,4194304] 0 2024-03-16T10:44:57.615 INFO:tasks.workunit.client.0.smithi002.stdout:8/388: dwrite d7/d9/f78 [0,4194304] 0 2024-03-16T10:44:57.626 INFO:tasks.workunit.client.0.smithi002.stdout:4/399: getdents d2/d33/da 0 2024-03-16T10:44:57.626 INFO:tasks.workunit.client.0.smithi002.stdout:7/334: getdents d3/d4/d1d/d3d 0 2024-03-16T10:44:57.627 INFO:tasks.workunit.client.0.smithi002.stdout:1/351: creat d2/f76 x:0 0 0 2024-03-16T10:44:57.627 INFO:tasks.workunit.client.0.smithi002.stdout:3/362: creat d1/dd/d28/d7b/d2c/d63/f88 x:0 0 0 2024-03-16T10:44:57.629 INFO:tasks.workunit.client.0.smithi002.stdout:0/427: rename d3/d28/d3d/f45 to d3/de/f7d 0 2024-03-16T10:44:57.630 INFO:tasks.workunit.client.0.smithi002.stdout:1/352: fsync d2/d3/f39 0 2024-03-16T10:44:57.631 INFO:tasks.workunit.client.0.smithi002.stdout:8/389: dwrite d7/d9/d17/d23/d39/f72 [0,4194304] 0 2024-03-16T10:44:57.632 INFO:tasks.workunit.client.0.smithi002.stdout:5/348: getdents d0/d16/d1d/d33/d65 0 2024-03-16T10:44:57.633 INFO:tasks.workunit.client.0.smithi002.stdout:3/363: dread - d1/dd/d28/d7b/d2c/f6d zero size 2024-03-16T10:44:57.634 INFO:tasks.workunit.client.0.smithi002.stdout:9/399: mknod d5/d9/c76 0 2024-03-16T10:44:57.634 INFO:tasks.workunit.client.0.smithi002.stdout:7/335: rmdir d3/d4/d1c/d44/d5c 39 2024-03-16T10:44:57.637 INFO:tasks.workunit.client.0.smithi002.stdout:4/400: stat d2/d33/d35/d59/l5c 0 2024-03-16T10:44:57.638 INFO:tasks.workunit.client.0.smithi002.stdout:6/412: getdents d15/d2d/d2c/d5d 0 2024-03-16T10:44:57.638 INFO:tasks.workunit.client.0.smithi002.stdout:0/428: mknod d3/d31/d75/c7e 0 2024-03-16T10:44:57.639 INFO:tasks.workunit.client.0.smithi002.stdout:7/336: write d3/d4/d1d/d45/f42 [3546149,72500] 0 2024-03-16T10:44:57.639 INFO:tasks.workunit.client.0.smithi002.stdout:3/364: unlink d1/c64 0 2024-03-16T10:44:57.640 INFO:tasks.workunit.client.0.smithi002.stdout:9/400: read d5/d9/d25/f71 [293486,51883] 0 2024-03-16T10:44:57.644 INFO:tasks.workunit.client.0.smithi002.stdout:5/349: symlink d0/d16/d4a/d55/l68 0 2024-03-16T10:44:57.644 INFO:tasks.workunit.client.0.smithi002.stdout:7/337: dread d3/f27 [0,4194304] 0 2024-03-16T10:44:57.649 INFO:tasks.workunit.client.0.smithi002.stdout:6/413: unlink d15/d2d/d2b/f68 0 2024-03-16T10:44:57.649 INFO:tasks.workunit.client.0.smithi002.stdout:6/414: dread - d15/d2d/d2c/d5d/f6f zero size 2024-03-16T10:44:57.651 INFO:tasks.workunit.client.0.smithi002.stdout:2/394: sync 2024-03-16T10:44:57.652 INFO:tasks.workunit.client.0.smithi002.stdout:9/401: mkdir d5/d9/d55/d77 0 2024-03-16T10:44:57.664 INFO:tasks.workunit.client.0.smithi002.stdout:5/350: rmdir d0/d16 39 2024-03-16T10:44:57.664 INFO:tasks.workunit.client.0.smithi002.stdout:7/338: write d3/d4/d1c/d44/d5c/d63/f3c [4153402,38424] 0 2024-03-16T10:44:57.664 INFO:tasks.workunit.client.0.smithi002.stdout:8/390: getdents d7/d9/d17/d23 0 2024-03-16T10:44:57.674 INFO:tasks.workunit.client.0.smithi002.stdout:1/353: link d2/df/d12/d29/l32 d2/df/d12/d29/d5c/l77 0 2024-03-16T10:44:57.675 INFO:tasks.workunit.client.0.smithi002.stdout:0/429: creat d3/d28/f7f x:0 0 0 2024-03-16T10:44:57.676 INFO:tasks.workunit.client.0.smithi002.stdout:7/339: dread d3/f11 [0,4194304] 0 2024-03-16T10:44:57.676 INFO:tasks.workunit.client.0.smithi002.stdout:3/365: truncate d1/dd/d28/d7b/d22/d3f/f47 2758390 0 2024-03-16T10:44:57.677 INFO:tasks.workunit.client.0.smithi002.stdout:2/395: dread d7/d2e/d28/d40/f52 [0,4194304] 0 2024-03-16T10:44:57.678 INFO:tasks.workunit.client.0.smithi002.stdout:5/351: fdatasync d0/d16/d1d/f3b 0 2024-03-16T10:44:57.681 INFO:tasks.workunit.client.0.smithi002.stdout:8/391: rename d7/d13/l28 to d7/d9/d17/d20/l88 0 2024-03-16T10:44:57.682 INFO:tasks.workunit.client.0.smithi002.stdout:6/415: mkdir d15/d2d/d73/d97/d98 0 2024-03-16T10:44:57.683 INFO:tasks.workunit.client.0.smithi002.stdout:8/392: truncate d7/d9/d17/d20/f5e 271089 0 2024-03-16T10:44:57.684 INFO:tasks.workunit.client.0.smithi002.stdout:4/401: getdents d2 0 2024-03-16T10:44:57.685 INFO:tasks.workunit.client.0.smithi002.stdout:7/340: chown d3/cf 97585130 1 2024-03-16T10:44:57.685 INFO:tasks.workunit.client.0.smithi002.stdout:8/393: chown d7/d9/d17/d23/f4d 25481201 1 2024-03-16T10:44:57.686 INFO:tasks.workunit.client.0.smithi002.stdout:2/396: creat d7/d2e/d1d/d38/f73 x:0 0 0 2024-03-16T10:44:57.687 INFO:tasks.workunit.client.0.smithi002.stdout:5/352: truncate d0/d16/d4a/d39/f59 1247482 0 2024-03-16T10:44:57.687 INFO:tasks.workunit.client.0.smithi002.stdout:2/397: fdatasync d7/db/d12/f11 0 2024-03-16T10:44:57.688 INFO:tasks.workunit.client.0.smithi002.stdout:2/398: readlink d7/d2e/d28/l61 0 2024-03-16T10:44:57.688 INFO:tasks.workunit.client.0.smithi002.stdout:7/341: fsync d3/d4/d1c/d44/d5c/d63/f3b 0 2024-03-16T10:44:57.691 INFO:tasks.workunit.client.0.smithi002.stdout:6/416: creat d15/d2d/d2b/d3f/d77/f99 x:0 0 0 2024-03-16T10:44:57.691 INFO:tasks.workunit.client.0.smithi002.stdout:1/354: symlink d2/df/d12/d74/l78 0 2024-03-16T10:44:57.691 INFO:tasks.workunit.client.0.smithi002.stdout:9/402: creat d5/d9/f78 x:0 0 0 2024-03-16T10:44:57.692 INFO:tasks.workunit.client.0.smithi002.stdout:4/402: dread d2/d33/da/d2e/f56 [0,4194304] 0 2024-03-16T10:44:57.693 INFO:tasks.workunit.client.0.smithi002.stdout:6/417: dread - d15/d2d/d2b/d3f/d77/f99 zero size 2024-03-16T10:44:57.694 INFO:tasks.workunit.client.0.smithi002.stdout:5/353: rmdir d0 39 2024-03-16T10:44:57.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.694+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98e0000bf0 con 0x7f98e806d3d0 2024-03-16T10:44:57.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.695+0000 7f98ef7fe700 1 -- 172.21.15.2:0/800190801 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f98e0000bf0 con 0x7f98e806d3d0 2024-03-16T10:44:57.699 INFO:tasks.workunit.client.0.smithi002.stdout:2/399: rename d7/f8 to d7/d2e/d1d/d2f/d4b/f74 0 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f98e806d3d0 msgr2=0x7f98e806f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f98e806d3d0 0x7f98e806f890 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98f0005fd0 tx=0x7f98f0009310 comp rx=0 tx=0).stop 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99000ffa10 msgr2=0x7f990019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99000ffa10 0x7f990019cb80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f98f4004700 tx=0x7f98f4005e40 comp rx=0 tx=0).stop 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 shutdown_connections 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f98e806d3d0 0x7f98e806f890 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99000ffa10 0x7f990019cb80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 --2- 172.21.15.2:0/800190801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9900100c10 0x7f990019d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 >> 172.21.15.2:0/800190801 conn(0x7f99000fafb0 msgr2=0x7f9900104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:57.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 shutdown_connections 2024-03-16T10:44:57.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.699+0000 7f9904bc6700 1 -- 172.21.15.2:0/800190801 wait complete. 2024-03-16T10:44:57.702 INFO:tasks.workunit.client.0.smithi002.stdout:0/430: creat d3/de/f80 x:0 0 0 2024-03-16T10:44:57.702 INFO:tasks.workunit.client.0.smithi002.stdout:0/431: stat d3/de/f51 0 2024-03-16T10:44:57.703 INFO:tasks.workunit.client.0.smithi002.stdout:1/355: unlink d2/d3/c20 0 2024-03-16T10:44:57.706 INFO:tasks.workunit.client.0.smithi002.stdout:9/403: mknod d5/d9/d34/c79 0 2024-03-16T10:44:57.708 INFO:tasks.workunit.client.0.smithi002.stdout:8/394: symlink d7/d9/d17/d83/l89 0 2024-03-16T10:44:57.709 INFO:tasks.workunit.client.0.smithi002.stdout:6/418: dread d15/d2d/d2c/f56 [0,4194304] 0 2024-03-16T10:44:57.713 INFO:tasks.workunit.client.0.smithi002.stdout:8/395: write d7/d9/d17/d23/d2e/d3a/f53 [838252,108541] 0 2024-03-16T10:44:57.715 INFO:tasks.workunit.client.0.smithi002.stdout:3/366: truncate d1/dd/d11/d15/f17 604623 0 2024-03-16T10:44:57.718 INFO:tasks.workunit.client.0.smithi002.stdout:7/342: getdents d3/d4/d1c/d44/d5c/d63/d3f/d75 0 2024-03-16T10:44:57.721 INFO:tasks.workunit.client.0.smithi002.stdout:0/432: chown d3/d28/f4d 1902 1 2024-03-16T10:44:57.723 INFO:tasks.workunit.client.0.smithi002.stdout:1/356: dread d2/f52 [0,4194304] 0 2024-03-16T10:44:57.723 INFO:tasks.workunit.client.0.smithi002.stdout:9/404: dwrite d5/d9/dc/d28/f5a [0,4194304] 0 2024-03-16T10:44:57.723 INFO:tasks.workunit.client.0.smithi002.stdout:6/419: dwrite d15/f4f [0,4194304] 0 2024-03-16T10:44:57.725 INFO:tasks.workunit.client.0.smithi002.stdout:5/354: symlink d0/d16/d4a/d55/l69 0 2024-03-16T10:44:57.726 INFO:tasks.workunit.client.0.smithi002.stdout:6/420: write fc [2234628,29931] 0 2024-03-16T10:44:57.727 INFO:tasks.workunit.client.0.smithi002.stdout:9/405: fsync d5/d9/f78 0 2024-03-16T10:44:57.727 INFO:tasks.workunit.client.0.smithi002.stdout:2/400: link d7/d2e/d1d/d2f/d4b/f62 d7/d2e/d1d/d2f/f75 0 2024-03-16T10:44:57.728 INFO:tasks.workunit.client.0.smithi002.stdout:4/403: write d2/d33/f3e [269224,99831] 0 2024-03-16T10:44:57.730 INFO:tasks.workunit.client.0.smithi002.stdout:6/421: fsync d15/d2d/f1f 0 2024-03-16T10:44:57.732 INFO:tasks.workunit.client.0.smithi002.stdout:8/396: write d7/d9/d25/f45 [3250506,13162] 0 2024-03-16T10:44:57.732 INFO:tasks.workunit.client.0.smithi002.stdout:4/404: dread - d2/d6/f73 zero size 2024-03-16T10:44:57.735 INFO:tasks.workunit.client.0.smithi002.stdout:1/357: dread d2/df/d12/f4e [0,4194304] 0 2024-03-16T10:44:57.736 INFO:tasks.workunit.client.0.smithi002.stdout:1/358: dread - d2/d58/f70 zero size 2024-03-16T10:44:57.740 INFO:tasks.workunit.client.0.smithi002.stdout:9/406: dwrite d5/d9/dc/d1e/d24/f30 [4194304,4194304] 0 2024-03-16T10:44:57.749 INFO:tasks.workunit.client.0.smithi002.stdout:6/422: rmdir d15/d2d/d8a/d7c 39 2024-03-16T10:44:57.754 INFO:tasks.workunit.client.0.smithi002.stdout:4/405: mknod d2/d33/d35/d59/c7c 0 2024-03-16T10:44:57.755 INFO:tasks.workunit.client.0.smithi002.stdout:4/406: fdatasync d2/d33/f64 0 2024-03-16T10:44:57.756 INFO:tasks.workunit.client.0.smithi002.stdout:1/359: dread d2/df/d12/f72 [0,4194304] 0 2024-03-16T10:44:57.758 INFO:tasks.workunit.client.0.smithi002.stdout:7/343: write d3/f27 [620962,34000] 0 2024-03-16T10:44:57.764 INFO:tasks.workunit.client.0.smithi002.stdout:6/423: rmdir d15 39 2024-03-16T10:44:57.768 INFO:tasks.workunit.client.0.smithi002.stdout:4/407: creat d2/d33/da/f7d x:0 0 0 2024-03-16T10:44:57.769 INFO:tasks.workunit.client.0.smithi002.stdout:4/408: write d2/d33/d1c/f1d [2729287,64519] 0 2024-03-16T10:44:57.769 INFO:tasks.workunit.client.0.smithi002.stdout:1/360: chown d2/d3/fe 2196779 1 2024-03-16T10:44:57.771 INFO:tasks.workunit.client.0.smithi002.stdout:1/361: write d2/d3/fc [5084378,49346] 0 2024-03-16T10:44:57.772 INFO:tasks.workunit.client.0.smithi002.stdout:1/362: read d2/f52 [1480670,102736] 0 2024-03-16T10:44:57.774 INFO:tasks.workunit.client.0.smithi002.stdout:3/367: rename d1/dd/d11 to d1/dd/d28/d7b/d89 0 2024-03-16T10:44:57.778 INFO:tasks.workunit.client.0.smithi002.stdout:9/407: creat d5/d9/d55/d77/f7a x:0 0 0 2024-03-16T10:44:57.781 INFO:tasks.workunit.client.0.smithi002.stdout:6/424: write d15/d2d/f22 [304894,21200] 0 2024-03-16T10:44:57.785 INFO:tasks.workunit.client.0.smithi002.stdout:4/409: creat d2/d6/f7e x:0 0 0 2024-03-16T10:44:57.790 INFO:tasks.workunit.client.0.smithi002.stdout:6/425: dwrite d15/d2d/f1f [0,4194304] 0 2024-03-16T10:44:57.790 INFO:tasks.workunit.client.0.smithi002.stdout:0/433: rename d3/de/f80 to d3/d31/d75/f81 0 2024-03-16T10:44:57.792 INFO:tasks.workunit.client.0.smithi002.stdout:7/344: mknod d3/d4/d1c/d44/d5c/d63/c7a 0 2024-03-16T10:44:57.794 INFO:tasks.workunit.client.0.smithi002.stdout:9/408: unlink d5/d9/fe 0 2024-03-16T10:44:57.795 INFO:tasks.workunit.client.0.smithi002.stdout:9/409: fsync d5/d9/dc/d1e/d24/f30 0 2024-03-16T10:44:57.795 INFO:tasks.workunit.client.0.smithi002.stdout:1/363: creat d2/d3/d1c/d5d/d71/f79 x:0 0 0 2024-03-16T10:44:57.797 INFO:tasks.workunit.client.0.smithi002.stdout:4/410: mkdir d2/d6/d32/d66/d7f 0 2024-03-16T10:44:57.805 INFO:tasks.workunit.client.0.smithi002.stdout:3/368: mkdir d1/dd/d28/d7b/d8a 0 2024-03-16T10:44:57.806 INFO:tasks.workunit.client.0.smithi002.stdout:0/434: dread d3/d28/d2a/f61 [0,4194304] 0 2024-03-16T10:44:57.809 INFO:tasks.workunit.client.0.smithi002.stdout:9/410: creat d5/d9/d55/f7b x:0 0 0 2024-03-16T10:44:57.810 INFO:tasks.workunit.client.0.smithi002.stdout:1/364: symlink d2/df/d12/d29/d5c/l7a 0 2024-03-16T10:44:57.812 INFO:tasks.workunit.client.0.smithi002.stdout:0/435: dwrite d3/d5/f23 [0,4194304] 0 2024-03-16T10:44:57.818 INFO:tasks.workunit.client.0.smithi002.stdout:5/355: rename d0/d16/d1d/l41 to d0/d1f/l6a 0 2024-03-16T10:44:57.823 INFO:tasks.workunit.client.0.smithi002.stdout:6/426: creat d15/f9a x:0 0 0 2024-03-16T10:44:57.824 INFO:tasks.workunit.client.0.smithi002.stdout:1/365: rmdir d2/d3/d1c/d5d 39 2024-03-16T10:44:57.825 INFO:tasks.workunit.client.0.smithi002.stdout:1/366: write d2/d58/f70 [14737,81835] 0 2024-03-16T10:44:57.833 INFO:tasks.workunit.client.0.smithi002.stdout:2/401: rename d7/d2e/d28/d40/f52 to d7/d2e/d28/d44/f76 0 2024-03-16T10:44:57.836 INFO:tasks.workunit.client.0.smithi002.stdout:1/367: stat d2/df/d12/l1b 0 2024-03-16T10:44:57.839 INFO:tasks.workunit.client.0.smithi002.stdout:3/369: link d1/dd/d28/d7b/d2c/l4c d1/dd/d28/d7b/d89/d15/d87/l8b 0 2024-03-16T10:44:57.842 INFO:tasks.workunit.client.0.smithi002.stdout:5/356: mkdir d0/d6b 0 2024-03-16T10:44:57.844 INFO:tasks.workunit.client.0.smithi002.stdout:8/397: rename d7/d9/d17/d20/d82/f85 to d7/d9/d17/d23/d2e/d5c/f8a 0 2024-03-16T10:44:57.845 INFO:tasks.workunit.client.0.smithi002.stdout:2/402: rmdir d7/db/d70 39 2024-03-16T10:44:57.847 INFO:tasks.workunit.client.0.smithi002.stdout:6/427: fsync fd 0 2024-03-16T10:44:57.848 INFO:tasks.workunit.client.0.smithi002.stdout:1/368: fsync d2/d3/f4b 0 2024-03-16T10:44:57.849 INFO:tasks.workunit.client.0.smithi002.stdout:3/370: dwrite d1/dd/d28/f33 [0,4194304] 0 2024-03-16T10:44:57.850 INFO:tasks.workunit.client.0.smithi002.stdout:6/428: readlink d15/l7d 0 2024-03-16T10:44:57.851 INFO:tasks.workunit.client.0.smithi002.stdout:9/411: readlink d5/d9/dc/d1e/d24/d31/l66 0 2024-03-16T10:44:57.853 INFO:tasks.workunit.client.0.smithi002.stdout:2/403: read d7/d2e/d28/d40/d50/f68 [653610,97523] 0 2024-03-16T10:44:57.853 INFO:tasks.workunit.client.0.smithi002.stdout:1/369: chown d2/d3/fe 13566 1 2024-03-16T10:44:57.856 INFO:tasks.workunit.client.0.smithi002.stdout:3/371: read d1/dd/d28/d7b/d22/d3f/f4d [700750,11567] 0 2024-03-16T10:44:57.857 INFO:tasks.workunit.client.0.smithi002.stdout:5/357: dwrite d0/d16/f5f [0,4194304] 0 2024-03-16T10:44:57.858 INFO:tasks.workunit.client.0.smithi002.stdout:0/436: dwrite d3/d28/d65/f6a [0,4194304] 0 2024-03-16T10:44:57.860 INFO:tasks.workunit.client.0.smithi002.stdout:5/358: truncate d0/d16/f4d 823473 0 2024-03-16T10:44:57.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 -- 172.21.15.2:0/2588961511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d8100bd0 msgr2=0x7fc3d8101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2588961511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8101030 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009910 tx=0x7fc3c4009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 -- 172.21.15.2:0/2588961511 shutdown_connections 2024-03-16T10:44:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2588961511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8101030 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2588961511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d80ff9d0 0x7fc3d80ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 -- 172.21.15.2:0/2588961511 >> 172.21.15.2:0/2588961511 conn(0x7fc3d80faf70 msgr2=0x7fc3d80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 -- 172.21.15.2:0/2588961511 shutdown_connections 2024-03-16T10:44:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.859+0000 7fc3ded05700 1 -- 172.21.15.2:0/2588961511 wait complete. 2024-03-16T10:44:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.860+0000 7fc3ded05700 1 Processor -- start 2024-03-16T10:44:57.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.860+0000 7fc3ded05700 1 -- start start 2024-03-16T10:44:57.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.860+0000 7fc3ded05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d80ff9d0 0x7fc3d8194300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:57.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.860+0000 7fc3ded05700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:57.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.861+0000 7fc3ded05700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8194e10 con 0x7fc3d80ff9d0 2024-03-16T10:44:57.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.861+0000 7fc3ded05700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8194f50 con 0x7fc3d8100bd0 2024-03-16T10:44:57.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.861+0000 7fc3d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:57.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.862+0000 7fc3d7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8194840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56850/0 (socket says 172.21.15.2:56850) 2024-03-16T10:44:57.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.862+0000 7fc3d7fff700 1 -- 172.21.15.2:0/2126860921 learned_addr learned my addr 172.21.15.2:0/2126860921 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:57.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.863+0000 7fc3dcaa1700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d80ff9d0 0x7fc3d8194300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:57.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.863+0000 7fc3d7fff700 1 -- 172.21.15.2:0/2126860921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d80ff9d0 msgr2=0x7fc3d8194300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:57.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.863+0000 7fc3d7fff700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d80ff9d0 0x7fc3d8194300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:57.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.863+0000 7fc3d7fff700 1 -- 172.21.15.2:0/2126860921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40095f0 con 0x7fc3d8100bd0 2024-03-16T10:44:57.864 INFO:tasks.workunit.client.0.smithi002.stdout:6/429: dread d15/d2d/f1f [0,4194304] 0 2024-03-16T10:44:57.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.864+0000 7fc3d7fff700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8194840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40092e0 tx=0x7fc3c400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:57.865 INFO:tasks.workunit.client.0.smithi002.stdout:3/372: read d1/dd/f10 [385511,37855] 0 2024-03-16T10:44:57.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.865+0000 7fc3d5ffb700 1 -- 172.21.15.2:0/2126860921 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c401d070 con 0x7fc3d8100bd0 2024-03-16T10:44:57.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.865+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d806a8c0 con 0x7fc3d8100bd0 2024-03-16T10:44:57.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.865+0000 7fc3d5ffb700 1 -- 172.21.15.2:0/2126860921 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3c400f460 con 0x7fc3d8100bd0 2024-03-16T10:44:57.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.865+0000 7fc3d5ffb700 1 -- 172.21.15.2:0/2126860921 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c4021640 con 0x7fc3d8100bd0 2024-03-16T10:44:57.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.865+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d806ad70 con 0x7fc3d8100bd0 2024-03-16T10:44:57.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.867+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8109520 con 0x7fc3d8100bd0 2024-03-16T10:44:57.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.868+0000 7fc3d5ffb700 1 -- 172.21.15.2:0/2126860921 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7fc3c400fa90 con 0x7fc3d8100bd0 2024-03-16T10:44:57.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.869+0000 7fc3d5ffb700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fc3c806d370 0x7fc3c806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:57.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.869+0000 7fc3d5ffb700 1 -- 172.21.15.2:0/2126860921 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc3c408df30 con 0x7fc3d8100bd0 2024-03-16T10:44:57.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.870+0000 7fc3dcaa1700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fc3c806d370 0x7fc3c806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:57.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.871+0000 7fc3dcaa1700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fc3c806d370 0x7fc3c806f830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8100a30 tx=0x7fc3cc00b410 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:57.872 INFO:tasks.workunit.client.0.smithi002.stdout:6/430: dwrite f8 [0,4194304] 0 2024-03-16T10:44:57.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:57.876+0000 7fc3d5ffb700 1 -- 172.21.15.2:0/2126860921 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fc3c405b3f0 con 0x7fc3d8100bd0 2024-03-16T10:44:57.877 INFO:tasks.workunit.client.0.smithi002.stdout:9/412: mknod d5/d9/dc/d1e/c7c 0 2024-03-16T10:44:57.883 INFO:tasks.workunit.client.0.smithi002.stdout:0/437: symlink d3/d28/d2a/d67/l82 0 2024-03-16T10:44:57.886 INFO:tasks.workunit.client.0.smithi002.stdout:9/413: dwrite d5/d9/d22/f6e [0,4194304] 0 2024-03-16T10:44:57.888 INFO:tasks.workunit.client.0.smithi002.stdout:0/438: write d3/de/f20 [2499069,45976] 0 2024-03-16T10:44:57.890 INFO:tasks.workunit.client.0.smithi002.stdout:6/431: dread fc [0,4194304] 0 2024-03-16T10:44:57.892 INFO:tasks.workunit.client.0.smithi002.stdout:2/404: creat d7/db/d70/f77 x:0 0 0 2024-03-16T10:44:57.893 INFO:tasks.workunit.client.0.smithi002.stdout:1/370: mknod d2/d3/d1c/d5d/c7b 0 2024-03-16T10:44:57.893 INFO:tasks.workunit.client.0.smithi002.stdout:6/432: write d15/d2d/d2c/f30 [2349113,67385] 0 2024-03-16T10:44:57.893 INFO:tasks.workunit.client.0.smithi002.stdout:8/398: creat d7/d9/d17/d23/d2e/f8b x:0 0 0 2024-03-16T10:44:57.895 INFO:tasks.workunit.client.0.smithi002.stdout:1/371: stat d2/d3/d1c/d5d/d71 0 2024-03-16T10:44:57.896 INFO:tasks.workunit.client.0.smithi002.stdout:5/359: symlink d0/d16/d1d/d33/d65/l6c 0 2024-03-16T10:44:57.896 INFO:tasks.workunit.client.0.smithi002.stdout:1/372: write d2/d3/f8 [1007384,105276] 0 2024-03-16T10:44:57.898 INFO:tasks.workunit.client.0.smithi002.stdout:3/373: mkdir d1/dd/d28/d7b/d89/d15/d87/d8c 0 2024-03-16T10:44:57.905 INFO:tasks.workunit.client.0.smithi002.stdout:3/374: dwrite d1/dd/d28/d7b/d22/d3f/f51 [0,4194304] 0 2024-03-16T10:44:57.908 INFO:tasks.workunit.client.0.smithi002.stdout:3/375: dread d1/dd/d28/f33 [0,4194304] 0 2024-03-16T10:44:57.909 INFO:tasks.workunit.client.0.smithi002.stdout:8/399: creat d7/d9/d17/d23/d39/f8c x:0 0 0 2024-03-16T10:44:57.913 INFO:tasks.workunit.client.0.smithi002.stdout:1/373: write d2/df/f45 [4826104,73088] 0 2024-03-16T10:44:57.913 INFO:tasks.workunit.client.0.smithi002.stdout:1/374: read - d2/f76 zero size 2024-03-16T10:44:57.923 INFO:tasks.workunit.client.0.smithi002.stdout:2/405: mknod d7/db/d70/d72/c78 0 2024-03-16T10:44:57.927 INFO:tasks.workunit.client.0.smithi002.stdout:3/376: readlink d1/dd/d28/d7b/l29 0 2024-03-16T10:44:57.928 INFO:tasks.workunit.client.0.smithi002.stdout:3/377: stat d1/dd/d28/d7b/l29 0 2024-03-16T10:44:57.928 INFO:tasks.workunit.client.0.smithi002.stdout:3/378: fdatasync d1/fb 0 2024-03-16T10:44:57.934 INFO:tasks.workunit.client.0.smithi002.stdout:0/439: creat d3/de/f83 x:0 0 0 2024-03-16T10:44:57.935 INFO:tasks.workunit.client.0.smithi002.stdout:0/440: chown d3/d28/d3d/l6f 6 1 2024-03-16T10:44:57.939 INFO:tasks.workunit.client.0.smithi002.stdout:2/406: mknod d7/db/d12/c79 0 2024-03-16T10:44:57.940 INFO:tasks.workunit.client.0.smithi002.stdout:6/433: creat d15/d2d/f9b x:0 0 0 2024-03-16T10:44:57.943 INFO:tasks.workunit.client.0.smithi002.stdout:3/379: creat d1/dd/d28/d37/f8d x:0 0 0 2024-03-16T10:44:57.944 INFO:tasks.workunit.client.0.smithi002.stdout:1/375: write d2/df/d12/d29/d5c/f6d [1720899,13098] 0 2024-03-16T10:44:57.945 INFO:tasks.workunit.client.0.smithi002.stdout:5/360: link d0/d16/d1d/c27 d0/d24/c6d 0 2024-03-16T10:44:57.947 INFO:tasks.workunit.client.0.smithi002.stdout:9/414: link d5/d9/d25/d23/c45 d5/d9/dc/d28/c7d 0 2024-03-16T10:44:57.949 INFO:tasks.workunit.client.0.smithi002.stdout:6/434: mknod d15/d2d/d2b/d3f/d77/c9c 0 2024-03-16T10:44:57.955 INFO:tasks.workunit.client.0.smithi002.stdout:0/441: mknod d3/d28/c84 0 2024-03-16T10:44:57.956 INFO:tasks.workunit.client.0.smithi002.stdout:2/407: creat d7/d2e/d28/d40/d6c/f7a x:0 0 0 2024-03-16T10:44:57.957 INFO:tasks.workunit.client.0.smithi002.stdout:2/408: write d7/f5e [239330,36014] 0 2024-03-16T10:44:57.959 INFO:tasks.workunit.client.0.smithi002.stdout:8/400: getdents d7/d9/d17/d23/d39 0 2024-03-16T10:44:57.960 INFO:tasks.workunit.client.0.smithi002.stdout:3/380: getdents d1/dd/d28/d7b/d8a 0 2024-03-16T10:44:57.961 INFO:tasks.workunit.client.0.smithi002.stdout:3/381: chown d1/dd/f5e 84618 1 2024-03-16T10:44:57.963 INFO:tasks.workunit.client.0.smithi002.stdout:5/361: mknod d0/d6b/c6e 0 2024-03-16T10:44:57.965 INFO:tasks.workunit.client.0.smithi002.stdout:2/409: symlink d7/d27/d3b/l7b 0 2024-03-16T10:44:57.966 INFO:tasks.workunit.client.0.smithi002.stdout:8/401: dwrite d7/d9/d17/d23/d39/f72 [0,4194304] 0 2024-03-16T10:44:57.970 INFO:tasks.workunit.client.0.smithi002.stdout:1/376: creat d2/d3/f7c x:0 0 0 2024-03-16T10:44:57.971 INFO:tasks.workunit.client.0.smithi002.stdout:3/382: symlink d1/dd/d28/d7b/d22/d3f/l8e 0 2024-03-16T10:44:57.973 INFO:tasks.workunit.client.0.smithi002.stdout:8/402: dread d7/f60 [0,4194304] 0 2024-03-16T10:44:57.973 INFO:tasks.workunit.client.0.smithi002.stdout:0/442: mknod d3/de/c85 0 2024-03-16T10:44:57.974 INFO:tasks.workunit.client.0.smithi002.stdout:3/383: write d1/dd/d74/f80 [3687953,46347] 0 2024-03-16T10:44:57.975 INFO:tasks.workunit.client.0.smithi002.stdout:0/443: chown d3/d28/d2a 1311291 1 2024-03-16T10:44:57.978 INFO:tasks.workunit.client.0.smithi002.stdout:9/415: truncate d5/d9/dc/d1e/d24/d31/f4a 4113404 0 2024-03-16T10:44:57.983 INFO:tasks.workunit.client.0.smithi002.stdout:8/403: rmdir d7/d9/d17/d23 39 2024-03-16T10:44:57.983 INFO:tasks.workunit.client.0.smithi002.stdout:8/404: read - d7/d13/f68 zero size 2024-03-16T10:44:57.984 INFO:tasks.workunit.client.0.smithi002.stdout:8/405: chown d7/d9/d17/d20/c22 0 1 2024-03-16T10:44:57.985 INFO:tasks.workunit.client.0.smithi002.stdout:3/384: creat d1/f8f x:0 0 0 2024-03-16T10:44:57.987 INFO:tasks.workunit.client.0.smithi002.stdout:3/385: dread - d1/dd/d28/d7b/d22/d3f/f60 zero size 2024-03-16T10:44:57.987 INFO:tasks.workunit.client.0.smithi002.stdout:1/377: dread d2/df/d12/f37 [0,4194304] 0 2024-03-16T10:44:57.988 INFO:tasks.workunit.client.0.smithi002.stdout:0/444: mknod d3/d28/d2a/d67/c86 0 2024-03-16T10:44:57.991 INFO:tasks.workunit.client.0.smithi002.stdout:5/362: creat d0/f6f x:0 0 0 2024-03-16T10:44:57.994 INFO:tasks.workunit.client.0.smithi002.stdout:2/410: creat d7/db/f7c x:0 0 0 2024-03-16T10:44:57.994 INFO:tasks.workunit.client.0.smithi002.stdout:2/411: write f6 [3554658,66137] 0 2024-03-16T10:44:57.996 INFO:tasks.workunit.client.0.smithi002.stdout:9/416: mkdir d5/d7e 0 2024-03-16T10:44:57.997 INFO:tasks.workunit.client.0.smithi002.stdout:0/445: mknod d3/d28/d2a/d67/c87 0 2024-03-16T10:44:58.000 INFO:tasks.workunit.client.0.smithi002.stdout:2/412: creat d7/d2e/d1d/d2f/f7d x:0 0 0 2024-03-16T10:44:58.001 INFO:tasks.workunit.client.0.smithi002.stdout:1/378: dwrite d2/d3/f65 [0,4194304] 0 2024-03-16T10:44:58.002 INFO:tasks.workunit.client.0.smithi002.stdout:2/413: chown d7/d2e/d28/f6a 40659026 1 2024-03-16T10:44:58.004 INFO:tasks.workunit.client.0.smithi002.stdout:8/406: truncate d7/d9/d17/d23/d39/f7a 1179390 0 2024-03-16T10:44:58.004 INFO:tasks.workunit.client.0.smithi002.stdout:9/417: mkdir d5/d9/d22/d3c/d4d/d7f 0 2024-03-16T10:44:58.007 INFO:tasks.workunit.client.0.smithi002.stdout:0/446: dwrite d3/de/f51 [0,4194304] 0 2024-03-16T10:44:58.007 INFO:tasks.workunit.client.0.smithi002.stdout:9/418: dread - d5/d9/dc/d28/f49 zero size 2024-03-16T10:44:58.009 INFO:tasks.workunit.client.0.smithi002.stdout:3/386: truncate d1/dd/d28/d7b/d22/d3f/f47 3141327 0 2024-03-16T10:44:58.010 INFO:tasks.workunit.client.0.smithi002.stdout:3/387: stat d1/dd/d28/d7b/d22/l70 0 2024-03-16T10:44:58.011 INFO:tasks.workunit.client.0.smithi002.stdout:1/379: unlink d2/d3/d51/f75 0 2024-03-16T10:44:58.016 INFO:tasks.workunit.client.0.smithi002.stdout:8/407: dwrite d7/d9/d17/d23/d2e/d34/f79 [0,4194304] 0 2024-03-16T10:44:58.018 INFO:tasks.workunit.client.0.smithi002.stdout:5/363: creat d0/d16/f70 x:0 0 0 2024-03-16T10:44:58.019 INFO:tasks.workunit.client.0.smithi002.stdout:0/447: mkdir d3/d31/d88 0 2024-03-16T10:44:58.021 INFO:tasks.workunit.client.0.smithi002.stdout:5/364: write d0/f6f [905540,33925] 0 2024-03-16T10:44:58.021 INFO:tasks.workunit.client.0.smithi002.stdout:0/448: write d3/f49 [5210333,120731] 0 2024-03-16T10:44:58.025 INFO:tasks.workunit.client.0.smithi002.stdout:2/414: dwrite d7/d2e/d28/d44/f64 [0,4194304] 0 2024-03-16T10:44:58.025 INFO:tasks.workunit.client.0.smithi002.stdout:0/449: truncate d3/f60 314885 0 2024-03-16T10:44:58.028 INFO:tasks.workunit.client.0.smithi002.stdout:3/388: creat d1/dd/d28/d7b/d2c/f90 x:0 0 0 2024-03-16T10:44:58.030 INFO:tasks.workunit.client.0.smithi002.stdout:1/380: creat d2/d3/d51/d56/f7d x:0 0 0 2024-03-16T10:44:58.034 INFO:tasks.workunit.client.0.smithi002.stdout:8/408: creat d7/d9/d17/d23/d2e/d34/f8d x:0 0 0 2024-03-16T10:44:58.035 INFO:tasks.workunit.client.0.smithi002.stdout:9/419: rename d5/d9/d25/l1b to d5/d9/d25/d23/l80 0 2024-03-16T10:44:58.035 INFO:tasks.workunit.client.0.smithi002.stdout:2/415: dread d7/d2e/d28/d44/f64 [0,4194304] 0 2024-03-16T10:44:58.059 INFO:tasks.workunit.client.0.smithi002.stdout:0/450: creat d3/d28/d2a/d67/f89 x:0 0 0 2024-03-16T10:44:58.071 INFO:tasks.workunit.client.0.smithi002.stdout:0/451: dread d3/d31/f4b [0,4194304] 0 2024-03-16T10:44:58.089 INFO:tasks.workunit.client.0.smithi002.stdout:3/389: creat d1/dd/d28/d7b/d22/d3f/f91 x:0 0 0 2024-03-16T10:44:58.090 INFO:tasks.workunit.client.0.smithi002.stdout:8/409: creat d7/d9/d25/d67/f8e x:0 0 0 2024-03-16T10:44:58.091 INFO:tasks.workunit.client.0.smithi002.stdout:9/420: mkdir d5/d9/d25/d23/d81 0 2024-03-16T10:44:58.093 INFO:tasks.workunit.client.0.smithi002.stdout:4/411: sync 2024-03-16T10:44:58.093 INFO:tasks.workunit.client.0.smithi002.stdout:7/345: sync 2024-03-16T10:44:58.094 INFO:tasks.workunit.client.0.smithi002.stdout:5/365: truncate d0/d1f/f2c 2081282 0 2024-03-16T10:44:58.108 INFO:tasks.workunit.client.0.smithi002.stdout:8/410: mkdir d7/d55/d8f 0 2024-03-16T10:44:58.109 INFO:tasks.workunit.client.0.smithi002.stdout:9/421: truncate d5/d9/dc/f16 521360 0 2024-03-16T10:44:58.110 INFO:tasks.workunit.client.0.smithi002.stdout:9/422: write d5/d9/f13 [4454856,96440] 0 2024-03-16T10:44:58.111 INFO:tasks.workunit.client.0.smithi002.stdout:7/346: symlink d3/d4/d1c/d5b/l7b 0 2024-03-16T10:44:58.114 INFO:tasks.workunit.client.0.smithi002.stdout:4/412: rename d2/d6/d32/d68/f5b to d2/d33/d1c/f80 0 2024-03-16T10:44:58.115 INFO:tasks.workunit.client.0.smithi002.stdout:4/413: write d2/d33/d35/f40 [1596796,43938] 0 2024-03-16T10:44:58.116 INFO:tasks.workunit.client.0.smithi002.stdout:4/414: fsync d2/d33/d1c/f6c 0 2024-03-16T10:44:58.116 INFO:tasks.workunit.client.0.smithi002.stdout:4/415: readlink d2/d33/l2b 0 2024-03-16T10:44:58.118 INFO:tasks.workunit.client.0.smithi002.stdout:4/416: read d2/d6/d32/f4e [602860,12448] 0 2024-03-16T10:44:58.119 INFO:tasks.workunit.client.0.smithi002.stdout:3/390: symlink d1/dd/d28/d7b/d73/l92 0 2024-03-16T10:44:58.121 INFO:tasks.workunit.client.0.smithi002.stdout:9/423: mkdir d5/d9/d55/d77/d82 0 2024-03-16T10:44:58.122 INFO:tasks.workunit.client.0.smithi002.stdout:9/424: write d5/f7 [1106588,107848] 0 2024-03-16T10:44:58.135 INFO:tasks.workunit.client.0.smithi002.stdout:3/391: dread d1/dd/d28/d7b/d22/d3f/f47 [0,4194304] 0 2024-03-16T10:44:58.139 INFO:tasks.workunit.client.0.smithi002.stdout:3/392: write d1/dd/d28/d7b/d2c/f6d [399398,99012] 0 2024-03-16T10:44:58.156 INFO:tasks.workunit.client.0.smithi002.stdout:1/381: creat d2/d3/f7e x:0 0 0 2024-03-16T10:44:58.156 INFO:tasks.workunit.client.0.smithi002.stdout:1/382: chown d2/d3/l2e 933563 1 2024-03-16T10:44:58.161 INFO:tasks.workunit.client.0.smithi002.stdout:1/383: dwrite d2/df/d12/f37 [0,4194304] 0 2024-03-16T10:44:58.167 INFO:tasks.workunit.client.0.smithi002.stdout:1/384: link d2/d3/d51/d56/f7d d2/d3/d1c/f7f 0 2024-03-16T10:44:58.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.169+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3d80611f0 con 0x7fc3c806d370 2024-03-16T10:44:58.172 INFO:tasks.workunit.client.0.smithi002.stdout:8/411: mkdir d7/d9/d17/d23/d2e/d90 0 2024-03-16T10:44:58.173 INFO:tasks.workunit.client.0.smithi002.stdout:8/412: chown d7/d9/f75 0 1 2024-03-16T10:44:58.173 INFO:tasks.workunit.client.0.smithi002.stdout:1/385: write d2/df/d12/d29/f34 [2815581,118423] 0 2024-03-16T10:44:58.174 INFO:tasks.workunit.client.0.smithi002.stdout:1/386: chown d2/d3/d51/d56/c5b 7847908 1 2024-03-16T10:44:58.177 INFO:tasks.workunit.client.0.smithi002.stdout:8/413: dread d7/d9/d17/d23/d39/f72 [0,4194304] 0 2024-03-16T10:44:58.178 INFO:tasks.workunit.client.0.smithi002.stdout:8/414: chown d7/d9/d17/d23/d2e/d34 43419 1 2024-03-16T10:44:58.179 INFO:tasks.workunit.client.0.smithi002.stdout:1/387: fdatasync d2/d3/d51/f5a 0 2024-03-16T10:44:58.180 INFO:tasks.workunit.client.0.smithi002.stdout:1/388: read d2/df/d12/d29/f34 [1638628,112786] 0 2024-03-16T10:44:58.182 INFO:tasks.workunit.client.0.smithi002.stdout:8/415: fdatasync d7/d9/d17/d20/f5e 0 2024-03-16T10:44:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.184+0000 7fc3d5ffb700 1 -- 172.21.15.2:0/2126860921 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc3d80611f0 con 0x7fc3c806d370 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (7m) 47s ago 8m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (8m) 47s ago 8m 7583k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (7m) 41s ago 7m 7596k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (8m) 47s ago 8m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (7m) 41s ago 7m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (7m) 47s ago 8m 86.9M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (2m) 47s ago 2m 15.9M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (3m) 47s ago 3m 12.8M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (3m) 41s ago 3m 63.7M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:44:58.185 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (2m) 41s ago 2m 14.6M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 starting - - - - 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (69s) 41s ago 7m 492M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5df99c885681 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (10m) 47s ago 10m 51.5M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (7m) 41s ago 7m 39.1M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (8m) 47s ago 8m 17.0M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (7m) 41s ago 7m 15.5M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (6m) 47s ago 6m 56.3M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (6m) 47s ago 6m 58.7M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (5m) 47s ago 5m 56.0M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (5m) 41s ago 5m 60.3M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (4m) 41s ago 4m 55.9M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (4m) 41s ago 4m 57.2M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:44:58.186 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 starting - - - - 2024-03-16T10:44:58.187 INFO:tasks.workunit.client.0.smithi002.stdout:1/389: dwrite d2/d3/f7c [0,4194304] 0 2024-03-16T10:44:58.187 INFO:tasks.workunit.client.0.smithi002.stdout:1/390: stat d2/df/d12/c1e 0 2024-03-16T10:44:58.187 INFO:tasks.workunit.client.0.smithi002.stdout:2/416: link d7/d2e/d1d/f4f d7/d2e/f7e 0 2024-03-16T10:44:58.188 INFO:tasks.workunit.client.0.smithi002.stdout:1/391: dread - d2/d3/d1c/f7f zero size 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fc3c806d370 msgr2=0x7fc3c806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fc3c806d370 0x7fc3c806f830 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8100a30 tx=0x7fc3cc00b410 comp rx=0 tx=0).stop 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d8100bd0 msgr2=0x7fc3d8194840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8194840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40092e0 tx=0x7fc3c400bd70 comp rx=0 tx=0).stop 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 shutdown_connections 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fc3c806d370 0x7fc3c806f830 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc3d80ff9d0 0x7fc3d8194300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 --2- 172.21.15.2:0/2126860921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3d8100bd0 0x7fc3d8194840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 >> 172.21.15.2:0/2126860921 conn(0x7fc3d80faf70 msgr2=0x7fc3d8104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.188+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 shutdown_connections 2024-03-16T10:44:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.189+0000 7fc3ded05700 1 -- 172.21.15.2:0/2126860921 wait complete. 2024-03-16T10:44:58.197 INFO:tasks.workunit.client.0.smithi002.stdout:2/417: mkdir d7/d27/d3f/d7f 0 2024-03-16T10:44:58.199 INFO:tasks.workunit.client.0.smithi002.stdout:1/392: dwrite d2/df/d12/f37 [4194304,4194304] 0 2024-03-16T10:44:58.201 INFO:tasks.workunit.client.0.smithi002.stdout:1/393: write d2/d3/d51/f5a [2408682,117183] 0 2024-03-16T10:44:58.208 INFO:tasks.workunit.client.0.smithi002.stdout:1/394: write d2/f13 [7788868,110910] 0 2024-03-16T10:44:58.215 INFO:tasks.workunit.client.0.smithi002.stdout:7/347: rename d3/d4/d1c/d44/d5c/d63/d3f/c4e to d3/c7c 0 2024-03-16T10:44:58.219 INFO:tasks.workunit.client.0.smithi002.stdout:3/393: rename d1/dd/d28/d7b/d22/d3f to d1/dd/d28/d7b/d8a/d93 0 2024-03-16T10:44:58.220 INFO:tasks.workunit.client.0.smithi002.stdout:6/435: sync 2024-03-16T10:44:58.220 INFO:tasks.workunit.client.0.smithi002.stdout:1/395: dread d2/d3/fc [0,4194304] 0 2024-03-16T10:44:58.224 INFO:tasks.workunit.client.0.smithi002.stdout:8/416: rename d7/d9/d17/d20/d82 to d7/d9/d25/d91 0 2024-03-16T10:44:58.228 INFO:tasks.workunit.client.0.smithi002.stdout:7/348: link d3/d4/d1c/d44/d5c/d63/d55/f77 d3/d61/f7d 0 2024-03-16T10:44:58.232 INFO:tasks.workunit.client.0.smithi002.stdout:1/396: mknod d2/d3/d1c/c80 0 2024-03-16T10:44:58.233 INFO:tasks.workunit.client.0.smithi002.stdout:1/397: write d2/f76 [721018,70153] 0 2024-03-16T10:44:58.240 INFO:tasks.workunit.client.0.smithi002.stdout:8/417: rename d7/d9/d25/f45 to d7/d9/d17/d23/d2e/f92 0 2024-03-16T10:44:58.243 INFO:tasks.workunit.client.0.smithi002.stdout:7/349: link d3/d4/d1c/d44/d5c/d63/f3b d3/d4/d1c/d44/f7e 0 2024-03-16T10:44:58.244 INFO:tasks.workunit.client.0.smithi002.stdout:1/398: symlink d2/d3/d1c/l81 0 2024-03-16T10:44:58.248 INFO:tasks.workunit.client.0.smithi002.stdout:5/366: dwrite d0/d1f/f2c [0,4194304] 0 2024-03-16T10:44:58.249 INFO:tasks.workunit.client.0.smithi002.stdout:8/418: symlink d7/d9/d17/d23/d2e/d90/l93 0 2024-03-16T10:44:58.254 INFO:tasks.workunit.client.0.smithi002.stdout:7/350: unlink d3/d4/d1d/l79 0 2024-03-16T10:44:58.259 INFO:tasks.workunit.client.0.smithi002.stdout:1/399: truncate d2/df/d12/f72 2467720 0 2024-03-16T10:44:58.260 INFO:tasks.workunit.client.0.smithi002.stdout:8/419: write d7/d13/f4c [361445,25532] 0 2024-03-16T10:44:58.263 INFO:tasks.workunit.client.0.smithi002.stdout:1/400: creat d2/d3/d1c/d5d/f82 x:0 0 0 2024-03-16T10:44:58.264 INFO:tasks.workunit.client.0.smithi002.stdout:7/351: creat d3/d4/d1c/d44/d5c/d63/f7f x:0 0 0 2024-03-16T10:44:58.266 INFO:tasks.workunit.client.0.smithi002.stdout:5/367: dread d0/d16/d1d/d2b/f4f [0,4194304] 0 2024-03-16T10:44:58.268 INFO:tasks.workunit.client.0.smithi002.stdout:1/401: dread d2/d3/f8 [0,4194304] 0 2024-03-16T10:44:58.272 INFO:tasks.workunit.client.0.smithi002.stdout:5/368: fsync d0/d16/d4a/d39/f53 0 2024-03-16T10:44:58.275 INFO:tasks.workunit.client.0.smithi002.stdout:7/352: symlink d3/d4/d1c/d44/d5c/d63/l80 0 2024-03-16T10:44:58.275 INFO:tasks.workunit.client.0.smithi002.stdout:7/353: readlink d3/l20 0 2024-03-16T10:44:58.276 INFO:tasks.workunit.client.0.smithi002.stdout:2/418: write d7/d2e/d28/d44/f76 [2319952,42311] 0 2024-03-16T10:44:58.278 INFO:tasks.workunit.client.0.smithi002.stdout:2/419: chown d7/d2e/d1d/d2f/f7d 60103 1 2024-03-16T10:44:58.280 INFO:tasks.workunit.client.0.smithi002.stdout:1/402: truncate d2/d3/f39 3188676 0 2024-03-16T10:44:58.281 INFO:tasks.workunit.client.0.smithi002.stdout:1/403: truncate d2/f68 807123 0 2024-03-16T10:44:58.283 INFO:tasks.workunit.client.0.smithi002.stdout:7/354: creat d3/d4/d1c/f81 x:0 0 0 2024-03-16T10:44:58.283 INFO:tasks.workunit.client.0.smithi002.stdout:1/404: chown d2/df/d12/c18 244 1 2024-03-16T10:44:58.285 INFO:tasks.workunit.client.0.smithi002.stdout:7/355: fsync d3/f57 0 2024-03-16T10:44:58.286 INFO:tasks.workunit.client.0.smithi002.stdout:2/420: dwrite d7/db/f35 [0,4194304] 0 2024-03-16T10:44:58.287 INFO:tasks.workunit.client.0.smithi002.stdout:5/369: getdents d0/d16/d1d/d43 0 2024-03-16T10:44:58.290 INFO:tasks.workunit.client.0.smithi002.stdout:5/370: fdatasync d0/d16/d4a/d39/f51 0 2024-03-16T10:44:58.293 INFO:tasks.workunit.client.0.smithi002.stdout:3/394: dwrite d1/dd/f10 [0,4194304] 0 2024-03-16T10:44:58.293 INFO:tasks.workunit.client.0.smithi002.stdout:2/421: creat d7/d27/f80 x:0 0 0 2024-03-16T10:44:58.296 INFO:tasks.workunit.client.0.smithi002.stdout:7/356: creat d3/d4/d1d/d45/f82 x:0 0 0 2024-03-16T10:44:58.297 INFO:tasks.workunit.client.0.smithi002.stdout:1/405: dwrite d2/d3/f7c [0,4194304] 0 2024-03-16T10:44:58.297 INFO:tasks.workunit.client.0.smithi002.stdout:3/395: write d1/dd/d28/f6a [104827,56105] 0 2024-03-16T10:44:58.300 INFO:tasks.workunit.client.0.smithi002.stdout:7/357: dread - d3/d61/f7d zero size 2024-03-16T10:44:58.302 INFO:tasks.workunit.client.0.smithi002.stdout:2/422: symlink d7/d2e/d1d/d2f/l81 0 2024-03-16T10:44:58.303 INFO:tasks.workunit.client.0.smithi002.stdout:2/423: fsync d7/db/f48 0 2024-03-16T10:44:58.305 INFO:tasks.workunit.client.0.smithi002.stdout:5/371: unlink d0/d16/d4a/d39/f59 0 2024-03-16T10:44:58.307 INFO:tasks.workunit.client.0.smithi002.stdout:2/424: write d7/db/f7c [578346,126410] 0 2024-03-16T10:44:58.312 INFO:tasks.workunit.client.0.smithi002.stdout:7/358: chown d3/d4/d1d/l68 0 1 2024-03-16T10:44:58.313 INFO:tasks.workunit.client.0.smithi002.stdout:3/396: dwrite d1/f1d [0,4194304] 0 2024-03-16T10:44:58.315 INFO:tasks.workunit.client.0.smithi002.stdout:7/359: dread - d3/d4/d1c/d44/d5c/d63/f7f zero size 2024-03-16T10:44:58.316 INFO:tasks.workunit.client.0.smithi002.stdout:2/425: symlink d7/d2e/d1d/d2f/l82 0 2024-03-16T10:44:58.318 INFO:tasks.workunit.client.0.smithi002.stdout:8/420: write d7/d9/d17/d23/d39/f7a [2188330,26446] 0 2024-03-16T10:44:58.319 INFO:tasks.workunit.client.0.smithi002.stdout:5/372: symlink d0/d1f/l71 0 2024-03-16T10:44:58.323 INFO:tasks.workunit.client.0.smithi002.stdout:8/421: write d7/d55/f7c [309946,77416] 0 2024-03-16T10:44:58.327 INFO:tasks.workunit.client.0.smithi002.stdout:7/360: link d3/f14 d3/d61/f83 0 2024-03-16T10:44:58.333 INFO:tasks.workunit.client.0.smithi002.stdout:0/452: sync 2024-03-16T10:44:58.334 INFO:tasks.workunit.client.0.smithi002.stdout:5/373: dread d0/d1f/f36 [0,4194304] 0 2024-03-16T10:44:58.338 INFO:tasks.workunit.client.0.smithi002.stdout:7/361: write d3/f52 [526914,38889] 0 2024-03-16T10:44:58.339 INFO:tasks.workunit.client.0.smithi002.stdout:0/453: creat d3/d5/d7c/f8a x:0 0 0 2024-03-16T10:44:58.341 INFO:tasks.workunit.client.0.smithi002.stdout:2/426: dread d7/f2b [0,4194304] 0 2024-03-16T10:44:58.344 INFO:tasks.workunit.client.0.smithi002.stdout:3/397: dwrite d1/dd/d28/d7b/d89/d15/f17 [0,4194304] 0 2024-03-16T10:44:58.344 INFO:tasks.workunit.client.0.smithi002.stdout:0/454: fdatasync d3/d5/f72 0 2024-03-16T10:44:58.344 INFO:tasks.workunit.client.0.smithi002.stdout:1/406: dwrite d2/d3/fe [0,4194304] 0 2024-03-16T10:44:58.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.344+0000 7fce53fff700 1 -- 172.21.15.2:0/1098271803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce5800bb10 con 0x7fce5c0ff160 2024-03-16T10:44:58.345 INFO:tasks.workunit.client.0.smithi002.stdout:2/427: fsync d7/f5e 0 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.344+0000 7fce635a6700 1 -- 172.21.15.2:0/1098271803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c0ff160 msgr2=0x7fce5c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.344+0000 7fce635a6700 1 --2- 172.21.15.2:0/1098271803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c0ff160 0x7fce5c0ff580 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fce58009870 tx=0x7fce58009b80 comp rx=0 tx=0).stop 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.345+0000 7fce635a6700 1 -- 172.21.15.2:0/1098271803 shutdown_connections 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.345+0000 7fce635a6700 1 --2- 172.21.15.2:0/1098271803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c100360 0x7fce5c1007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.345+0000 7fce635a6700 1 --2- 172.21.15.2:0/1098271803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c0ff160 0x7fce5c0ff580 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.345+0000 7fce635a6700 1 -- 172.21.15.2:0/1098271803 >> 172.21.15.2:0/1098271803 conn(0x7fce5c0fa700 msgr2=0x7fce5c0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.345+0000 7fce635a6700 1 -- 172.21.15.2:0/1098271803 shutdown_connections 2024-03-16T10:44:58.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.345+0000 7fce635a6700 1 -- 172.21.15.2:0/1098271803 wait complete. 2024-03-16T10:44:58.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.347+0000 7fce635a6700 1 Processor -- start 2024-03-16T10:44:58.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.347+0000 7fce635a6700 1 -- start start 2024-03-16T10:44:58.348 INFO:tasks.workunit.client.0.smithi002.stdout:7/362: creat d3/d4/d1d/d3d/d5e/f84 x:0 0 0 2024-03-16T10:44:58.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.348+0000 7fce635a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0ff160 0x7fce5c195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:58.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.348+0000 7fce635a6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c100360 0x7fce5c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:58.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.349+0000 7fce635a6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c195fb0 con 0x7fce5c100360 2024-03-16T10:44:58.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.349+0000 7fce635a6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c1960f0 con 0x7fce5c0ff160 2024-03-16T10:44:58.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.349+0000 7fce61342700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0ff160 0x7fce5c195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:58.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.349+0000 7fce61342700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0ff160 0x7fce5c195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56870/0 (socket says 172.21.15.2:56870) 2024-03-16T10:44:58.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.349+0000 7fce61342700 1 -- 172.21.15.2:0/3191199488 learned_addr learned my addr 172.21.15.2:0/3191199488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:58.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.349+0000 7fce60b41700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c100360 0x7fce5c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:58.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.350+0000 7fce60b41700 1 -- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0ff160 msgr2=0x7fce5c195450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.350+0000 7fce60b41700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0ff160 0x7fce5c195450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.350+0000 7fce60b41700 1 -- 172.21.15.2:0/3191199488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0095f0 con 0x7fce5c100360 2024-03-16T10:44:58.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.350+0000 7fce60b41700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c100360 0x7fce5c195990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00eb10 tx=0x7fce4c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:58.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.350+0000 7fce527fc700 1 -- 172.21.15.2:0/3191199488 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce4c00ccc0 con 0x7fce5c100360 2024-03-16T10:44:58.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.350+0000 7fce527fc700 1 -- 172.21.15.2:0/3191199488 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce4c00ce20 con 0x7fce5c100360 2024-03-16T10:44:58.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.350+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce58009520 con 0x7fce5c100360 2024-03-16T10:44:58.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.351+0000 7fce527fc700 1 -- 172.21.15.2:0/3191199488 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce4c003ea0 con 0x7fce5c100360 2024-03-16T10:44:58.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.351+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c108c70 con 0x7fce5c100360 2024-03-16T10:44:58.352 INFO:tasks.workunit.client.0.smithi002.stdout:2/428: chown d7/d2e/d28/d44 1103 1 2024-03-16T10:44:58.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.353+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c1086c0 con 0x7fce5c100360 2024-03-16T10:44:58.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.354+0000 7fce527fc700 1 -- 172.21.15.2:0/3191199488 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7fce4c0038a0 con 0x7fce5c100360 2024-03-16T10:44:58.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.354+0000 7fce527fc700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fce4806d390 0x7fce4806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:58.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.355+0000 7fce527fc700 1 -- 172.21.15.2:0/3191199488 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fce4c014070 con 0x7fce5c100360 2024-03-16T10:44:58.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.355+0000 7fce61342700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fce4806d390 0x7fce4806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:58.356 INFO:tasks.workunit.client.0.smithi002.stdout:5/374: link d0/d16/d1d/c52 d0/d16/d1d/d2b/c72 0 2024-03-16T10:44:58.356 INFO:tasks.workunit.client.0.smithi002.stdout:0/455: fdatasync d3/d28/d3d/d44/f53 0 2024-03-16T10:44:58.357 INFO:tasks.workunit.client.0.smithi002.stdout:3/398: dwrite d1/dd/d28/d37/f8d [0,4194304] 0 2024-03-16T10:44:58.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.357+0000 7fce61342700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fce4806d390 0x7fce4806f850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce58009f50 tx=0x7fce5801b040 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:58.358 INFO:tasks.workunit.client.0.smithi002.stdout:0/456: fsync d3/de/f83 0 2024-03-16T10:44:58.359 INFO:tasks.workunit.client.0.smithi002.stdout:7/363: rmdir d3/d4/d1d/d3d 39 2024-03-16T10:44:58.361 INFO:tasks.workunit.client.0.smithi002.stdout:7/364: chown d3/d4/d1c/d5b/d64/l6f 1 1 2024-03-16T10:44:58.361 INFO:tasks.workunit.client.0.smithi002.stdout:1/407: fsync d2/d3/d1c/f7f 0 2024-03-16T10:44:58.363 INFO:tasks.workunit.client.0.smithi002.stdout:2/429: rmdir d7/db/d70/d72 39 2024-03-16T10:44:58.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.363+0000 7fce527fc700 1 -- 172.21.15.2:0/3191199488 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fce4c05a5e0 con 0x7fce5c100360 2024-03-16T10:44:58.368 INFO:tasks.workunit.client.0.smithi002.stdout:0/457: chown d3/d5/c25 0 1 2024-03-16T10:44:58.369 INFO:tasks.workunit.client.0.smithi002.stdout:7/365: dwrite d3/d61/d66/f4f [4194304,4194304] 0 2024-03-16T10:44:58.370 INFO:tasks.workunit.client.0.smithi002.stdout:5/375: mkdir d0/d16/d1d/d2b/d5d/d73 0 2024-03-16T10:44:58.373 INFO:tasks.workunit.client.0.smithi002.stdout:0/458: write d3/d28/f3e [104523,53544] 0 2024-03-16T10:44:58.374 INFO:tasks.workunit.client.0.smithi002.stdout:1/408: unlink d2/d3/d51/c57 0 2024-03-16T10:44:58.375 INFO:tasks.workunit.client.0.smithi002.stdout:2/430: creat d7/d2e/d1d/d2f/f83 x:0 0 0 2024-03-16T10:44:58.375 INFO:tasks.workunit.client.0.smithi002.stdout:3/399: mknod d1/dd/d28/d7b/d89/d15/d87/d8c/c94 0 2024-03-16T10:44:58.376 INFO:tasks.workunit.client.0.smithi002.stdout:7/366: mkdir d3/d61/d85 0 2024-03-16T10:44:58.377 INFO:tasks.workunit.client.0.smithi002.stdout:3/400: creat d1/d5a/f95 x:0 0 0 2024-03-16T10:44:58.381 INFO:tasks.workunit.client.0.smithi002.stdout:0/459: write d3/d28/d2a/d67/f89 [677260,13415] 0 2024-03-16T10:44:58.383 INFO:tasks.workunit.client.0.smithi002.stdout:2/431: write d7/d2e/f67 [405186,4109] 0 2024-03-16T10:44:58.384 INFO:tasks.workunit.client.0.smithi002.stdout:0/460: dread - d3/f2d zero size 2024-03-16T10:44:58.385 INFO:tasks.workunit.client.0.smithi002.stdout:1/409: creat d2/df/d12/d29/d5c/f83 x:0 0 0 2024-03-16T10:44:58.391 INFO:tasks.workunit.client.0.smithi002.stdout:0/461: creat d3/d31/d88/f8b x:0 0 0 2024-03-16T10:44:58.391 INFO:tasks.workunit.client.0.smithi002.stdout:5/376: dwrite d0/d1f/f36 [0,4194304] 0 2024-03-16T10:44:58.394 INFO:tasks.workunit.client.0.smithi002.stdout:1/410: mkdir d2/df/d12/d84 0 2024-03-16T10:44:58.400 INFO:tasks.workunit.client.0.smithi002.stdout:1/411: read d2/df/d12/f37 [3567777,10337] 0 2024-03-16T10:44:58.401 INFO:tasks.workunit.client.0.smithi002.stdout:0/462: creat d3/d31/d75/f8c x:0 0 0 2024-03-16T10:44:58.406 INFO:tasks.workunit.client.0.smithi002.stdout:7/367: dwrite d3/d4/d1c/d44/d5c/d63/d3f/f41 [0,4194304] 0 2024-03-16T10:44:58.408 INFO:tasks.workunit.client.0.smithi002.stdout:2/432: dwrite d7/d2e/d28/d44/f76 [0,4194304] 0 2024-03-16T10:44:58.408 INFO:tasks.workunit.client.0.smithi002.stdout:3/401: dwrite d1/dd/d28/d7b/d25/f26 [0,4194304] 0 2024-03-16T10:44:58.410 INFO:tasks.workunit.client.0.smithi002.stdout:9/425: sync 2024-03-16T10:44:58.411 INFO:tasks.workunit.client.0.smithi002.stdout:4/417: sync 2024-03-16T10:44:58.411 INFO:tasks.workunit.client.0.smithi002.stdout:6/436: sync 2024-03-16T10:44:58.414 INFO:tasks.workunit.client.0.smithi002.stdout:3/402: read - d1/dd/d28/d7b/f7c zero size 2024-03-16T10:44:58.417 INFO:tasks.workunit.client.0.smithi002.stdout:7/368: chown d3/d4/d1c/d44/f7e 1 1 2024-03-16T10:44:58.424 INFO:tasks.workunit.client.0.smithi002.stdout:4/418: write d2/d6/f2d [1007120,104539] 0 2024-03-16T10:44:58.426 INFO:tasks.workunit.client.0.smithi002.stdout:7/369: chown d3/d4/d1d/d45/f82 495 1 2024-03-16T10:44:58.428 INFO:tasks.workunit.client.0.smithi002.stdout:5/377: dwrite d0/d16/f5f [4194304,4194304] 0 2024-03-16T10:44:58.429 INFO:tasks.workunit.client.0.smithi002.stdout:2/433: getdents d7/d2e/d1d/d2f/d4b 0 2024-03-16T10:44:58.431 INFO:tasks.workunit.client.0.smithi002.stdout:1/412: truncate d2/d3/f1f 3028767 0 2024-03-16T10:44:58.433 INFO:tasks.workunit.client.0.smithi002.stdout:3/403: dread d1/dd/d28/d37/f49 [0,4194304] 0 2024-03-16T10:44:58.434 INFO:tasks.workunit.client.0.smithi002.stdout:4/419: mknod d2/d33/da/d2e/c81 0 2024-03-16T10:44:58.434 INFO:tasks.workunit.client.0.smithi002.stdout:0/463: dwrite d3/d27/f55 [0,4194304] 0 2024-03-16T10:44:58.441 INFO:tasks.workunit.client.0.smithi002.stdout:2/434: mknod d7/d2e/d1d/d2f/c84 0 2024-03-16T10:44:58.444 INFO:tasks.workunit.client.0.smithi002.stdout:0/464: creat d3/d28/d2a/d67/f8d x:0 0 0 2024-03-16T10:44:58.446 INFO:tasks.workunit.client.0.smithi002.stdout:7/370: mkdir d3/d4/d1c/d44/d5c/d63/d3f/d75/d86 0 2024-03-16T10:44:58.448 INFO:tasks.workunit.client.0.smithi002.stdout:1/413: fdatasync d2/df/d12/d29/f34 0 2024-03-16T10:44:58.449 INFO:tasks.workunit.client.0.smithi002.stdout:0/465: truncate d3/d28/d65/f7a 1025098 0 2024-03-16T10:44:58.451 INFO:tasks.workunit.client.0.smithi002.stdout:5/378: rename d0/d16/d1d/c27 to d0/d16/d1d/d2b/d5d/d73/c74 0 2024-03-16T10:44:58.452 INFO:tasks.workunit.client.0.smithi002.stdout:3/404: write d1/dd/d28/d7b/f7c [1550,121705] 0 2024-03-16T10:44:58.456 INFO:tasks.workunit.client.0.smithi002.stdout:7/371: dread d3/f27 [0,4194304] 0 2024-03-16T10:44:58.456 INFO:tasks.workunit.client.0.smithi002.stdout:1/414: truncate d2/d3/f7e 336387 0 2024-03-16T10:44:58.460 INFO:tasks.workunit.client.0.smithi002.stdout:0/466: creat d3/d28/f8e x:0 0 0 2024-03-16T10:44:58.462 INFO:tasks.workunit.client.0.smithi002.stdout:2/435: dwrite d7/d2e/d1d/f41 [0,4194304] 0 2024-03-16T10:44:58.462 INFO:tasks.workunit.client.0.smithi002.stdout:7/372: chown d3/d37/d6a/f72 3 1 2024-03-16T10:44:58.462 INFO:tasks.workunit.client.0.smithi002.stdout:0/467: stat d3/d28/d3d 0 2024-03-16T10:44:58.463 INFO:tasks.workunit.client.0.smithi002.stdout:1/415: mknod d2/df/d12/d84/c85 0 2024-03-16T10:44:58.464 INFO:tasks.workunit.client.0.smithi002.stdout:9/426: dread d5/d9/d25/f71 [0,4194304] 0 2024-03-16T10:44:58.466 INFO:tasks.workunit.client.0.smithi002.stdout:4/420: dread d2/d33/d35/f63 [0,4194304] 0 2024-03-16T10:44:58.466 INFO:tasks.workunit.client.0.smithi002.stdout:3/405: unlink d1/dd/d28/d37/f49 0 2024-03-16T10:44:58.468 INFO:tasks.workunit.client.0.smithi002.stdout:9/427: write d5/f29 [3895478,65552] 0 2024-03-16T10:44:58.470 INFO:tasks.workunit.client.0.smithi002.stdout:9/428: read - d5/d9/dc/d1e/f6c zero size 2024-03-16T10:44:58.470 INFO:tasks.workunit.client.0.smithi002.stdout:2/436: read d7/d2e/d28/d44/f64 [2974800,101570] 0 2024-03-16T10:44:58.471 INFO:tasks.workunit.client.0.smithi002.stdout:1/416: rmdir d2/df/d12/d29 39 2024-03-16T10:44:58.472 INFO:tasks.workunit.client.0.smithi002.stdout:7/373: rmdir d3/d4/d1c/d44/d5c/d63/d3f 39 2024-03-16T10:44:58.473 INFO:tasks.workunit.client.0.smithi002.stdout:1/417: readlink d2/d3/l2d 0 2024-03-16T10:44:58.474 INFO:tasks.workunit.client.0.smithi002.stdout:4/421: creat d2/d6/d32/d66/f82 x:0 0 0 2024-03-16T10:44:58.475 INFO:tasks.workunit.client.0.smithi002.stdout:2/437: read d7/f5e [24158,118769] 0 2024-03-16T10:44:58.477 INFO:tasks.workunit.client.0.smithi002.stdout:0/468: getdents d3/d31 0 2024-03-16T10:44:58.479 INFO:tasks.workunit.client.0.smithi002.stdout:2/438: creat d7/db/d12/f85 x:0 0 0 2024-03-16T10:44:58.479 INFO:tasks.workunit.client.0.smithi002.stdout:5/379: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:58.480 INFO:tasks.workunit.client.0.smithi002.stdout:1/418: mkdir d2/df/d12/d86 0 2024-03-16T10:44:58.480 INFO:tasks.workunit.client.0.smithi002.stdout:3/406: dwrite d1/dd/f5e [0,4194304] 0 2024-03-16T10:44:58.483 INFO:tasks.workunit.client.0.smithi002.stdout:2/439: write d7/db/d12/f5d [4230087,28507] 0 2024-03-16T10:44:58.483 INFO:tasks.workunit.client.0.smithi002.stdout:3/407: write d1/dd/d28/d37/f8d [5189132,42295] 0 2024-03-16T10:44:58.484 INFO:tasks.workunit.client.0.smithi002.stdout:9/429: truncate d5/d9/dc/d28/f4c 1095546 0 2024-03-16T10:44:58.485 INFO:tasks.workunit.client.0.smithi002.stdout:9/430: chown d5/d9/dc/d1e/f6b 77 1 2024-03-16T10:44:58.489 INFO:tasks.workunit.client.0.smithi002.stdout:1/419: mkdir d2/df/d12/d84/d87 0 2024-03-16T10:44:58.493 INFO:tasks.workunit.client.0.smithi002.stdout:1/420: dread d2/d3/d51/f5a [4194304,4194304] 0 2024-03-16T10:44:58.495 INFO:tasks.workunit.client.0.smithi002.stdout:0/469: dread d3/f3a [0,4194304] 0 2024-03-16T10:44:58.496 INFO:tasks.workunit.client.0.smithi002.stdout:5/380: mknod d0/d16/d1d/d2b/c75 0 2024-03-16T10:44:58.498 INFO:tasks.workunit.client.0.smithi002.stdout:3/408: mknod d1/dd/d28/d7b/c96 0 2024-03-16T10:44:58.502 INFO:tasks.workunit.client.0.smithi002.stdout:0/470: rmdir d3/d5 39 2024-03-16T10:44:58.503 INFO:tasks.workunit.client.0.smithi002.stdout:8/422: sync 2024-03-16T10:44:58.505 INFO:tasks.workunit.client.0.smithi002.stdout:5/381: mkdir d0/d16/d1d/d2b/d5d/d73/d76 0 2024-03-16T10:44:58.506 INFO:tasks.workunit.client.0.smithi002.stdout:5/382: truncate d0/d16/f63 98334 0 2024-03-16T10:44:58.508 INFO:tasks.workunit.client.0.smithi002.stdout:3/409: mkdir d1/dd/d28/d7b/d2c/d97 0 2024-03-16T10:44:58.509 INFO:tasks.workunit.client.0.smithi002.stdout:1/421: mknod d2/df/d12/d86/c88 0 2024-03-16T10:44:58.511 INFO:tasks.workunit.client.0.smithi002.stdout:2/440: getdents d7/d27/d3f 0 2024-03-16T10:44:58.512 INFO:tasks.workunit.client.0.smithi002.stdout:5/383: write d0/f57 [3405848,96678] 0 2024-03-16T10:44:58.514 INFO:tasks.workunit.client.0.smithi002.stdout:9/431: getdents d5/d9/dc/d1e/d24 0 2024-03-16T10:44:58.516 INFO:tasks.workunit.client.0.smithi002.stdout:1/422: dread d2/d3/f65 [0,4194304] 0 2024-03-16T10:44:58.516 INFO:tasks.workunit.client.0.smithi002.stdout:2/441: mknod d7/d2e/d28/d40/d6c/c86 0 2024-03-16T10:44:58.517 INFO:tasks.workunit.client.0.smithi002.stdout:9/432: mkdir d5/d9/d55/d77/d83 0 2024-03-16T10:44:58.518 INFO:tasks.workunit.client.0.smithi002.stdout:0/471: creat d3/de/f8f x:0 0 0 2024-03-16T10:44:58.520 INFO:tasks.workunit.client.0.smithi002.stdout:8/423: rename d7/d9/d17/d23/d2e/d34/f8d to d7/d9/f94 0 2024-03-16T10:44:58.520 INFO:tasks.workunit.client.0.smithi002.stdout:3/410: link d1/dd/d28/d7b/d89/d15/l21 d1/dd/d28/d7b/d8a/l98 0 2024-03-16T10:44:58.522 INFO:tasks.workunit.client.0.smithi002.stdout:9/433: creat d5/d9/d55/d77/f84 x:0 0 0 2024-03-16T10:44:58.523 INFO:tasks.workunit.client.0.smithi002.stdout:1/423: rename d2/df/d12/c1e to d2/d58/c89 0 2024-03-16T10:44:58.525 INFO:tasks.workunit.client.0.smithi002.stdout:8/424: symlink d7/d55/l95 0 2024-03-16T10:44:58.525 INFO:tasks.workunit.client.0.smithi002.stdout:3/411: chown d1/dd/d28/d7b/d89/c42 762 1 2024-03-16T10:44:58.526 INFO:tasks.workunit.client.0.smithi002.stdout:5/384: dwrite d0/f6f [0,4194304] 0 2024-03-16T10:44:58.529 INFO:tasks.workunit.client.0.smithi002.stdout:1/424: symlink d2/d3/l8a 0 2024-03-16T10:44:58.529 INFO:tasks.workunit.client.0.smithi002.stdout:5/385: fsync d0/d16/f37 0 2024-03-16T10:44:58.529 INFO:tasks.workunit.client.0.smithi002.stdout:2/442: dwrite d7/f4c [4194304,4194304] 0 2024-03-16T10:44:58.532 INFO:tasks.workunit.client.0.smithi002.stdout:8/425: creat d7/d9/d25/d91/f96 x:0 0 0 2024-03-16T10:44:58.532 INFO:tasks.workunit.client.0.smithi002.stdout:9/434: link d5/l6 d5/d9/d22/l85 0 2024-03-16T10:44:58.533 INFO:tasks.workunit.client.0.smithi002.stdout:9/435: dread - d5/f6f zero size 2024-03-16T10:44:58.534 INFO:tasks.workunit.client.0.smithi002.stdout:8/426: write d7/d55/f61 [591470,116620] 0 2024-03-16T10:44:58.536 INFO:tasks.workunit.client.0.smithi002.stdout:3/412: write d1/dd/d28/d7b/d89/d15/f43 [835907,20642] 0 2024-03-16T10:44:58.537 INFO:tasks.workunit.client.0.smithi002.stdout:9/436: write d5/d9/d55/f7b [498901,23034] 0 2024-03-16T10:44:58.538 INFO:tasks.workunit.client.0.smithi002.stdout:5/386: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:58.541 INFO:tasks.workunit.client.0.smithi002.stdout:3/413: write d1/dd/f10 [107072,29511] 0 2024-03-16T10:44:58.541 INFO:tasks.workunit.client.0.smithi002.stdout:9/437: dread - d5/d9/dc/d28/f49 zero size 2024-03-16T10:44:58.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:58 smithi002 ceph-mon[29248]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:58.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:58 smithi002 ceph-mon[29248]: pgmap v32: 65 pgs: 65 active+clean; 770 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 105 MiB/s wr, 442 op/s 2024-03-16T10:44:58.548 INFO:tasks.workunit.client.0.smithi002.stdout:1/425: getdents d2/d3/d1c 0 2024-03-16T10:44:58.550 INFO:tasks.workunit.client.0.smithi002.stdout:9/438: chown d5/d9/d25/d23/c57 117176345 1 2024-03-16T10:44:58.550 INFO:tasks.workunit.client.0.smithi002.stdout:5/387: dwrite d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:44:58.551 INFO:tasks.workunit.client.0.smithi002.stdout:9/439: readlink d5/l12 0 2024-03-16T10:44:58.551 INFO:tasks.workunit.client.0.smithi002.stdout:8/427: dwrite d7/d9/d17/d23/d2e/d3a/f5f [0,4194304] 0 2024-03-16T10:44:58.560 INFO:tasks.workunit.client.0.smithi002.stdout:3/414: getdents d1/dd/d28 0 2024-03-16T10:44:58.560 INFO:tasks.workunit.client.0.smithi002.stdout:9/440: mkdir d5/d9/dc/d1e/d86 0 2024-03-16T10:44:58.564 INFO:tasks.workunit.client.0.smithi002.stdout:1/426: fdatasync d2/d3/f7 0 2024-03-16T10:44:58.565 INFO:tasks.workunit.client.0.smithi002.stdout:9/441: chown d5/d9/d22/d3c/d4d/l6d 0 1 2024-03-16T10:44:58.566 INFO:tasks.workunit.client.0.smithi002.stdout:5/388: creat d0/d1f/f77 x:0 0 0 2024-03-16T10:44:58.567 INFO:tasks.workunit.client.0.smithi002.stdout:1/427: fsync d2/d3/f65 0 2024-03-16T10:44:58.570 INFO:tasks.workunit.client.0.smithi002.stdout:5/389: read d0/d16/d1d/f23 [3187571,57764] 0 2024-03-16T10:44:58.570 INFO:tasks.workunit.client.0.smithi002.stdout:9/442: mknod d5/d9/dc/d1e/c87 0 2024-03-16T10:44:58.572 INFO:tasks.workunit.client.0.smithi002.stdout:5/390: chown d0/f57 133 1 2024-03-16T10:44:58.573 INFO:tasks.workunit.client.0.smithi002.stdout:3/415: dwrite d1/f16 [0,4194304] 0 2024-03-16T10:44:58.574 INFO:tasks.workunit.client.0.smithi002.stdout:2/443: truncate d7/db/f58 1115997 0 2024-03-16T10:44:58.576 INFO:tasks.workunit.client.0.smithi002.stdout:5/391: write d0/d16/f63 [1056196,86183] 0 2024-03-16T10:44:58.581 INFO:tasks.workunit.client.0.smithi002.stdout:3/416: chown d1/dd/f68 2688375 1 2024-03-16T10:44:58.583 INFO:tasks.workunit.client.0.smithi002.stdout:8/428: dread d7/d9/d17/d23/d2e/f92 [0,4194304] 0 2024-03-16T10:44:58.584 INFO:tasks.workunit.client.0.smithi002.stdout:2/444: write d7/d2e/d1d/d38/f3a [688849,7216] 0 2024-03-16T10:44:58.586 INFO:tasks.workunit.client.0.smithi002.stdout:1/428: dwrite d2/d58/f70 [0,4194304] 0 2024-03-16T10:44:58.588 INFO:tasks.workunit.client.0.smithi002.stdout:5/392: creat d0/d16/d1d/d2b/f78 x:0 0 0 2024-03-16T10:44:58.589 INFO:tasks.workunit.client.0.smithi002.stdout:8/429: mknod d7/d9/d17/d23/c97 0 2024-03-16T10:44:58.593 INFO:tasks.workunit.client.0.smithi002.stdout:8/430: chown d7/d9/d17/d23/c97 191 1 2024-03-16T10:44:58.593 INFO:tasks.workunit.client.0.smithi002.stdout:3/417: rename d1/dd/d28/c6e to d1/dd/d28/d7b/d89/d15/c99 0 2024-03-16T10:44:58.596 INFO:tasks.workunit.client.0.smithi002.stdout:1/429: getdents d2/df/d12/d84 0 2024-03-16T10:44:58.596 INFO:tasks.workunit.client.0.smithi002.stdout:8/431: stat d7/d9/d17/d23/d2e/d90/l93 0 2024-03-16T10:44:58.597 INFO:tasks.workunit.client.0.smithi002.stdout:1/430: fsync d2/d3/f4b 0 2024-03-16T10:44:58.597 INFO:tasks.workunit.client.0.smithi002.stdout:5/393: mknod d0/d16/d1d/d2b/d5d/d73/c79 0 2024-03-16T10:44:58.598 INFO:tasks.workunit.client.0.smithi002.stdout:3/418: truncate d1/dd/d28/d7b/d89/d15/f43 1305406 0 2024-03-16T10:44:58.600 INFO:tasks.workunit.client.0.smithi002.stdout:5/394: truncate d0/d16/f70 796429 0 2024-03-16T10:44:58.601 INFO:tasks.workunit.client.0.smithi002.stdout:8/432: creat d7/d55/f98 x:0 0 0 2024-03-16T10:44:58.602 INFO:tasks.workunit.client.0.smithi002.stdout:1/431: creat d2/d3/d51/d56/f8b x:0 0 0 2024-03-16T10:44:58.602 INFO:tasks.workunit.client.0.smithi002.stdout:2/445: link d7/d2e/d1d/d2f/d4b/f62 d7/d2e/d28/d40/d6c/f87 0 2024-03-16T10:44:58.606 INFO:tasks.workunit.client.0.smithi002.stdout:5/395: creat d0/d16/d1d/d33/d65/f7a x:0 0 0 2024-03-16T10:44:58.606 INFO:tasks.workunit.client.0.smithi002.stdout:9/443: dread d5/d9/dc/f1f [0,4194304] 0 2024-03-16T10:44:58.611 INFO:tasks.workunit.client.0.smithi002.stdout:2/446: stat d7/db/l30 0 2024-03-16T10:44:58.611 INFO:tasks.workunit.client.0.smithi002.stdout:8/433: write d7/d9/f51 [882479,41652] 0 2024-03-16T10:44:58.611 INFO:tasks.workunit.client.0.smithi002.stdout:5/396: mknod d0/d16/c7b 0 2024-03-16T10:44:58.611 INFO:tasks.workunit.client.0.smithi002.stdout:3/419: dwrite d1/dd/d28/d37/f8d [0,4194304] 0 2024-03-16T10:44:58.611 INFO:tasks.workunit.client.0.smithi002.stdout:6/437: sync 2024-03-16T10:44:58.613 INFO:tasks.workunit.client.0.smithi002.stdout:3/420: write d1/f52 [3749612,98444] 0 2024-03-16T10:44:58.614 INFO:tasks.workunit.client.0.smithi002.stdout:5/397: write d0/f18 [366951,50831] 0 2024-03-16T10:44:58.614 INFO:tasks.workunit.client.0.smithi002.stdout:3/421: chown d1/dd/d28/d7b/d8a/d93/f47 390097 1 2024-03-16T10:44:58.618 INFO:tasks.workunit.client.0.smithi002.stdout:2/447: rename d7/d2e/d28/d40/l47 to d7/d27/d3f/l88 0 2024-03-16T10:44:58.618 INFO:tasks.workunit.client.0.smithi002.stdout:2/448: readlink d7/l43 0 2024-03-16T10:44:58.620 INFO:tasks.workunit.client.0.smithi002.stdout:2/449: write d7/db/d12/f5d [4472556,60577] 0 2024-03-16T10:44:58.620 INFO:tasks.workunit.client.0.smithi002.stdout:8/434: symlink d7/d9/d17/d23/d2e/l99 0 2024-03-16T10:44:58.622 INFO:tasks.workunit.client.0.smithi002.stdout:6/438: rmdir d15/d2d/d8a/d7c/d87 39 2024-03-16T10:44:58.623 INFO:tasks.workunit.client.0.smithi002.stdout:3/422: rmdir d1/dd/d28/d7b/d2c/d63 39 2024-03-16T10:44:58.626 INFO:tasks.workunit.client.0.smithi002.stdout:5/398: fsync d0/d16/f19 0 2024-03-16T10:44:58.628 INFO:tasks.workunit.client.0.smithi002.stdout:8/435: unlink d7/d9/d17/d23/d2e/d3a/f53 0 2024-03-16T10:44:58.629 INFO:tasks.workunit.client.0.smithi002.stdout:5/399: read - d0/d16/d1d/d2b/f78 zero size 2024-03-16T10:44:58.630 INFO:tasks.workunit.client.0.smithi002.stdout:1/432: dread d2/d3/f39 [0,4194304] 0 2024-03-16T10:44:58.631 INFO:tasks.workunit.client.0.smithi002.stdout:8/436: chown d7/d9/d17/d20/f63 178 1 2024-03-16T10:44:58.631 INFO:tasks.workunit.client.0.smithi002.stdout:6/439: dwrite d15/d2d/d2b/d3f/f52 [0,4194304] 0 2024-03-16T10:44:58.632 INFO:tasks.workunit.client.0.smithi002.stdout:3/423: rmdir d1/dd/d28 39 2024-03-16T10:44:58.643 INFO:tasks.workunit.client.0.smithi002.stdout:2/450: truncate d7/db/f48 697798 0 2024-03-16T10:44:58.646 INFO:tasks.workunit.client.0.smithi002.stdout:1/433: creat d2/d3/d51/d56/f8c x:0 0 0 2024-03-16T10:44:58.649 INFO:tasks.workunit.client.0.smithi002.stdout:5/400: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:44:58.652 INFO:tasks.workunit.client.0.smithi002.stdout:2/451: symlink d7/d2e/d1d/d38/l89 0 2024-03-16T10:44:58.653 INFO:tasks.workunit.client.0.smithi002.stdout:5/401: write d0/d16/f70 [836427,96806] 0 2024-03-16T10:44:58.653 INFO:tasks.workunit.client.0.smithi002.stdout:8/437: symlink d7/l9a 0 2024-03-16T10:44:58.655 INFO:tasks.workunit.client.0.smithi002.stdout:6/440: rename d15/d2d/d8a/d7c/d87/c8d to d15/d82/c9d 0 2024-03-16T10:44:58.660 INFO:tasks.workunit.client.0.smithi002.stdout:5/402: dread d0/f6 [0,4194304] 0 2024-03-16T10:44:58.661 INFO:tasks.workunit.client.0.smithi002.stdout:2/452: dwrite d7/d2e/f67 [0,4194304] 0 2024-03-16T10:44:58.666 INFO:tasks.workunit.client.0.smithi002.stdout:2/453: read d7/db/f57 [409671,121537] 0 2024-03-16T10:44:58.666 INFO:tasks.workunit.client.0.smithi002.stdout:8/438: stat d7/d9/d17/d23/d2e/l62 0 2024-03-16T10:44:58.670 INFO:tasks.workunit.client.0.smithi002.stdout:3/424: mknod d1/dd/d28/d7b/d25/c9a 0 2024-03-16T10:44:58.671 INFO:tasks.workunit.client.0.smithi002.stdout:3/425: dread - d1/dd/d28/d7b/d89/f7f zero size 2024-03-16T10:44:58.676 INFO:tasks.workunit.client.0.smithi002.stdout:8/439: rename d7/d9/d25/c43 to d7/d9/d17/d23/c9b 0 2024-03-16T10:44:58.676 INFO:tasks.workunit.client.0.smithi002.stdout:8/440: fsync d7/d9/f51 0 2024-03-16T10:44:58.677 INFO:tasks.workunit.client.0.smithi002.stdout:8/441: read - d7/d55/f98 zero size 2024-03-16T10:44:58.677 INFO:tasks.workunit.client.0.smithi002.stdout:6/441: symlink d15/d2d/d2b/l9e 0 2024-03-16T10:44:58.684 INFO:tasks.workunit.client.0.smithi002.stdout:8/442: mknod d7/d9/d17/d23/d2e/d3a/c9c 0 2024-03-16T10:44:58.689 INFO:tasks.workunit.client.0.smithi002.stdout:6/442: rename d15/d82 to d15/d2d/d2b/d3f/d77/d9f 0 2024-03-16T10:44:58.692 INFO:tasks.workunit.client.0.smithi002.stdout:1/434: truncate d2/df/f45 2239022 0 2024-03-16T10:44:58.694 INFO:tasks.workunit.client.0.smithi002.stdout:3/426: dwrite d1/dd/d28/d7b/d2c/d63/f79 [0,4194304] 0 2024-03-16T10:44:58.696 INFO:tasks.workunit.client.0.smithi002.stdout:1/435: dread d2/f68 [0,4194304] 0 2024-03-16T10:44:58.700 INFO:tasks.workunit.client.0.smithi002.stdout:6/443: dwrite d15/d2d/d2c/f36 [0,4194304] 0 2024-03-16T10:44:58.700 INFO:tasks.workunit.client.0.smithi002.stdout:1/436: dread - d2/d3/d1c/d5d/d71/f79 zero size 2024-03-16T10:44:58.703 INFO:tasks.workunit.client.0.smithi002.stdout:8/443: dwrite d7/d9/d17/f27 [4194304,4194304] 0 2024-03-16T10:44:58.708 INFO:tasks.workunit.client.0.smithi002.stdout:1/437: write d2/d3/d51/d56/f8b [827739,56232] 0 2024-03-16T10:44:58.713 INFO:tasks.workunit.client.0.smithi002.stdout:3/427: dwrite d1/d5a/f95 [0,4194304] 0 2024-03-16T10:44:58.715 INFO:tasks.workunit.client.0.smithi002.stdout:2/454: dwrite d7/f2b [0,4194304] 0 2024-03-16T10:44:58.719 INFO:tasks.workunit.client.0.smithi002.stdout:6/444: dwrite d15/d2d/f9b [0,4194304] 0 2024-03-16T10:44:58.721 INFO:tasks.workunit.client.0.smithi002.stdout:6/445: stat l1 0 2024-03-16T10:44:58.723 INFO:tasks.workunit.client.0.smithi002.stdout:6/446: truncate d15/f5f 390750 0 2024-03-16T10:44:58.723 INFO:tasks.workunit.client.0.smithi002.stdout:6/447: chown d15/f5f 380 1 2024-03-16T10:44:58.728 INFO:tasks.workunit.client.0.smithi002.stdout:3/428: symlink d1/dd/l9b 0 2024-03-16T10:44:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:58 smithi026 ceph-mon[37041]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:58.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:58 smithi026 ceph-mon[37041]: pgmap v32: 65 pgs: 65 active+clean; 770 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 105 MiB/s wr, 442 op/s 2024-03-16T10:44:58.735 INFO:tasks.workunit.client.0.smithi002.stdout:3/429: write d1/dd/d28/d7b/d89/d15/f2e [692310,40122] 0 2024-03-16T10:44:58.736 INFO:tasks.workunit.client.0.smithi002.stdout:6/448: dwrite fa [12582912,4194304] 0 2024-03-16T10:44:58.744 INFO:tasks.workunit.client.0.smithi002.stdout:1/438: dwrite d2/d3/f7 [4194304,4194304] 0 2024-03-16T10:44:58.748 INFO:tasks.workunit.client.0.smithi002.stdout:1/439: dread - d2/d3/d1c/d5d/f82 zero size 2024-03-16T10:44:58.751 INFO:tasks.workunit.client.0.smithi002.stdout:3/430: dwrite d1/dd/d28/d7b/d8a/d93/f91 [0,4194304] 0 2024-03-16T10:44:58.753 INFO:tasks.workunit.client.0.smithi002.stdout:2/455: dwrite d7/db/f58 [0,4194304] 0 2024-03-16T10:44:58.759 INFO:tasks.workunit.client.0.smithi002.stdout:1/440: mknod d2/c8d 0 2024-03-16T10:44:58.759 INFO:tasks.workunit.client.0.smithi002.stdout:1/441: read d2/d3/fe [1582209,53083] 0 2024-03-16T10:44:58.762 INFO:tasks.workunit.client.0.smithi002.stdout:1/442: write d2/d3/d1c/d5d/f6b [707519,93670] 0 2024-03-16T10:44:58.762 INFO:tasks.workunit.client.0.smithi002.stdout:6/449: truncate d15/d2d/d2c/f50 226938 0 2024-03-16T10:44:58.763 INFO:tasks.workunit.client.0.smithi002.stdout:3/431: dwrite d1/dd/d28/d7b/d2c/f90 [0,4194304] 0 2024-03-16T10:44:58.768 INFO:tasks.workunit.client.0.smithi002.stdout:2/456: rename d7/d2e/d1d/d2f/d4b/c5b to d7/d2e/d28/d44/d6d/c8a 0 2024-03-16T10:44:58.771 INFO:tasks.workunit.client.0.smithi002.stdout:1/443: read d2/f68 [41076,19153] 0 2024-03-16T10:44:58.773 INFO:tasks.workunit.client.0.smithi002.stdout:6/450: mknod d15/d2d/d2c/d5d/ca0 0 2024-03-16T10:44:58.774 INFO:tasks.workunit.client.0.smithi002.stdout:6/451: truncate d15/d2d/d73/f95 714800 0 2024-03-16T10:44:58.774 INFO:tasks.workunit.client.0.smithi002.stdout:3/432: unlink d1/dd/d28/d37/l58 0 2024-03-16T10:44:58.775 INFO:tasks.workunit.client.0.smithi002.stdout:3/433: fdatasync d1/f8f 0 2024-03-16T10:44:58.778 INFO:tasks.workunit.client.0.smithi002.stdout:2/457: mknod d7/d2e/d1d/d2f/c8b 0 2024-03-16T10:44:58.782 INFO:tasks.workunit.client.0.smithi002.stdout:3/434: stat d1/dd/d28/d7b/d89/c3d 0 2024-03-16T10:44:58.784 INFO:tasks.workunit.client.0.smithi002.stdout:3/435: truncate d1/dd/d28/d7b/d89/f30 4911765 0 2024-03-16T10:44:58.785 INFO:tasks.workunit.client.0.smithi002.stdout:1/444: mknod d2/c8e 0 2024-03-16T10:44:58.788 INFO:tasks.workunit.client.0.smithi002.stdout:6/452: link d15/d2d/f34 d15/d2d/d2b/d3f/d77/d9f/fa1 0 2024-03-16T10:44:58.788 INFO:tasks.workunit.client.0.smithi002.stdout:2/458: mkdir d7/d2e/d1d/d38/d49/d8c 0 2024-03-16T10:44:58.789 INFO:tasks.workunit.client.0.smithi002.stdout:3/436: readlink d1/dd/d28/d7b/l5d 0 2024-03-16T10:44:58.792 INFO:tasks.workunit.client.0.smithi002.stdout:2/459: mknod d7/d2e/d1d/d2f/c8d 0 2024-03-16T10:44:58.794 INFO:tasks.workunit.client.0.smithi002.stdout:1/445: dread d2/df/d12/f4e [0,4194304] 0 2024-03-16T10:44:58.795 INFO:tasks.workunit.client.0.smithi002.stdout:3/437: link d1/dd/d28/f55 d1/dd/d28/f9c 0 2024-03-16T10:44:58.797 INFO:tasks.workunit.client.0.smithi002.stdout:2/460: mkdir d7/d27/d3f/d7f/d8e 0 2024-03-16T10:44:58.798 INFO:tasks.workunit.client.0.smithi002.stdout:1/446: unlink d2/d58/f70 0 2024-03-16T10:44:58.803 INFO:tasks.workunit.client.0.smithi002.stdout:1/447: creat d2/d3/d1c/d5d/d71/f8f x:0 0 0 2024-03-16T10:44:58.804 INFO:tasks.workunit.client.0.smithi002.stdout:2/461: mknod d7/d27/d3f/d7f/d8e/c8f 0 2024-03-16T10:44:58.804 INFO:tasks.workunit.client.0.smithi002.stdout:1/448: fsync d2/d3/d1c/d5d/d71/f79 0 2024-03-16T10:44:58.805 INFO:tasks.workunit.client.0.smithi002.stdout:2/462: write d7/d2e/f25 [796993,128868] 0 2024-03-16T10:44:58.806 INFO:tasks.workunit.client.0.smithi002.stdout:3/438: getdents d1/dd/d28/d7b/d8a/d93 0 2024-03-16T10:44:58.808 INFO:tasks.workunit.client.0.smithi002.stdout:1/449: rename d2/d3/f7 to d2/df/d12/d74/f90 0 2024-03-16T10:44:58.808 INFO:tasks.workunit.client.0.smithi002.stdout:1/450: dread - d2/d3/d51/d56/f8c zero size 2024-03-16T10:44:58.810 INFO:tasks.workunit.client.0.smithi002.stdout:3/439: creat d1/dd/d28/f9d x:0 0 0 2024-03-16T10:44:58.827 INFO:tasks.workunit.client.0.smithi002.stdout:7/374: sync 2024-03-16T10:44:58.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.831+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce5c109120 con 0x7fce5c100360 2024-03-16T10:44:58.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.832+0000 7fce527fc700 1 -- 172.21.15.2:0/3191199488 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fce4c05a170 con 0x7fce5c100360 2024-03-16T10:44:58.833 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:44:58.833 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:44:58.833 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 1, 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 13, 2024-03-16T10:44:58.834 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T10:44:58.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:44:58.835 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:44:58.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fce4806d390 msgr2=0x7fce4806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fce4806d390 0x7fce4806f850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce58009f50 tx=0x7fce5801b040 comp rx=0 tx=0).stop 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c100360 msgr2=0x7fce5c195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c100360 0x7fce5c195990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00eb10 tx=0x7fce4c00eed0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 shutdown_connections 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7fce4806d390 0x7fce4806f850 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce5c0ff160 0x7fce5c195450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 --2- 172.21.15.2:0/3191199488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce5c100360 0x7fce5c195990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 >> 172.21.15.2:0/3191199488 conn(0x7fce5c0fa700 msgr2=0x7fce5c0fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 shutdown_connections 2024-03-16T10:44:58.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.835+0000 7fce635a6700 1 -- 172.21.15.2:0/3191199488 wait complete. 2024-03-16T10:44:58.838 INFO:tasks.workunit.client.0.smithi002.stdout:7/375: rename d3/d4/d1c/d44 to d3/d4/d1d/d87 0 2024-03-16T10:44:58.840 INFO:tasks.workunit.client.0.smithi002.stdout:3/440: getdents d1/d5a 0 2024-03-16T10:44:58.845 INFO:tasks.workunit.client.0.smithi002.stdout:7/376: rename d3/d4/d1d/d87/d5c/d63/d3f/d75/d86 to d3/d61/d66/d88 0 2024-03-16T10:44:58.895 INFO:tasks.workunit.client.0.smithi002.stdout:3/441: creat d1/dd/f9e x:0 0 0 2024-03-16T10:44:58.895 INFO:tasks.workunit.client.0.smithi002.stdout:7/377: creat d3/d4/d1d/d87/d5c/d63/d3f/f89 x:0 0 0 2024-03-16T10:44:58.901 INFO:tasks.workunit.client.0.smithi002.stdout:7/378: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f43 [0,4194304] 0 2024-03-16T10:44:58.902 INFO:tasks.workunit.client.0.smithi002.stdout:2/463: dread f6 [0,4194304] 0 2024-03-16T10:44:58.904 INFO:tasks.workunit.client.0.smithi002.stdout:2/464: read d7/d2e/f25 [1672585,128613] 0 2024-03-16T10:44:58.905 INFO:tasks.workunit.client.0.smithi002.stdout:7/379: mknod d3/d4/d1d/d87/d5c/d63/d3f/c8a 0 2024-03-16T10:44:58.906 INFO:tasks.workunit.client.0.smithi002.stdout:7/380: chown d3/d4/d1d/d45/f82 1458784 1 2024-03-16T10:44:58.914 INFO:tasks.workunit.client.0.smithi002.stdout:2/465: dread d7/d2e/f16 [0,4194304] 0 2024-03-16T10:44:58.915 INFO:tasks.workunit.client.0.smithi002.stdout:2/466: stat d7/d27/d3f/d7f/d8e 0 2024-03-16T10:44:58.915 INFO:tasks.workunit.client.0.smithi002.stdout:7/381: link d3/d4/d1d/d87/d5c/d63/d3f/l47 d3/d4/d1d/d45/l8b 0 2024-03-16T10:44:58.916 INFO:tasks.workunit.client.0.smithi002.stdout:7/382: read - d3/d4/d1d/d45/f82 zero size 2024-03-16T10:44:58.921 INFO:tasks.workunit.client.0.smithi002.stdout:2/467: creat d7/f90 x:0 0 0 2024-03-16T10:44:58.921 INFO:tasks.workunit.client.0.smithi002.stdout:2/468: stat d7/d2e/d1d/d38 0 2024-03-16T10:44:58.922 INFO:tasks.workunit.client.0.smithi002.stdout:7/383: mkdir d3/d4/d1d/d3d/d8c 0 2024-03-16T10:44:58.922 INFO:tasks.workunit.client.0.smithi002.stdout:2/469: write d7/d2e/d28/d40/f5a [295667,16645] 0 2024-03-16T10:44:58.928 INFO:tasks.workunit.client.0.smithi002.stdout:7/384: getdents d3/d4/d1d/d87 0 2024-03-16T10:44:58.930 INFO:tasks.workunit.client.0.smithi002.stdout:2/470: dwrite d7/d2e/d1d/d38/f73 [0,4194304] 0 2024-03-16T10:44:58.930 INFO:tasks.workunit.client.0.smithi002.stdout:4/422: sync 2024-03-16T10:44:58.931 INFO:tasks.workunit.client.0.smithi002.stdout:2/471: fsync d7/d27/f56 0 2024-03-16T10:44:58.934 INFO:tasks.workunit.client.0.smithi002.stdout:7/385: creat d3/d5f/f8d x:0 0 0 2024-03-16T10:44:58.937 INFO:tasks.workunit.client.0.smithi002.stdout:4/423: rmdir d2/d33/da/d5e 39 2024-03-16T10:44:58.937 INFO:tasks.workunit.client.0.smithi002.stdout:4/424: readlink d2/d33/l2b 0 2024-03-16T10:44:58.938 INFO:tasks.workunit.client.0.smithi002.stdout:3/442: truncate d1/dd/f10 3848660 0 2024-03-16T10:44:58.943 INFO:tasks.workunit.client.0.smithi002.stdout:4/425: creat d2/d33/d1c/f83 x:0 0 0 2024-03-16T10:44:58.945 INFO:tasks.workunit.client.0.smithi002.stdout:7/386: symlink d3/d4/d1d/d87/d5c/d63/l8e 0 2024-03-16T10:44:58.946 INFO:tasks.workunit.client.0.smithi002.stdout:7/387: read - d3/d4/d1d/d87/d5c/d63/d55/f5d zero size 2024-03-16T10:44:58.947 INFO:tasks.workunit.client.0.smithi002.stdout:2/472: rename d7/d2e/d1d/d2f/l37 to d7/db/l91 0 2024-03-16T10:44:58.950 INFO:tasks.workunit.client.0.smithi002.stdout:4/426: dwrite d2/d33/da/f28 [0,4194304] 0 2024-03-16T10:44:58.954 INFO:tasks.workunit.client.0.smithi002.stdout:2/473: write d7/d2e/f20 [539093,23216] 0 2024-03-16T10:44:58.956 INFO:tasks.workunit.client.0.smithi002.stdout:3/443: rename d1/dd/l9b to d1/dd/d28/d7b/d89/d15/l9f 0 2024-03-16T10:44:58.958 INFO:tasks.workunit.client.0.smithi002.stdout:7/388: link d3/d4/d1d/d87/d5c/d63/f7f d3/d37/f8f 0 2024-03-16T10:44:58.966 INFO:tasks.workunit.client.0.smithi002.stdout:2/474: link d7/db/d12/l14 d7/d2e/d28/l92 0 2024-03-16T10:44:58.969 INFO:tasks.workunit.client.0.smithi002.stdout:2/475: creat d7/d2e/d28/d40/f93 x:0 0 0 2024-03-16T10:44:58.970 INFO:tasks.workunit.client.0.smithi002.stdout:2/476: chown d7/d2e/d1d/f26 3 1 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 -- 172.21.15.2:0/122110246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c100bd0 msgr2=0x7f4c6c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 --2- 172.21.15.2:0/122110246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c101030 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009910 tx=0x7f4c5c009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 -- 172.21.15.2:0/122110246 shutdown_connections 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 --2- 172.21.15.2:0/122110246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c101030 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 --2- 172.21.15.2:0/122110246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c0ff9d0 0x7f4c6c0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 -- 172.21.15.2:0/122110246 >> 172.21.15.2:0/122110246 conn(0x7f4c6c0faf70 msgr2=0x7f4c6c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 -- 172.21.15.2:0/122110246 shutdown_connections 2024-03-16T10:44:58.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.973+0000 7f4c7399f700 1 -- 172.21.15.2:0/122110246 wait complete. 2024-03-16T10:44:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c7399f700 1 Processor -- start 2024-03-16T10:44:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c7399f700 1 -- start start 2024-03-16T10:44:58.975 INFO:tasks.workunit.client.0.smithi002.stdout:3/444: dread d1/dd/d28/d7b/d89/d15/f62 [0,4194304] 0 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c7399f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c0ff9d0 0x7f4c6c068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c7399f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c7399f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c19b7b0 con 0x7f4c6c0ff9d0 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c7399f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c19b920 con 0x7f4c6c100bd0 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c70f3a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c7173b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c0ff9d0 0x7f4c6c068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.974+0000 7f4c70f3a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:56882/0 (socket says 172.21.15.2:56882) 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.975+0000 7f4c70f3a700 1 -- 172.21.15.2:0/2134852486 learned_addr learned my addr 172.21.15.2:0/2134852486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.975+0000 7f4c70f3a700 1 -- 172.21.15.2:0/2134852486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c0ff9d0 msgr2=0x7f4c6c068f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.975+0000 7f4c70f3a700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c0ff9d0 0x7f4c6c068f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:58.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.975+0000 7f4c70f3a700 1 -- 172.21.15.2:0/2134852486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0095f0 con 0x7f4c6c100bd0 2024-03-16T10:44:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.975+0000 7f4c70f3a700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c19b270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0092e0 tx=0x7f4c5c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.975+0000 7f4c627fc700 1 -- 172.21.15.2:0/2134852486 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c01d070 con 0x7f4c6c100bd0 2024-03-16T10:44:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.975+0000 7f4c627fc700 1 -- 172.21.15.2:0/2134852486 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c5c00f460 con 0x7f4c6c100bd0 2024-03-16T10:44:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.976+0000 7f4c627fc700 1 -- 172.21.15.2:0/2134852486 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5c021620 con 0x7f4c6c100bd0 2024-03-16T10:44:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.976+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c19bba0 con 0x7f4c6c100bd0 2024-03-16T10:44:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.976+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c195290 con 0x7f4c6c100bd0 2024-03-16T10:44:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.977+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c18f550 con 0x7f4c6c100bd0 2024-03-16T10:44:58.990 INFO:tasks.workunit.client.0.smithi002.stdout:2/477: rename d7/d2e/f1a to d7/d27/d3b/f94 0 2024-03-16T10:44:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.978+0000 7f4c627fc700 1 -- 172.21.15.2:0/2134852486 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 91211+0+0 (secure 0 0 0) 0x7f4c5c00f5d0 con 0x7f4c6c100bd0 2024-03-16T10:44:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.978+0000 7f4c627fc700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4c580717f0 0x7f4c58073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.978+0000 7f4c627fc700 1 -- 172.21.15.2:0/2134852486 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4c5c08ded0 con 0x7f4c6c100bd0 2024-03-16T10:44:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.978+0000 7f4c7173b700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4c580717f0 0x7f4c58073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.979+0000 7f4c7173b700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4c580717f0 0x7f4c58073cb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c100a30 tx=0x7f4c6800b410 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:59.001 INFO:tasks.workunit.client.0.smithi002.stdout:2/478: write d7/d2e/f67 [1822576,107741] 0 2024-03-16T10:44:59.001 INFO:tasks.workunit.client.0.smithi002.stdout:3/445: dwrite d1/d5a/f95 [4194304,4194304] 0 2024-03-16T10:44:59.001 INFO:tasks.workunit.client.0.smithi002.stdout:2/479: mkdir d7/d27/d3b/d95 0 2024-03-16T10:44:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:58.985+0000 7f4c627fc700 1 -- 172.21.15.2:0/2134852486 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f4c5c05b260 con 0x7f4c6c100bd0 2024-03-16T10:44:59.002 INFO:tasks.workunit.client.0.smithi002.stdout:3/446: write d1/dd/d28/d37/f78 [5211968,101352] 0 2024-03-16T10:44:59.002 INFO:tasks.workunit.client.0.smithi002.stdout:2/480: mknod d7/d2e/d28/d44/d6d/c96 0 2024-03-16T10:44:59.002 INFO:tasks.workunit.client.0.smithi002.stdout:3/447: symlink d1/dd/d28/d7b/d8a/d93/la0 0 2024-03-16T10:44:59.002 INFO:tasks.workunit.client.0.smithi002.stdout:3/448: write d1/dd/f9e [630547,10767] 0 2024-03-16T10:44:59.002 INFO:tasks.workunit.client.0.smithi002.stdout:3/449: creat d1/dd/d28/d7b/d22/fa1 x:0 0 0 2024-03-16T10:44:59.002 INFO:tasks.workunit.client.0.smithi002.stdout:2/481: symlink d7/d2e/d46/l97 0 2024-03-16T10:44:59.002 INFO:tasks.workunit.client.0.smithi002.stdout:3/450: chown d1/dd/d28/d7b/d89/f7f 123 1 2024-03-16T10:44:59.003 INFO:tasks.workunit.client.0.smithi002.stdout:3/451: readlink d1/l7 0 2024-03-16T10:44:59.009 INFO:tasks.workunit.client.0.smithi002.stdout:3/452: read d1/d5a/f75 [264311,21990] 0 2024-03-16T10:44:59.010 INFO:tasks.workunit.client.0.smithi002.stdout:3/453: rmdir d1/dd/d28/d7b/d8a/d93 39 2024-03-16T10:44:59.012 INFO:tasks.workunit.client.0.smithi002.stdout:3/454: rename d1/dd/d28/d7b/d22/fa1 to d1/dd/d28/d7b/d8a/d93/fa2 0 2024-03-16T10:44:59.019 INFO:tasks.workunit.client.0.smithi002.stdout:2/482: dread d7/f4c [0,4194304] 0 2024-03-16T10:44:59.022 INFO:tasks.workunit.client.0.smithi002.stdout:2/483: creat d7/db/d70/f98 x:0 0 0 2024-03-16T10:44:59.026 INFO:tasks.workunit.client.0.smithi002.stdout:3/455: dwrite d1/dd/d28/d7b/d2c/d63/f88 [0,4194304] 0 2024-03-16T10:44:59.026 INFO:tasks.workunit.client.0.smithi002.stdout:3/456: fsync d1/dd/d28/d7b/d8a/d93/f51 0 2024-03-16T10:44:59.027 INFO:tasks.workunit.client.0.smithi002.stdout:2/484: dread d7/db/f58 [0,4194304] 0 2024-03-16T10:44:59.034 INFO:tasks.workunit.client.0.smithi002.stdout:2/485: unlink d7/d27/d3b/f5c 0 2024-03-16T10:44:59.034 INFO:tasks.workunit.client.0.smithi002.stdout:2/486: readlink d7/l54 0 2024-03-16T10:44:59.036 INFO:tasks.workunit.client.0.smithi002.stdout:3/457: creat d1/dd/d28/d7b/d2c/fa3 x:0 0 0 2024-03-16T10:44:59.038 INFO:tasks.workunit.client.0.smithi002.stdout:3/458: readlink d1/dd/d28/d37/l4f 0 2024-03-16T10:44:59.039 INFO:tasks.workunit.client.0.smithi002.stdout:2/487: rename d7/d27/d3f/d7f/d8e/c8f to d7/d27/d3f/d7f/c99 0 2024-03-16T10:44:59.039 INFO:tasks.workunit.client.0.smithi002.stdout:3/459: write d1/dd/d28/d7b/d89/d15/f17 [3028630,5566] 0 2024-03-16T10:44:59.039 INFO:tasks.workunit.client.0.smithi002.stdout:3/460: readlink d1/dd/d28/d7b/l29 0 2024-03-16T10:44:59.045 INFO:tasks.workunit.client.0.smithi002.stdout:3/461: creat d1/dd/d28/d7b/d2c/d97/fa4 x:0 0 0 2024-03-16T10:44:59.047 INFO:tasks.workunit.client.0.smithi002.stdout:3/462: fdatasync d1/dd/d28/d7b/d2c/f90 0 2024-03-16T10:44:59.047 INFO:tasks.workunit.client.0.smithi002.stdout:3/463: mkdir d1/da5 0 2024-03-16T10:44:59.048 INFO:tasks.workunit.client.0.smithi002.stdout:3/464: fdatasync d1/dd/d28/f9c 0 2024-03-16T10:44:59.066 INFO:tasks.workunit.client.0.smithi002.stdout:3/465: write d1/dd/d28/f31 [2130992,128536] 0 2024-03-16T10:44:59.069 INFO:tasks.workunit.client.0.smithi002.stdout:2/488: dread d7/d27/d3b/f94 [0,4194304] 0 2024-03-16T10:44:59.069 INFO:tasks.workunit.client.0.smithi002.stdout:3/466: dwrite d1/dd/d28/d7b/d89/d15/f62 [0,4194304] 0 2024-03-16T10:44:59.071 INFO:tasks.workunit.client.0.smithi002.stdout:2/489: dwrite d7/d2e/d28/d40/f93 [0,4194304] 0 2024-03-16T10:44:59.072 INFO:tasks.workunit.client.0.smithi002.stdout:2/490: chown d7/d2e/d1d/d2f/d4b 2297222 1 2024-03-16T10:44:59.072 INFO:tasks.workunit.client.0.smithi002.stdout:2/491: chown d7/d2e/d1d/d2f/d4b/f62 0 1 2024-03-16T10:44:59.075 INFO:tasks.workunit.client.0.smithi002.stdout:2/492: rename d7/db/d12/f11 to d7/d2e/d1d/d2f/f9a 0 2024-03-16T10:44:59.075 INFO:tasks.workunit.client.0.smithi002.stdout:3/467: truncate d1/dd/d28/f31 802309 0 2024-03-16T10:44:59.077 INFO:tasks.workunit.client.0.smithi002.stdout:2/493: mknod d7/d27/d3b/c9b 0 2024-03-16T10:44:59.087 INFO:tasks.workunit.client.0.smithi002.stdout:3/468: getdents d1/dd/d74 0 2024-03-16T10:44:59.089 INFO:tasks.workunit.client.0.smithi002.stdout:3/469: symlink d1/dd/d28/d7b/d89/d15/d87/d8c/la6 0 2024-03-16T10:44:59.091 INFO:tasks.workunit.client.0.smithi002.stdout:3/470: creat d1/dd/d28/d7b/d89/d15/fa7 x:0 0 0 2024-03-16T10:44:59.092 INFO:tasks.workunit.client.0.smithi002.stdout:3/471: creat d1/dd/d74/fa8 x:0 0 0 2024-03-16T10:44:59.092 INFO:tasks.workunit.client.0.smithi002.stdout:3/472: chown d1/d5a/l7a 6 1 2024-03-16T10:44:59.101 INFO:tasks.workunit.client.0.smithi002.stdout:3/473: dwrite d1/fb [0,4194304] 0 2024-03-16T10:44:59.101 INFO:tasks.workunit.client.0.smithi002.stdout:3/474: dwrite d1/dd/f68 [0,4194304] 0 2024-03-16T10:44:59.114 INFO:tasks.workunit.client.0.smithi002.stdout:3/475: read d1/f16 [4745073,118311] 0 2024-03-16T10:44:59.118 INFO:tasks.workunit.client.0.smithi002.stdout:3/476: dread d1/dd/d28/f6a [0,4194304] 0 2024-03-16T10:44:59.120 INFO:tasks.workunit.client.0.smithi002.stdout:3/477: fsync d1/dd/d28/d7b/d8a/d93/f57 0 2024-03-16T10:44:59.121 INFO:tasks.workunit.client.0.smithi002.stdout:3/478: dwrite d1/f52 [0,4194304] 0 2024-03-16T10:44:59.122 INFO:tasks.workunit.client.0.smithi002.stdout:3/479: dwrite d1/dd/d28/d37/f78 [0,4194304] 0 2024-03-16T10:44:59.126 INFO:tasks.workunit.client.0.smithi002.stdout:3/480: creat d1/dd/d28/d7b/d89/d15/d87/d8c/fa9 x:0 0 0 2024-03-16T10:44:59.137 INFO:tasks.workunit.client.0.smithi002.stdout:3/481: chown d1/d5a/f95 21595 1 2024-03-16T10:44:59.139 INFO:tasks.workunit.client.0.smithi002.stdout:3/482: write d1/dd/d74/f80 [4223467,28757] 0 2024-03-16T10:44:59.139 INFO:tasks.workunit.client.0.smithi002.stdout:3/483: mknod d1/dd/d28/d7b/d2c/d63/caa 0 2024-03-16T10:44:59.139 INFO:tasks.workunit.client.0.smithi002.stdout:3/484: read d1/dd/f68 [428821,62072] 0 2024-03-16T10:44:59.140 INFO:tasks.workunit.client.0.smithi002.stdout:3/485: write d1/dd/d28/d7b/d22/f39 [5921128,117518] 0 2024-03-16T10:44:59.141 INFO:tasks.workunit.client.0.smithi002.stdout:3/486: fdatasync d1/dd/d28/d7b/d2c/d63/f88 0 2024-03-16T10:44:59.141 INFO:tasks.workunit.client.0.smithi002.stdout:3/487: symlink d1/dd/d28/d7b/d89/lab 0 2024-03-16T10:44:59.147 INFO:tasks.workunit.client.0.smithi002.stdout:3/488: truncate d1/dd/d28/d7b/d8a/d93/f91 4805757 0 2024-03-16T10:44:59.147 INFO:tasks.workunit.client.0.smithi002.stdout:3/489: symlink d1/dd/d74/lac 0 2024-03-16T10:44:59.148 INFO:tasks.workunit.client.0.smithi002.stdout:3/490: unlink d1/dd/d28/d7b/d2c/l4c 0 2024-03-16T10:44:59.155 INFO:tasks.workunit.client.0.smithi002.stdout:3/491: read d1/dd/d28/d7b/d89/f4a [462630,76995] 0 2024-03-16T10:44:59.160 INFO:tasks.workunit.client.0.smithi002.stdout:3/492: dread d1/dd/d28/d7b/d2c/d63/f79 [0,4194304] 0 2024-03-16T10:44:59.160 INFO:tasks.workunit.client.0.smithi002.stdout:3/493: truncate d1/dd/d28/f9c 717669 0 2024-03-16T10:44:59.181 INFO:tasks.workunit.client.0.smithi002.stdout:5/403: sync 2024-03-16T10:44:59.181 INFO:tasks.workunit.client.0.smithi002.stdout:4/427: sync 2024-03-16T10:44:59.181 INFO:tasks.workunit.client.0.smithi002.stdout:9/444: sync 2024-03-16T10:44:59.181 INFO:tasks.workunit.client.0.smithi002.stdout:8/444: sync 2024-03-16T10:44:59.181 INFO:tasks.workunit.client.0.smithi002.stdout:6/453: sync 2024-03-16T10:44:59.182 INFO:tasks.workunit.client.0.smithi002.stdout:0/472: sync 2024-03-16T10:44:59.182 INFO:tasks.workunit.client.0.smithi002.stdout:1/451: sync 2024-03-16T10:44:59.197 INFO:tasks.workunit.client.0.smithi002.stdout:0/473: write d3/f49 [5327218,107197] 0 2024-03-16T10:44:59.197 INFO:tasks.workunit.client.0.smithi002.stdout:9/445: dread - d5/d9/dc/d1e/f6b zero size 2024-03-16T10:44:59.198 INFO:tasks.workunit.client.0.smithi002.stdout:5/404: fsync d0/d16/d1d/d33/d65/f7a 0 2024-03-16T10:44:59.200 INFO:tasks.workunit.client.0.smithi002.stdout:8/445: rename d7/l4b to d7/d9/d17/d20/l9d 0 2024-03-16T10:44:59.201 INFO:tasks.workunit.client.0.smithi002.stdout:5/405: creat d0/d16/d4a/d39/f7c x:0 0 0 2024-03-16T10:44:59.201 INFO:tasks.workunit.client.0.smithi002.stdout:1/452: write d2/d3/f55 [1855943,53962] 0 2024-03-16T10:44:59.202 INFO:tasks.workunit.client.0.smithi002.stdout:4/428: creat d2/d33/da/f84 x:0 0 0 2024-03-16T10:44:59.205 INFO:tasks.workunit.client.0.smithi002.stdout:8/446: creat d7/d9/d17/d23/d2e/d34/f9e x:0 0 0 2024-03-16T10:44:59.209 INFO:tasks.workunit.client.0.smithi002.stdout:4/429: fsync d2/d33/da/f25 0 2024-03-16T10:44:59.210 INFO:tasks.workunit.client.0.smithi002.stdout:9/446: link d5/d9/dc/c15 d5/d9/d34/c88 0 2024-03-16T10:44:59.211 INFO:tasks.workunit.client.0.smithi002.stdout:6/454: dwrite d15/f4f [0,4194304] 0 2024-03-16T10:44:59.212 INFO:tasks.workunit.client.0.smithi002.stdout:1/453: fsync d2/df/d12/d29/d5c/f6d 0 2024-03-16T10:44:59.212 INFO:tasks.workunit.client.0.smithi002.stdout:8/447: write d7/d9/d17/f2d [4373601,80242] 0 2024-03-16T10:44:59.212 INFO:tasks.workunit.client.0.smithi002.stdout:0/474: dwrite d3/f3a [4194304,4194304] 0 2024-03-16T10:44:59.223 INFO:tasks.workunit.client.0.smithi002.stdout:4/430: chown d2/d33/da/f4f 245 1 2024-03-16T10:44:59.224 INFO:tasks.workunit.client.0.smithi002.stdout:5/406: rename d0/d16/d1d/d2b/d5d/d73 to d0/d16/d4a/d7d 0 2024-03-16T10:44:59.235 INFO:tasks.workunit.client.0.smithi002.stdout:1/454: read d2/df/d12/f3f [88296,51182] 0 2024-03-16T10:44:59.235 INFO:tasks.workunit.client.0.smithi002.stdout:9/447: truncate d5/d9/dc/f1f 1884606 0 2024-03-16T10:44:59.235 INFO:tasks.workunit.client.0.smithi002.stdout:6/455: symlink d15/d2d/d8a/d7c/d87/la2 0 2024-03-16T10:44:59.235 INFO:tasks.workunit.client.0.smithi002.stdout:0/475: mkdir d3/d27/d90 0 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:8/448: mknod d7/d55/c9f 0 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:8/449: write d7/d9/d17/d23/d2e/d3a/f47 [889057,116445] 0 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:4/431: creat d2/d33/d35/f85 x:0 0 0 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:4/432: chown d2/d33/d35/d59/f60 15468642 1 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:9/448: rename d5/d9/dc/l1c to d5/d7e/l89 0 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:9/449: stat d5/d9/dc/d28/c51 0 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:8/450: dwrite d7/d9/f78 [0,4194304] 0 2024-03-16T10:44:59.236 INFO:tasks.workunit.client.0.smithi002.stdout:1/455: symlink d2/df/l91 0 2024-03-16T10:44:59.242 INFO:tasks.workunit.client.0.smithi002.stdout:9/450: dread d5/d9/d25/d23/f56 [0,4194304] 0 2024-03-16T10:44:59.243 INFO:tasks.workunit.client.0.smithi002.stdout:4/433: rename d2/d6/d32/d68/l65 to d2/l86 0 2024-03-16T10:44:59.244 INFO:tasks.workunit.client.0.smithi002.stdout:1/456: dwrite d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:44:59.245 INFO:tasks.workunit.client.0.smithi002.stdout:0/476: mknod d3/d28/c91 0 2024-03-16T10:44:59.256 INFO:tasks.workunit.client.0.smithi002.stdout:6/456: dread d15/d2d/d2c/f56 [0,4194304] 0 2024-03-16T10:44:59.262 INFO:tasks.workunit.client.0.smithi002.stdout:9/451: rename d5/d9/d22/d3c/d4d/l6d to d5/d9/d55/d77/d82/l8a 0 2024-03-16T10:44:59.262 INFO:tasks.workunit.client.0.smithi002.stdout:8/451: symlink d7/d55/d8f/la0 0 2024-03-16T10:44:59.263 INFO:tasks.workunit.client.0.smithi002.stdout:6/457: dwrite d15/d2d/d2c/d5d/f72 [0,4194304] 0 2024-03-16T10:44:59.263 INFO:tasks.workunit.client.0.smithi002.stdout:8/452: fsync d7/d9/d17/d23/f4d 0 2024-03-16T10:44:59.263 INFO:tasks.workunit.client.0.smithi002.stdout:6/458: fsync d15/d2d/d2b/d3f/d77/f99 0 2024-03-16T10:44:59.263 INFO:tasks.workunit.client.0.smithi002.stdout:7/389: sync 2024-03-16T10:44:59.263 INFO:tasks.workunit.client.0.smithi002.stdout:0/477: creat d3/d27/f92 x:0 0 0 2024-03-16T10:44:59.264 INFO:tasks.workunit.client.0.smithi002.stdout:1/457: creat d2/d3/d1c/d5d/f92 x:0 0 0 2024-03-16T10:44:59.264 INFO:tasks.workunit.client.0.smithi002.stdout:6/459: stat d15/f29 0 2024-03-16T10:44:59.264 INFO:tasks.workunit.client.0.smithi002.stdout:7/390: readlink d3/d4/d1d/d87/d5c/d63/l29 0 2024-03-16T10:44:59.264 INFO:tasks.workunit.client.0.smithi002.stdout:8/453: dread - d7/d9/d17/d23/d2e/d34/f6e zero size 2024-03-16T10:44:59.264 INFO:tasks.workunit.client.0.smithi002.stdout:8/454: write d7/d55/f61 [290317,84535] 0 2024-03-16T10:44:59.265 INFO:tasks.workunit.client.0.smithi002.stdout:5/407: link d0/d16/d4a/c2d d0/c7e 0 2024-03-16T10:44:59.265 INFO:tasks.workunit.client.0.smithi002.stdout:5/408: write d0/d16/f70 [270257,69400] 0 2024-03-16T10:44:59.277 INFO:tasks.workunit.client.0.smithi002.stdout:7/391: rmdir d3/d4/d1c/d5b/d64 39 2024-03-16T10:44:59.278 INFO:tasks.workunit.client.0.smithi002.stdout:6/460: rmdir d15/d2d/d73/d97 39 2024-03-16T10:44:59.278 INFO:tasks.workunit.client.0.smithi002.stdout:6/461: chown d15/l48 97 1 2024-03-16T10:44:59.280 INFO:tasks.workunit.client.0.smithi002.stdout:8/455: creat d7/d9/d17/d23/d2e/d5c/fa1 x:0 0 0 2024-03-16T10:44:59.283 INFO:tasks.workunit.client.0.smithi002.stdout:5/409: stat d0/d1f/l6a 0 2024-03-16T10:44:59.286 INFO:tasks.workunit.client.0.smithi002.stdout:8/456: dwrite d7/d9/d17/d23/d2e/d3a/f5f [0,4194304] 0 2024-03-16T10:44:59.292 INFO:tasks.workunit.client.0.smithi002.stdout:1/458: symlink d2/l93 0 2024-03-16T10:44:59.292 INFO:tasks.workunit.client.0.smithi002.stdout:8/457: fsync d7/d9/d17/f27 0 2024-03-16T10:44:59.294 INFO:tasks.workunit.client.0.smithi002.stdout:7/392: rename d3/d4/d1d/d87/d5c/d63/f3b to d3/d4/d1c/f90 0 2024-03-16T10:44:59.294 INFO:tasks.workunit.client.0.smithi002.stdout:6/462: mknod d15/d2d/d2b/d75/ca3 0 2024-03-16T10:44:59.294 INFO:tasks.workunit.client.0.smithi002.stdout:9/452: write d5/d9/dc/d1e/d24/d31/f32 [3877951,114374] 0 2024-03-16T10:44:59.297 INFO:tasks.workunit.client.0.smithi002.stdout:0/478: creat d3/d28/f93 x:0 0 0 2024-03-16T10:44:59.304 INFO:tasks.workunit.client.0.smithi002.stdout:8/458: mkdir d7/d9/d17/d83/da2 0 2024-03-16T10:44:59.305 INFO:tasks.workunit.client.0.smithi002.stdout:6/463: creat d15/d2d/d8a/d7c/d87/fa4 x:0 0 0 2024-03-16T10:44:59.305 INFO:tasks.workunit.client.0.smithi002.stdout:6/464: readlink d15/d2d/d8a/d7c/d87/la2 0 2024-03-16T10:44:59.311 INFO:tasks.workunit.client.0.smithi002.stdout:6/465: chown d15/d2d/f22 812734 1 2024-03-16T10:44:59.314 INFO:tasks.workunit.client.0.smithi002.stdout:5/410: rmdir d0/d16/d4a/d7d/d76 0 2024-03-16T10:44:59.314 INFO:tasks.workunit.client.0.smithi002.stdout:0/479: dread d3/d28/d3d/f46 [0,4194304] 0 2024-03-16T10:44:59.314 INFO:tasks.workunit.client.0.smithi002.stdout:0/480: fdatasync d3/d31/d75/f8c 0 2024-03-16T10:44:59.315 INFO:tasks.workunit.client.0.smithi002.stdout:1/459: link d2/df/d12/d74/l78 d2/df/d12/d84/l94 0 2024-03-16T10:44:59.316 INFO:tasks.workunit.client.0.smithi002.stdout:6/466: fdatasync d15/d2d/d2c/f3c 0 2024-03-16T10:44:59.321 INFO:tasks.workunit.client.0.smithi002.stdout:9/453: truncate d5/d9/dc/d1e/d24/d31/f32 3266442 0 2024-03-16T10:44:59.328 INFO:tasks.workunit.client.0.smithi002.stdout:9/454: write d5/d9/d34/f61 [73541,16409] 0 2024-03-16T10:44:59.330 INFO:tasks.workunit.client.0.smithi002.stdout:6/467: rename d15/d2d/d2b/c5e to d15/d2d/d2c/d86/ca5 0 2024-03-16T10:44:59.330 INFO:tasks.workunit.client.0.smithi002.stdout:4/434: link d2/d6/l30 d2/d33/da/d5e/l87 0 2024-03-16T10:44:59.331 INFO:tasks.workunit.client.0.smithi002.stdout:5/411: creat d0/d16/d1d/d43/d48/f7f x:0 0 0 2024-03-16T10:44:59.331 INFO:tasks.workunit.client.0.smithi002.stdout:0/481: symlink d3/d28/l94 0 2024-03-16T10:44:59.331 INFO:tasks.workunit.client.0.smithi002.stdout:7/393: link d3/d61/f83 d3/f91 0 2024-03-16T10:44:59.332 INFO:tasks.workunit.client.0.smithi002.stdout:5/412: readlink d0/d16/d1d/d2b/d5d/l5c 0 2024-03-16T10:44:59.333 INFO:tasks.workunit.client.0.smithi002.stdout:9/455: symlink d5/d9/dc/d1e/d24/d31/l8b 0 2024-03-16T10:44:59.333 INFO:tasks.workunit.client.0.smithi002.stdout:0/482: stat d3/d5/f23 0 2024-03-16T10:44:59.334 INFO:tasks.workunit.client.0.smithi002.stdout:1/460: dread d2/d3/f1f [0,4194304] 0 2024-03-16T10:44:59.335 INFO:tasks.workunit.client.0.smithi002.stdout:7/394: write d3/d4/d1d/f23 [4823539,109183] 0 2024-03-16T10:44:59.342 INFO:tasks.workunit.client.0.smithi002.stdout:5/413: readlink d0/d16/d1d/d43/l47 0 2024-03-16T10:44:59.343 INFO:tasks.workunit.client.0.smithi002.stdout:1/461: rmdir d2 39 2024-03-16T10:44:59.344 INFO:tasks.workunit.client.0.smithi002.stdout:9/456: mknod d5/d9/d22/d3c/c8c 0 2024-03-16T10:44:59.344 INFO:tasks.workunit.client.0.smithi002.stdout:6/468: dwrite d15/d2d/d2b/f63 [4194304,4194304] 0 2024-03-16T10:44:59.344 INFO:tasks.workunit.client.0.smithi002.stdout:5/414: dread - d0/d16/d1d/d33/d65/f7a zero size 2024-03-16T10:44:59.345 INFO:tasks.workunit.client.0.smithi002.stdout:6/469: chown d15/d2d/d73/d97 119 1 2024-03-16T10:44:59.351 INFO:tasks.workunit.client.0.smithi002.stdout:0/483: dwrite d3/d5/f2f [0,4194304] 0 2024-03-16T10:44:59.352 INFO:tasks.workunit.client.0.smithi002.stdout:1/462: creat d2/df/d12/d86/f95 x:0 0 0 2024-03-16T10:44:59.352 INFO:tasks.workunit.client.0.smithi002.stdout:0/484: chown d3/c68 251 1 2024-03-16T10:44:59.356 INFO:tasks.workunit.client.0.smithi002.stdout:7/395: getdents d3/d4/d1d/d3d/d8c 0 2024-03-16T10:44:59.358 INFO:tasks.workunit.client.0.smithi002.stdout:9/457: dwrite d5/d9/d55/f69 [0,4194304] 0 2024-03-16T10:44:59.361 INFO:tasks.workunit.client.0.smithi002.stdout:0/485: rmdir d3/d28/d2a/d67 39 2024-03-16T10:44:59.374 INFO:tasks.workunit.client.0.smithi002.stdout:7/396: unlink d3/d4/d1d/f23 0 2024-03-16T10:44:59.375 INFO:tasks.workunit.client.0.smithi002.stdout:1/463: mkdir d2/df/d12/d84/d87/d96 0 2024-03-16T10:44:59.376 INFO:tasks.workunit.client.0.smithi002.stdout:0/486: write d3/d28/d2a/d67/f89 [1017307,122989] 0 2024-03-16T10:44:59.378 INFO:tasks.workunit.client.0.smithi002.stdout:5/415: write d0/d1f/f58 [1808561,41526] 0 2024-03-16T10:44:59.384 INFO:tasks.workunit.client.0.smithi002.stdout:6/470: creat d15/d2d/d73/fa6 x:0 0 0 2024-03-16T10:44:59.385 INFO:tasks.workunit.client.0.smithi002.stdout:0/487: dread d3/d28/f3b [0,4194304] 0 2024-03-16T10:44:59.389 INFO:tasks.workunit.client.0.smithi002.stdout:5/416: rename d0/d16/f70 to d0/d16/d4a/d39/f80 0 2024-03-16T10:44:59.390 INFO:tasks.workunit.client.0.smithi002.stdout:1/464: mknod d2/df/d12/d84/d87/d96/c97 0 2024-03-16T10:44:59.390 INFO:tasks.workunit.client.0.smithi002.stdout:5/417: chown d0/c4e 3 1 2024-03-16T10:44:59.391 INFO:tasks.workunit.client.0.smithi002.stdout:7/397: truncate d3/d61/f83 7489507 0 2024-03-16T10:44:59.391 INFO:tasks.workunit.client.0.smithi002.stdout:0/488: dwrite d3/d31/d75/f8c [0,4194304] 0 2024-03-16T10:44:59.393 INFO:tasks.workunit.client.0.smithi002.stdout:6/471: mknod d15/d2d/d73/d97/ca7 0 2024-03-16T10:44:59.394 INFO:tasks.workunit.client.0.smithi002.stdout:3/494: sync 2024-03-16T10:44:59.394 INFO:tasks.workunit.client.0.smithi002.stdout:2/494: sync 2024-03-16T10:44:59.396 INFO:tasks.workunit.client.0.smithi002.stdout:7/398: truncate d3/d4/d1c/f32 744318 0 2024-03-16T10:44:59.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.395+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4c6c04ea80 con 0x7f4c6c100bd0 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.396+0000 7f4c627fc700 1 -- 172.21.15.2:0/2134852486 <== mon.1 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1841 (secure 0 0 0) 0x7f4c5c05adf0 con 0x7f4c6c100bd0 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:44:59.398 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:44:59.399 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:59.400 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:44:59.401 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:44:59.401 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:59.401 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:59.401 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:44:59.402 INFO:tasks.workunit.client.0.smithi002.stdout:5/418: read d0/d16/f5f [8000814,80514] 0 2024-03-16T10:44:59.403 INFO:tasks.workunit.client.0.smithi002.stdout:7/399: write d3/d4/d1d/d87/d5c/d63/d3f/f43 [642056,93145] 0 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4c580717f0 msgr2=0x7f4c58073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4c580717f0 0x7f4c58073cb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c100a30 tx=0x7f4c6800b410 comp rx=0 tx=0).stop 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c100bd0 msgr2=0x7f4c6c19b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c19b270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0092e0 tx=0x7f4c5c00bd70 comp rx=0 tx=0).stop 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 shutdown_connections 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f4c580717f0 0x7f4c58073cb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c6c0ff9d0 0x7f4c6c068f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 --2- 172.21.15.2:0/2134852486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6c100bd0 0x7f4c6c19b270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.400+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 >> 172.21.15.2:0/2134852486 conn(0x7f4c6c0faf70 msgr2=0x7f4c6c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.401+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 shutdown_connections 2024-03-16T10:44:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.401+0000 7f4c7399f700 1 -- 172.21.15.2:0/2134852486 wait complete. 2024-03-16T10:44:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:44:59.410 INFO:tasks.workunit.client.0.smithi002.stdout:1/465: creat d2/df/d12/d29/d5c/f98 x:0 0 0 2024-03-16T10:44:59.410 INFO:tasks.workunit.client.0.smithi002.stdout:6/472: dwrite d15/d2d/d2b/d3f/d77/d7e/f83 [0,4194304] 0 2024-03-16T10:44:59.410 INFO:tasks.workunit.client.0.smithi002.stdout:0/489: rename d3/d28/d3d/c48 to d3/de/d6c/c95 0 2024-03-16T10:44:59.422 INFO:tasks.workunit.client.0.smithi002.stdout:0/490: fdatasync d3/d28/d3d/d44/f53 0 2024-03-16T10:44:59.424 INFO:tasks.workunit.client.0.smithi002.stdout:0/491: write d3/d5/f23 [819865,110798] 0 2024-03-16T10:44:59.424 INFO:tasks.workunit.client.0.smithi002.stdout:0/492: dread - d3/d28/d65/f77 zero size 2024-03-16T10:44:59.425 INFO:tasks.workunit.client.0.smithi002.stdout:7/400: unlink d3/l13 0 2024-03-16T10:44:59.425 INFO:tasks.workunit.client.0.smithi002.stdout:9/458: chown d5/d9/d25/f71 153876 1 2024-03-16T10:44:59.425 INFO:tasks.workunit.client.0.smithi002.stdout:2/495: dread d7/d2e/d1d/f3e [0,4194304] 0 2024-03-16T10:44:59.430 INFO:tasks.workunit.client.0.smithi002.stdout:2/496: write d7/f90 [724938,80452] 0 2024-03-16T10:44:59.437 INFO:tasks.workunit.client.0.smithi002.stdout:3/495: rename d1/dd/d28/d37/l3e to d1/dd/d28/d7b/d73/lad 0 2024-03-16T10:44:59.437 INFO:tasks.workunit.client.0.smithi002.stdout:5/419: symlink d0/d1f/l81 0 2024-03-16T10:44:59.437 INFO:tasks.workunit.client.0.smithi002.stdout:5/420: write d0/d16/d1d/d2b/d5d/f64 [672233,27060] 0 2024-03-16T10:44:59.438 INFO:tasks.workunit.client.0.smithi002.stdout:5/421: write d0/d16/f4d [976898,13497] 0 2024-03-16T10:44:59.438 INFO:tasks.workunit.client.0.smithi002.stdout:7/401: fsync d3/d4/d1d/d87/f7e 0 2024-03-16T10:44:59.438 INFO:tasks.workunit.client.0.smithi002.stdout:2/497: symlink d7/d27/d3f/d7f/l9c 0 2024-03-16T10:44:59.439 INFO:tasks.workunit.client.0.smithi002.stdout:0/493: rename d3/d28/d2a/f61 to d3/d27/f96 0 2024-03-16T10:44:59.446 INFO:tasks.workunit.client.0.smithi002.stdout:6/473: mkdir d15/d2d/da8 0 2024-03-16T10:44:59.448 INFO:tasks.workunit.client.0.smithi002.stdout:5/422: creat d0/d16/d1d/d43/d48/f82 x:0 0 0 2024-03-16T10:44:59.448 INFO:tasks.workunit.client.0.smithi002.stdout:5/423: write d0/f18 [2682899,53872] 0 2024-03-16T10:44:59.449 INFO:tasks.workunit.client.0.smithi002.stdout:8/459: sync 2024-03-16T10:44:59.451 INFO:tasks.workunit.client.0.smithi002.stdout:9/459: link d5/d9/d55/f7b d5/d9/d25/d23/f8d 0 2024-03-16T10:44:59.451 INFO:tasks.workunit.client.0.smithi002.stdout:8/460: chown d7/d55/d8f/la0 0 1 2024-03-16T10:44:59.451 INFO:tasks.workunit.client.0.smithi002.stdout:2/498: truncate d7/d2e/d1d/d2f/d4b/f62 660604 0 2024-03-16T10:44:59.461 INFO:tasks.workunit.client.0.smithi002.stdout:0/494: symlink d3/d31/d75/l97 0 2024-03-16T10:44:59.463 INFO:tasks.workunit.client.0.smithi002.stdout:6/474: creat d15/d2d/d2b/d3f/d77/fa9 x:0 0 0 2024-03-16T10:44:59.463 INFO:tasks.workunit.client.0.smithi002.stdout:1/466: dwrite d2/df/d12/f72 [0,4194304] 0 2024-03-16T10:44:59.463 INFO:tasks.workunit.client.0.smithi002.stdout:7/402: fdatasync d3/d37/f48 0 2024-03-16T10:44:59.464 INFO:tasks.workunit.client.0.smithi002.stdout:2/499: write f6 [714328,104567] 0 2024-03-16T10:44:59.464 INFO:tasks.workunit.client.0.smithi002.stdout:9/460: dwrite d5/d9/d55/d77/f7a [0,4194304] 0 2024-03-16T10:44:59.467 INFO:tasks.workunit.client.0.smithi002.stdout:7/403: stat d3/d4/d1d/d87/d5c/d63/d3f/f89 0 2024-03-16T10:44:59.469 INFO:tasks.workunit.client.0.smithi002.stdout:9/461: chown d5/d9/d55/f7b 302254 1 2024-03-16T10:44:59.469 INFO:tasks.workunit.client.0.smithi002.stdout:1/467: unlink d2/d3/d51/d56/c5b 0 2024-03-16T10:44:59.482 INFO:tasks.workunit.client.0.smithi002.stdout:8/461: mknod d7/d9/d17/d23/d2e/ca3 0 2024-03-16T10:44:59.484 INFO:tasks.workunit.client.0.smithi002.stdout:2/500: creat d7/d27/f9d x:0 0 0 2024-03-16T10:44:59.485 INFO:tasks.workunit.client.0.smithi002.stdout:3/496: link d1/dd/c82 d1/da5/cae 0 2024-03-16T10:44:59.486 INFO:tasks.workunit.client.0.smithi002.stdout:3/497: chown d1/dd/d74/f80 13533506 1 2024-03-16T10:44:59.488 INFO:tasks.workunit.client.0.smithi002.stdout:5/424: dread d0/d16/f5f [0,4194304] 0 2024-03-16T10:44:59.488 INFO:tasks.workunit.client.0.smithi002.stdout:3/498: write d1/dd/d28/d7b/d8a/d93/f51 [2332688,25439] 0 2024-03-16T10:44:59.489 INFO:tasks.workunit.client.0.smithi002.stdout:6/475: mknod d15/d2d/d2b/caa 0 2024-03-16T10:44:59.489 INFO:tasks.workunit.client.0.smithi002.stdout:5/425: dread d0/d16/d4a/d39/f80 [0,4194304] 0 2024-03-16T10:44:59.489 INFO:tasks.workunit.client.0.smithi002.stdout:7/404: rmdir d3/d4/d1d/d87/d5c/d63 39 2024-03-16T10:44:59.489 INFO:tasks.workunit.client.0.smithi002.stdout:7/405: write d3/d61/d66/f50 [2259176,5620] 0 2024-03-16T10:44:59.489 INFO:tasks.workunit.client.0.smithi002.stdout:9/462: mkdir d5/d9/dc/d28/d8e 0 2024-03-16T10:44:59.489 INFO:tasks.workunit.client.0.smithi002.stdout:1/468: mknod d2/df/d12/d84/d87/d96/c99 0 2024-03-16T10:44:59.489 INFO:tasks.workunit.client.0.smithi002.stdout:8/462: symlink d7/d9/d17/d23/la4 0 2024-03-16T10:44:59.498 INFO:tasks.workunit.client.0.smithi002.stdout:2/501: rmdir d7/db 39 2024-03-16T10:44:59.498 INFO:tasks.workunit.client.0.smithi002.stdout:2/502: dwrite d7/d27/d3f/f69 [4194304,4194304] 0 2024-03-16T10:44:59.498 INFO:tasks.workunit.client.0.smithi002.stdout:6/476: unlink d15/d2d/f2a 0 2024-03-16T10:44:59.509 INFO:tasks.workunit.client.0.smithi002.stdout:2/503: dread d7/d2e/d28/d44/f76 [0,4194304] 0 2024-03-16T10:44:59.509 INFO:tasks.workunit.client.0.smithi002.stdout:6/477: dwrite d15/f29 [0,4194304] 0 2024-03-16T10:44:59.509 INFO:tasks.workunit.client.0.smithi002.stdout:0/495: truncate d3/d28/d3d/f50 39616 0 2024-03-16T10:44:59.509 INFO:tasks.workunit.client.0.smithi002.stdout:9/463: creat d5/d7e/f8f x:0 0 0 2024-03-16T10:44:59.510 INFO:tasks.workunit.client.0.smithi002.stdout:1/469: creat d2/d3/d1c/d5d/f9a x:0 0 0 2024-03-16T10:44:59.510 INFO:tasks.workunit.client.0.smithi002.stdout:1/470: write d2/d3/d1c/d5d/f92 [785126,80658] 0 2024-03-16T10:44:59.510 INFO:tasks.workunit.client.0.smithi002.stdout:8/463: creat d7/d9/d17/d23/d2e/d3a/fa5 x:0 0 0 2024-03-16T10:44:59.515 INFO:tasks.workunit.client.0.smithi002.stdout:7/406: fdatasync d3/f14 0 2024-03-16T10:44:59.525 INFO:tasks.workunit.client.0.smithi002.stdout:9/464: mkdir d5/d9/dc/d28/d90 0 2024-03-16T10:44:59.525 INFO:tasks.workunit.client.0.smithi002.stdout:9/465: readlink d5/d9/d25/d23/l52 0 2024-03-16T10:44:59.526 INFO:tasks.workunit.client.0.smithi002.stdout:1/471: symlink d2/df/d12/d29/d5c/l9b 0 2024-03-16T10:44:59.526 INFO:tasks.workunit.client.0.smithi002.stdout:9/466: dread - d5/d9/dc/f74 zero size 2024-03-16T10:44:59.527 INFO:tasks.workunit.client.0.smithi002.stdout:9/467: read - d5/d9/d34/f68 zero size 2024-03-16T10:44:59.530 INFO:tasks.workunit.client.0.smithi002.stdout:8/464: mknod d7/d9/d17/d23/d2e/d5c/ca6 0 2024-03-16T10:44:59.530 INFO:tasks.workunit.client.0.smithi002.stdout:5/426: getdents d0/d16/d1d/d43 0 2024-03-16T10:44:59.531 INFO:tasks.workunit.client.0.smithi002.stdout:7/407: creat d3/d4/d1d/d87/f92 x:0 0 0 2024-03-16T10:44:59.531 INFO:tasks.workunit.client.0.smithi002.stdout:6/478: mkdir d15/dab 0 2024-03-16T10:44:59.549 INFO:tasks.workunit.client.0.smithi002.stdout:0/496: truncate d3/d28/d3d/d44/f53 4986884 0 2024-03-16T10:44:59.550 INFO:tasks.workunit.client.0.smithi002.stdout:1/472: readlink d2/df/d12/d29/l6c 0 2024-03-16T10:44:59.550 INFO:tasks.workunit.client.0.smithi002.stdout:1/473: write d2/d3/f65 [5014863,58470] 0 2024-03-16T10:44:59.554 INFO:tasks.workunit.client.0.smithi002.stdout:9/468: rename d5/d9/d55/d77 to d5/d9/dc/d1e/d91 0 2024-03-16T10:44:59.554 INFO:tasks.workunit.client.0.smithi002.stdout:8/465: fdatasync d7/d9/d17/d23/d39/f72 0 2024-03-16T10:44:59.554 INFO:tasks.workunit.client.0.smithi002.stdout:0/497: dwrite d3/d28/f29 [0,4194304] 0 2024-03-16T10:44:59.554 INFO:tasks.workunit.client.0.smithi002.stdout:8/466: dread - d7/d9/d17/d23/d2e/d34/f9e zero size 2024-03-16T10:44:59.554 INFO:tasks.workunit.client.0.smithi002.stdout:6/479: creat d15/d80/fac x:0 0 0 2024-03-16T10:44:59.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 -- 172.21.15.2:0/3800611651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378074de0 msgr2=0x7f4378073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:59.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 --2- 172.21.15.2:0/3800611651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378074de0 0x7f4378073240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4368009910 tx=0x7f4368009c20 comp rx=0 tx=0).stop 2024-03-16T10:44:59.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 -- 172.21.15.2:0/3800611651 shutdown_connections 2024-03-16T10:44:59.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 --2- 172.21.15.2:0/3800611651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4378073810 0x7f4378073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 --2- 172.21.15.2:0/3800611651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378074de0 0x7f4378073240 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 -- 172.21.15.2:0/3800611651 >> 172.21.15.2:0/3800611651 conn(0x7f43780fc0a0 msgr2=0x7f43780fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:59.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 -- 172.21.15.2:0/3800611651 shutdown_connections 2024-03-16T10:44:59.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.549+0000 7f437e2c0700 1 -- 172.21.15.2:0/3800611651 wait complete. 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3191199488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: Standby manager daemon smithi002.zevrtm restarted 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: Standby manager daemon smithi002.zevrtm started 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/crt"}]: dispatch 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/key"}]: dispatch 2024-03-16T10:44:59.557 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:44:59.558 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:44:59 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2134852486' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:44:59.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.550+0000 7f437e2c0700 1 Processor -- start 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.550+0000 7f437e2c0700 1 -- start start 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.550+0000 7f437e2c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378073810 0x7f437819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.550+0000 7f437e2c0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4378074de0 0x7f437819d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.550+0000 7f437e2c0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819d750 con 0x7f4378073810 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.550+0000 7f437e2c0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819d890 con 0x7f4378074de0 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.550+0000 7f4377fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378073810 0x7f437819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f4377fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378073810 0x7f437819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57564/0 (socket says 172.21.15.2:57564) 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f4377fff700 1 -- 172.21.15.2:0/2200716146 learned_addr learned my addr 172.21.15.2:0/2200716146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:44:59.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f4377fff700 1 -- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4378074de0 msgr2=0x7f437819d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f4377fff700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4378074de0 0x7f437819d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f4377fff700 1 -- 172.21.15.2:0/2200716146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680095f0 con 0x7f4378073810 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f4377fff700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378073810 0x7f437819cbf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f43680059d0 tx=0x7f436800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f43757fa700 1 -- 172.21.15.2:0/2200716146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f436801d070 con 0x7f4378073810 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f43757fa700 1 -- 172.21.15.2:0/2200716146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f436800f460 con 0x7f4378073810 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43781a2290 con 0x7f4378073810 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f43757fa700 1 -- 172.21.15.2:0/2200716146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4368021620 con 0x7f4378073810 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.551+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43781a2800 con 0x7f4378073810 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.552+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437804ea80 con 0x7f4378073810 2024-03-16T10:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.553+0000 7f43757fa700 1 -- 172.21.15.2:0/2200716146 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f436802b440 con 0x7f4378073810 2024-03-16T10:44:59.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.553+0000 7f43757fa700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f436006dd60 0x7f4360070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:44:59.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.553+0000 7f43757fa700 1 -- 172.21.15.2:0/2200716146 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f436808ece0 con 0x7f4378073810 2024-03-16T10:44:59.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.553+0000 7f43777fe700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f436006dd60 0x7f4360070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:44:59.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.554+0000 7f43777fe700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f436006dd60 0x7f4360070220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4378074b10 tx=0x7f436c006d20 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:44:59.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.558+0000 7f43757fa700 1 -- 172.21.15.2:0/2200716146 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f436805c100 con 0x7f4378073810 2024-03-16T10:44:59.563 INFO:tasks.workunit.client.0.smithi002.stdout:0/498: unlink d3/d28/c91 0 2024-03-16T10:44:59.570 INFO:tasks.workunit.client.0.smithi002.stdout:0/499: chown d3/d31/d75/f81 461 1 2024-03-16T10:44:59.570 INFO:tasks.workunit.client.0.smithi002.stdout:8/467: creat d7/d9/d17/d83/fa7 x:0 0 0 2024-03-16T10:44:59.573 INFO:tasks.workunit.client.0.smithi002.stdout:6/480: chown d15/d2d/d2c/d5d/c62 1979932 1 2024-03-16T10:44:59.573 INFO:tasks.workunit.client.0.smithi002.stdout:1/474: truncate d2/df/f45 3102796 0 2024-03-16T10:44:59.573 INFO:tasks.workunit.client.0.smithi002.stdout:3/499: getdents d1/dd/d28/d7b/d89/d15/d87/d8c 0 2024-03-16T10:44:59.573 INFO:tasks.workunit.client.0.smithi002.stdout:9/469: fsync d5/d9/dc/f11 0 2024-03-16T10:44:59.573 INFO:tasks.workunit.client.0.smithi002.stdout:3/500: dread - d1/dd/d28/d7b/d2c/fa3 zero size 2024-03-16T10:44:59.573 INFO:tasks.workunit.client.0.smithi002.stdout:8/468: creat d7/d9/d25/d91/fa8 x:0 0 0 2024-03-16T10:44:59.573 INFO:tasks.workunit.client.0.smithi002.stdout:6/481: dwrite d15/f5f [0,4194304] 0 2024-03-16T10:44:59.575 INFO:tasks.workunit.client.0.smithi002.stdout:3/501: read d1/dd/d28/d7b/d89/d15/f53 [1633404,110] 0 2024-03-16T10:44:59.577 INFO:tasks.workunit.client.0.smithi002.stdout:6/482: write d15/d2d/d2c/f3d [3887390,96446] 0 2024-03-16T10:44:59.578 INFO:tasks.workunit.client.0.smithi002.stdout:9/470: dwrite d5/f29 [0,4194304] 0 2024-03-16T10:44:59.578 INFO:tasks.workunit.client.0.smithi002.stdout:3/502: fsync d1/dd/d28/d7b/d8a/d93/f91 0 2024-03-16T10:44:59.580 INFO:tasks.workunit.client.0.smithi002.stdout:0/500: link d3/d28/f8e d3/d31/d75/f98 0 2024-03-16T10:44:59.581 INFO:tasks.workunit.client.0.smithi002.stdout:3/503: write d1/dd/d28/d7b/d2c/f90 [2427140,86094] 0 2024-03-16T10:44:59.582 INFO:tasks.workunit.client.0.smithi002.stdout:1/475: rename d2/d3/d1c/d5d/f9a to d2/df/d12/d84/d87/d96/f9c 0 2024-03-16T10:44:59.585 INFO:tasks.workunit.client.0.smithi002.stdout:4/435: sync 2024-03-16T10:44:59.596 INFO:tasks.workunit.client.0.smithi002.stdout:3/504: dwrite d1/f52 [0,4194304] 0 2024-03-16T10:44:59.597 INFO:tasks.workunit.client.0.smithi002.stdout:8/469: dread d7/d9/d17/d20/f6b [0,4194304] 0 2024-03-16T10:44:59.598 INFO:tasks.workunit.client.0.smithi002.stdout:9/471: creat d5/d9/dc/f92 x:0 0 0 2024-03-16T10:44:59.606 INFO:tasks.workunit.client.0.smithi002.stdout:9/472: write d5/f7 [3220083,102616] 0 2024-03-16T10:44:59.609 INFO:tasks.workunit.client.0.smithi002.stdout:1/476: dread d2/d3/fc [0,4194304] 0 2024-03-16T10:44:59.611 INFO:tasks.workunit.client.0.smithi002.stdout:4/436: dread - d2/f79 zero size 2024-03-16T10:44:59.611 INFO:tasks.workunit.client.0.smithi002.stdout:6/483: mknod d15/dab/cad 0 2024-03-16T10:44:59.613 INFO:tasks.workunit.client.0.smithi002.stdout:4/437: dread d2/d33/f6a [0,4194304] 0 2024-03-16T10:44:59.613 INFO:tasks.workunit.client.0.smithi002.stdout:3/505: creat d1/dd/d28/d7b/d2c/d63/faf x:0 0 0 2024-03-16T10:44:59.613 INFO:tasks.workunit.client.0.smithi002.stdout:4/438: write d2/d33/da/f41 [4874091,45689] 0 2024-03-16T10:44:59.613 INFO:tasks.workunit.client.0.smithi002.stdout:4/439: stat d2/d33/d35/d59/f60 0 2024-03-16T10:44:59.614 INFO:tasks.workunit.client.0.smithi002.stdout:6/484: dread d15/f25 [12582912,4194304] 0 2024-03-16T10:44:59.614 INFO:tasks.workunit.client.0.smithi002.stdout:7/408: creat d3/d4/d1d/d87/d5c/d63/f93 x:0 0 0 2024-03-16T10:44:59.618 INFO:tasks.workunit.client.0.smithi002.stdout:3/506: creat d1/fb0 x:0 0 0 2024-03-16T10:44:59.634 INFO:tasks.workunit.client.0.smithi002.stdout:0/501: write d3/d5/f30 [96929,96500] 0 2024-03-16T10:44:59.634 INFO:tasks.workunit.client.0.smithi002.stdout:9/473: truncate d5/d9/dc/d1e/d24/d31/f4a 767319 0 2024-03-16T10:44:59.634 INFO:tasks.workunit.client.0.smithi002.stdout:7/409: dread d3/d37/d6a/f6e [0,4194304] 0 2024-03-16T10:44:59.634 INFO:tasks.workunit.client.0.smithi002.stdout:4/440: mkdir d2/d33/da/d88 0 2024-03-16T10:44:59.669 INFO:tasks.workunit.client.0.smithi002.stdout:1/477: creat d2/d3/f9d x:0 0 0 2024-03-16T10:44:59.671 INFO:tasks.workunit.client.0.smithi002.stdout:0/502: creat d3/d31/f99 x:0 0 0 2024-03-16T10:44:59.673 INFO:tasks.workunit.client.0.smithi002.stdout:6/485: creat d15/d2d/d8a/fae x:0 0 0 2024-03-16T10:44:59.673 INFO:tasks.workunit.client.0.smithi002.stdout:4/441: creat d2/d6/f89 x:0 0 0 2024-03-16T10:44:59.673 INFO:tasks.workunit.client.0.smithi002.stdout:8/470: dwrite d7/d9/d17/d23/d2e/d34/f71 [0,4194304] 0 2024-03-16T10:44:59.675 INFO:tasks.workunit.client.0.smithi002.stdout:1/478: readlink d2/df/l26 0 2024-03-16T10:44:59.684 INFO:tasks.workunit.client.0.smithi002.stdout:4/442: read d2/d6/d32/f50 [2485184,111954] 0 2024-03-16T10:44:59.684 INFO:tasks.workunit.client.0.smithi002.stdout:8/471: read d7/d9/d17/f2d [3376095,1307] 0 2024-03-16T10:44:59.686 INFO:tasks.workunit.client.0.smithi002.stdout:9/474: dwrite d5/d9/dc/d1e/f36 [0,4194304] 0 2024-03-16T10:44:59.691 INFO:tasks.workunit.client.0.smithi002.stdout:7/410: dwrite d3/d4/d1d/d87/d5c/d63/d34/f6b [0,4194304] 0 2024-03-16T10:44:59.693 INFO:tasks.workunit.client.0.smithi002.stdout:1/479: symlink d2/d3/d1c/d5d/d71/l9e 0 2024-03-16T10:44:59.694 INFO:tasks.workunit.client.0.smithi002.stdout:4/443: mknod d2/d33/da/d2e/c8a 0 2024-03-16T10:44:59.694 INFO:tasks.workunit.client.0.smithi002.stdout:1/480: stat d2/f13 0 2024-03-16T10:44:59.695 INFO:tasks.workunit.client.0.smithi002.stdout:6/486: rename d15/d2d/d2b/d3f/l60 to d15/d2d/d73/laf 0 2024-03-16T10:44:59.696 INFO:tasks.workunit.client.0.smithi002.stdout:1/481: fsync d2/df/d12/f72 0 2024-03-16T10:44:59.696 INFO:tasks.workunit.client.0.smithi002.stdout:0/503: dwrite d3/d27/f55 [4194304,4194304] 0 2024-03-16T10:44:59.697 INFO:tasks.workunit.client.0.smithi002.stdout:7/411: creat d3/d61/f94 x:0 0 0 2024-03-16T10:44:59.697 INFO:tasks.workunit.client.0.smithi002.stdout:9/475: truncate d5/d9/dc/d1e/f36 4600881 0 2024-03-16T10:44:59.698 INFO:tasks.workunit.client.0.smithi002.stdout:4/444: write d2/d33/d1c/f1d [3816935,128418] 0 2024-03-16T10:44:59.703 INFO:tasks.workunit.client.0.smithi002.stdout:7/412: chown d3/d61 1 1 2024-03-16T10:44:59.704 INFO:tasks.workunit.client.0.smithi002.stdout:8/472: dwrite d7/d9/d17/d23/f4d [0,4194304] 0 2024-03-16T10:44:59.707 INFO:tasks.workunit.client.0.smithi002.stdout:1/482: chown d2/d3/f39 364892232 1 2024-03-16T10:44:59.713 INFO:tasks.workunit.client.0.smithi002.stdout:6/487: dread d15/d2d/f2e [0,4194304] 0 2024-03-16T10:44:59.713 INFO:tasks.workunit.client.0.smithi002.stdout:4/445: creat d2/d33/da/d5e/f8b x:0 0 0 2024-03-16T10:44:59.713 INFO:tasks.workunit.client.0.smithi002.stdout:0/504: symlink d3/d28/l9a 0 2024-03-16T10:44:59.716 INFO:tasks.workunit.client.0.smithi002.stdout:9/476: mknod d5/d9/dc/d28/d8e/c93 0 2024-03-16T10:44:59.717 INFO:tasks.workunit.client.0.smithi002.stdout:8/473: rename d7/d9/d17/d23/d2e/d34/f71 to d7/d9/d17/d23/fa9 0 2024-03-16T10:44:59.717 INFO:tasks.workunit.client.0.smithi002.stdout:6/488: rmdir d15 39 2024-03-16T10:44:59.719 INFO:tasks.workunit.client.0.smithi002.stdout:1/483: dwrite d2/d3/d1c/d5d/f92 [0,4194304] 0 2024-03-16T10:44:59.720 INFO:tasks.workunit.client.0.smithi002.stdout:7/413: creat d3/d4/d1d/d3d/d8c/f95 x:0 0 0 2024-03-16T10:44:59.721 INFO:tasks.workunit.client.0.smithi002.stdout:0/505: write d3/d27/f96 [59201,3897] 0 2024-03-16T10:44:59.722 INFO:tasks.workunit.client.0.smithi002.stdout:8/474: symlink d7/d9/d25/d91/laa 0 2024-03-16T10:44:59.726 INFO:tasks.workunit.client.0.smithi002.stdout:7/414: chown d3/d4/c31 85009 1 2024-03-16T10:44:59.727 INFO:tasks.workunit.client.0.smithi002.stdout:1/484: chown d2/d3/d51/f5a 1841 1 2024-03-16T10:44:59.728 INFO:tasks.workunit.client.0.smithi002.stdout:4/446: dwrite d2/d33/d35/f6f [0,4194304] 0 2024-03-16T10:44:59.728 INFO:tasks.workunit.client.0.smithi002.stdout:0/506: write d3/d28/d2a/d67/f8d [59606,54078] 0 2024-03-16T10:44:59.731 INFO:tasks.workunit.client.0.smithi002.stdout:9/477: dread d5/d9/d55/f63 [0,4194304] 0 2024-03-16T10:44:59.731 INFO:tasks.workunit.client.0.smithi002.stdout:1/485: dread - d2/df/d12/d29/d5c/f83 zero size 2024-03-16T10:44:59.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:59.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:44:59.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3191199488' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:44:59.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: Standby manager daemon smithi002.zevrtm restarted 2024-03-16T10:44:59.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: Standby manager daemon smithi002.zevrtm started 2024-03-16T10:44:59.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/crt"}]: dispatch 2024-03-16T10:44:59.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:44:59.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi002.zevrtm/key"}]: dispatch 2024-03-16T10:44:59.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:44:59.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:44:59 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2134852486' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:44:59.733 INFO:tasks.workunit.client.0.smithi002.stdout:8/475: creat d7/d9/d17/d83/da2/fab x:0 0 0 2024-03-16T10:44:59.735 INFO:tasks.workunit.client.0.smithi002.stdout:0/507: truncate d3/d28/f4d 2575349 0 2024-03-16T10:44:59.736 INFO:tasks.workunit.client.0.smithi002.stdout:7/415: truncate d3/d4/d1d/d3d/d5e/f84 424690 0 2024-03-16T10:44:59.737 INFO:tasks.workunit.client.0.smithi002.stdout:7/416: chown d3/d4/d1c/f81 7 1 2024-03-16T10:44:59.737 INFO:tasks.workunit.client.0.smithi002.stdout:4/447: creat d2/d33/da/d88/f8c x:0 0 0 2024-03-16T10:44:59.739 INFO:tasks.workunit.client.0.smithi002.stdout:7/417: dread - d3/d4/d1d/d87/d5c/d63/d3f/f89 zero size 2024-03-16T10:44:59.741 INFO:tasks.workunit.client.0.smithi002.stdout:1/486: rename d2/df/l26 to d2/d3/d1c/d5d/l9f 0 2024-03-16T10:44:59.747 INFO:tasks.workunit.client.0.smithi002.stdout:8/476: truncate d7/d9/f51 695063 0 2024-03-16T10:44:59.747 INFO:tasks.workunit.client.0.smithi002.stdout:4/448: creat d2/d33/d1c/f8d x:0 0 0 2024-03-16T10:44:59.751 INFO:tasks.workunit.client.0.smithi002.stdout:1/487: mkdir d2/d3/d1c/da0 0 2024-03-16T10:44:59.751 INFO:tasks.workunit.client.0.smithi002.stdout:9/478: truncate d5/d9/d25/f42 2169705 0 2024-03-16T10:44:59.753 INFO:tasks.workunit.client.0.smithi002.stdout:0/508: dread d3/de/f51 [0,4194304] 0 2024-03-16T10:44:59.758 INFO:tasks.workunit.client.0.smithi002.stdout:1/488: mknod d2/d3/d51/ca1 0 2024-03-16T10:44:59.759 INFO:tasks.workunit.client.0.smithi002.stdout:1/489: truncate d2/d3/d51/d56/f8c 627721 0 2024-03-16T10:44:59.759 INFO:tasks.workunit.client.0.smithi002.stdout:9/479: mkdir d5/d9/dc/d1e/d94 0 2024-03-16T10:44:59.765 INFO:tasks.workunit.client.0.smithi002.stdout:1/490: dwrite d2/d3/f7c [0,4194304] 0 2024-03-16T10:44:59.766 INFO:tasks.workunit.client.0.smithi002.stdout:7/418: rename d3/d4/d1d/d3d to d3/d4/d1d/d87/d96 0 2024-03-16T10:44:59.768 INFO:tasks.workunit.client.0.smithi002.stdout:8/477: write d7/d9/d17/d23/d39/f72 [2135636,105538] 0 2024-03-16T10:44:59.771 INFO:tasks.workunit.client.0.smithi002.stdout:4/449: write d2/d33/da/f5f [1007683,61080] 0 2024-03-16T10:44:59.774 INFO:tasks.workunit.client.0.smithi002.stdout:4/450: write d2/d33/da/d5e/f75 [761806,56279] 0 2024-03-16T10:44:59.775 INFO:tasks.workunit.client.0.smithi002.stdout:1/491: dwrite d2/d3/d51/d56/f8c [0,4194304] 0 2024-03-16T10:44:59.783 INFO:tasks.workunit.client.0.smithi002.stdout:8/478: rename d7/f48 to d7/d9/d17/d23/d2e/d3a/fac 0 2024-03-16T10:44:59.785 INFO:tasks.workunit.client.0.smithi002.stdout:7/419: dwrite d3/d4/d1d/d87/d5c/d63/f4c [0,4194304] 0 2024-03-16T10:44:59.786 INFO:tasks.workunit.client.0.smithi002.stdout:8/479: write d7/d9/d17/d23/d39/f72 [3539841,64833] 0 2024-03-16T10:44:59.788 INFO:tasks.workunit.client.0.smithi002.stdout:7/420: write d3/d4/d1d/d87/d5c/d63/d3f/f43 [3040249,71224] 0 2024-03-16T10:44:59.788 INFO:tasks.workunit.client.0.smithi002.stdout:9/480: creat d5/d9/dc/d1e/d94/f95 x:0 0 0 2024-03-16T10:44:59.791 INFO:tasks.workunit.client.0.smithi002.stdout:0/509: creat d3/d28/f9b x:0 0 0 2024-03-16T10:44:59.803 INFO:tasks.workunit.client.0.smithi002.stdout:9/481: dread - d5/d9/d25/d23/f33 zero size 2024-03-16T10:44:59.804 INFO:tasks.workunit.client.0.smithi002.stdout:9/482: chown d5/d9/d25/d23/l52 14 1 2024-03-16T10:44:59.805 INFO:tasks.workunit.client.0.smithi002.stdout:0/510: mkdir d3/de/d6c/d9c 0 2024-03-16T10:44:59.807 INFO:tasks.workunit.client.0.smithi002.stdout:1/492: rename d2/df/c42 to d2/df/d12/d84/ca2 0 2024-03-16T10:44:59.824 INFO:tasks.workunit.client.0.smithi002.stdout:5/427: sync 2024-03-16T10:44:59.824 INFO:tasks.workunit.client.0.smithi002.stdout:2/504: sync 2024-03-16T10:44:59.838 INFO:tasks.workunit.client.0.smithi002.stdout:0/511: dread d3/d28/d3d/f74 [0,4194304] 0 2024-03-16T10:44:59.839 INFO:tasks.workunit.client.0.smithi002.stdout:5/428: fsync d0/d16/d4a/d39/f51 0 2024-03-16T10:44:59.843 INFO:tasks.workunit.client.0.smithi002.stdout:1/493: write d2/f68 [46136,71468] 0 2024-03-16T10:44:59.843 INFO:tasks.workunit.client.0.smithi002.stdout:4/451: getdents d2/d33/d35/d59 0 2024-03-16T10:44:59.843 INFO:tasks.workunit.client.0.smithi002.stdout:8/480: rename l3 to d7/d9/d25/d67/lad 0 2024-03-16T10:44:59.843 INFO:tasks.workunit.client.0.smithi002.stdout:1/494: stat d2/df/d12/f60 0 2024-03-16T10:44:59.843 INFO:tasks.workunit.client.0.smithi002.stdout:5/429: dread - d0/d16/d1d/d33/f42 zero size 2024-03-16T10:44:59.843 INFO:tasks.workunit.client.0.smithi002.stdout:2/505: write d7/d2e/d1d/d2f/f9a [7687212,85155] 0 2024-03-16T10:44:59.844 INFO:tasks.workunit.client.0.smithi002.stdout:8/481: read - d7/d9/d25/d67/f81 zero size 2024-03-16T10:44:59.844 INFO:tasks.workunit.client.0.smithi002.stdout:4/452: creat d2/d33/da/d2e/f8e x:0 0 0 2024-03-16T10:44:59.847 INFO:tasks.workunit.client.0.smithi002.stdout:9/483: dread d5/d9/d22/d3c/d4d/f54 [0,4194304] 0 2024-03-16T10:44:59.851 INFO:tasks.workunit.client.0.smithi002.stdout:8/482: dwrite d7/d9/d17/d23/f33 [0,4194304] 0 2024-03-16T10:44:59.852 INFO:tasks.workunit.client.0.smithi002.stdout:5/430: mknod d0/c83 0 2024-03-16T10:44:59.853 INFO:tasks.workunit.client.0.smithi002.stdout:4/453: write d2/d33/da/d2e/f56 [4081779,73131] 0 2024-03-16T10:44:59.855 INFO:tasks.workunit.client.0.smithi002.stdout:4/454: write d2/d33/da/d88/f8c [113493,102017] 0 2024-03-16T10:44:59.855 INFO:tasks.workunit.client.0.smithi002.stdout:2/506: dread d7/f2b [0,4194304] 0 2024-03-16T10:44:59.866 INFO:tasks.workunit.client.0.smithi002.stdout:8/483: dread d7/d9/d17/d23/d2e/f92 [0,4194304] 0 2024-03-16T10:44:59.867 INFO:tasks.workunit.client.0.smithi002.stdout:8/484: stat d7/d55/l66 0 2024-03-16T10:44:59.867 INFO:tasks.workunit.client.0.smithi002.stdout:1/495: link d2/c8d d2/df/d12/d29/d5c/ca3 0 2024-03-16T10:44:59.869 INFO:tasks.workunit.client.0.smithi002.stdout:2/507: dread d7/d2e/d1d/f41 [0,4194304] 0 2024-03-16T10:44:59.875 INFO:tasks.workunit.client.0.smithi002.stdout:1/496: write d2/df/d12/d84/d87/d96/f9c [400933,84180] 0 2024-03-16T10:44:59.876 INFO:tasks.workunit.client.0.smithi002.stdout:4/455: dread d2/d33/da/f4f [0,4194304] 0 2024-03-16T10:44:59.876 INFO:tasks.workunit.client.0.smithi002.stdout:4/456: readlink d2/d6/d32/d66/l6b 0 2024-03-16T10:44:59.877 INFO:tasks.workunit.client.0.smithi002.stdout:4/457: write d2/d6/f53 [3954683,130753] 0 2024-03-16T10:44:59.878 INFO:tasks.workunit.client.0.smithi002.stdout:4/458: chown d2/d33/d1c/c57 5643 1 2024-03-16T10:44:59.881 INFO:tasks.workunit.client.0.smithi002.stdout:1/497: fsync d2/f52 0 2024-03-16T10:44:59.881 INFO:tasks.workunit.client.0.smithi002.stdout:1/498: readlink d2/df/l73 0 2024-03-16T10:44:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.887+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 --> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4378103b90 con 0x7f436006dd60 2024-03-16T10:44:59.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.889+0000 7f43757fa700 1 -- 172.21.15.2:0/2200716146 <== mgr.24375 v2:172.21.15.26:6828/274393035 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f4378103b90 con 0x7f436006dd60 2024-03-16T10:44:59.890 INFO:tasks.workunit.client.0.smithi002.stdout:1/499: unlink d2/df/d12/d29/d5c/f83 0 2024-03-16T10:44:59.890 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:44:59.890 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:44:59.891 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:44:59.891 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:44:59.891 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [], 2024-03-16T10:44:59.891 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "", 2024-03-16T10:44:59.891 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading mgr daemons", 2024-03-16T10:44:59.891 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:44:59.891 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:44:59.891 INFO:tasks.workunit.client.0.smithi002.stdout:8/485: link d7/d9/d17/d23/d2e/l42 d7/d9/d17/d20/lae 0 2024-03-16T10:44:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.891+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f436006dd60 msgr2=0x7f4360070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.891+0000 7f437e2c0700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f436006dd60 0x7f4360070220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4378074b10 tx=0x7f436c006d20 comp rx=0 tx=0).stop 2024-03-16T10:44:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.891+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378073810 msgr2=0x7f437819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.891+0000 7f437e2c0700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378073810 0x7f437819cbf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f43680059d0 tx=0x7f436800be20 comp rx=0 tx=0).stop 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.892+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 shutdown_connections 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.892+0000 7f437e2c0700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f436006dd60 0x7f4360070220 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.892+0000 7f437e2c0700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4378073810 0x7f437819cbf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.892+0000 7f437e2c0700 1 --2- 172.21.15.2:0/2200716146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4378074de0 0x7f437819d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.892+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 >> 172.21.15.2:0/2200716146 conn(0x7f43780fc0a0 msgr2=0x7f4378102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.892+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 shutdown_connections 2024-03-16T10:44:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:44:59.892+0000 7f437e2c0700 1 -- 172.21.15.2:0/2200716146 wait complete. 2024-03-16T10:44:59.893 INFO:tasks.workunit.client.0.smithi002.stdout:4/459: getdents d2/d6/d32/d66/d7f 0 2024-03-16T10:44:59.896 INFO:tasks.workunit.client.0.smithi002.stdout:1/500: rmdir d2/df/d12/d86 39 2024-03-16T10:44:59.899 INFO:tasks.workunit.client.0.smithi002.stdout:8/486: rmdir d7/d9/d17/d23/d2e/d3a 39 2024-03-16T10:44:59.905 INFO:tasks.workunit.client.0.smithi002.stdout:8/487: chown d7/d13/l1b 28577 1 2024-03-16T10:44:59.914 INFO:tasks.workunit.client.0.smithi002.stdout:4/460: rename d2/d6/d32/d68/f52 to d2/d6/d32/f8f 0 2024-03-16T10:44:59.914 INFO:tasks.workunit.client.0.smithi002.stdout:4/461: readlink d2/d33/l2b 0 2024-03-16T10:44:59.919 INFO:tasks.workunit.client.0.smithi002.stdout:4/462: symlink d2/l90 0 2024-03-16T10:44:59.920 INFO:tasks.workunit.client.0.smithi002.stdout:9/484: truncate d5/d9/f13 2162195 0 2024-03-16T10:44:59.922 INFO:tasks.workunit.client.0.smithi002.stdout:2/508: dwrite d7/d2e/d1d/d2f/f59 [4194304,4194304] 0 2024-03-16T10:44:59.926 INFO:tasks.workunit.client.0.smithi002.stdout:4/463: rmdir d2/d33/d35/d59 39 2024-03-16T10:44:59.928 INFO:tasks.workunit.client.0.smithi002.stdout:1/501: write d2/d3/d51/f5a [3666696,125778] 0 2024-03-16T10:44:59.929 INFO:tasks.workunit.client.0.smithi002.stdout:3/507: sync 2024-03-16T10:44:59.929 INFO:tasks.workunit.client.0.smithi002.stdout:6/489: sync 2024-03-16T10:44:59.929 INFO:tasks.workunit.client.0.smithi002.stdout:2/509: creat d7/d2e/d1d/f9e x:0 0 0 2024-03-16T10:44:59.931 INFO:tasks.workunit.client.0.smithi002.stdout:3/508: write d1/dd/f9e [391375,7699] 0 2024-03-16T10:44:59.931 INFO:tasks.workunit.client.0.smithi002.stdout:2/510: write f6 [2818404,109325] 0 2024-03-16T10:44:59.932 INFO:tasks.workunit.client.0.smithi002.stdout:4/464: creat d2/d33/d1c/d45/f91 x:0 0 0 2024-03-16T10:44:59.936 INFO:tasks.workunit.client.0.smithi002.stdout:6/490: fsync d15/f2f 0 2024-03-16T10:44:59.941 INFO:tasks.workunit.client.0.smithi002.stdout:3/509: mknod d1/dd/d74/cb1 0 2024-03-16T10:44:59.943 INFO:tasks.workunit.client.0.smithi002.stdout:3/510: fsync d1/dd/d28/d7b/d89/f7f 0 2024-03-16T10:44:59.943 INFO:tasks.workunit.client.0.smithi002.stdout:2/511: write d7/d2e/d28/d44/f76 [829087,39511] 0 2024-03-16T10:44:59.945 INFO:tasks.workunit.client.0.smithi002.stdout:3/511: write d1/dd/d28/d7b/d8a/d93/f57 [408837,56675] 0 2024-03-16T10:44:59.947 INFO:tasks.workunit.client.0.smithi002.stdout:6/491: read - d15/d2d/d2c/f3b zero size 2024-03-16T10:44:59.947 INFO:tasks.workunit.client.0.smithi002.stdout:3/512: fdatasync d1/dd/d28/d7b/d2c/fa3 0 2024-03-16T10:44:59.948 INFO:tasks.workunit.client.0.smithi002.stdout:9/485: getdents d5/d9/dc/d28 0 2024-03-16T10:44:59.948 INFO:tasks.workunit.client.0.smithi002.stdout:6/492: write d15/d2d/d8a/d7c/d87/fa4 [591052,62501] 0 2024-03-16T10:44:59.951 INFO:tasks.workunit.client.0.smithi002.stdout:1/502: creat d2/fa4 x:0 0 0 2024-03-16T10:44:59.952 INFO:tasks.workunit.client.0.smithi002.stdout:6/493: dread d15/f18 [0,4194304] 0 2024-03-16T10:44:59.953 INFO:tasks.workunit.client.0.smithi002.stdout:8/488: write d7/d9/d17/d23/d39/f5d [676515,31380] 0 2024-03-16T10:44:59.953 INFO:tasks.workunit.client.0.smithi002.stdout:3/513: symlink d1/dd/d28/d7b/d89/d15/d87/d8c/lb2 0 2024-03-16T10:44:59.954 INFO:tasks.workunit.client.0.smithi002.stdout:1/503: readlink d2/df/d12/d29/d5c/l77 0 2024-03-16T10:44:59.957 INFO:tasks.workunit.client.0.smithi002.stdout:1/504: fdatasync d2/d3/f7c 0 2024-03-16T10:44:59.958 INFO:tasks.workunit.client.0.smithi002.stdout:8/489: chown d7/d9/d17/f2d 1728962583 1 2024-03-16T10:44:59.959 INFO:tasks.workunit.client.0.smithi002.stdout:9/486: dwrite d5/d9/d22/d3c/d4d/f75 [0,4194304] 0 2024-03-16T10:44:59.959 INFO:tasks.workunit.client.0.smithi002.stdout:2/512: link d7/d2e/d28/d40/f63 d7/d2e/d28/d44/d6d/f9f 0 2024-03-16T10:44:59.960 INFO:tasks.workunit.client.0.smithi002.stdout:1/505: read d2/df/d12/d29/f34 [2579905,34930] 0 2024-03-16T10:44:59.962 INFO:tasks.workunit.client.0.smithi002.stdout:6/494: creat d15/d2d/d8a/d7c/d87/fb0 x:0 0 0 2024-03-16T10:44:59.962 INFO:tasks.workunit.client.0.smithi002.stdout:6/495: chown d15/d2d/d73/fa6 8 1 2024-03-16T10:44:59.972 INFO:tasks.workunit.client.0.smithi002.stdout:4/465: rename d2/d6/d32/f8f to d2/f92 0 2024-03-16T10:44:59.973 INFO:tasks.workunit.client.0.smithi002.stdout:3/514: creat d1/dd/d28/d7b/d89/d15/d87/d8c/fb3 x:0 0 0 2024-03-16T10:44:59.973 INFO:tasks.workunit.client.0.smithi002.stdout:2/513: rmdir d7/d2e/d1d/d38/d49 39 2024-03-16T10:44:59.975 INFO:tasks.workunit.client.0.smithi002.stdout:6/496: write d15/f19 [2889019,91699] 0 2024-03-16T10:44:59.976 INFO:tasks.workunit.client.0.smithi002.stdout:3/515: mknod d1/dd/d28/d37/cb4 0 2024-03-16T10:44:59.981 INFO:tasks.workunit.client.0.smithi002.stdout:2/514: creat d7/d2e/d1d/d2f/d4b/fa0 x:0 0 0 2024-03-16T10:44:59.983 INFO:tasks.workunit.client.0.smithi002.stdout:4/466: truncate d2/d33/da/f25 400738 0 2024-03-16T10:44:59.987 INFO:tasks.workunit.client.0.smithi002.stdout:1/506: creat d2/df/d12/fa5 x:0 0 0 2024-03-16T10:44:59.991 INFO:tasks.workunit.client.0.smithi002.stdout:4/467: write d2/d33/d1c/f67 [759217,119150] 0 2024-03-16T10:44:59.992 INFO:tasks.workunit.client.0.smithi002.stdout:1/507: dwrite d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:44:59.994 INFO:tasks.workunit.client.0.smithi002.stdout:8/490: getdents d7/d9 0 2024-03-16T10:44:59.994 INFO:tasks.workunit.client.0.smithi002.stdout:1/508: fsync d2/d3/f4b 0 2024-03-16T10:45:00.000 INFO:tasks.workunit.client.0.smithi002.stdout:4/468: mkdir d2/d33/da/d5e/d93 0 2024-03-16T10:45:00.001 INFO:tasks.workunit.client.0.smithi002.stdout:4/469: chown d2/d33/da/d2e 0 1 2024-03-16T10:45:00.002 INFO:tasks.workunit.client.0.smithi002.stdout:8/491: dwrite d7/d9/d17/d23/f33 [0,4194304] 0 2024-03-16T10:45:00.005 INFO:tasks.workunit.client.0.smithi002.stdout:3/516: getdents d1/dd/d28/d7b/d89/d15 0 2024-03-16T10:45:00.010 INFO:tasks.workunit.client.0.smithi002.stdout:4/470: dwrite d2/d33/da/f84 [0,4194304] 0 2024-03-16T10:45:00.011 INFO:tasks.workunit.client.0.smithi002.stdout:3/517: readlink d1/dd/d28/d7b/d89/l18 0 2024-03-16T10:45:00.013 INFO:tasks.workunit.client.0.smithi002.stdout:2/515: dwrite d7/d2e/d1d/f4f [0,4194304] 0 2024-03-16T10:45:00.015 INFO:tasks.workunit.client.0.smithi002.stdout:4/471: truncate d2/d33/d1c/f1d 4748356 0 2024-03-16T10:45:00.023 INFO:tasks.workunit.client.0.smithi002.stdout:8/492: truncate d7/d9/d17/f2d 2062012 0 2024-03-16T10:45:00.023 INFO:tasks.workunit.client.0.smithi002.stdout:1/509: write d2/d3/fe [3251500,25145] 0 2024-03-16T10:45:00.023 INFO:tasks.workunit.client.0.smithi002.stdout:2/516: dread d7/f2b [0,4194304] 0 2024-03-16T10:45:00.027 INFO:tasks.workunit.client.0.smithi002.stdout:8/493: symlink d7/d13/laf 0 2024-03-16T10:45:00.029 INFO:tasks.workunit.client.0.smithi002.stdout:1/510: creat d2/d3/d51/fa6 x:0 0 0 2024-03-16T10:45:00.030 INFO:tasks.workunit.client.0.smithi002.stdout:2/517: dread d7/d27/d3f/f69 [4194304,4194304] 0 2024-03-16T10:45:00.031 INFO:tasks.workunit.client.0.smithi002.stdout:4/472: creat d2/d33/f94 x:0 0 0 2024-03-16T10:45:00.031 INFO:tasks.workunit.client.0.smithi002.stdout:1/511: chown d2/d3/ld 279 1 2024-03-16T10:45:00.036 INFO:tasks.workunit.client.0.smithi002.stdout:3/518: dwrite d1/dd/d28/f9c [0,4194304] 0 2024-03-16T10:45:00.036 INFO:tasks.workunit.client.0.smithi002.stdout:1/512: readlink d2/df/d12/d74/l78 0 2024-03-16T10:45:00.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 -- 172.21.15.2:0/999490002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a10100bd0 msgr2=0x7f8a10101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/999490002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a10100bd0 0x7f8a10101030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009910 tx=0x7f8a04009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 -- 172.21.15.2:0/999490002 shutdown_connections 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/999490002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a10100bd0 0x7f8a10101030 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/999490002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a100ff9d0 0x7f8a100ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 -- 172.21.15.2:0/999490002 >> 172.21.15.2:0/999490002 conn(0x7f8a100faf70 msgr2=0x7f8a100fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 -- 172.21.15.2:0/999490002 shutdown_connections 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.035+0000 7f8a15c9f700 1 -- 172.21.15.2:0/999490002 wait complete. 2024-03-16T10:45:00.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.036+0000 7f8a15c9f700 1 Processor -- start 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a15c9f700 1 -- start start 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a15c9f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a100ff9d0 0x7f8a10195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a15c9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a10100bd0 0x7f8a10195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a15c9f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10195f70 con 0x7f8a100ff9d0 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a15c9f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a101960b0 con 0x7f8a10100bd0 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a100ff9d0 0x7f8a10195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a100ff9d0 0x7f8a10195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57594/0 (socket says 172.21.15.2:57594) 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0f7fe700 1 -- 172.21.15.2:0/3083509888 learned_addr learned my addr 172.21.15.2:0/3083509888 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0effd700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a10100bd0 0x7f8a10195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0f7fe700 1 -- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a10100bd0 msgr2=0x7f8a10195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0f7fe700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a10100bd0 0x7f8a10195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0f7fe700 1 -- 172.21.15.2:0/3083509888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040095f0 con 0x7f8a100ff9d0 2024-03-16T10:45:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.037+0000 7f8a0effd700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a10100bd0 0x7f8a10195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:45:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.038+0000 7f8a0f7fe700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a100ff9d0 0x7f8a10195410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000ea80 tx=0x7f8a0000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.038+0000 7f8a0cff9700 1 -- 172.21.15.2:0/3083509888 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a00013070 con 0x7f8a100ff9d0 2024-03-16T10:45:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.038+0000 7f8a0cff9700 1 -- 172.21.15.2:0/3083509888 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a000044e0 con 0x7f8a100ff9d0 2024-03-16T10:45:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.038+0000 7f8a0cff9700 1 -- 172.21.15.2:0/3083509888 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a00003d90 con 0x7f8a100ff9d0 2024-03-16T10:45:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.038+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a100684e0 con 0x7f8a100ff9d0 2024-03-16T10:45:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.038+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a10068ab0 con 0x7f8a100ff9d0 2024-03-16T10:45:00.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.039+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1004ea80 con 0x7f8a100ff9d0 2024-03-16T10:45:00.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.040+0000 7f8a0cff9700 1 -- 172.21.15.2:0/3083509888 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) v1 ==== 91796+0+0 (secure 0 0 0) 0x7f8a0000bbd0 con 0x7f8a100ff9d0 2024-03-16T10:45:00.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.041+0000 7f8a0cff9700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f89f806de60 0x7f89f8070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:00.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.041+0000 7f8a0cff9700 1 -- 172.21.15.2:0/3083509888 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8a0008f210 con 0x7f8a100ff9d0 2024-03-16T10:45:00.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.042+0000 7f8a0effd700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f89f806de60 0x7f89f8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:00.043 INFO:tasks.workunit.client.0.smithi002.stdout:8/494: creat d7/d9/d17/fb0 x:0 0 0 2024-03-16T10:45:00.043 INFO:tasks.workunit.client.0.smithi002.stdout:7/421: sync 2024-03-16T10:45:00.044 INFO:tasks.workunit.client.0.smithi002.stdout:1/513: dread d2/df/d12/d84/d87/d96/f9c [0,4194304] 0 2024-03-16T10:45:00.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.044+0000 7f8a0effd700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f89f806de60 0x7f89f8070320 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006010 tx=0x7f8a04009f90 comp rx=0 tx=0).ready entity=mgr.24375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:00.046 INFO:tasks.workunit.client.0.smithi002.stdout:1/514: write d2/d3/fe [2783387,70123] 0 2024-03-16T10:45:00.047 INFO:tasks.workunit.client.0.smithi002.stdout:2/518: getdents d7/d2e/d1d/d2f/d4b 0 2024-03-16T10:45:00.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.047+0000 7f8a0cff9700 1 -- 172.21.15.2:0/3083509888 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f8a0005c620 con 0x7f8a100ff9d0 2024-03-16T10:45:00.050 INFO:tasks.workunit.client.0.smithi002.stdout:1/515: mknod d2/df/d12/d74/ca7 0 2024-03-16T10:45:00.050 INFO:tasks.workunit.client.0.smithi002.stdout:2/519: read d7/d2e/d1d/f21 [7411668,71270] 0 2024-03-16T10:45:00.051 INFO:tasks.workunit.client.0.smithi002.stdout:1/516: stat d2/df/d12/d74/ca7 0 2024-03-16T10:45:00.052 INFO:tasks.workunit.client.0.smithi002.stdout:2/520: chown d7/d27/d3f/d7f/d8e 29691 1 2024-03-16T10:45:00.053 INFO:tasks.workunit.client.0.smithi002.stdout:1/517: truncate d2/d3/f50 972048 0 2024-03-16T10:45:00.053 INFO:tasks.workunit.client.0.smithi002.stdout:7/422: rename d3/d4/d1d/d87/l54 to d3/d4/d1d/d87/d5c/d63/d34/l97 0 2024-03-16T10:45:00.055 INFO:tasks.workunit.client.0.smithi002.stdout:7/423: write d3/d61/d66/f4d [732896,44927] 0 2024-03-16T10:45:00.056 INFO:tasks.workunit.client.0.smithi002.stdout:8/495: link d7/fd d7/d13/fb1 0 2024-03-16T10:45:00.058 INFO:tasks.workunit.client.0.smithi002.stdout:3/519: dread d1/f16 [4194304,4194304] 0 2024-03-16T10:45:00.059 INFO:tasks.workunit.client.0.smithi002.stdout:2/521: chown d7/db/f58 1 1 2024-03-16T10:45:00.061 INFO:tasks.workunit.client.0.smithi002.stdout:7/424: creat d3/d4/d1d/d87/d5c/d63/d34/f98 x:0 0 0 2024-03-16T10:45:00.062 INFO:tasks.workunit.client.0.smithi002.stdout:8/496: mkdir d7/d9/d25/db2 0 2024-03-16T10:45:00.067 INFO:tasks.workunit.client.0.smithi002.stdout:1/518: rmdir d2/d3/d1c/d5d 39 2024-03-16T10:45:00.068 INFO:tasks.workunit.client.0.smithi002.stdout:8/497: mknod d7/d9/d25/d67/cb3 0 2024-03-16T10:45:00.070 INFO:tasks.workunit.client.0.smithi002.stdout:2/522: unlink d7/d2e/l33 0 2024-03-16T10:45:00.071 INFO:tasks.workunit.client.0.smithi002.stdout:2/523: fdatasync d7/d2e/f32 0 2024-03-16T10:45:00.072 INFO:tasks.workunit.client.0.smithi002.stdout:7/425: mkdir d3/d4/d1c/d5b/d64/d99 0 2024-03-16T10:45:00.073 INFO:tasks.workunit.client.0.smithi002.stdout:2/524: write d7/d2e/d1d/f41 [3116732,29013] 0 2024-03-16T10:45:00.074 INFO:tasks.workunit.client.0.smithi002.stdout:3/520: dread d1/dd/d28/d7b/d89/f30 [0,4194304] 0 2024-03-16T10:45:00.074 INFO:tasks.workunit.client.0.smithi002.stdout:2/525: stat d7/db 0 2024-03-16T10:45:00.075 INFO:tasks.workunit.client.0.smithi002.stdout:1/519: write d2/d3/d1c/d5d/d71/f79 [196366,83551] 0 2024-03-16T10:45:00.076 INFO:tasks.workunit.client.0.smithi002.stdout:7/426: read d3/d4/d1d/d87/d5c/d63/d3f/f43 [3541942,58919] 0 2024-03-16T10:45:00.077 INFO:tasks.workunit.client.0.smithi002.stdout:1/520: readlink d2/d3/ld 0 2024-03-16T10:45:00.079 INFO:tasks.workunit.client.0.smithi002.stdout:7/427: write d3/d4/d1d/d87/d5c/d63/d55/f5d [417508,26763] 0 2024-03-16T10:45:00.081 INFO:tasks.workunit.client.0.smithi002.stdout:2/526: creat d7/d2e/d28/d40/d50/fa1 x:0 0 0 2024-03-16T10:45:00.084 INFO:tasks.workunit.client.0.smithi002.stdout:1/521: chown d2/d3/d1c/c40 14 1 2024-03-16T10:45:00.086 INFO:tasks.workunit.client.0.smithi002.stdout:7/428: mknod d3/d4/d1d/d87/d96/d8c/c9a 0 2024-03-16T10:45:00.086 INFO:tasks.workunit.client.0.smithi002.stdout:8/498: write d7/d9/d17/d23/d2e/f92 [4375553,46322] 0 2024-03-16T10:45:00.088 INFO:tasks.workunit.client.0.smithi002.stdout:3/521: dwrite d1/dd/d28/d7b/d2c/d63/f79 [0,4194304] 0 2024-03-16T10:45:00.090 INFO:tasks.workunit.client.0.smithi002.stdout:3/522: write d1/dd/d28/d7b/d89/d15/d87/d8c/fa9 [668280,50587] 0 2024-03-16T10:45:00.090 INFO:tasks.workunit.client.0.smithi002.stdout:1/522: mkdir d2/df/d12/d84/d87/d96/da8 0 2024-03-16T10:45:00.091 INFO:tasks.workunit.client.0.smithi002.stdout:8/499: rmdir d7/d55/d8f 39 2024-03-16T10:45:00.091 INFO:tasks.workunit.client.0.smithi002.stdout:1/523: fdatasync d2/d3/d51/f5a 0 2024-03-16T10:45:00.093 INFO:tasks.workunit.client.0.smithi002.stdout:1/524: readlink d2/df/d12/d29/d5c/l9b 0 2024-03-16T10:45:00.093 INFO:tasks.workunit.client.0.smithi002.stdout:7/429: creat d3/d4/d1d/d87/d96/d8c/f9b x:0 0 0 2024-03-16T10:45:00.097 INFO:tasks.workunit.client.0.smithi002.stdout:3/523: dwrite d1/f8f [0,4194304] 0 2024-03-16T10:45:00.099 INFO:tasks.workunit.client.0.smithi002.stdout:8/500: creat d7/d9/d17/d23/d39/fb4 x:0 0 0 2024-03-16T10:45:00.101 INFO:tasks.workunit.client.0.smithi002.stdout:3/524: rmdir d1/da5 39 2024-03-16T10:45:00.106 INFO:tasks.workunit.client.0.smithi002.stdout:3/525: dread - d1/dd/d28/d7b/d89/d15/f67 zero size 2024-03-16T10:45:00.106 INFO:tasks.workunit.client.0.smithi002.stdout:8/501: chown d7/d9/d17/d23/d2e/d3a/f47 100 1 2024-03-16T10:45:00.107 INFO:tasks.workunit.client.0.smithi002.stdout:7/430: dwrite d3/d4/d1d/d87/f92 [0,4194304] 0 2024-03-16T10:45:00.108 INFO:tasks.workunit.client.0.smithi002.stdout:2/527: write d7/d2e/d28/d40/d6c/f87 [383711,92495] 0 2024-03-16T10:45:00.110 INFO:tasks.workunit.client.0.smithi002.stdout:8/502: dread - d7/d9/d17/d23/d39/f8c zero size 2024-03-16T10:45:00.113 INFO:tasks.workunit.client.0.smithi002.stdout:3/526: unlink d1/dd/d28/d7b/d89/d15/f62 0 2024-03-16T10:45:00.117 INFO:tasks.workunit.client.0.smithi002.stdout:1/525: dread d2/d3/f4b [0,4194304] 0 2024-03-16T10:45:00.120 INFO:tasks.workunit.client.0.smithi002.stdout:3/527: creat d1/dd/d74/fb5 x:0 0 0 2024-03-16T10:45:00.120 INFO:tasks.workunit.client.0.smithi002.stdout:7/431: write d3/d37/f48 [1073534,125447] 0 2024-03-16T10:45:00.123 INFO:tasks.workunit.client.0.smithi002.stdout:2/528: dwrite d7/d2e/f20 [0,4194304] 0 2024-03-16T10:45:00.127 INFO:tasks.workunit.client.0.smithi002.stdout:8/503: dwrite d7/d9/d17/d23/d2e/d34/f9e [0,4194304] 0 2024-03-16T10:45:00.128 INFO:tasks.workunit.client.0.smithi002.stdout:7/432: chown d3/d4/d1d/d87/d5c/d63/d3f/f46 2005380 1 2024-03-16T10:45:00.131 INFO:tasks.workunit.client.0.smithi002.stdout:3/528: rename d1/dd/d28/d7b/d89 to d1/db6 0 2024-03-16T10:45:00.132 INFO:tasks.workunit.client.0.smithi002.stdout:2/529: mknod d7/d2e/d28/d40/d50/ca2 0 2024-03-16T10:45:00.135 INFO:tasks.workunit.client.0.smithi002.stdout:1/526: dwrite d2/df/d12/d29/d5c/f6e [0,4194304] 0 2024-03-16T10:45:00.137 INFO:tasks.workunit.client.0.smithi002.stdout:8/504: rmdir d7/d9/d25 39 2024-03-16T10:45:00.139 INFO:tasks.workunit.client.0.smithi002.stdout:7/433: mkdir d3/d4/d1c/d5b/d64/d99/d9c 0 2024-03-16T10:45:00.142 INFO:tasks.workunit.client.0.smithi002.stdout:5/431: sync 2024-03-16T10:45:00.142 INFO:tasks.workunit.client.0.smithi002.stdout:0/512: sync 2024-03-16T10:45:00.143 INFO:tasks.workunit.client.0.smithi002.stdout:7/434: dread - d3/d4/d1d/d87/d5c/d63/f93 zero size 2024-03-16T10:45:00.146 INFO:tasks.workunit.client.0.smithi002.stdout:1/527: creat d2/df/d12/d84/d87/d96/da8/fa9 x:0 0 0 2024-03-16T10:45:00.146 INFO:tasks.workunit.client.0.smithi002.stdout:2/530: dwrite d7/db/f7c [0,4194304] 0 2024-03-16T10:45:00.148 INFO:tasks.workunit.client.0.smithi002.stdout:5/432: readlink d0/d24/l40 0 2024-03-16T10:45:00.150 INFO:tasks.workunit.client.0.smithi002.stdout:3/529: unlink d1/db6/d15/d87/d8c/fa9 0 2024-03-16T10:45:00.150 INFO:tasks.workunit.client.0.smithi002.stdout:3/530: stat d1/db6/c3d 0 2024-03-16T10:45:00.150 INFO:tasks.workunit.client.0.smithi002.stdout:2/531: fdatasync d7/d2e/d28/d44/f64 0 2024-03-16T10:45:00.159 INFO:tasks.workunit.client.0.smithi002.stdout:3/531: creat d1/dd/d28/d7b/d8a/d93/fb7 x:0 0 0 2024-03-16T10:45:00.165 INFO:tasks.workunit.client.0.smithi002.stdout:3/532: dwrite d1/f52 [4194304,4194304] 0 2024-03-16T10:45:00.166 INFO:tasks.workunit.client.0.smithi002.stdout:1/528: rename d2/d3/d1c/d5d/f92 to d2/df/d12/d86/faa 0 2024-03-16T10:45:00.167 INFO:tasks.workunit.client.0.smithi002.stdout:8/505: creat d7/d9/fb5 x:0 0 0 2024-03-16T10:45:00.173 INFO:tasks.workunit.client.0.smithi002.stdout:2/532: write d7/d2e/f16 [716946,77855] 0 2024-03-16T10:45:00.173 INFO:tasks.workunit.client.0.smithi002.stdout:7/435: truncate d3/d4/d1d/d45/f42 2640028 0 2024-03-16T10:45:00.178 INFO:tasks.workunit.client.0.smithi002.stdout:6/497: sync 2024-03-16T10:45:00.178 INFO:tasks.workunit.client.0.smithi002.stdout:4/473: sync 2024-03-16T10:45:00.178 INFO:tasks.workunit.client.0.smithi002.stdout:9/487: sync 2024-03-16T10:45:00.179 INFO:tasks.workunit.client.0.smithi002.stdout:6/498: chown d15/d2d/d2c/f7a 101879 1 2024-03-16T10:45:00.180 INFO:tasks.workunit.client.0.smithi002.stdout:5/433: truncate d0/d1f/f2c 38656 0 2024-03-16T10:45:00.183 INFO:tasks.workunit.client.0.smithi002.stdout:2/533: creat d7/d2e/d28/d40/fa3 x:0 0 0 2024-03-16T10:45:00.185 INFO:tasks.workunit.client.0.smithi002.stdout:7/436: mknod d3/d37/c9d 0 2024-03-16T10:45:00.186 INFO:tasks.workunit.client.0.smithi002.stdout:7/437: write d3/f52 [648990,125452] 0 2024-03-16T10:45:00.191 INFO:tasks.workunit.client.0.smithi002.stdout:4/474: write d2/d33/d35/f63 [1195958,27108] 0 2024-03-16T10:45:00.192 INFO:tasks.workunit.client.0.smithi002.stdout:5/434: unlink d0/d16/d1d/d2b/c54 0 2024-03-16T10:45:00.194 INFO:tasks.workunit.client.0.smithi002.stdout:1/529: dwrite d2/d3/f39 [0,4194304] 0 2024-03-16T10:45:00.195 INFO:tasks.workunit.client.0.smithi002.stdout:6/499: dread d15/d2d/d2b/f53 [0,4194304] 0 2024-03-16T10:45:00.196 INFO:tasks.workunit.client.0.smithi002.stdout:8/506: truncate d7/d9/f94 78118 0 2024-03-16T10:45:00.198 INFO:tasks.workunit.client.0.smithi002.stdout:3/533: creat d1/dd/fb8 x:0 0 0 2024-03-16T10:45:00.198 INFO:tasks.workunit.client.0.smithi002.stdout:1/530: truncate d2/df/d12/d29/d5c/f98 80273 0 2024-03-16T10:45:00.200 INFO:tasks.workunit.client.0.smithi002.stdout:9/488: mknod d5/d9/d25/c96 0 2024-03-16T10:45:00.208 INFO:tasks.workunit.client.0.smithi002.stdout:9/489: chown d5/l8 14278226 1 2024-03-16T10:45:00.209 INFO:tasks.workunit.client.0.smithi002.stdout:1/531: write d2/df/d12/d29/d5c/f6e [4648681,63554] 0 2024-03-16T10:45:00.209 INFO:tasks.workunit.client.0.smithi002.stdout:5/435: dwrite d0/d16/d1d/d2b/d5d/f64 [0,4194304] 0 2024-03-16T10:45:00.210 INFO:tasks.workunit.client.0.smithi002.stdout:6/500: mknod d15/d2d/d2b/d3f/d77/d7e/cb1 0 2024-03-16T10:45:00.211 INFO:tasks.workunit.client.0.smithi002.stdout:7/438: symlink d3/d4/d1d/d87/l9e 0 2024-03-16T10:45:00.214 INFO:tasks.workunit.client.0.smithi002.stdout:6/501: stat d15/d2d/d2c/d5d/c62 0 2024-03-16T10:45:00.218 INFO:tasks.workunit.client.0.smithi002.stdout:3/534: dwrite d1/f52 [8388608,4194304] 0 2024-03-16T10:45:00.226 INFO:tasks.workunit.client.0.smithi002.stdout:1/532: dwrite d2/f68 [0,4194304] 0 2024-03-16T10:45:00.227 INFO:tasks.workunit.client.0.smithi002.stdout:7/439: dwrite d3/d4/d1d/d87/f92 [0,4194304] 0 2024-03-16T10:45:00.228 INFO:tasks.workunit.client.0.smithi002.stdout:4/475: mknod d2/d33/d35/c95 0 2024-03-16T10:45:00.229 INFO:tasks.workunit.client.0.smithi002.stdout:1/533: write d2/d3/f4b [1121044,10399] 0 2024-03-16T10:45:00.230 INFO:tasks.workunit.client.0.smithi002.stdout:2/534: creat d7/d2e/d1d/d38/fa4 x:0 0 0 2024-03-16T10:45:00.231 INFO:tasks.workunit.client.0.smithi002.stdout:4/476: stat d2/d33/f1b 0 2024-03-16T10:45:00.231 INFO:tasks.workunit.client.0.smithi002.stdout:8/507: creat d7/d9/d17/d23/d2e/d5c/fb6 x:0 0 0 2024-03-16T10:45:00.234 INFO:tasks.workunit.client.0.smithi002.stdout:6/502: creat d15/d2d/d2b/d3f/d77/d7e/fb2 x:0 0 0 2024-03-16T10:45:00.234 INFO:tasks.workunit.client.0.smithi002.stdout:1/534: truncate d2/df/d12/f72 4367672 0 2024-03-16T10:45:00.235 INFO:tasks.workunit.client.0.smithi002.stdout:3/535: symlink d1/dd/d28/d37/lb9 0 2024-03-16T10:45:00.239 INFO:tasks.workunit.client.0.smithi002.stdout:1/535: write d2/df/d12/d29/d5c/f6e [5488934,72514] 0 2024-03-16T10:45:00.244 INFO:tasks.workunit.client.0.smithi002.stdout:6/503: mknod d15/d2d/d2c/d8f/cb3 0 2024-03-16T10:45:00.244 INFO:tasks.workunit.client.0.smithi002.stdout:1/536: write d2/df/d12/d84/d87/d96/da8/fa9 [1039418,23183] 0 2024-03-16T10:45:00.245 INFO:tasks.workunit.client.0.smithi002.stdout:3/536: dwrite d1/dd/f9e [0,4194304] 0 2024-03-16T10:45:00.246 INFO:tasks.workunit.client.0.smithi002.stdout:8/508: dwrite d7/d9/d17/d23/d39/f5d [0,4194304] 0 2024-03-16T10:45:00.250 INFO:tasks.workunit.client.0.smithi002.stdout:2/535: rename d7/d27/d3f/d7f/d8e to d7/d27/d3b/d95/da5 0 2024-03-16T10:45:00.251 INFO:tasks.workunit.client.0.smithi002.stdout:3/537: write d1/dd/d74/fa8 [850653,112281] 0 2024-03-16T10:45:00.253 INFO:tasks.workunit.client.0.smithi002.stdout:6/504: truncate d15/d2d/d2b/d3f/d77/d9f/fa1 462908 0 2024-03-16T10:45:00.254 INFO:tasks.workunit.client.0.smithi002.stdout:9/490: getdents d5/d9/d22/d3c/d4d 0 2024-03-16T10:45:00.254 INFO:tasks.workunit.client.0.smithi002.stdout:2/536: read d7/d2e/d1d/f3e [2542467,74948] 0 2024-03-16T10:45:00.255 INFO:tasks.workunit.client.0.smithi002.stdout:9/491: fsync d5/f29 0 2024-03-16T10:45:00.258 INFO:tasks.workunit.client.0.smithi002.stdout:1/537: dwrite d2/df/d12/f60 [0,4194304] 0 2024-03-16T10:45:00.258 INFO:tasks.workunit.client.0.smithi002.stdout:6/505: dread d15/d2d/f2e [0,4194304] 0 2024-03-16T10:45:00.259 INFO:tasks.workunit.client.0.smithi002.stdout:3/538: mknod d1/d5a/cba 0 2024-03-16T10:45:00.260 INFO:tasks.workunit.client.0.smithi002.stdout:2/537: fsync d7/d2e/d28/d40/f5a 0 2024-03-16T10:45:00.260 INFO:tasks.workunit.client.0.smithi002.stdout:5/436: link d0/l61 d0/l84 0 2024-03-16T10:45:00.262 INFO:tasks.workunit.client.0.smithi002.stdout:5/437: read d0/d1f/f36 [1784858,20391] 0 2024-03-16T10:45:00.269 INFO:tasks.workunit.client.0.smithi002.stdout:5/438: dwrite d0/d16/d1d/d2b/d5d/f5a [0,4194304] 0 2024-03-16T10:45:00.270 INFO:tasks.workunit.client.0.smithi002.stdout:5/439: dread - d0/d16/d4a/d39/f51 zero size 2024-03-16T10:45:00.277 INFO:tasks.workunit.client.0.smithi002.stdout:5/440: dwrite d0/d16/d1d/d33/d65/f7a [0,4194304] 0 2024-03-16T10:45:00.281 INFO:tasks.workunit.client.0.smithi002.stdout:2/538: symlink d7/la6 0 2024-03-16T10:45:00.282 INFO:tasks.workunit.client.0.smithi002.stdout:2/539: chown d7/d2e/d1d/d2f/c84 658837 1 2024-03-16T10:45:00.282 INFO:tasks.workunit.client.0.smithi002.stdout:2/540: write f6 [4038983,93747] 0 2024-03-16T10:45:00.283 INFO:tasks.workunit.client.0.smithi002.stdout:2/541: chown d7/l54 33 1 2024-03-16T10:45:00.287 INFO:tasks.workunit.client.0.smithi002.stdout:1/538: creat d2/d3/fab x:0 0 0 2024-03-16T10:45:00.289 INFO:tasks.workunit.client.0.smithi002.stdout:9/492: rename d5/d9/d25/f73 to d5/f97 0 2024-03-16T10:45:00.289 INFO:tasks.workunit.client.0.smithi002.stdout:1/539: read d2/d3/d51/f5a [4497436,128677] 0 2024-03-16T10:45:00.291 INFO:tasks.workunit.client.0.smithi002.stdout:5/441: creat d0/d16/d1d/f85 x:0 0 0 2024-03-16T10:45:00.293 INFO:tasks.workunit.client.0.smithi002.stdout:9/493: mknod d5/d9/d22/d3c/d4d/d7f/c98 0 2024-03-16T10:45:00.293 INFO:tasks.workunit.client.0.smithi002.stdout:6/506: write d15/d2d/d2b/d3f/d77/d9f/fa1 [648939,93584] 0 2024-03-16T10:45:00.295 INFO:tasks.workunit.client.0.smithi002.stdout:8/509: dwrite d7/d9/d17/f2d [0,4194304] 0 2024-03-16T10:45:00.302 INFO:tasks.workunit.client.0.smithi002.stdout:1/540: truncate d2/d3/f55 1035141 0 2024-03-16T10:45:00.308 INFO:tasks.workunit.client.0.smithi002.stdout:6/507: dwrite d15/d2d/d2c/f36 [0,4194304] 0 2024-03-16T10:45:00.310 INFO:tasks.workunit.client.0.smithi002.stdout:1/541: write d2/d3/f8 [782043,88594] 0 2024-03-16T10:45:00.310 INFO:tasks.workunit.client.0.smithi002.stdout:9/494: dwrite d5/d9/dc/d1e/d91/f7a [0,4194304] 0 2024-03-16T10:45:00.313 INFO:tasks.workunit.client.0.smithi002.stdout:8/510: dwrite d7/d9/d17/d23/f33 [0,4194304] 0 2024-03-16T10:45:00.315 INFO:tasks.workunit.client.0.smithi002.stdout:6/508: rmdir d15/d2d/d2b/d75 39 2024-03-16T10:45:00.316 INFO:tasks.workunit.client.0.smithi002.stdout:8/511: fdatasync d7/d9/d17/d83/da2/fab 0 2024-03-16T10:45:00.316 INFO:tasks.workunit.client.0.smithi002.stdout:1/542: symlink d2/df/d12/d74/lac 0 2024-03-16T10:45:00.317 INFO:tasks.workunit.client.0.smithi002.stdout:5/442: dwrite d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:45:00.318 INFO:tasks.workunit.client.0.smithi002.stdout:8/512: chown d7/fc 23233113 1 2024-03-16T10:45:00.320 INFO:tasks.workunit.client.0.smithi002.stdout:6/509: unlink l1 0 2024-03-16T10:45:00.321 INFO:tasks.workunit.client.0.smithi002.stdout:6/510: readlink d15/d2d/d8a/l7f 0 2024-03-16T10:45:00.325 INFO:tasks.workunit.client.0.smithi002.stdout:6/511: write d15/d2d/d2b/d3f/d77/d9f/f91 [816186,45835] 0 2024-03-16T10:45:00.326 INFO:tasks.workunit.client.0.smithi002.stdout:6/512: dread - d15/d2d/f28 zero size 2024-03-16T10:45:00.327 INFO:tasks.workunit.client.0.smithi002.stdout:6/513: fsync d15/d2d/d2b/d3f/d77/f99 0 2024-03-16T10:45:00.331 INFO:tasks.workunit.client.0.smithi002.stdout:9/495: dread d5/d9/dc/d28/f2d [0,4194304] 0 2024-03-16T10:45:00.336 INFO:tasks.workunit.client.0.smithi002.stdout:8/513: dread d7/d9/d17/d23/d2e/d5c/f8a [0,4194304] 0 2024-03-16T10:45:00.338 INFO:tasks.workunit.client.0.smithi002.stdout:8/514: truncate d7/d9/d17/d23/d39/f7a 3244890 0 2024-03-16T10:45:00.344 INFO:tasks.workunit.client.0.smithi002.stdout:6/514: write d15/f1a [2920,125421] 0 2024-03-16T10:45:00.345 INFO:tasks.workunit.client.0.smithi002.stdout:5/443: dread d0/f6 [0,4194304] 0 2024-03-16T10:45:00.348 INFO:tasks.workunit.client.0.smithi002.stdout:6/515: write d15/d2d/d2c/d5d/f72 [1055300,130417] 0 2024-03-16T10:45:00.351 INFO:tasks.workunit.client.0.smithi002.stdout:9/496: dwrite d5/d9/d25/d23/f33 [0,4194304] 0 2024-03-16T10:45:00.353 INFO:tasks.workunit.client.0.smithi002.stdout:5/444: unlink d0/d16/d4a/d7d/c79 0 2024-03-16T10:45:00.354 INFO:tasks.workunit.client.0.smithi002.stdout:5/445: truncate d0/f57 4557276 0 2024-03-16T10:45:00.358 INFO:tasks.workunit.client.0.smithi002.stdout:8/515: dread d7/d9/d17/d23/d2e/d3a/f5f [0,4194304] 0 2024-03-16T10:45:00.363 INFO:tasks.workunit.client.0.smithi002.stdout:9/497: dwrite d5/d9/d25/f62 [0,4194304] 0 2024-03-16T10:45:00.364 INFO:tasks.workunit.client.0.smithi002.stdout:8/516: mkdir d7/d9/d17/d23/d39/db7 0 2024-03-16T10:45:00.364 INFO:tasks.workunit.client.0.smithi002.stdout:8/517: readlink d7/d55/l66 0 2024-03-16T10:45:00.365 INFO:tasks.workunit.client.0.smithi002.stdout:5/446: rename d0/d16/d1d/d43/d48/f82 to d0/f86 0 2024-03-16T10:45:00.367 INFO:tasks.workunit.client.0.smithi002.stdout:5/447: write d0/d16/f63 [668135,112323] 0 2024-03-16T10:45:00.368 INFO:tasks.workunit.client.0.smithi002.stdout:9/498: readlink d5/d9/dc/d1e/d24/l38 0 2024-03-16T10:45:00.369 INFO:tasks.workunit.client.0.smithi002.stdout:5/448: fsync d0/d16/d1d/f85 0 2024-03-16T10:45:00.370 INFO:tasks.workunit.client.0.smithi002.stdout:5/449: dread - d0/d1f/f77 zero size 2024-03-16T10:45:00.374 INFO:tasks.workunit.client.0.smithi002.stdout:8/518: rename d7/d55 to d7/d13/db8 0 2024-03-16T10:45:00.375 INFO:tasks.workunit.client.0.smithi002.stdout:8/519: dread - d7/d9/d17/d23/d2e/d34/f6e zero size 2024-03-16T10:45:00.377 INFO:tasks.workunit.client.0.smithi002.stdout:8/520: readlink d7/d9/d25/d91/laa 0 2024-03-16T10:45:00.378 INFO:tasks.workunit.client.0.smithi002.stdout:5/450: rename d0/d1f/c22 to d0/d16/d4a/d55/c87 0 2024-03-16T10:45:00.378 INFO:tasks.workunit.client.0.smithi002.stdout:8/521: write d7/d9/d17/d23/f4d [3523258,94090] 0 2024-03-16T10:45:00.386 INFO:tasks.workunit.client.0.smithi002.stdout:5/451: rename d0/d16/c56 to d0/c88 0 2024-03-16T10:45:00.389 INFO:tasks.workunit.client.0.smithi002.stdout:5/452: symlink d0/d16/d4a/d7d/l89 0 2024-03-16T10:45:00.391 INFO:tasks.workunit.client.0.smithi002.stdout:9/499: dwrite d5/d9/dc/f11 [0,4194304] 0 2024-03-16T10:45:00.395 INFO:tasks.workunit.client.0.smithi002.stdout:8/522: dread d7/d9/d17/d23/d2e/d34/f79 [0,4194304] 0 2024-03-16T10:45:00.399 INFO:tasks.workunit.client.0.smithi002.stdout:8/523: chown d7/d9/d17/l2a 545996 1 2024-03-16T10:45:00.403 INFO:tasks.workunit.client.0.smithi002.stdout:5/453: truncate d0/f57 2004884 0 2024-03-16T10:45:00.407 INFO:tasks.workunit.client.0.smithi002.stdout:8/524: dwrite d7/d9/d17/d23/f4d [0,4194304] 0 2024-03-16T10:45:00.407 INFO:tasks.workunit.client.0.smithi002.stdout:5/454: mknod d0/d16/d1d/d33/d4b/c8a 0 2024-03-16T10:45:00.412 INFO:tasks.workunit.client.0.smithi002.stdout:5/455: write d0/f6 [2428582,118344] 0 2024-03-16T10:45:00.415 INFO:tasks.workunit.client.0.smithi002.stdout:8/525: mknod d7/d9/d17/d23/d2e/d90/cb9 0 2024-03-16T10:45:00.415 INFO:tasks.workunit.client.0.smithi002.stdout:9/500: dread d5/d9/dc/d1e/d24/f30 [8388608,4194304] 0 2024-03-16T10:45:00.415 INFO:tasks.workunit.client.0.smithi002.stdout:0/513: sync 2024-03-16T10:45:00.419 INFO:tasks.workunit.client.0.smithi002.stdout:0/514: fsync d3/d28/f33 0 2024-03-16T10:45:00.422 INFO:tasks.workunit.client.0.smithi002.stdout:8/526: creat d7/d9/d17/d23/d2e/d90/fba x:0 0 0 2024-03-16T10:45:00.426 INFO:tasks.workunit.client.0.smithi002.stdout:5/456: dwrite d0/d16/d1d/d33/d65/f7a [0,4194304] 0 2024-03-16T10:45:00.429 INFO:tasks.workunit.client.0.smithi002.stdout:8/527: rmdir d7/d9/d17/d23/d2e 39 2024-03-16T10:45:00.429 INFO:tasks.workunit.client.0.smithi002.stdout:9/501: dwrite d5/d9/dc/d1e/f6b [0,4194304] 0 2024-03-16T10:45:00.432 INFO:tasks.workunit.client.0.smithi002.stdout:5/457: write d0/d16/d1d/d2b/d5d/f5a [4930092,11542] 0 2024-03-16T10:45:00.433 INFO:tasks.workunit.client.0.smithi002.stdout:0/515: dwrite d3/d28/f29 [0,4194304] 0 2024-03-16T10:45:00.439 INFO:tasks.workunit.client.0.smithi002.stdout:0/516: rmdir d3/d28/d65 39 2024-03-16T10:45:00.443 INFO:tasks.workunit.client.0.smithi002.stdout:9/502: link d5/d9/f78 d5/d9/d34/f99 0 2024-03-16T10:45:00.446 INFO:tasks.workunit.client.0.smithi002.stdout:9/503: chown d5/d7e/l89 39425 1 2024-03-16T10:45:00.449 INFO:tasks.workunit.client.0.smithi002.stdout:5/458: getdents d0/d16/d4a/d55 0 2024-03-16T10:45:00.450 INFO:tasks.workunit.client.0.smithi002.stdout:9/504: dread d5/d9/dc/f16 [0,4194304] 0 2024-03-16T10:45:00.451 INFO:tasks.workunit.client.0.smithi002.stdout:0/517: creat d3/d28/f9d x:0 0 0 2024-03-16T10:45:00.451 INFO:tasks.workunit.client.0.smithi002.stdout:5/459: read d0/d16/d1d/d2b/d5d/f64 [2712291,22216] 0 2024-03-16T10:45:00.452 INFO:tasks.workunit.client.0.smithi002.stdout:9/505: write d5/d9/d25/f62 [1447965,38793] 0 2024-03-16T10:45:00.453 INFO:tasks.workunit.client.0.smithi002.stdout:8/528: write d7/d9/d17/d23/d2e/f4e [541058,119598] 0 2024-03-16T10:45:00.455 INFO:tasks.workunit.client.0.smithi002.stdout:8/529: write d7/d9/d25/d91/fa8 [804814,17589] 0 2024-03-16T10:45:00.456 INFO:tasks.workunit.client.0.smithi002.stdout:8/530: chown d7/d9/d17/l36 6775028 1 2024-03-16T10:45:00.458 INFO:tasks.workunit.client.0.smithi002.stdout:5/460: fdatasync d0/d16/d1d/d2b/d5d/f64 0 2024-03-16T10:45:00.458 INFO:tasks.workunit.client.0.smithi002.stdout:5/461: dread - d0/f86 zero size 2024-03-16T10:45:00.459 INFO:tasks.workunit.client.0.smithi002.stdout:9/506: symlink d5/d9/d55/l9a 0 2024-03-16T10:45:00.460 INFO:tasks.workunit.client.0.smithi002.stdout:9/507: chown d5/d9/d34 37710 1 2024-03-16T10:45:00.461 INFO:tasks.workunit.client.0.smithi002.stdout:0/518: creat d3/d27/d90/f9e x:0 0 0 2024-03-16T10:45:00.463 INFO:tasks.workunit.client.0.smithi002.stdout:0/519: write d3/d28/d3d/f74 [1117949,121641] 0 2024-03-16T10:45:00.463 INFO:tasks.workunit.client.0.smithi002.stdout:8/531: dread d7/d9/f6f [0,4194304] 0 2024-03-16T10:45:00.464 INFO:tasks.workunit.client.0.smithi002.stdout:5/462: dread d0/f6 [0,4194304] 0 2024-03-16T10:45:00.465 INFO:tasks.workunit.client.0.smithi002.stdout:9/508: fsync d5/d9/d55/f63 0 2024-03-16T10:45:00.466 INFO:tasks.workunit.client.0.smithi002.stdout:5/463: chown d0/d16/d4a/d55/l69 2434958 1 2024-03-16T10:45:00.467 INFO:tasks.workunit.client.0.smithi002.stdout:9/509: write d5/d9/d22/d3c/d4d/f75 [367830,126999] 0 2024-03-16T10:45:00.472 INFO:tasks.workunit.client.0.smithi002.stdout:9/510: chown d5/d9/d34/f61 275636727 1 2024-03-16T10:45:00.472 INFO:tasks.workunit.client.0.smithi002.stdout:0/520: rename d3/de/f20 to d3/d28/d2a/d67/f9f 0 2024-03-16T10:45:00.475 INFO:tasks.workunit.client.0.smithi002.stdout:9/511: dwrite d5/fd [0,4194304] 0 2024-03-16T10:45:00.477 INFO:tasks.workunit.client.0.smithi002.stdout:0/521: dwrite d3/d28/f93 [0,4194304] 0 2024-03-16T10:45:00.478 INFO:tasks.workunit.client.0.smithi002.stdout:9/512: read - d5/f6f zero size 2024-03-16T10:45:00.481 INFO:tasks.workunit.client.0.smithi002.stdout:8/532: rename d7/d9/f78 to d7/d9/d17/d23/d39/fbb 0 2024-03-16T10:45:00.489 INFO:tasks.workunit.client.0.smithi002.stdout:9/513: dwrite d5/d9/dc/d28/f2d [0,4194304] 0 2024-03-16T10:45:00.489 INFO:tasks.workunit.client.0.smithi002.stdout:5/464: rename d0/d16/d1d/d33/d4b/c8a to d0/d16/d1d/d43/d48/c8b 0 2024-03-16T10:45:00.492 INFO:tasks.workunit.client.0.smithi002.stdout:0/522: dwrite d3/d5/fd [4194304,4194304] 0 2024-03-16T10:45:00.498 INFO:tasks.workunit.client.0.smithi002.stdout:0/523: dread - d3/d31/d75/f98 zero size 2024-03-16T10:45:00.499 INFO:tasks.workunit.client.0.smithi002.stdout:0/524: write d3/f3a [2618798,34118] 0 2024-03-16T10:45:00.499 INFO:tasks.workunit.client.0.smithi002.stdout:8/533: mknod d7/d9/d25/d91/cbc 0 2024-03-16T10:45:00.503 INFO:tasks.workunit.client.0.smithi002.stdout:0/525: mkdir d3/da0 0 2024-03-16T10:45:00.508 INFO:tasks.workunit.client.0.smithi002.stdout:9/514: write d5/d9/dc/d1e/d24/d31/f32 [3432804,89567] 0 2024-03-16T10:45:00.510 INFO:tasks.workunit.client.0.smithi002.stdout:0/526: dread d3/fc [4194304,4194304] 0 2024-03-16T10:45:00.516 INFO:tasks.workunit.client.0.smithi002.stdout:8/534: dwrite d7/d9/fa [0,4194304] 0 2024-03-16T10:45:00.516 INFO:tasks.workunit.client.0.smithi002.stdout:0/527: rename d3/de/d6c to d3/d28/d2a/da1 0 2024-03-16T10:45:00.521 INFO:tasks.workunit.client.0.smithi002.stdout:0/528: mkdir d3/de/da2 0 2024-03-16T10:45:00.523 INFO:tasks.workunit.client.0.smithi002.stdout:8/535: dwrite d7/f21 [0,4194304] 0 2024-03-16T10:45:00.523 INFO:tasks.workunit.client.0.smithi002.stdout:9/515: dread d5/d9/d22/d3c/d4d/f75 [0,4194304] 0 2024-03-16T10:45:00.525 INFO:tasks.workunit.client.0.smithi002.stdout:0/529: creat d3/da0/fa3 x:0 0 0 2024-03-16T10:45:00.525 INFO:tasks.workunit.client.0.smithi002.stdout:9/516: write d5/d9/dc/f74 [512562,82847] 0 2024-03-16T10:45:00.530 INFO:tasks.workunit.client.0.smithi002.stdout:0/530: mknod d3/d5/ca4 0 2024-03-16T10:45:00.533 INFO:tasks.workunit.client.0.smithi002.stdout:9/517: dwrite d5/d9/dc/d1e/d91/f7a [0,4194304] 0 2024-03-16T10:45:00.535 INFO:tasks.workunit.client.0.smithi002.stdout:0/531: unlink d3/d28/f9d 0 2024-03-16T10:45:00.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.538+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8a1018f140 con 0x7f8a100ff9d0 2024-03-16T10:45:00.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.539+0000 7f8a0cff9700 1 -- 172.21.15.2:0/3083509888 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8a0005c1b0 con 0x7f8a100ff9d0 2024-03-16T10:45:00.541 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:45:00.541 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:45:00.542 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:45:00.543 INFO:tasks.workunit.client.0.smithi002.stdout:9/518: creat d5/d9/dc/d28/d90/f9b x:0 0 0 2024-03-16T10:45:00.543 INFO:tasks.workunit.client.0.smithi002.stdout:9/519: chown d5/d9/d55 195 1 2024-03-16T10:45:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f89f806de60 msgr2=0x7f89f8070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f89f806de60 0x7f89f8070320 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006010 tx=0x7f8a04009f90 comp rx=0 tx=0).stop 2024-03-16T10:45:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a100ff9d0 msgr2=0x7f8a10195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a100ff9d0 0x7f8a10195410 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000ea80 tx=0x7f8a0000ee40 comp rx=0 tx=0).stop 2024-03-16T10:45:00.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 shutdown_connections 2024-03-16T10:45:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:6828/274393035,v1:172.21.15.26:6829/274393035] conn(0x7f89f806de60 0x7f89f8070320 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a100ff9d0 0x7f8a10195410 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 --2- 172.21.15.2:0/3083509888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a10100bd0 0x7f8a10195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 >> 172.21.15.2:0/3083509888 conn(0x7f8a100faf70 msgr2=0x7f8a10104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:00.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 shutdown_connections 2024-03-16T10:45:00.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:00.542+0000 7f8a15c9f700 1 -- 172.21.15.2:0/3083509888 wait complete. 2024-03-16T10:45:00.549 INFO:tasks.workunit.client.0.smithi002.stdout:9/520: unlink d5/d9/d22/l85 0 2024-03-16T10:45:00.549 INFO:tasks.workunit.client.0.smithi002.stdout:9/521: dread - d5/f6f zero size 2024-03-16T10:45:00.550 INFO:tasks.workunit.client.0.smithi002.stdout:9/522: read d5/d9/dc/d1e/d91/f7a [3834784,114801] 0 2024-03-16T10:45:00.550 INFO:tasks.workunit.client.0.smithi002.stdout:0/532: dread d3/d27/f55 [4194304,4194304] 0 2024-03-16T10:45:00.552 INFO:tasks.workunit.client.0.smithi002.stdout:8/536: truncate d7/d9/d17/d23/d2e/f92 2650917 0 2024-03-16T10:45:00.552 INFO:tasks.workunit.client.0.smithi002.stdout:0/533: write d3/de/f51 [3034110,29571] 0 2024-03-16T10:45:00.553 INFO:tasks.workunit.client.0.smithi002.stdout:9/523: rename d5/d9/dc/d1e/d24/d31/l8b to d5/d9/d22/l9c 0 2024-03-16T10:45:00.557 INFO:tasks.workunit.client.0.smithi002.stdout:0/534: creat d3/fa5 x:0 0 0 2024-03-16T10:45:00.560 INFO:tasks.workunit.client.0.smithi002.stdout:9/524: dwrite d5/d9/d25/f71 [0,4194304] 0 2024-03-16T10:45:00.560 INFO:tasks.workunit.client.0.smithi002.stdout:8/537: link d7/d13/db8/d8f/la0 d7/d9/d17/d23/d2e/lbd 0 2024-03-16T10:45:00.560 INFO:tasks.workunit.client.0.smithi002.stdout:0/535: write d3/d5/f18 [654204,64817] 0 2024-03-16T10:45:00.564 INFO:tasks.workunit.client.0.smithi002.stdout:9/525: unlink d5/d9/d25/f71 0 2024-03-16T10:45:00.567 INFO:tasks.workunit.client.0.smithi002.stdout:8/538: dread d7/d13/db8/f61 [0,4194304] 0 2024-03-16T10:45:00.567 INFO:tasks.workunit.client.0.smithi002.stdout:0/536: creat d3/d28/d65/fa6 x:0 0 0 2024-03-16T10:45:00.569 INFO:tasks.workunit.client.0.smithi002.stdout:9/526: mknod d5/d9/dc/d1e/d86/c9d 0 2024-03-16T10:45:00.570 INFO:tasks.workunit.client.0.smithi002.stdout:0/537: mknod d3/ca7 0 2024-03-16T10:45:00.572 INFO:tasks.workunit.client.0.smithi002.stdout:8/539: dread d7/d9/f51 [0,4194304] 0 2024-03-16T10:45:00.578 INFO:tasks.workunit.client.0.smithi002.stdout:9/527: dread d5/d9/dc/f1f [0,4194304] 0 2024-03-16T10:45:00.578 INFO:tasks.workunit.client.0.smithi002.stdout:0/538: link d3/l24 d3/d5/la8 0 2024-03-16T10:45:00.579 INFO:tasks.workunit.client.0.smithi002.stdout:0/539: stat d3/d28/d65/c78 0 2024-03-16T10:45:00.580 INFO:tasks.workunit.client.0.smithi002.stdout:8/540: dwrite d7/d9/d25/d67/f8e [0,4194304] 0 2024-03-16T10:45:00.583 INFO:tasks.workunit.client.0.smithi002.stdout:7/440: sync 2024-03-16T10:45:00.584 INFO:tasks.workunit.client.0.smithi002.stdout:7/441: chown d3/d4/d1d/d87/d5c/d63 254 1 2024-03-16T10:45:00.590 INFO:tasks.workunit.client.0.smithi002.stdout:0/540: mkdir d3/d28/d2a/da1/d9c/da9 0 2024-03-16T10:45:00.592 INFO:tasks.workunit.client.0.smithi002.stdout:7/442: mknod d3/d4/d1d/d87/d96/d8c/c9f 0 2024-03-16T10:45:00.594 INFO:tasks.workunit.client.0.smithi002.stdout:0/541: mkdir d3/d31/daa 0 2024-03-16T10:45:00.595 INFO:tasks.workunit.client.0.smithi002.stdout:7/443: read d3/d37/f48 [829139,15289] 0 2024-03-16T10:45:00.595 INFO:tasks.workunit.client.0.smithi002.stdout:0/542: dread - d3/d28/f7f zero size 2024-03-16T10:45:00.596 INFO:tasks.workunit.client.0.smithi002.stdout:0/543: readlink d3/de/l4a 0 2024-03-16T10:45:00.596 INFO:tasks.workunit.client.0.smithi002.stdout:8/541: getdents d7/d9/d17/d23 0 2024-03-16T10:45:00.597 INFO:tasks.workunit.client.0.smithi002.stdout:8/542: stat d7/d9/d17/d23/d39/f5d 0 2024-03-16T10:45:00.598 INFO:tasks.workunit.client.0.smithi002.stdout:7/444: symlink d3/d4/d1d/d87/d5c/d63/d3f/la0 0 2024-03-16T10:45:00.599 INFO:tasks.workunit.client.0.smithi002.stdout:8/543: write d7/d9/d17/d23/f4d [3396911,90530] 0 2024-03-16T10:45:00.599 INFO:tasks.workunit.client.0.smithi002.stdout:0/544: creat d3/d31/d88/fab x:0 0 0 2024-03-16T10:45:00.600 INFO:tasks.workunit.client.0.smithi002.stdout:7/445: chown d3/d4/d1d/d87/d5c/d63/d34 926 1 2024-03-16T10:45:00.600 INFO:tasks.workunit.client.0.smithi002.stdout:8/544: fdatasync d7/d13/f68 0 2024-03-16T10:45:00.601 INFO:tasks.workunit.client.0.smithi002.stdout:8/545: write d7/d13/db8/f7c [187034,130050] 0 2024-03-16T10:45:00.603 INFO:tasks.workunit.client.0.smithi002.stdout:8/546: write d7/d9/d17/d20/f63 [3277010,55014] 0 2024-03-16T10:45:00.603 INFO:tasks.workunit.client.0.smithi002.stdout:7/446: truncate d3/f14 908994 0 2024-03-16T10:45:00.605 INFO:tasks.workunit.client.0.smithi002.stdout:8/547: fsync d7/d9/d17/d23/d2e/d5c/fa1 0 2024-03-16T10:45:00.605 INFO:tasks.workunit.client.0.smithi002.stdout:0/545: creat d3/d28/d3d/fac x:0 0 0 2024-03-16T10:45:00.606 INFO:tasks.workunit.client.0.smithi002.stdout:8/548: fdatasync d7/d13/db8/f86 0 2024-03-16T10:45:00.609 INFO:tasks.workunit.client.0.smithi002.stdout:8/549: write d7/d13/db8/f61 [413742,96398] 0 2024-03-16T10:45:00.610 INFO:tasks.workunit.client.0.smithi002.stdout:7/447: rename d3/d4/d1d/d45/f42 to d3/d4/d1d/d45/fa1 0 2024-03-16T10:45:00.610 INFO:tasks.workunit.client.0.smithi002.stdout:0/546: getdents d3/d31/daa 0 2024-03-16T10:45:00.611 INFO:tasks.workunit.client.0.smithi002.stdout:8/550: creat d7/d9/d25/d91/fbe x:0 0 0 2024-03-16T10:45:00.611 INFO:tasks.workunit.client.0.smithi002.stdout:0/547: truncate d3/de/f8f 433210 0 2024-03-16T10:45:00.612 INFO:tasks.workunit.client.0.smithi002.stdout:8/551: stat d7/d9/d17/d23/d39/c5b 0 2024-03-16T10:45:00.613 INFO:tasks.workunit.client.0.smithi002.stdout:0/548: write d3/d31/d88/fab [879699,11842] 0 2024-03-16T10:45:00.613 INFO:tasks.workunit.client.0.smithi002.stdout:7/448: creat d3/d4/d1d/d87/d5c/d63/fa2 x:0 0 0 2024-03-16T10:45:00.614 INFO:tasks.workunit.client.0.smithi002.stdout:8/552: read d7/d9/d17/d20/f5e [30420,14120] 0 2024-03-16T10:45:00.628 INFO:tasks.workunit.client.0.smithi002.stdout:0/549: dread d3/d28/f34 [0,4194304] 0 2024-03-16T10:45:00.629 INFO:tasks.workunit.client.0.smithi002.stdout:0/550: dread d3/de/f8f [0,4194304] 0 2024-03-16T10:45:00.634 INFO:tasks.workunit.client.0.smithi002.stdout:7/449: write d3/d37/f8f [610070,29640] 0 2024-03-16T10:45:00.634 INFO:tasks.workunit.client.0.smithi002.stdout:7/450: stat d3/d4/d1c/c26 0 2024-03-16T10:45:00.635 INFO:tasks.workunit.client.0.smithi002.stdout:8/553: truncate d7/d9/fa 2045794 0 2024-03-16T10:45:00.636 INFO:tasks.workunit.client.0.smithi002.stdout:8/554: chown d7/l9a 62799 1 2024-03-16T10:45:00.637 INFO:tasks.workunit.client.0.smithi002.stdout:7/451: symlink d3/d4/d1d/d45/la3 0 2024-03-16T10:45:00.637 INFO:tasks.workunit.client.0.smithi002.stdout:7/452: chown d3/d61/d85 2 1 2024-03-16T10:45:00.639 INFO:tasks.workunit.client.0.smithi002.stdout:8/555: mkdir d7/d9/d17/d23/d2e/d34/dbf 0 2024-03-16T10:45:00.641 INFO:tasks.workunit.client.0.smithi002.stdout:8/556: truncate d7/d9/d17/d23/d2e/d5c/fb6 641221 0 2024-03-16T10:45:00.643 INFO:tasks.workunit.client.0.smithi002.stdout:7/453: fsync d3/d4/f1e 0 2024-03-16T10:45:00.645 INFO:tasks.workunit.client.0.smithi002.stdout:0/551: truncate d3/d5/f19 1832745 0 2024-03-16T10:45:00.648 INFO:tasks.workunit.client.0.smithi002.stdout:7/454: creat d3/d4/d1d/d87/d96/d8c/fa4 x:0 0 0 2024-03-16T10:45:00.649 INFO:tasks.workunit.client.0.smithi002.stdout:0/552: dread d3/d28/d65/f7a [0,4194304] 0 2024-03-16T10:45:00.649 INFO:tasks.workunit.client.0.smithi002.stdout:8/557: dwrite d7/d9/d17/d23/d2e/d34/f9e [4194304,4194304] 0 2024-03-16T10:45:00.650 INFO:tasks.workunit.client.0.smithi002.stdout:8/558: stat d7/d13/f68 0 2024-03-16T10:45:00.651 INFO:tasks.workunit.client.0.smithi002.stdout:8/559: chown d7/d9/d17/l36 26028711 1 2024-03-16T10:45:00.659 INFO:tasks.workunit.client.0.smithi002.stdout:0/553: getdents d3/d31/d88 0 2024-03-16T10:45:00.659 INFO:tasks.workunit.client.0.smithi002.stdout:7/455: dwrite d3/d4/d1d/d87/d5c/d63/d55/f5d [0,4194304] 0 2024-03-16T10:45:00.663 INFO:tasks.workunit.client.0.smithi002.stdout:7/456: truncate d3/d4/d1d/d87/d96/d8c/f9b 587323 0 2024-03-16T10:45:00.665 INFO:tasks.workunit.client.0.smithi002.stdout:7/457: chown d3/d4/d1c/f33 117 1 2024-03-16T10:45:00.666 INFO:tasks.workunit.client.0.smithi002.stdout:8/560: dwrite d7/d9/d25/d91/fbe [0,4194304] 0 2024-03-16T10:45:00.669 INFO:tasks.workunit.client.0.smithi002.stdout:8/561: read d7/d13/f4c [3388952,59566] 0 2024-03-16T10:45:00.670 INFO:tasks.workunit.client.0.smithi002.stdout:7/458: symlink d3/d4/d1d/d87/d5c/d63/d55/la5 0 2024-03-16T10:45:00.671 INFO:tasks.workunit.client.0.smithi002.stdout:7/459: fsync d3/d61/d66/f4d 0 2024-03-16T10:45:00.675 INFO:tasks.workunit.client.0.smithi002.stdout:7/460: mknod d3/d4/d1d/d87/d5c/d63/d3f/d75/ca6 0 2024-03-16T10:45:00.676 INFO:tasks.workunit.client.0.smithi002.stdout:8/562: truncate d7/d9/d17/d23/d2e/f92 1633362 0 2024-03-16T10:45:00.679 INFO:tasks.workunit.client.0.smithi002.stdout:8/563: creat d7/d9/d25/d67/fc0 x:0 0 0 2024-03-16T10:45:00.682 INFO:tasks.workunit.client.0.smithi002.stdout:7/461: dwrite d3/d4/d1d/d87/d5c/d63/d34/f78 [0,4194304] 0 2024-03-16T10:45:00.683 INFO:tasks.workunit.client.0.smithi002.stdout:7/462: fsync d3/f15 0 2024-03-16T10:45:00.685 INFO:tasks.workunit.client.0.smithi002.stdout:7/463: write d3/d4/d1d/d87/d96/d5e/f60 [456055,78756] 0 2024-03-16T10:45:00.687 INFO:tasks.workunit.client.0.smithi002.stdout:8/564: creat d7/d9/d17/d23/d2e/fc1 x:0 0 0 2024-03-16T10:45:00.694 INFO:tasks.workunit.client.0.smithi002.stdout:8/565: mkdir d7/d9/d17/d23/d39/db7/dc2 0 2024-03-16T10:45:00.698 INFO:tasks.workunit.client.0.smithi002.stdout:7/464: dread d3/d37/f48 [0,4194304] 0 2024-03-16T10:45:00.699 INFO:tasks.workunit.client.0.smithi002.stdout:8/566: dread d7/d9/d17/d23/d2e/d3a/f47 [0,4194304] 0 2024-03-16T10:45:00.701 INFO:tasks.workunit.client.0.smithi002.stdout:8/567: mkdir d7/d9/d17/dc3 0 2024-03-16T10:45:00.702 INFO:tasks.workunit.client.0.smithi002.stdout:8/568: readlink d7/d9/d17/d83/l89 0 2024-03-16T10:45:00.702 INFO:tasks.workunit.client.0.smithi002.stdout:8/569: readlink d7/d9/d17/d23/d2e/l73 0 2024-03-16T10:45:00.704 INFO:tasks.workunit.client.0.smithi002.stdout:8/570: creat d7/d9/d17/d23/d39/db7/fc4 x:0 0 0 2024-03-16T10:45:00.706 INFO:tasks.workunit.client.0.smithi002.stdout:8/571: creat d7/d9/d17/d23/d39/fc5 x:0 0 0 2024-03-16T10:45:00.720 INFO:tasks.workunit.client.0.smithi002.stdout:8/572: dwrite d7/d13/fb1 [0,4194304] 0 2024-03-16T10:45:00.720 INFO:tasks.workunit.client.0.smithi002.stdout:8/573: chown d7/d9/d25/c46 1 1 2024-03-16T10:45:00.721 INFO:tasks.workunit.client.0.smithi002.stdout:8/574: dread - d7/d9/d17/d23/d39/db7/fc4 zero size 2024-03-16T10:45:00.722 INFO:tasks.workunit.client.0.smithi002.stdout:8/575: chown d7/d13/db8/f7b 283530827 1 2024-03-16T10:45:00.726 INFO:tasks.workunit.client.0.smithi002.stdout:8/576: dread d7/d9/d17/d23/d2e/d3a/f5f [0,4194304] 0 2024-03-16T10:45:00.728 INFO:tasks.workunit.client.0.smithi002.stdout:8/577: mknod d7/d9/d25/d91/cc6 0 2024-03-16T10:45:00.729 INFO:tasks.workunit.client.0.smithi002.stdout:8/578: chown d7/d9/d17/d23/d2e/d90 25506347 1 2024-03-16T10:45:00.729 INFO:tasks.workunit.client.0.smithi002.stdout:8/579: dread - d7/d9/d17/d23/d39/fb4 zero size 2024-03-16T10:45:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:00 smithi026 ceph-mon[37041]: pgmap v33: 65 pgs: 65 active+clean; 770 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 103 MiB/s wr, 318 op/s 2024-03-16T10:45:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:00 smithi026 ceph-mon[37041]: mgrmap e26: smithi026.omjkpo(active, since 60s), standbys: smithi002.zevrtm 2024-03-16T10:45:00.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:00 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:00.732 INFO:tasks.workunit.client.0.smithi002.stdout:8/580: creat d7/d9/d17/dc3/fc7 x:0 0 0 2024-03-16T10:45:00.733 INFO:tasks.workunit.client.0.smithi002.stdout:3/539: sync 2024-03-16T10:45:00.733 INFO:tasks.workunit.client.0.smithi002.stdout:4/477: sync 2024-03-16T10:45:00.735 INFO:tasks.workunit.client.0.smithi002.stdout:8/581: rename d7/d9/d25/d91/fa8 to d7/d9/d17/d23/d2e/d34/fc8 0 2024-03-16T10:45:00.736 INFO:tasks.workunit.client.0.smithi002.stdout:8/582: truncate d7/d9/d17/d83/da2/fab 235372 0 2024-03-16T10:45:00.739 INFO:tasks.workunit.client.0.smithi002.stdout:4/478: unlink d2/c18 0 2024-03-16T10:45:00.741 INFO:tasks.workunit.client.0.smithi002.stdout:3/540: dread d1/dd/d28/f31 [0,4194304] 0 2024-03-16T10:45:00.742 INFO:tasks.workunit.client.0.smithi002.stdout:8/583: dread d7/d9/d17/d23/d39/f5d [0,4194304] 0 2024-03-16T10:45:00.745 INFO:tasks.workunit.client.0.smithi002.stdout:8/584: rmdir d7/d13 39 2024-03-16T10:45:00.746 INFO:tasks.workunit.client.0.smithi002.stdout:4/479: dwrite d2/d33/f94 [0,4194304] 0 2024-03-16T10:45:00.749 INFO:tasks.workunit.client.0.smithi002.stdout:8/585: mkdir d7/d9/d25/dc9 0 2024-03-16T10:45:00.756 INFO:tasks.workunit.client.0.smithi002.stdout:8/586: readlink d7/d9/d17/d20/lae 0 2024-03-16T10:45:00.758 INFO:tasks.workunit.client.0.smithi002.stdout:4/480: creat d2/d6/d32/d66/d7f/f96 x:0 0 0 2024-03-16T10:45:00.759 INFO:tasks.workunit.client.0.smithi002.stdout:4/481: chown d2/d33/d1c/f6c 478 1 2024-03-16T10:45:00.760 INFO:tasks.workunit.client.0.smithi002.stdout:8/587: unlink d7/d9/f75 0 2024-03-16T10:45:00.761 INFO:tasks.workunit.client.0.smithi002.stdout:3/541: truncate d1/dd/d28/d37/f8d 4470232 0 2024-03-16T10:45:00.764 INFO:tasks.workunit.client.0.smithi002.stdout:8/588: readlink d7/d9/d17/d20/l9d 0 2024-03-16T10:45:00.765 INFO:tasks.workunit.client.0.smithi002.stdout:3/542: creat d1/dd/d28/d7b/d2c/fbb x:0 0 0 2024-03-16T10:45:00.766 INFO:tasks.workunit.client.0.smithi002.stdout:8/589: mknod d7/d9/d17/d23/cca 0 2024-03-16T10:45:00.767 INFO:tasks.workunit.client.0.smithi002.stdout:3/543: symlink d1/dd/d28/d7b/d25/lbc 0 2024-03-16T10:45:00.772 INFO:tasks.workunit.client.0.smithi002.stdout:3/544: dread d1/db6/f4a [0,4194304] 0 2024-03-16T10:45:00.773 INFO:tasks.workunit.client.0.smithi002.stdout:4/482: dread d2/d33/da/f41 [0,4194304] 0 2024-03-16T10:45:00.774 INFO:tasks.workunit.client.0.smithi002.stdout:4/483: chown d2/d33/f94 0 1 2024-03-16T10:45:00.790 INFO:tasks.workunit.client.0.smithi002.stdout:8/590: dwrite d7/d13/f4c [0,4194304] 0 2024-03-16T10:45:00.791 INFO:tasks.workunit.client.0.smithi002.stdout:3/545: getdents d1/dd/d28/d7b/d2c/d63 0 2024-03-16T10:45:00.791 INFO:tasks.workunit.client.0.smithi002.stdout:3/546: readlink d1/dd/d28/d7b/d2c/l41 0 2024-03-16T10:45:00.792 INFO:tasks.workunit.client.0.smithi002.stdout:4/484: getdents d2/d6/d32/d66 0 2024-03-16T10:45:00.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:00 smithi002 ceph-mon[29248]: pgmap v33: 65 pgs: 65 active+clean; 770 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 103 MiB/s wr, 318 op/s 2024-03-16T10:45:00.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:00 smithi002 ceph-mon[29248]: mgrmap e26: smithi026.omjkpo(active, since 60s), standbys: smithi002.zevrtm 2024-03-16T10:45:00.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:00 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:00.802 INFO:tasks.workunit.client.0.smithi002.stdout:8/591: unlink d7/d9/d25/f70 0 2024-03-16T10:45:00.806 INFO:tasks.workunit.client.0.smithi002.stdout:8/592: symlink d7/d9/d17/d83/lcb 0 2024-03-16T10:45:00.807 INFO:tasks.workunit.client.0.smithi002.stdout:3/547: link d1/dd/d74/lac d1/dd/d28/lbd 0 2024-03-16T10:45:00.810 INFO:tasks.workunit.client.0.smithi002.stdout:4/485: link d2/l4 d2/d33/da/d5e/d93/l97 0 2024-03-16T10:45:00.812 INFO:tasks.workunit.client.0.smithi002.stdout:8/593: read - d7/d9/d17/d23/d2e/d34/f6e zero size 2024-03-16T10:45:00.813 INFO:tasks.workunit.client.0.smithi002.stdout:3/548: mknod d1/cbe 0 2024-03-16T10:45:00.814 INFO:tasks.workunit.client.0.smithi002.stdout:8/594: dread d7/d9/d17/d23/d2e/d3a/f47 [0,4194304] 0 2024-03-16T10:45:00.815 INFO:tasks.workunit.client.0.smithi002.stdout:3/549: creat d1/dd/d28/d7b/d8a/d93/fbf x:0 0 0 2024-03-16T10:45:00.817 INFO:tasks.workunit.client.0.smithi002.stdout:8/595: mknod d7/d9/d25/d91/ccc 0 2024-03-16T10:45:00.818 INFO:tasks.workunit.client.0.smithi002.stdout:3/550: fdatasync d1/db6/d15/f7d 0 2024-03-16T10:45:00.819 INFO:tasks.workunit.client.0.smithi002.stdout:3/551: readlink d1/la 0 2024-03-16T10:45:00.822 INFO:tasks.workunit.client.0.smithi002.stdout:3/552: mkdir d1/dd/d28/d7b/d8a/d93/dc0 0 2024-03-16T10:45:00.822 INFO:tasks.workunit.client.0.smithi002.stdout:3/553: readlink d1/la 0 2024-03-16T10:45:00.825 INFO:tasks.workunit.client.0.smithi002.stdout:3/554: mknod d1/dd/d28/d7b/d25/cc1 0 2024-03-16T10:45:00.837 INFO:tasks.workunit.client.0.smithi002.stdout:4/486: write d2/d33/d1c/d45/f54 [500437,78065] 0 2024-03-16T10:45:00.838 INFO:tasks.workunit.client.0.smithi002.stdout:4/487: fsync d2/d6/f7e 0 2024-03-16T10:45:00.838 INFO:tasks.workunit.client.0.smithi002.stdout:3/555: dread d1/dd/d28/d7b/d22/f39 [0,4194304] 0 2024-03-16T10:45:00.844 INFO:tasks.workunit.client.0.smithi002.stdout:4/488: rename d2/d6/d32/d68/l44 to d2/d33/da/d5e/l98 0 2024-03-16T10:45:00.846 INFO:tasks.workunit.client.0.smithi002.stdout:3/556: creat d1/dd/d28/d7b/d8a/d93/fc2 x:0 0 0 2024-03-16T10:45:00.850 INFO:tasks.workunit.client.0.smithi002.stdout:3/557: creat d1/da5/fc3 x:0 0 0 2024-03-16T10:45:00.851 INFO:tasks.workunit.client.0.smithi002.stdout:8/596: dwrite d7/f60 [0,4194304] 0 2024-03-16T10:45:00.852 INFO:tasks.workunit.client.0.smithi002.stdout:8/597: chown d7/d9/d17/d23/d2e/d34/fc8 25 1 2024-03-16T10:45:00.852 INFO:tasks.workunit.client.0.smithi002.stdout:8/598: stat d7/d9/d25 0 2024-03-16T10:45:00.855 INFO:tasks.workunit.client.0.smithi002.stdout:8/599: chown d7/f60 114551090 1 2024-03-16T10:45:00.858 INFO:tasks.workunit.client.0.smithi002.stdout:3/558: dwrite d1/dd/d28/d7b/d2c/fbb [0,4194304] 0 2024-03-16T10:45:00.863 INFO:tasks.workunit.client.0.smithi002.stdout:3/559: creat d1/dd/d28/d7b/d2c/d97/fc4 x:0 0 0 2024-03-16T10:45:00.865 INFO:tasks.workunit.client.0.smithi002.stdout:2/542: sync 2024-03-16T10:45:00.867 INFO:tasks.workunit.client.0.smithi002.stdout:2/543: fdatasync d7/d2e/d1d/f26 0 2024-03-16T10:45:00.867 INFO:tasks.workunit.client.0.smithi002.stdout:8/600: creat d7/d9/d25/dc9/fcd x:0 0 0 2024-03-16T10:45:00.869 INFO:tasks.workunit.client.0.smithi002.stdout:8/601: read - d7/d9/d17/d23/d2e/d34/f6e zero size 2024-03-16T10:45:00.870 INFO:tasks.workunit.client.0.smithi002.stdout:3/560: dwrite d1/dd/d28/d37/f78 [0,4194304] 0 2024-03-16T10:45:00.871 INFO:tasks.workunit.client.0.smithi002.stdout:8/602: dread - d7/d13/f68 zero size 2024-03-16T10:45:00.873 INFO:tasks.workunit.client.0.smithi002.stdout:3/561: dread - d1/dd/d28/d7b/d8a/d93/fc2 zero size 2024-03-16T10:45:00.875 INFO:tasks.workunit.client.0.smithi002.stdout:2/544: dwrite d7/db/d70/f77 [0,4194304] 0 2024-03-16T10:45:00.876 INFO:tasks.workunit.client.0.smithi002.stdout:2/545: write d7/db/d70/f98 [573059,44578] 0 2024-03-16T10:45:00.880 INFO:tasks.workunit.client.0.smithi002.stdout:8/603: creat d7/d9/d17/d20/fce x:0 0 0 2024-03-16T10:45:00.881 INFO:tasks.workunit.client.0.smithi002.stdout:8/604: chown d7/d9/d17/d23/d2e/l99 2 1 2024-03-16T10:45:00.881 INFO:tasks.workunit.client.0.smithi002.stdout:4/489: write d2/d33/f29 [874063,92608] 0 2024-03-16T10:45:00.883 INFO:tasks.workunit.client.0.smithi002.stdout:3/562: mknod d1/dd/d74/cc5 0 2024-03-16T10:45:00.883 INFO:tasks.workunit.client.0.smithi002.stdout:4/490: write d2/d33/da/f7d [877251,6578] 0 2024-03-16T10:45:00.886 INFO:tasks.workunit.client.0.smithi002.stdout:2/546: symlink d7/d2e/d46/la7 0 2024-03-16T10:45:00.889 INFO:tasks.workunit.client.0.smithi002.stdout:3/563: rmdir d1/da5 39 2024-03-16T10:45:00.892 INFO:tasks.workunit.client.0.smithi002.stdout:4/491: rmdir d2/d33/d35 39 2024-03-16T10:45:00.893 INFO:tasks.workunit.client.0.smithi002.stdout:4/492: dread - d2/d6/d32/d66/f82 zero size 2024-03-16T10:45:00.894 INFO:tasks.workunit.client.0.smithi002.stdout:4/493: fsync d2/d33/f29 0 2024-03-16T10:45:00.894 INFO:tasks.workunit.client.0.smithi002.stdout:4/494: truncate d2/d6/d32/f50 4833954 0 2024-03-16T10:45:00.898 INFO:tasks.workunit.client.0.smithi002.stdout:2/547: dread f6 [0,4194304] 0 2024-03-16T10:45:00.899 INFO:tasks.workunit.client.0.smithi002.stdout:8/605: symlink d7/d9/d17/d23/d39/db7/dc2/lcf 0 2024-03-16T10:45:00.901 INFO:tasks.workunit.client.0.smithi002.stdout:4/495: creat d2/d33/d1c/d45/f99 x:0 0 0 2024-03-16T10:45:00.902 INFO:tasks.workunit.client.0.smithi002.stdout:3/564: mknod d1/db6/d15/d87/cc6 0 2024-03-16T10:45:00.906 INFO:tasks.workunit.client.0.smithi002.stdout:9/528: sync 2024-03-16T10:45:00.906 INFO:tasks.workunit.client.0.smithi002.stdout:7/465: sync 2024-03-16T10:45:00.906 INFO:tasks.workunit.client.0.smithi002.stdout:6/516: sync 2024-03-16T10:45:00.906 INFO:tasks.workunit.client.0.smithi002.stdout:1/543: sync 2024-03-16T10:45:00.906 INFO:tasks.workunit.client.0.smithi002.stdout:0/554: sync 2024-03-16T10:45:00.906 INFO:tasks.workunit.client.0.smithi002.stdout:5/465: sync 2024-03-16T10:45:00.906 INFO:tasks.workunit.client.0.smithi002.stdout:8/606: rename d7/d9/d17/d20/l88 to d7/d9/d17/d20/ld0 0 2024-03-16T10:45:00.909 INFO:tasks.workunit.client.0.smithi002.stdout:4/496: write d2/d33/da/f4f [458550,33484] 0 2024-03-16T10:45:00.913 INFO:tasks.workunit.client.0.smithi002.stdout:2/548: dread d7/db/f35 [0,4194304] 0 2024-03-16T10:45:00.913 INFO:tasks.workunit.client.0.smithi002.stdout:7/466: unlink d3/d4/d1d/f70 0 2024-03-16T10:45:00.915 INFO:tasks.workunit.client.0.smithi002.stdout:0/555: dwrite d3/fa5 [0,4194304] 0 2024-03-16T10:45:00.916 INFO:tasks.workunit.client.0.smithi002.stdout:6/517: truncate d15/f33 389648 0 2024-03-16T10:45:00.917 INFO:tasks.workunit.client.0.smithi002.stdout:6/518: chown d15/d2d/d2c/d8f/d96 39071119 1 2024-03-16T10:45:00.918 INFO:tasks.workunit.client.0.smithi002.stdout:8/607: creat d7/d9/d17/d23/d39/db7/fd1 x:0 0 0 2024-03-16T10:45:00.919 INFO:tasks.workunit.client.0.smithi002.stdout:8/608: truncate d7/d13/f68 75362 0 2024-03-16T10:45:00.920 INFO:tasks.workunit.client.0.smithi002.stdout:1/544: dread d2/d3/d51/f5a [4194304,4194304] 0 2024-03-16T10:45:00.920 INFO:tasks.workunit.client.0.smithi002.stdout:8/609: write d7/d13/db8/f98 [742591,129319] 0 2024-03-16T10:45:00.921 INFO:tasks.workunit.client.0.smithi002.stdout:8/610: chown d7/d9/d17/d23 1451 1 2024-03-16T10:45:00.923 INFO:tasks.workunit.client.0.smithi002.stdout:4/497: rmdir d2/d6/d32/d66/d7f 39 2024-03-16T10:45:00.923 INFO:tasks.workunit.client.0.smithi002.stdout:8/611: dread - d7/d9/d17/d23/d39/f8c zero size 2024-03-16T10:45:00.925 INFO:tasks.workunit.client.0.smithi002.stdout:8/612: truncate d7/d9/d17/d23/d39/f8c 434737 0 2024-03-16T10:45:00.927 INFO:tasks.workunit.client.0.smithi002.stdout:7/467: chown d3/f14 823 1 2024-03-16T10:45:00.931 INFO:tasks.workunit.client.0.smithi002.stdout:6/519: rmdir d15/d2d/d2b 39 2024-03-16T10:45:00.933 INFO:tasks.workunit.client.0.smithi002.stdout:3/565: write d1/dd/f68 [4638773,129013] 0 2024-03-16T10:45:00.942 INFO:tasks.workunit.client.0.smithi002.stdout:7/468: dread d3/d4/d1d/d87/d5c/d63/f4c [0,4194304] 0 2024-03-16T10:45:00.943 INFO:tasks.workunit.client.0.smithi002.stdout:5/466: getdents d0/d6b 0 2024-03-16T10:45:00.944 INFO:tasks.workunit.client.0.smithi002.stdout:1/545: mknod d2/d3/d1c/cad 0 2024-03-16T10:45:00.948 INFO:tasks.workunit.client.0.smithi002.stdout:9/529: getdents d5/d9/d55 0 2024-03-16T10:45:00.949 INFO:tasks.workunit.client.0.smithi002.stdout:6/520: unlink d15/d2d/d2b/d3f/d77/f99 0 2024-03-16T10:45:00.951 INFO:tasks.workunit.client.0.smithi002.stdout:7/469: creat d3/d4/d1d/d87/d5c/d63/d55/fa7 x:0 0 0 2024-03-16T10:45:00.952 INFO:tasks.workunit.client.0.smithi002.stdout:8/613: rename d7/d9/d25/d67/lad to d7/d13/db8/ld2 0 2024-03-16T10:45:00.954 INFO:tasks.workunit.client.0.smithi002.stdout:1/546: unlink d2/df/d12/d84/d87/d96/c97 0 2024-03-16T10:45:00.954 INFO:tasks.workunit.client.0.smithi002.stdout:7/470: truncate d3/d61/d66/f50 5009518 0 2024-03-16T10:45:00.954 INFO:tasks.workunit.client.0.smithi002.stdout:6/521: dwrite d15/d2d/d2c/f3d [0,4194304] 0 2024-03-16T10:45:00.956 INFO:tasks.workunit.client.0.smithi002.stdout:2/549: getdents d7/d2e/d28/d44/d6d 0 2024-03-16T10:45:00.962 INFO:tasks.workunit.client.0.smithi002.stdout:7/471: creat d3/d4/d1c/d5b/d64/fa8 x:0 0 0 2024-03-16T10:45:00.962 INFO:tasks.workunit.client.0.smithi002.stdout:6/522: symlink d15/d2d/d2c/d8f/lb4 0 2024-03-16T10:45:00.963 INFO:tasks.workunit.client.0.smithi002.stdout:7/472: write d3/d37/f8f [1468167,58921] 0 2024-03-16T10:45:00.963 INFO:tasks.workunit.client.0.smithi002.stdout:9/530: creat d5/d9/dc/d1e/d91/d83/f9e x:0 0 0 2024-03-16T10:45:00.965 INFO:tasks.workunit.client.0.smithi002.stdout:3/566: rename d1/c65 to d1/db6/d15/d87/d8c/cc7 0 2024-03-16T10:45:00.965 INFO:tasks.workunit.client.0.smithi002.stdout:7/473: write d3/d4/d1d/d87/d96/d8c/f9b [844120,2620] 0 2024-03-16T10:45:00.966 INFO:tasks.workunit.client.0.smithi002.stdout:1/547: getdents d2/d3/d1c/da0 0 2024-03-16T10:45:00.968 INFO:tasks.workunit.client.0.smithi002.stdout:3/567: write d1/dd/d28/d7b/d2c/d63/f79 [5232967,120825] 0 2024-03-16T10:45:00.970 INFO:tasks.workunit.client.0.smithi002.stdout:5/467: getdents d0/d16/d5b 0 2024-03-16T10:45:00.971 INFO:tasks.workunit.client.0.smithi002.stdout:8/614: rmdir d7/d9/d25/db2 0 2024-03-16T10:45:00.973 INFO:tasks.workunit.client.0.smithi002.stdout:9/531: rename d5/d9/c19 to d5/d9/dc/d1e/d91/d83/c9f 0 2024-03-16T10:45:00.974 INFO:tasks.workunit.client.0.smithi002.stdout:7/474: mknod d3/d61/d66/d88/ca9 0 2024-03-16T10:45:00.976 INFO:tasks.workunit.client.0.smithi002.stdout:7/475: write d3/d4/d1d/d45/f53 [214073,39980] 0 2024-03-16T10:45:00.976 INFO:tasks.workunit.client.0.smithi002.stdout:8/615: dread d7/d9/d17/f27 [0,4194304] 0 2024-03-16T10:45:00.978 INFO:tasks.workunit.client.0.smithi002.stdout:7/476: chown d3/d4/d1d/d45/l71 0 1 2024-03-16T10:45:00.978 INFO:tasks.workunit.client.0.smithi002.stdout:9/532: dread d5/d9/dc/d28/f65 [0,4194304] 0 2024-03-16T10:45:00.981 INFO:tasks.workunit.client.0.smithi002.stdout:2/550: dread d7/d2e/d1d/d2f/d4b/f74 [4194304,4194304] 0 2024-03-16T10:45:00.986 INFO:tasks.workunit.client.0.smithi002.stdout:7/477: truncate d3/f14 294944 0 2024-03-16T10:45:00.990 INFO:tasks.workunit.client.0.smithi002.stdout:2/551: read d7/db/f2a [963166,14353] 0 2024-03-16T10:45:00.991 INFO:tasks.workunit.client.0.smithi002.stdout:3/568: creat d1/dd/d28/d7b/d8a/fc8 x:0 0 0 2024-03-16T10:45:00.993 INFO:tasks.workunit.client.0.smithi002.stdout:7/478: mkdir d3/d4/d1d/d87/d5c/d63/d3f/daa 0 2024-03-16T10:45:00.993 INFO:tasks.workunit.client.0.smithi002.stdout:1/548: getdents d2/d3/d1c/d5d 0 2024-03-16T10:45:00.995 INFO:tasks.workunit.client.0.smithi002.stdout:8/616: rename d7/d9/d17/d23/d39/fc5 to d7/d9/d17/d23/fd3 0 2024-03-16T10:45:00.996 INFO:tasks.workunit.client.0.smithi002.stdout:6/523: dwrite d15/f69 [0,4194304] 0 2024-03-16T10:45:00.998 INFO:tasks.workunit.client.0.smithi002.stdout:3/569: fdatasync d1/db6/d15/f2e 0 2024-03-16T10:45:01.001 INFO:tasks.workunit.client.0.smithi002.stdout:3/570: write d1/dd/d28/d7b/d2c/f90 [3727030,70564] 0 2024-03-16T10:45:01.002 INFO:tasks.workunit.client.0.smithi002.stdout:9/533: dread d5/d9/d22/f6e [0,4194304] 0 2024-03-16T10:45:01.004 INFO:tasks.workunit.client.0.smithi002.stdout:2/552: truncate d7/f5e 157092 0 2024-03-16T10:45:01.008 INFO:tasks.workunit.client.0.smithi002.stdout:8/617: mkdir d7/d9/d17/d23/d39/dd4 0 2024-03-16T10:45:01.008 INFO:tasks.workunit.client.0.smithi002.stdout:2/553: chown d7/d2e/d1d/d2f/c8d 167 1 2024-03-16T10:45:01.009 INFO:tasks.workunit.client.0.smithi002.stdout:6/524: dwrite d15/d2d/d8a/d7c/d87/fb0 [0,4194304] 0 2024-03-16T10:45:01.010 INFO:tasks.workunit.client.0.smithi002.stdout:9/534: dread d5/d9/dc/f74 [0,4194304] 0 2024-03-16T10:45:01.012 INFO:tasks.workunit.client.0.smithi002.stdout:2/554: chown d7/d2e/d28/f6a 438 1 2024-03-16T10:45:01.012 INFO:tasks.workunit.client.0.smithi002.stdout:6/525: chown d15/f25 1421237 1 2024-03-16T10:45:01.013 INFO:tasks.workunit.client.0.smithi002.stdout:3/571: dwrite d1/f8f [0,4194304] 0 2024-03-16T10:45:01.014 INFO:tasks.workunit.client.0.smithi002.stdout:1/549: unlink d2/d3/d1c/l46 0 2024-03-16T10:45:01.016 INFO:tasks.workunit.client.0.smithi002.stdout:1/550: chown d2/d3/d1c/c40 217036 1 2024-03-16T10:45:01.016 INFO:tasks.workunit.client.0.smithi002.stdout:3/572: dread - d1/db6/d15/d87/d8c/fb3 zero size 2024-03-16T10:45:01.018 INFO:tasks.workunit.client.0.smithi002.stdout:3/573: write d1/dd/d28/d7b/d2c/d63/faf [100490,3538] 0 2024-03-16T10:45:01.024 INFO:tasks.workunit.client.0.smithi002.stdout:8/618: unlink d7/d9/d17/d83/fa7 0 2024-03-16T10:45:01.027 INFO:tasks.workunit.client.0.smithi002.stdout:6/526: truncate d15/d2d/f20 870424 0 2024-03-16T10:45:01.028 INFO:tasks.workunit.client.0.smithi002.stdout:6/527: chown d15/d2d/da8 2 1 2024-03-16T10:45:01.029 INFO:tasks.workunit.client.0.smithi002.stdout:6/528: write d15/f5f [3599755,88817] 0 2024-03-16T10:45:01.032 INFO:tasks.workunit.client.0.smithi002.stdout:7/479: rename d3/f27 to d3/d4/d1d/d87/d5c/d63/fab 0 2024-03-16T10:45:01.035 INFO:tasks.workunit.client.0.smithi002.stdout:1/551: dread d2/d3/f7c [0,4194304] 0 2024-03-16T10:45:01.037 INFO:tasks.workunit.client.0.smithi002.stdout:6/529: dwrite d15/f1a [0,4194304] 0 2024-03-16T10:45:01.037 INFO:tasks.workunit.client.0.smithi002.stdout:1/552: dread - d2/d3/fab zero size 2024-03-16T10:45:01.039 INFO:tasks.workunit.client.0.smithi002.stdout:2/555: mkdir d7/d2e/d1d/d38/da8 0 2024-03-16T10:45:01.045 INFO:tasks.workunit.client.0.smithi002.stdout:7/480: creat d3/d4/d1d/d87/d5c/d63/d3f/fac x:0 0 0 2024-03-16T10:45:01.047 INFO:tasks.workunit.client.0.smithi002.stdout:7/481: write d3/d4/d1d/d87/d96/d8c/f95 [328426,114249] 0 2024-03-16T10:45:01.050 INFO:tasks.workunit.client.0.smithi002.stdout:7/482: write d3/d37/d6a/f72 [848394,58639] 0 2024-03-16T10:45:01.051 INFO:tasks.workunit.client.0.smithi002.stdout:8/619: write d7/d9/d17/f3d [84596,49959] 0 2024-03-16T10:45:01.057 INFO:tasks.workunit.client.0.smithi002.stdout:3/574: getdents d1/dd/d28/d7b/d73 0 2024-03-16T10:45:01.058 INFO:tasks.workunit.client.0.smithi002.stdout:6/530: dread d15/d2d/f1f [0,4194304] 0 2024-03-16T10:45:01.058 INFO:tasks.workunit.client.0.smithi002.stdout:7/483: symlink d3/d4/d1c/d5b/d64/lad 0 2024-03-16T10:45:01.059 INFO:tasks.workunit.client.0.smithi002.stdout:1/553: dread d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:45:01.059 INFO:tasks.workunit.client.0.smithi002.stdout:2/556: dread d7/d2e/d1d/f4f [0,4194304] 0 2024-03-16T10:45:01.060 INFO:tasks.workunit.client.0.smithi002.stdout:9/535: truncate d5/d9/dc/d1e/d24/d31/f32 3129781 0 2024-03-16T10:45:01.066 INFO:tasks.workunit.client.0.smithi002.stdout:1/554: unlink d2/fa4 0 2024-03-16T10:45:01.067 INFO:tasks.workunit.client.0.smithi002.stdout:7/484: chown d3/d4/d1d/d87/d5c/d63/c25 440878 1 2024-03-16T10:45:01.069 INFO:tasks.workunit.client.0.smithi002.stdout:2/557: mknod d7/d2e/d28/d44/d6d/ca9 0 2024-03-16T10:45:01.069 INFO:tasks.workunit.client.0.smithi002.stdout:8/620: link d7/d9/d17/d20/c77 d7/d9/d17/d23/d2e/d34/cd5 0 2024-03-16T10:45:01.070 INFO:tasks.workunit.client.0.smithi002.stdout:2/558: write d7/d2e/f67 [3076439,104130] 0 2024-03-16T10:45:01.073 INFO:tasks.workunit.client.0.smithi002.stdout:2/559: write d7/d2e/d1d/f26 [5123113,104947] 0 2024-03-16T10:45:01.073 INFO:tasks.workunit.client.0.smithi002.stdout:3/575: rmdir d1/dd/d28/d7b/d8a/d93/dc0 0 2024-03-16T10:45:01.074 INFO:tasks.workunit.client.0.smithi002.stdout:1/555: mkdir d2/d3/dae 0 2024-03-16T10:45:01.077 INFO:tasks.workunit.client.0.smithi002.stdout:3/576: dread d1/dd/d28/f31 [0,4194304] 0 2024-03-16T10:45:01.077 INFO:tasks.workunit.client.0.smithi002.stdout:2/560: creat d7/d2e/d28/d40/d50/faa x:0 0 0 2024-03-16T10:45:01.079 INFO:tasks.workunit.client.0.smithi002.stdout:9/536: rename d5/d9/d25/f42 to d5/fa0 0 2024-03-16T10:45:01.083 INFO:tasks.workunit.client.0.smithi002.stdout:1/556: mknod d2/df/d12/d29/d5c/caf 0 2024-03-16T10:45:01.093 INFO:tasks.workunit.client.0.smithi002.stdout:9/537: creat d5/d9/d25/d23/d81/fa1 x:0 0 0 2024-03-16T10:45:01.093 INFO:tasks.workunit.client.0.smithi002.stdout:8/621: getdents d7/d9/d17/d23/d39/db7/dc2 0 2024-03-16T10:45:01.096 INFO:tasks.workunit.client.0.smithi002.stdout:8/622: stat d7/d9/d25/d91/f96 0 2024-03-16T10:45:01.099 INFO:tasks.workunit.client.0.smithi002.stdout:1/557: getdents d2/d58 0 2024-03-16T10:45:01.099 INFO:tasks.workunit.client.0.smithi002.stdout:9/538: creat d5/d9/dc/fa2 x:0 0 0 2024-03-16T10:45:01.101 INFO:tasks.workunit.client.0.smithi002.stdout:2/561: write d7/d27/d3f/f69 [2443066,120388] 0 2024-03-16T10:45:01.102 INFO:tasks.workunit.client.0.smithi002.stdout:3/577: dread d1/f35 [4194304,4194304] 0 2024-03-16T10:45:01.103 INFO:tasks.workunit.client.0.smithi002.stdout:8/623: chown d7/d9/d17/d23/d39/fbb 76227 1 2024-03-16T10:45:01.104 INFO:tasks.workunit.client.0.smithi002.stdout:2/562: fsync d7/db/f58 0 2024-03-16T10:45:01.104 INFO:tasks.workunit.client.0.smithi002.stdout:7/485: dwrite d3/f14 [0,4194304] 0 2024-03-16T10:45:01.106 INFO:tasks.workunit.client.0.smithi002.stdout:8/624: chown d7/d9/d17/dc3 111022 1 2024-03-16T10:45:01.107 INFO:tasks.workunit.client.0.smithi002.stdout:9/539: creat d5/d9/dc/d1e/d94/fa3 x:0 0 0 2024-03-16T10:45:01.108 INFO:tasks.workunit.client.0.smithi002.stdout:1/558: symlink d2/df/d12/lb0 0 2024-03-16T10:45:01.111 INFO:tasks.workunit.client.0.smithi002.stdout:1/559: write d2/df/d12/f3f [701702,8696] 0 2024-03-16T10:45:01.112 INFO:tasks.workunit.client.0.smithi002.stdout:9/540: dread d5/d9/d34/f61 [0,4194304] 0 2024-03-16T10:45:01.115 INFO:tasks.workunit.client.0.smithi002.stdout:1/560: fsync d2/d3/d51/d56/f8c 0 2024-03-16T10:45:01.115 INFO:tasks.workunit.client.0.smithi002.stdout:7/486: creat d3/d4/d1c/d5b/d64/d99/d9c/fae x:0 0 0 2024-03-16T10:45:01.116 INFO:tasks.workunit.client.0.smithi002.stdout:2/563: dwrite d7/f90 [0,4194304] 0 2024-03-16T10:45:01.118 INFO:tasks.workunit.client.0.smithi002.stdout:1/561: read d2/f76 [382209,18358] 0 2024-03-16T10:45:01.123 INFO:tasks.workunit.client.0.smithi002.stdout:7/487: creat d3/d61/faf x:0 0 0 2024-03-16T10:45:01.125 INFO:tasks.workunit.client.0.smithi002.stdout:2/564: unlink d7/d2e/f32 0 2024-03-16T10:45:01.127 INFO:tasks.workunit.client.0.smithi002.stdout:8/625: link d7/d9/d17/d23/c9b d7/d9/d17/d23/d2e/d34/cd6 0 2024-03-16T10:45:01.128 INFO:tasks.workunit.client.0.smithi002.stdout:3/578: truncate d1/dd/d28/d7b/f24 2734203 0 2024-03-16T10:45:01.132 INFO:tasks.workunit.client.0.smithi002.stdout:1/562: rename d2/df/d12/d86 to d2/d3/d1c/da0/db1 0 2024-03-16T10:45:01.132 INFO:tasks.workunit.client.0.smithi002.stdout:9/541: dread d5/d9/d25/d23/f3b [0,4194304] 0 2024-03-16T10:45:01.133 INFO:tasks.workunit.client.0.smithi002.stdout:8/626: symlink d7/d9/d25/d91/ld7 0 2024-03-16T10:45:01.134 INFO:tasks.workunit.client.0.smithi002.stdout:8/627: dread - d7/d9/d17/d23/d2e/fc1 zero size 2024-03-16T10:45:01.135 INFO:tasks.workunit.client.0.smithi002.stdout:1/563: mknod d2/df/d12/d84/d87/d96/da8/cb2 0 2024-03-16T10:45:01.137 INFO:tasks.workunit.client.0.smithi002.stdout:8/628: write d7/d9/d17/d20/fce [764553,34642] 0 2024-03-16T10:45:01.139 INFO:tasks.workunit.client.0.smithi002.stdout:1/564: mknod d2/d3/dae/cb3 0 2024-03-16T10:45:01.140 INFO:tasks.workunit.client.0.smithi002.stdout:0/556: sync 2024-03-16T10:45:01.141 INFO:tasks.workunit.client.0.smithi002.stdout:9/542: dwrite d5/d9/dc/d1e/d91/d83/f9e [0,4194304] 0 2024-03-16T10:45:01.142 INFO:tasks.workunit.client.0.smithi002.stdout:1/565: chown d2/df/d12/f60 6 1 2024-03-16T10:45:01.143 INFO:tasks.workunit.client.0.smithi002.stdout:2/565: dwrite f6 [0,4194304] 0 2024-03-16T10:45:01.147 INFO:tasks.workunit.client.0.smithi002.stdout:2/566: chown d7/d27/d3f/f69 176830 1 2024-03-16T10:45:01.147 INFO:tasks.workunit.client.0.smithi002.stdout:9/543: unlink d5/d9/d55/l9a 0 2024-03-16T10:45:01.150 INFO:tasks.workunit.client.0.smithi002.stdout:1/566: rmdir d2/df/d12/d74 39 2024-03-16T10:45:01.153 INFO:tasks.workunit.client.0.smithi002.stdout:8/629: rename d7/d9/d17/d23/f4d to d7/d9/d17/d23/d39/db7/fd8 0 2024-03-16T10:45:01.153 INFO:tasks.workunit.client.0.smithi002.stdout:0/557: dwrite d3/d5/f2f [0,4194304] 0 2024-03-16T10:45:01.154 INFO:tasks.workunit.client.0.smithi002.stdout:9/544: rmdir d5/d9/dc/d1e/d86 39 2024-03-16T10:45:01.162 INFO:tasks.workunit.client.0.smithi002.stdout:9/545: mkdir d5/d9/d22/d3c/d4d/da4 0 2024-03-16T10:45:01.255 INFO:tasks.workunit.client.0.smithi002.stdout:0/558: rename d3/f47 to d3/d31/daa/fad 0 2024-03-16T10:45:01.256 INFO:tasks.workunit.client.0.smithi002.stdout:9/546: creat d5/d9/dc/d1e/d24/fa5 x:0 0 0 2024-03-16T10:45:01.256 INFO:tasks.workunit.client.0.smithi002.stdout:8/630: truncate d7/d9/d17/d23/d2e/f92 24612 0 2024-03-16T10:45:01.256 INFO:tasks.workunit.client.0.smithi002.stdout:0/559: chown d3/d59/l5e 310 1 2024-03-16T10:45:01.257 INFO:tasks.workunit.client.0.smithi002.stdout:8/631: dread - d7/d9/d17/d23/d2e/d90/fba zero size 2024-03-16T10:45:01.258 INFO:tasks.workunit.client.0.smithi002.stdout:8/632: fdatasync d7/d13/db8/f61 0 2024-03-16T10:45:01.260 INFO:tasks.workunit.client.0.smithi002.stdout:1/567: dread d2/d3/f39 [0,4194304] 0 2024-03-16T10:45:01.261 INFO:tasks.workunit.client.0.smithi002.stdout:1/568: read d2/d3/d51/f5a [6564628,90700] 0 2024-03-16T10:45:01.266 INFO:tasks.workunit.client.0.smithi002.stdout:1/569: dread d2/d3/f8 [0,4194304] 0 2024-03-16T10:45:01.266 INFO:tasks.workunit.client.0.smithi002.stdout:9/547: fsync d5/d9/dc/f1f 0 2024-03-16T10:45:01.270 INFO:tasks.workunit.client.0.smithi002.stdout:1/570: write d2/d3/f4b [3994225,48319] 0 2024-03-16T10:45:01.273 INFO:tasks.workunit.client.0.smithi002.stdout:8/633: mkdir d7/d9/d17/d23/d2e/d34/dbf/dd9 0 2024-03-16T10:45:01.278 INFO:tasks.workunit.client.0.smithi002.stdout:0/560: creat d3/fae x:0 0 0 2024-03-16T10:45:01.279 INFO:tasks.workunit.client.0.smithi002.stdout:9/548: symlink d5/d9/d22/d3c/d4d/da4/la6 0 2024-03-16T10:45:01.281 INFO:tasks.workunit.client.0.smithi002.stdout:0/561: write d3/de/f7d [102231,69808] 0 2024-03-16T10:45:01.283 INFO:tasks.workunit.client.0.smithi002.stdout:8/634: symlink d7/d9/d17/d23/d2e/d34/dbf/dd9/lda 0 2024-03-16T10:45:01.284 INFO:tasks.workunit.client.0.smithi002.stdout:9/549: dwrite d5/d9/dc/d1e/d91/d83/f9e [0,4194304] 0 2024-03-16T10:45:01.285 INFO:tasks.workunit.client.0.smithi002.stdout:4/498: sync 2024-03-16T10:45:01.285 INFO:tasks.workunit.client.0.smithi002.stdout:5/468: sync 2024-03-16T10:45:01.285 INFO:tasks.workunit.client.0.smithi002.stdout:6/531: sync 2024-03-16T10:45:01.286 INFO:tasks.workunit.client.0.smithi002.stdout:4/499: chown d2/d33/da/f49 820 1 2024-03-16T10:45:01.286 INFO:tasks.workunit.client.0.smithi002.stdout:0/562: creat d3/d28/d3d/d44/faf x:0 0 0 2024-03-16T10:45:01.290 INFO:tasks.workunit.client.0.smithi002.stdout:5/469: mkdir d0/d16/d1d/d33/d8c 0 2024-03-16T10:45:01.291 INFO:tasks.workunit.client.0.smithi002.stdout:9/550: dread d5/d9/d55/f7b [0,4194304] 0 2024-03-16T10:45:01.291 INFO:tasks.workunit.client.0.smithi002.stdout:8/635: getdents d7/d9/d17/d23/d39/dd4 0 2024-03-16T10:45:01.292 INFO:tasks.workunit.client.0.smithi002.stdout:4/500: creat d2/d33/da/d5e/f9a x:0 0 0 2024-03-16T10:45:01.293 INFO:tasks.workunit.client.0.smithi002.stdout:9/551: write d5/d9/d22/f47 [2646944,59675] 0 2024-03-16T10:45:01.294 INFO:tasks.workunit.client.0.smithi002.stdout:6/532: mkdir d15/d2d/d2c/d8f/d96/db5 0 2024-03-16T10:45:01.297 INFO:tasks.workunit.client.0.smithi002.stdout:0/563: rename d3/d28/f7f to d3/d59/fb0 0 2024-03-16T10:45:01.298 INFO:tasks.workunit.client.0.smithi002.stdout:5/470: dwrite d0/d16/d1d/d43/d48/f7f [0,4194304] 0 2024-03-16T10:45:01.299 INFO:tasks.workunit.client.0.smithi002.stdout:0/564: chown d3/d28/d2a/f4f 138104681 1 2024-03-16T10:45:01.299 INFO:tasks.workunit.client.0.smithi002.stdout:0/565: chown d3/d28/d2a/d67/f89 25 1 2024-03-16T10:45:01.302 INFO:tasks.workunit.client.0.smithi002.stdout:9/552: rmdir d5/d9/d25/d23 39 2024-03-16T10:45:01.310 INFO:tasks.workunit.client.0.smithi002.stdout:4/501: rename d2/d6/f46 to d2/d33/da/d88/f9b 0 2024-03-16T10:45:01.311 INFO:tasks.workunit.client.0.smithi002.stdout:6/533: dwrite d15/d2d/f2e [0,4194304] 0 2024-03-16T10:45:01.313 INFO:tasks.workunit.client.0.smithi002.stdout:0/566: creat d3/da0/fb1 x:0 0 0 2024-03-16T10:45:01.315 INFO:tasks.workunit.client.0.smithi002.stdout:9/553: rmdir d5/d9/dc/d1e/d91/d83 39 2024-03-16T10:45:01.321 INFO:tasks.workunit.client.0.smithi002.stdout:5/471: mkdir d0/d16/d8d 0 2024-03-16T10:45:01.323 INFO:tasks.workunit.client.0.smithi002.stdout:4/502: symlink d2/d33/d1c/l9c 0 2024-03-16T10:45:01.324 INFO:tasks.workunit.client.0.smithi002.stdout:6/534: creat d15/dab/fb6 x:0 0 0 2024-03-16T10:45:01.328 INFO:tasks.workunit.client.0.smithi002.stdout:0/567: write d3/d31/daa/fad [1176272,58054] 0 2024-03-16T10:45:01.328 INFO:tasks.workunit.client.0.smithi002.stdout:9/554: write d5/d9/dc/d1e/f6c [1020881,118775] 0 2024-03-16T10:45:01.330 INFO:tasks.workunit.client.0.smithi002.stdout:5/472: write d0/f6 [1463517,49400] 0 2024-03-16T10:45:01.332 INFO:tasks.workunit.client.0.smithi002.stdout:4/503: write d2/d33/d35/f63 [3607569,82820] 0 2024-03-16T10:45:01.334 INFO:tasks.workunit.client.0.smithi002.stdout:8/636: getdents d7 0 2024-03-16T10:45:01.335 INFO:tasks.workunit.client.0.smithi002.stdout:8/637: chown d7/d13/db8/f98 3 1 2024-03-16T10:45:01.336 INFO:tasks.workunit.client.0.smithi002.stdout:9/555: readlink d5/d9/d25/l44 0 2024-03-16T10:45:01.342 INFO:tasks.workunit.client.0.smithi002.stdout:8/638: creat d7/d13/db8/d8f/fdb x:0 0 0 2024-03-16T10:45:01.343 INFO:tasks.workunit.client.0.smithi002.stdout:8/639: write d7/d9/d17/d23/d2e/d34/f9e [4343590,5076] 0 2024-03-16T10:45:01.346 INFO:tasks.workunit.client.0.smithi002.stdout:6/535: rename d15/d2d/d2b/d3f/f45 to d15/d2d/d2b/fb7 0 2024-03-16T10:45:01.347 INFO:tasks.workunit.client.0.smithi002.stdout:4/504: mknod d2/c9d 0 2024-03-16T10:45:01.353 INFO:tasks.workunit.client.0.smithi002.stdout:8/640: fdatasync d7/d9/d17/d23/f33 0 2024-03-16T10:45:01.356 INFO:tasks.workunit.client.0.smithi002.stdout:6/536: unlink d15/d2d/d2c/f3c 0 2024-03-16T10:45:01.358 INFO:tasks.workunit.client.0.smithi002.stdout:4/505: symlink d2/d6/l9e 0 2024-03-16T10:45:01.360 INFO:tasks.workunit.client.0.smithi002.stdout:5/473: getdents d0/d16/d1d/d2b 0 2024-03-16T10:45:01.363 INFO:tasks.workunit.client.0.smithi002.stdout:8/641: creat d7/d9/d17/d23/d39/fdc x:0 0 0 2024-03-16T10:45:01.364 INFO:tasks.workunit.client.0.smithi002.stdout:8/642: chown d7/d9/d25/d91/cc6 4606231 1 2024-03-16T10:45:01.365 INFO:tasks.workunit.client.0.smithi002.stdout:9/556: rename d5/d9/d55/f63 to d5/d9/dc/d28/d90/fa7 0 2024-03-16T10:45:01.367 INFO:tasks.workunit.client.0.smithi002.stdout:6/537: creat d15/d2d/d2c/d8f/fb8 x:0 0 0 2024-03-16T10:45:01.371 INFO:tasks.workunit.client.0.smithi002.stdout:8/643: fdatasync d7/d9/d17/d20/f5e 0 2024-03-16T10:45:01.374 INFO:tasks.workunit.client.0.smithi002.stdout:8/644: write d7/d9/d25/d91/fbe [869405,78806] 0 2024-03-16T10:45:01.375 INFO:tasks.workunit.client.0.smithi002.stdout:5/474: dread d0/f6f [0,4194304] 0 2024-03-16T10:45:01.375 INFO:tasks.workunit.client.0.smithi002.stdout:9/557: creat d5/d9/dc/d28/fa8 x:0 0 0 2024-03-16T10:45:01.375 INFO:tasks.workunit.client.0.smithi002.stdout:8/645: stat d7/d9/d17/f3d 0 2024-03-16T10:45:01.378 INFO:tasks.workunit.client.0.smithi002.stdout:6/538: symlink d15/d2d/d2c/d5d/lb9 0 2024-03-16T10:45:01.382 INFO:tasks.workunit.client.0.smithi002.stdout:5/475: dread d0/d1f/f2c [0,4194304] 0 2024-03-16T10:45:01.384 INFO:tasks.workunit.client.0.smithi002.stdout:8/646: mknod d7/d9/d17/d23/d2e/d34/cdd 0 2024-03-16T10:45:01.385 INFO:tasks.workunit.client.0.smithi002.stdout:6/539: creat d15/d2d/d73/d97/fba x:0 0 0 2024-03-16T10:45:01.386 INFO:tasks.workunit.client.0.smithi002.stdout:6/540: write d15/dab/fb6 [523902,54558] 0 2024-03-16T10:45:01.386 INFO:tasks.workunit.client.0.smithi002.stdout:9/558: mkdir d5/d9/d25/d23/d81/da9 0 2024-03-16T10:45:01.388 INFO:tasks.workunit.client.0.smithi002.stdout:7/488: sync 2024-03-16T10:45:01.388 INFO:tasks.workunit.client.0.smithi002.stdout:4/506: getdents d2/d33/da 0 2024-03-16T10:45:01.390 INFO:tasks.workunit.client.0.smithi002.stdout:4/507: chown d2/d6/d32/d68/c76 1031421 1 2024-03-16T10:45:01.390 INFO:tasks.workunit.client.0.smithi002.stdout:5/476: rename d0/d16/f63 to d0/d16/d1d/d43/d48/f8e 0 2024-03-16T10:45:01.391 INFO:tasks.workunit.client.0.smithi002.stdout:8/647: readlink d7/d9/d17/d20/ld0 0 2024-03-16T10:45:01.394 INFO:tasks.workunit.client.0.smithi002.stdout:9/559: mknod d5/d9/d22/caa 0 2024-03-16T10:45:01.394 INFO:tasks.workunit.client.0.smithi002.stdout:0/568: dwrite d3/d31/d75/f98 [0,4194304] 0 2024-03-16T10:45:01.396 INFO:tasks.workunit.client.0.smithi002.stdout:7/489: creat d3/d4/d1c/d5b/d64/d99/fb0 x:0 0 0 2024-03-16T10:45:01.441 INFO:tasks.workunit.client.0.smithi002.stdout:4/508: dwrite d2/d6/f2d [0,4194304] 0 2024-03-16T10:45:01.443 INFO:tasks.workunit.client.0.smithi002.stdout:6/541: write d15/f18 [50020,4874] 0 2024-03-16T10:45:01.444 INFO:tasks.workunit.client.0.smithi002.stdout:9/560: write d5/d9/d25/d23/f56 [1039141,125118] 0 2024-03-16T10:45:01.445 INFO:tasks.workunit.client.0.smithi002.stdout:0/569: symlink d3/de/da2/lb2 0 2024-03-16T10:45:01.447 INFO:tasks.workunit.client.0.smithi002.stdout:5/477: getdents d0/d16/d1d/d43 0 2024-03-16T10:45:01.449 INFO:tasks.workunit.client.0.smithi002.stdout:9/561: fsync d5/d9/d25/d23/f48 0 2024-03-16T10:45:01.449 INFO:tasks.workunit.client.0.smithi002.stdout:7/490: dwrite d3/d61/f83 [0,4194304] 0 2024-03-16T10:45:01.451 INFO:tasks.workunit.client.0.smithi002.stdout:0/570: mknod d3/d31/d88/cb3 0 2024-03-16T10:45:01.451 INFO:tasks.workunit.client.0.smithi002.stdout:9/562: dread - d5/d9/d34/f68 zero size 2024-03-16T10:45:01.453 INFO:tasks.workunit.client.0.smithi002.stdout:5/478: creat d0/d16/d1d/d43/d48/f8f x:0 0 0 2024-03-16T10:45:01.455 INFO:tasks.workunit.client.0.smithi002.stdout:0/571: write d3/d27/d90/f9e [136227,73360] 0 2024-03-16T10:45:01.456 INFO:tasks.workunit.client.0.smithi002.stdout:6/542: dwrite d15/f5f [0,4194304] 0 2024-03-16T10:45:01.460 INFO:tasks.workunit.client.0.smithi002.stdout:0/572: mkdir d3/d31/d75/db4 0 2024-03-16T10:45:01.461 INFO:tasks.workunit.client.0.smithi002.stdout:0/573: stat d3/d28/d2a/f4f 0 2024-03-16T10:45:01.463 INFO:tasks.workunit.client.0.smithi002.stdout:4/509: link d2/d33/f3e d2/d6/d32/d66/d7f/f9f 0 2024-03-16T10:45:01.471 INFO:tasks.workunit.client.0.smithi002.stdout:6/543: dwrite d15/d2d/d8a/d7c/d87/fa4 [0,4194304] 0 2024-03-16T10:45:01.472 INFO:tasks.workunit.client.0.smithi002.stdout:4/510: rename d2/d33/d1c/l4c to d2/d33/la0 0 2024-03-16T10:45:01.475 INFO:tasks.workunit.client.0.smithi002.stdout:0/574: dwrite d3/d5/f18 [0,4194304] 0 2024-03-16T10:45:01.479 INFO:tasks.workunit.client.0.smithi002.stdout:5/479: dread d0/d16/d1d/d33/d65/f7a [0,4194304] 0 2024-03-16T10:45:01.480 INFO:tasks.workunit.client.0.smithi002.stdout:3/579: sync 2024-03-16T10:45:01.480 INFO:tasks.workunit.client.0.smithi002.stdout:2/567: sync 2024-03-16T10:45:01.480 INFO:tasks.workunit.client.0.smithi002.stdout:1/571: sync 2024-03-16T10:45:01.481 INFO:tasks.workunit.client.0.smithi002.stdout:7/491: dread d3/d4/d1d/d87/d5c/d63/d3f/f41 [0,4194304] 0 2024-03-16T10:45:01.482 INFO:tasks.workunit.client.0.smithi002.stdout:4/511: dwrite d2/d33/da/f74 [0,4194304] 0 2024-03-16T10:45:01.485 INFO:tasks.workunit.client.0.smithi002.stdout:3/580: write d1/dd/d28/d7b/d2c/d97/fa4 [697216,126204] 0 2024-03-16T10:45:01.485 INFO:tasks.workunit.client.0.smithi002.stdout:0/575: mknod d3/d28/d2a/cb5 0 2024-03-16T10:45:01.492 INFO:tasks.workunit.client.0.smithi002.stdout:2/568: rename d7/d2e/d1d/f41 to d7/d2e/d1d/d2f/d4b/d6e/fab 0 2024-03-16T10:45:01.494 INFO:tasks.workunit.client.0.smithi002.stdout:6/544: getdents d15/d2d/d2b/d3f/d77 0 2024-03-16T10:45:01.496 INFO:tasks.workunit.client.0.smithi002.stdout:3/581: stat d1/db6/d15/fa7 0 2024-03-16T10:45:01.498 INFO:tasks.workunit.client.0.smithi002.stdout:7/492: dwrite d3/d4/d1c/f32 [0,4194304] 0 2024-03-16T10:45:01.500 INFO:tasks.workunit.client.0.smithi002.stdout:0/576: symlink d3/de/da2/lb6 0 2024-03-16T10:45:01.501 INFO:tasks.workunit.client.0.smithi002.stdout:4/512: dwrite d2/d33/da/f5f [0,4194304] 0 2024-03-16T10:45:01.507 INFO:tasks.workunit.client.0.smithi002.stdout:2/569: write d7/d2e/d1d/d2f/d4b/d6e/fab [1622540,81063] 0 2024-03-16T10:45:01.507 INFO:tasks.workunit.client.0.smithi002.stdout:6/545: mkdir d15/d2d/d2c/d8f/d96/dbb 0 2024-03-16T10:45:01.507 INFO:tasks.workunit.client.0.smithi002.stdout:4/513: chown d2/d6/f53 0 1 2024-03-16T10:45:01.508 INFO:tasks.workunit.client.0.smithi002.stdout:0/577: write d3/d5/f30 [1024091,24255] 0 2024-03-16T10:45:01.508 INFO:tasks.workunit.client.0.smithi002.stdout:4/514: chown d2/d33/d1c/l9c 85 1 2024-03-16T10:45:01.509 INFO:tasks.workunit.client.0.smithi002.stdout:9/563: dwrite d5/d9/d55/f7b [0,4194304] 0 2024-03-16T10:45:01.510 INFO:tasks.workunit.client.0.smithi002.stdout:3/582: creat d1/dd/d28/d7b/d25/fc9 x:0 0 0 2024-03-16T10:45:01.510 INFO:tasks.workunit.client.0.smithi002.stdout:2/570: stat d7/db/f58 0 2024-03-16T10:45:01.511 INFO:tasks.workunit.client.0.smithi002.stdout:2/571: readlink d7/l43 0 2024-03-16T10:45:01.515 INFO:tasks.workunit.client.0.smithi002.stdout:2/572: fsync d7/d2e/d1d/d2f/d4b/fa0 0 2024-03-16T10:45:01.517 INFO:tasks.workunit.client.0.smithi002.stdout:3/583: truncate d1/da5/fc3 341308 0 2024-03-16T10:45:01.517 INFO:tasks.workunit.client.0.smithi002.stdout:7/493: rename d3/d4/d1d/d87/d5c/d63/c3e to d3/d61/d66/cb1 0 2024-03-16T10:45:01.518 INFO:tasks.workunit.client.0.smithi002.stdout:1/572: dread d2/df/d12/d74/f90 [4194304,4194304] 0 2024-03-16T10:45:01.522 INFO:tasks.workunit.client.0.smithi002.stdout:6/546: mkdir d15/d2d/d2c/d8f/d96/dbc 0 2024-03-16T10:45:01.523 INFO:tasks.workunit.client.0.smithi002.stdout:6/547: read - d15/d2d/d2c/d5d/f6f zero size 2024-03-16T10:45:01.527 INFO:tasks.workunit.client.0.smithi002.stdout:3/584: mknod d1/dd/d28/d7b/d2c/d97/cca 0 2024-03-16T10:45:01.528 INFO:tasks.workunit.client.0.smithi002.stdout:3/585: write d1/dd/d28/d7b/d8a/d93/f91 [1785712,3590] 0 2024-03-16T10:45:01.530 INFO:tasks.workunit.client.0.smithi002.stdout:7/494: rmdir d3/d4/d1d/d87/d5c/d63/d3f 39 2024-03-16T10:45:01.532 INFO:tasks.workunit.client.0.smithi002.stdout:1/573: mknod d2/d3/dae/cb4 0 2024-03-16T10:45:01.533 INFO:tasks.workunit.client.0.smithi002.stdout:0/578: write d3/d28/f36 [710963,12205] 0 2024-03-16T10:45:01.541 INFO:tasks.workunit.client.0.smithi002.stdout:7/495: symlink d3/d4/d1d/d45/lb2 0 2024-03-16T10:45:01.541 INFO:tasks.workunit.client.0.smithi002.stdout:4/515: truncate d2/d33/d35/f40 301179 0 2024-03-16T10:45:01.542 INFO:tasks.workunit.client.0.smithi002.stdout:7/496: read d3/d4/d1d/d87/d96/d8c/f95 [371713,111113] 0 2024-03-16T10:45:01.543 INFO:tasks.workunit.client.0.smithi002.stdout:9/564: truncate d5/d9/d25/d23/f33 81374 0 2024-03-16T10:45:01.544 INFO:tasks.workunit.client.0.smithi002.stdout:9/565: stat d5/d9/d25/c96 0 2024-03-16T10:45:01.545 INFO:tasks.workunit.client.0.smithi002.stdout:2/573: truncate d7/f90 2133017 0 2024-03-16T10:45:01.546 INFO:tasks.workunit.client.0.smithi002.stdout:3/586: fsync d1/db6/d15/f1b 0 2024-03-16T10:45:01.549 INFO:tasks.workunit.client.0.smithi002.stdout:1/574: write d2/df/d12/f4e [575567,73360] 0 2024-03-16T10:45:01.549 INFO:tasks.workunit.client.0.smithi002.stdout:4/516: symlink d2/d33/da/la1 0 2024-03-16T10:45:01.550 INFO:tasks.workunit.client.0.smithi002.stdout:1/575: fsync d2/df/d12/d29/f34 0 2024-03-16T10:45:01.551 INFO:tasks.workunit.client.0.smithi002.stdout:4/517: write d2/d33/d1c/d45/f91 [824113,120957] 0 2024-03-16T10:45:01.555 INFO:tasks.workunit.client.0.smithi002.stdout:1/576: dread d2/d3/f39 [0,4194304] 0 2024-03-16T10:45:01.557 INFO:tasks.workunit.client.0.smithi002.stdout:2/574: rename d7/d27/f9d to d7/d27/d3f/fac 0 2024-03-16T10:45:01.561 INFO:tasks.workunit.client.0.smithi002.stdout:3/587: dread d1/d5a/f75 [0,4194304] 0 2024-03-16T10:45:01.562 INFO:tasks.workunit.client.0.smithi002.stdout:3/588: dread d1/da5/fc3 [0,4194304] 0 2024-03-16T10:45:01.568 INFO:tasks.workunit.client.0.smithi002.stdout:1/577: rename d2/d3 to d2/df/d12/d84/d87/d96/db5 0 2024-03-16T10:45:01.573 INFO:tasks.workunit.client.0.smithi002.stdout:7/497: write d3/d4/d1d/d45/fa1 [1936002,108839] 0 2024-03-16T10:45:01.574 INFO:tasks.workunit.client.0.smithi002.stdout:0/579: getdents d3/d31/d88 0 2024-03-16T10:45:01.574 INFO:tasks.workunit.client.0.smithi002.stdout:9/566: write d5/d9/dc/d28/f4c [403735,4318] 0 2024-03-16T10:45:01.577 INFO:tasks.workunit.client.0.smithi002.stdout:0/580: write d3/d28/d3d/f74 [1257070,35752] 0 2024-03-16T10:45:01.578 INFO:tasks.workunit.client.0.smithi002.stdout:1/578: unlink d2/df/d12/d84/d87/d96/db5/dae/cb4 0 2024-03-16T10:45:01.585 INFO:tasks.workunit.client.0.smithi002.stdout:0/581: mknod d3/d27/cb7 0 2024-03-16T10:45:01.587 INFO:tasks.workunit.client.0.smithi002.stdout:0/582: read d3/d31/d75/f8c [4057859,52859] 0 2024-03-16T10:45:01.587 INFO:tasks.workunit.client.0.smithi002.stdout:4/518: dwrite d2/d33/d35/d59/f60 [0,4194304] 0 2024-03-16T10:45:01.588 INFO:tasks.workunit.client.0.smithi002.stdout:1/579: mknod d2/df/d12/d84/d87/d96/db5/d1c/cb6 0 2024-03-16T10:45:01.589 INFO:tasks.workunit.client.0.smithi002.stdout:1/580: dread - d2/df/d12/d84/d87/d96/db5/fab zero size 2024-03-16T10:45:01.591 INFO:tasks.workunit.client.0.smithi002.stdout:7/498: rename d3/d4/d1d/d87/d5c/d63/d34/l97 to d3/d4/d1d/d87/d5c/d63/d3f/daa/lb3 0 2024-03-16T10:45:01.591 INFO:tasks.workunit.client.0.smithi002.stdout:9/567: link d5/d9/d22/d3c/c3e d5/d9/dc/cab 0 2024-03-16T10:45:01.592 INFO:tasks.workunit.client.0.smithi002.stdout:9/568: write d5/f53 [3289180,14960] 0 2024-03-16T10:45:01.594 INFO:tasks.workunit.client.0.smithi002.stdout:1/581: truncate d2/df/d12/d84/d87/d96/db5/d1c/d5d/f6b 1828695 0 2024-03-16T10:45:01.594 INFO:tasks.workunit.client.0.smithi002.stdout:9/569: write d5/d9/d25/f62 [2281203,104232] 0 2024-03-16T10:45:01.602 INFO:tasks.workunit.client.0.smithi002.stdout:7/499: rename d3/d4/d1c/f32 to d3/d61/d66/fb4 0 2024-03-16T10:45:01.605 INFO:tasks.workunit.client.0.smithi002.stdout:0/583: symlink d3/d28/d2a/da1/d9c/da9/lb8 0 2024-03-16T10:45:01.611 INFO:tasks.workunit.client.0.smithi002.stdout:4/519: dread d2/fe [0,4194304] 0 2024-03-16T10:45:01.611 INFO:tasks.workunit.client.0.smithi002.stdout:9/570: mkdir d5/d9/d25/dac 0 2024-03-16T10:45:01.615 INFO:tasks.workunit.client.0.smithi002.stdout:9/571: creat d5/d9/dc/d1e/d94/fad x:0 0 0 2024-03-16T10:45:01.616 INFO:tasks.workunit.client.0.smithi002.stdout:0/584: dread d3/d28/f29 [0,4194304] 0 2024-03-16T10:45:01.618 INFO:tasks.workunit.client.0.smithi002.stdout:4/520: rename d2/d6/f53 to d2/d33/da/d5e/d93/fa2 0 2024-03-16T10:45:01.619 INFO:tasks.workunit.client.0.smithi002.stdout:9/572: creat d5/d9/dc/d28/d8e/fae x:0 0 0 2024-03-16T10:45:01.624 INFO:tasks.workunit.client.0.smithi002.stdout:0/585: link d3/d28/d2a/f4f d3/d28/d2a/da1/d9c/fb9 0 2024-03-16T10:45:01.624 INFO:tasks.workunit.client.0.smithi002.stdout:0/586: read d3/f60 [143034,78521] 0 2024-03-16T10:45:01.625 INFO:tasks.workunit.client.0.smithi002.stdout:4/521: creat d2/fa3 x:0 0 0 2024-03-16T10:45:01.626 INFO:tasks.workunit.client.0.smithi002.stdout:9/573: creat d5/d9/faf x:0 0 0 2024-03-16T10:45:01.626 INFO:tasks.workunit.client.0.smithi002.stdout:4/522: dread - d2/d33/da/d5e/f9a zero size 2024-03-16T10:45:01.628 INFO:tasks.workunit.client.0.smithi002.stdout:0/587: mknod d3/d59/cba 0 2024-03-16T10:45:01.630 INFO:tasks.workunit.client.0.smithi002.stdout:0/588: fdatasync d3/de/f5c 0 2024-03-16T10:45:01.634 INFO:tasks.workunit.client.0.smithi002.stdout:4/523: rename d2/d33/d35/d59/f60 to d2/d6/d32/d66/fa4 0 2024-03-16T10:45:01.637 INFO:tasks.workunit.client.0.smithi002.stdout:9/574: rename d5/d9/d25/d23/d81/da9 to d5/d9/d22/d3c/d4d/d7f/db0 0 2024-03-16T10:45:01.638 INFO:tasks.workunit.client.0.smithi002.stdout:9/575: fdatasync d5/d9/d22/f47 0 2024-03-16T10:45:01.639 INFO:tasks.workunit.client.0.smithi002.stdout:0/589: dread d3/fc [0,4194304] 0 2024-03-16T10:45:01.640 INFO:tasks.workunit.client.0.smithi002.stdout:4/524: mknod d2/d6/d32/ca5 0 2024-03-16T10:45:01.645 INFO:tasks.workunit.client.0.smithi002.stdout:0/590: mkdir d3/d27/d90/dbb 0 2024-03-16T10:45:01.645 INFO:tasks.workunit.client.0.smithi002.stdout:4/525: rmdir d2/d33/d35/d59 39 2024-03-16T10:45:01.648 INFO:tasks.workunit.client.0.smithi002.stdout:9/576: unlink d5/d9/dc/d28/c7d 0 2024-03-16T10:45:01.649 INFO:tasks.workunit.client.0.smithi002.stdout:0/591: dread - d3/d5/f1c zero size 2024-03-16T10:45:01.649 INFO:tasks.workunit.client.0.smithi002.stdout:9/577: read d5/f7 [2929699,94514] 0 2024-03-16T10:45:01.652 INFO:tasks.workunit.client.0.smithi002.stdout:0/592: symlink d3/d27/d90/lbc 0 2024-03-16T10:45:01.657 INFO:tasks.workunit.client.0.smithi002.stdout:9/578: getdents d5/d9/d25/d23 0 2024-03-16T10:45:01.658 INFO:tasks.workunit.client.0.smithi002.stdout:9/579: rmdir d5/d9/dc/d1e/d91 39 2024-03-16T10:45:01.660 INFO:tasks.workunit.client.0.smithi002.stdout:4/526: dread d2/d33/da/ff [0,4194304] 0 2024-03-16T10:45:01.661 INFO:tasks.workunit.client.0.smithi002.stdout:4/527: readlink d2/d6/d32/l4d 0 2024-03-16T10:45:01.663 INFO:tasks.workunit.client.0.smithi002.stdout:4/528: chown d2/d33/da/d2e 180290407 1 2024-03-16T10:45:01.663 INFO:tasks.workunit.client.0.smithi002.stdout:9/580: dread d5/d9/dc/d28/f4c [0,4194304] 0 2024-03-16T10:45:01.664 INFO:tasks.workunit.client.0.smithi002.stdout:4/529: write d2/d6/f73 [186439,87121] 0 2024-03-16T10:45:01.669 INFO:tasks.workunit.client.0.smithi002.stdout:9/581: write d5/f7 [1971399,79236] 0 2024-03-16T10:45:01.669 INFO:tasks.workunit.client.0.smithi002.stdout:4/530: rmdir d2/d33/da 39 2024-03-16T10:45:01.670 INFO:tasks.workunit.client.0.smithi002.stdout:4/531: write d2/d33/d35/f63 [1914767,5173] 0 2024-03-16T10:45:01.672 INFO:tasks.workunit.client.0.smithi002.stdout:9/582: mkdir d5/db1 0 2024-03-16T10:45:01.673 INFO:tasks.workunit.client.0.smithi002.stdout:9/583: write d5/d9/faf [1036783,31558] 0 2024-03-16T10:45:01.674 INFO:tasks.workunit.client.0.smithi002.stdout:9/584: readlink d5/l12 0 2024-03-16T10:45:01.676 INFO:tasks.workunit.client.0.smithi002.stdout:9/585: mknod d5/d9/dc/cb2 0 2024-03-16T10:45:01.677 INFO:tasks.workunit.client.0.smithi002.stdout:0/593: write d3/d28/d65/f7a [1055259,108824] 0 2024-03-16T10:45:01.679 INFO:tasks.workunit.client.0.smithi002.stdout:9/586: mkdir d5/d9/dc/d1e/d24/d31/db3 0 2024-03-16T10:45:01.680 INFO:tasks.workunit.client.0.smithi002.stdout:9/587: write d5/d9/d25/f62 [556285,75724] 0 2024-03-16T10:45:01.683 INFO:tasks.workunit.client.0.smithi002.stdout:0/594: symlink d3/d28/d2a/lbd 0 2024-03-16T10:45:01.687 INFO:tasks.workunit.client.0.smithi002.stdout:0/595: rename d3/d31/d75/f98 to d3/d5/d7c/fbe 0 2024-03-16T10:45:01.688 INFO:tasks.workunit.client.0.smithi002.stdout:0/596: write d3/d31/daa/fad [1010090,101489] 0 2024-03-16T10:45:01.693 INFO:tasks.workunit.client.0.smithi002.stdout:0/597: mknod d3/d28/cbf 0 2024-03-16T10:45:01.693 INFO:tasks.workunit.client.0.smithi002.stdout:4/532: dwrite d2/fe [0,4194304] 0 2024-03-16T10:45:01.700 INFO:tasks.workunit.client.1.smithi026.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-16T10:45:01.702 INFO:tasks.workunit.client.0.smithi002.stdout:5/480: sync 2024-03-16T10:45:01.702 INFO:tasks.workunit.client.0.smithi002.stdout:8/648: sync 2024-03-16T10:45:01.704 INFO:tasks.workunit.client.0.smithi002.stdout:5/481: fdatasync d0/d16/d1d/d2b/d5d/f5a 0 2024-03-16T10:45:01.705 INFO:tasks.workunit.client.0.smithi002.stdout:9/588: dwrite d5/d9/dc/d1e/d24/d31/f4a [0,4194304] 0 2024-03-16T10:45:01.707 INFO:tasks.workunit.client.1.smithi026.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-16T10:45:01.707 INFO:tasks.workunit.client.1.smithi026.stderr:+ make 2024-03-16T10:45:01.712 INFO:tasks.workunit.client.0.smithi002.stdout:5/482: rmdir d0/d24 39 2024-03-16T10:45:01.713 INFO:tasks.workunit.client.0.smithi002.stdout:8/649: dwrite d7/f60 [0,4194304] 0 2024-03-16T10:45:01.716 INFO:tasks.workunit.client.0.smithi002.stdout:4/533: dread d2/d33/d1c/f67 [0,4194304] 0 2024-03-16T10:45:01.716 INFO:tasks.workunit.client.0.smithi002.stdout:9/589: stat d5/d9/dc/d1e/d91/d83/c9f 0 2024-03-16T10:45:01.718 INFO:tasks.workunit.client.0.smithi002.stdout:5/483: getdents d0/d16/d8d 0 2024-03-16T10:45:01.720 INFO:tasks.workunit.client.0.smithi002.stdout:8/650: rmdir d7/d9/d17/d23/d39 39 2024-03-16T10:45:01.720 INFO:tasks.workunit.client.0.smithi002.stdout:0/598: dread d3/d31/d75/f8c [0,4194304] 0 2024-03-16T10:45:01.720 INFO:tasks.workunit.client.0.smithi002.stdout:8/651: fdatasync d7/f21 0 2024-03-16T10:45:01.721 INFO:tasks.workunit.client.0.smithi002.stdout:9/590: mkdir d5/d9/dc/d1e/d91/d82/db4 0 2024-03-16T10:45:01.722 INFO:tasks.workunit.client.0.smithi002.stdout:9/591: read - d5/f6f zero size 2024-03-16T10:45:01.725 INFO:tasks.workunit.client.0.smithi002.stdout:5/484: creat d0/d16/d4a/d7d/f90 x:0 0 0 2024-03-16T10:45:01.728 INFO:tasks.workunit.client.0.smithi002.stdout:8/652: mkdir d7/d9/d17/d23/d2e/d34/dde 0 2024-03-16T10:45:01.729 INFO:tasks.workunit.client.0.smithi002.stdout:9/592: readlink d5/d9/d25/d23/l80 0 2024-03-16T10:45:01.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:01 smithi026 ceph-mon[37041]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:01.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:01 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3083509888' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:45:01.732 INFO:tasks.workunit.client.0.smithi002.stdout:0/599: dwrite d3/d28/d3d/f46 [0,4194304] 0 2024-03-16T10:45:01.734 INFO:tasks.workunit.client.0.smithi002.stdout:5/485: creat d0/d16/d1d/f91 x:0 0 0 2024-03-16T10:45:01.734 INFO:tasks.workunit.client.0.smithi002.stdout:9/593: mkdir d5/d9/dc/d28/d8e/db5 0 2024-03-16T10:45:01.735 INFO:tasks.workunit.client.0.smithi002.stdout:0/600: creat d3/d28/d2a/da1/d9c/da9/fc0 x:0 0 0 2024-03-16T10:45:01.736 INFO:tasks.workunit.client.0.smithi002.stdout:8/653: dwrite d7/d9/d25/d91/fbe [0,4194304] 0 2024-03-16T10:45:01.742 INFO:tasks.workunit.client.0.smithi002.stdout:9/594: dwrite d5/fd [4194304,4194304] 0 2024-03-16T10:45:01.743 INFO:tasks.workunit.client.0.smithi002.stdout:5/486: creat d0/d16/d1d/d2b/d5d/f92 x:0 0 0 2024-03-16T10:45:01.748 INFO:tasks.workunit.client.0.smithi002.stdout:0/601: symlink d3/d31/d75/db4/lc1 0 2024-03-16T10:45:01.748 INFO:tasks.workunit.client.0.smithi002.stdout:9/595: dread d5/d9/dc/d28/f65 [0,4194304] 0 2024-03-16T10:45:01.748 INFO:tasks.workunit.client.0.smithi002.stdout:4/534: truncate d2/d33/da/f41 1372919 0 2024-03-16T10:45:01.750 INFO:tasks.workunit.client.0.smithi002.stdout:5/487: rename d0/d16/d1d/f91 to d0/d16/d4a/d39/f93 0 2024-03-16T10:45:01.756 INFO:tasks.workunit.client.0.smithi002.stdout:6/548: sync 2024-03-16T10:45:01.756 INFO:tasks.workunit.client.0.smithi002.stdout:0/602: dwrite d3/d31/f99 [0,4194304] 0 2024-03-16T10:45:01.758 INFO:tasks.workunit.client.0.smithi002.stdout:4/535: creat d2/d6/d32/fa6 x:0 0 0 2024-03-16T10:45:01.759 INFO:tasks.workunit.client.0.smithi002.stdout:0/603: stat d3/d31/l5a 0 2024-03-16T10:45:01.760 INFO:tasks.workunit.client.0.smithi002.stdout:8/654: dwrite d7/d9/f58 [0,4194304] 0 2024-03-16T10:45:01.760 INFO:tasks.workunit.client.0.smithi002.stdout:5/488: rmdir d0/d16/d8d 0 2024-03-16T10:45:01.764 INFO:tasks.workunit.client.1.smithi026.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-16T10:45:01.767 INFO:tasks.workunit.client.0.smithi002.stdout:6/549: creat d15/d2d/fbd x:0 0 0 2024-03-16T10:45:01.767 INFO:tasks.workunit.client.0.smithi002.stdout:9/596: dread d5/d9/dc/d1e/d24/d31/f32 [0,4194304] 0 2024-03-16T10:45:01.770 INFO:tasks.workunit.client.0.smithi002.stdout:5/489: rename d0/d16/d1d/d33/d65 to d0/d16/d1d/d2b/d94 0 2024-03-16T10:45:01.772 INFO:tasks.workunit.client.0.smithi002.stdout:8/655: dwrite d7/d9/d17/d23/d2e/d5c/f8a [4194304,4194304] 0 2024-03-16T10:45:01.775 INFO:tasks.workunit.client.0.smithi002.stdout:0/604: link d3/d31/d88/f8b d3/d27/fc2 0 2024-03-16T10:45:01.776 INFO:tasks.workunit.client.0.smithi002.stdout:6/550: mkdir d15/d80/dbe 0 2024-03-16T10:45:01.778 INFO:tasks.workunit.client.0.smithi002.stdout:4/536: link d2/d33/d35/f63 d2/d33/da/d5e/d93/fa7 0 2024-03-16T10:45:01.779 INFO:tasks.workunit.client.0.smithi002.stdout:9/597: dwrite d5/d7e/f8f [0,4194304] 0 2024-03-16T10:45:01.779 INFO:tasks.workunit.client.0.smithi002.stdout:6/551: stat d15/d2d/d2b/d3f/f6d 0 2024-03-16T10:45:01.781 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:01 smithi002 ceph-mon[29248]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:01.781 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:01 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3083509888' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:45:01.785 INFO:tasks.workunit.client.0.smithi002.stdout:5/490: creat d0/d16/d1d/d33/d8c/f95 x:0 0 0 2024-03-16T10:45:01.789 INFO:tasks.workunit.client.0.smithi002.stdout:5/491: dread - d0/d16/d4a/d39/f93 zero size 2024-03-16T10:45:01.789 INFO:tasks.workunit.client.0.smithi002.stdout:4/537: dwrite d2/d33/da/d2e/f8e [0,4194304] 0 2024-03-16T10:45:01.790 INFO:tasks.workunit.client.0.smithi002.stdout:8/656: dread d7/f8 [0,4194304] 0 2024-03-16T10:45:01.791 INFO:tasks.workunit.client.0.smithi002.stdout:6/552: creat d15/d2d/d73/d97/fbf x:0 0 0 2024-03-16T10:45:01.791 INFO:tasks.workunit.client.0.smithi002.stdout:4/538: read - d2/d6/f89 zero size 2024-03-16T10:45:01.794 INFO:tasks.workunit.client.0.smithi002.stdout:4/539: write d2/d33/da/d5e/f75 [1334041,70578] 0 2024-03-16T10:45:01.795 INFO:tasks.workunit.client.0.smithi002.stdout:8/657: truncate d7/d9/d25/d67/fc0 504850 0 2024-03-16T10:45:01.796 INFO:tasks.workunit.client.0.smithi002.stdout:9/598: dread d5/f7 [0,4194304] 0 2024-03-16T10:45:01.800 INFO:tasks.workunit.client.0.smithi002.stdout:5/492: symlink d0/d16/d1d/d33/d8c/l96 0 2024-03-16T10:45:01.808 INFO:tasks.workunit.client.0.smithi002.stdout:4/540: read d2/d6/f10 [3366667,11274] 0 2024-03-16T10:45:01.810 INFO:tasks.workunit.client.0.smithi002.stdout:8/658: creat d7/d9/d25/fdf x:0 0 0 2024-03-16T10:45:01.812 INFO:tasks.workunit.client.0.smithi002.stdout:9/599: readlink d5/d9/dc/d1e/d24/d31/l5b 0 2024-03-16T10:45:01.817 INFO:tasks.workunit.client.0.smithi002.stdout:5/493: write d0/d16/d4a/d39/f93 [481810,10852] 0 2024-03-16T10:45:01.819 INFO:tasks.workunit.client.0.smithi002.stdout:4/541: rmdir d2/d33/d35 39 2024-03-16T10:45:01.821 INFO:tasks.workunit.client.0.smithi002.stdout:6/553: dwrite d15/d2d/d2c/f46 [4194304,4194304] 0 2024-03-16T10:45:01.821 INFO:tasks.workunit.client.0.smithi002.stdout:8/659: fdatasync d7/d9/d17/d23/d2e/d3a/fac 0 2024-03-16T10:45:01.822 INFO:tasks.workunit.client.0.smithi002.stdout:6/554: stat d15/d2d/d73/fa6 0 2024-03-16T10:45:01.824 INFO:tasks.workunit.client.0.smithi002.stdout:9/600: mknod d5/d9/dc/d1e/d94/cb6 0 2024-03-16T10:45:01.825 INFO:tasks.workunit.client.0.smithi002.stdout:9/601: write d5/d9/dc/d28/d8e/fae [974714,88528] 0 2024-03-16T10:45:01.825 INFO:tasks.workunit.client.0.smithi002.stdout:5/494: mknod d0/d16/d4a/d39/c97 0 2024-03-16T10:45:01.828 INFO:tasks.workunit.client.0.smithi002.stdout:4/542: unlink d2/d33/f29 0 2024-03-16T10:45:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:3/589: sync 2024-03-16T10:45:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:2/575: sync 2024-03-16T10:45:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:1/582: sync 2024-03-16T10:45:01.847 INFO:tasks.workunit.client.0.smithi002.stdout:7/500: sync 2024-03-16T10:45:01.852 INFO:tasks.workunit.client.0.smithi002.stdout:6/555: creat d15/d2d/d2b/d3f/fc0 x:0 0 0 2024-03-16T10:45:01.856 INFO:tasks.workunit.client.0.smithi002.stdout:5/495: dwrite d0/d16/d4a/d7d/f90 [0,4194304] 0 2024-03-16T10:45:01.857 INFO:tasks.workunit.client.0.smithi002.stdout:7/501: dread d3/d4/d1c/f2b [0,4194304] 0 2024-03-16T10:45:01.858 INFO:tasks.workunit.client.0.smithi002.stdout:3/590: rmdir d1/db6 39 2024-03-16T10:45:01.865 INFO:tasks.workunit.client.0.smithi002.stdout:6/556: dread d15/f1a [0,4194304] 0 2024-03-16T10:45:01.865 INFO:tasks.workunit.client.0.smithi002.stdout:4/543: symlink d2/d33/da/d88/la8 0 2024-03-16T10:45:01.867 INFO:tasks.workunit.client.0.smithi002.stdout:2/576: dwrite d7/d2e/d1d/d2f/f7d [0,4194304] 0 2024-03-16T10:45:01.871 INFO:tasks.workunit.client.0.smithi002.stdout:5/496: dwrite d0/d16/f37 [0,4194304] 0 2024-03-16T10:45:01.871 INFO:tasks.workunit.client.0.smithi002.stdout:1/583: dwrite d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f79 [0,4194304] 0 2024-03-16T10:45:01.871 INFO:tasks.workunit.client.0.smithi002.stdout:3/591: rmdir d1/da5 39 2024-03-16T10:45:01.881 INFO:tasks.workunit.client.0.smithi002.stdout:5/497: dwrite d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:45:01.889 INFO:tasks.workunit.client.0.smithi002.stdout:2/577: mknod d7/d2e/d28/d40/d50/cad 0 2024-03-16T10:45:01.890 INFO:tasks.workunit.client.0.smithi002.stdout:5/498: dwrite d0/d16/d4a/d39/f93 [0,4194304] 0 2024-03-16T10:45:01.891 INFO:tasks.workunit.client.0.smithi002.stdout:2/578: write d7/d2e/d1d/d2f/f59 [4696674,111709] 0 2024-03-16T10:45:01.891 INFO:tasks.workunit.client.0.smithi002.stdout:5/499: chown d0/f18 5067 1 2024-03-16T10:45:01.898 INFO:tasks.workunit.client.0.smithi002.stdout:9/602: truncate d5/d9/dc/d1e/d24/d31/f4a 173247 0 2024-03-16T10:45:01.899 INFO:tasks.workunit.client.0.smithi002.stdout:0/605: sync 2024-03-16T10:45:01.899 INFO:tasks.workunit.client.0.smithi002.stdout:0/606: dread - d3/d59/fb0 zero size 2024-03-16T10:45:01.900 INFO:tasks.workunit.client.0.smithi002.stdout:0/607: write d3/fae [929048,107203] 0 2024-03-16T10:45:01.902 INFO:tasks.workunit.client.0.smithi002.stdout:1/584: symlink d2/df/d12/d84/d87/d96/db5/dae/lb7 0 2024-03-16T10:45:01.905 INFO:tasks.workunit.client.0.smithi002.stdout:6/557: mkdir d15/d2d/d2b/dc1 0 2024-03-16T10:45:01.914 INFO:tasks.workunit.client.0.smithi002.stdout:0/608: rmdir d3/d28/d3d/d44 39 2024-03-16T10:45:01.915 INFO:tasks.workunit.client.0.smithi002.stdout:0/609: dread - d3/d5/f1c zero size 2024-03-16T10:45:01.916 INFO:tasks.workunit.client.0.smithi002.stdout:0/610: read - d3/da0/fb1 zero size 2024-03-16T10:45:01.918 INFO:tasks.workunit.client.0.smithi002.stdout:4/544: rename d2/d33/da/d5e/f8b to d2/d33/fa9 0 2024-03-16T10:45:01.919 INFO:tasks.workunit.client.0.smithi002.stdout:4/545: dread - d2/d33/da/d5e/f9a zero size 2024-03-16T10:45:01.921 INFO:tasks.workunit.client.0.smithi002.stdout:1/585: dwrite d2/df/d12/d74/f90 [0,4194304] 0 2024-03-16T10:45:01.922 INFO:tasks.workunit.client.0.smithi002.stdout:2/579: mknod d7/db/d70/d72/cae 0 2024-03-16T10:45:01.927 INFO:tasks.workunit.client.0.smithi002.stdout:7/502: getdents d3/d4/d1c/d5b 0 2024-03-16T10:45:01.933 INFO:tasks.workunit.client.0.smithi002.stdout:6/558: mknod d15/d2d/cc2 0 2024-03-16T10:45:01.936 INFO:tasks.workunit.client.0.smithi002.stdout:4/546: write d2/d33/da/f42 [3309425,68950] 0 2024-03-16T10:45:01.938 INFO:tasks.workunit.client.0.smithi002.stdout:4/547: dread - d2/d33/d1c/d45/f99 zero size 2024-03-16T10:45:01.940 INFO:tasks.workunit.client.0.smithi002.stdout:2/580: symlink d7/d2e/d28/d44/d6d/laf 0 2024-03-16T10:45:01.943 INFO:tasks.workunit.client.0.smithi002.stdout:3/592: creat d1/db6/d15/d87/fcb x:0 0 0 2024-03-16T10:45:01.946 INFO:tasks.workunit.client.0.smithi002.stdout:2/581: dwrite d7/d2e/d1d/d2f/d4b/d6e/fab [0,4194304] 0 2024-03-16T10:45:01.946 INFO:tasks.workunit.client.0.smithi002.stdout:7/503: rmdir d3/d4/d1d/d45 39 2024-03-16T10:45:01.948 INFO:tasks.workunit.client.0.smithi002.stdout:7/504: dread - d3/d4/d1d/d87/d5c/d63/fa2 zero size 2024-03-16T10:45:01.950 INFO:tasks.workunit.client.0.smithi002.stdout:9/603: rmdir d5/d9/d22/d3c/d4d/d7f/db0 0 2024-03-16T10:45:01.951 INFO:tasks.workunit.client.0.smithi002.stdout:6/559: rmdir d15/d2d/d2c/d8f/d96 39 2024-03-16T10:45:01.959 INFO:tasks.workunit.client.0.smithi002.stdout:6/560: dwrite d15/d2d/f34 [0,4194304] 0 2024-03-16T10:45:01.959 INFO:tasks.workunit.client.0.smithi002.stdout:1/586: symlink d2/df/d12/d84/d87/d96/db5/lb8 0 2024-03-16T10:45:01.961 INFO:tasks.workunit.client.0.smithi002.stdout:5/500: dwrite d0/f57 [0,4194304] 0 2024-03-16T10:45:01.963 INFO:tasks.workunit.client.0.smithi002.stdout:2/582: mknod d7/d2e/d1d/d2f/d4b/cb0 0 2024-03-16T10:45:01.977 INFO:tasks.workunit.client.0.smithi002.stdout:6/561: symlink d15/d2d/d2b/d3f/d77/lc3 0 2024-03-16T10:45:01.979 INFO:tasks.workunit.client.0.smithi002.stdout:1/587: creat d2/df/d12/d84/d87/d96/db5/d1c/d5d/fb9 x:0 0 0 2024-03-16T10:45:01.979 INFO:tasks.workunit.client.0.smithi002.stdout:4/548: write d2/d33/f6a [850485,83689] 0 2024-03-16T10:45:01.980 INFO:tasks.workunit.client.0.smithi002.stdout:5/501: rmdir d0/d16/d1d/d33 39 2024-03-16T10:45:01.980 INFO:tasks.workunit.client.0.smithi002.stdout:4/549: readlink d2/d6/d32/l4d 0 2024-03-16T10:45:01.983 INFO:tasks.workunit.client.0.smithi002.stdout:5/502: chown d0/d16/d1d 906202 1 2024-03-16T10:45:01.983 INFO:tasks.workunit.client.0.smithi002.stdout:2/583: write d7/d2e/f7e [4391757,96715] 0 2024-03-16T10:45:01.983 INFO:tasks.workunit.client.0.smithi002.stdout:9/604: truncate d5/fa0 1893271 0 2024-03-16T10:45:01.986 INFO:tasks.workunit.client.0.smithi002.stdout:6/562: write d15/f25 [7213189,73050] 0 2024-03-16T10:45:01.991 INFO:tasks.workunit.client.0.smithi002.stdout:4/550: truncate d2/f77 151201 0 2024-03-16T10:45:01.991 INFO:tasks.workunit.client.0.smithi002.stdout:9/605: symlink d5/d9/dc/d28/lb7 0 2024-03-16T10:45:01.992 INFO:tasks.workunit.client.0.smithi002.stdout:7/505: link d3/d4/d1d/d87/d5c/d63/c25 d3/d61/d66/d88/cb5 0 2024-03-16T10:45:01.994 INFO:tasks.workunit.client.0.smithi002.stdout:5/503: creat d0/d16/d1d/d33/d8c/f98 x:0 0 0 2024-03-16T10:45:01.995 INFO:tasks.workunit.client.0.smithi002.stdout:6/563: dwrite d15/d2d/fbd [0,4194304] 0 2024-03-16T10:45:01.996 INFO:tasks.workunit.client.0.smithi002.stdout:7/506: write d3/d4/d1d/d87/d5c/d63/fa2 [323330,38093] 0 2024-03-16T10:45:02.001 INFO:tasks.workunit.client.0.smithi002.stdout:6/564: mkdir d15/dab/dc4 0 2024-03-16T10:45:02.003 INFO:tasks.workunit.client.0.smithi002.stdout:2/584: write d7/d2e/d28/d40/f63 [1627461,59660] 0 2024-03-16T10:45:02.004 INFO:tasks.workunit.client.0.smithi002.stdout:1/588: dwrite d2/df/d12/d84/d87/d96/db5/d1c/f4a [0,4194304] 0 2024-03-16T10:45:02.006 INFO:tasks.workunit.client.0.smithi002.stdout:8/660: sync 2024-03-16T10:45:02.008 INFO:tasks.workunit.client.0.smithi002.stdout:2/585: write d7/d2e/d1d/d2f/f9a [1502248,37746] 0 2024-03-16T10:45:02.010 INFO:tasks.workunit.client.0.smithi002.stdout:9/606: write d5/d9/dc/d28/f65 [1699088,63768] 0 2024-03-16T10:45:02.010 INFO:tasks.workunit.client.0.smithi002.stdout:8/661: fsync d7/d9/d17/d23/d2e/f4e 0 2024-03-16T10:45:02.012 INFO:tasks.workunit.client.0.smithi002.stdout:4/551: dread d2/d33/da/d2e/f47 [0,4194304] 0 2024-03-16T10:45:02.013 INFO:tasks.workunit.client.0.smithi002.stdout:8/662: read - d7/d9/d25/fdf zero size 2024-03-16T10:45:02.014 INFO:tasks.workunit.client.0.smithi002.stdout:1/589: write d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/f95 [828713,74367] 0 2024-03-16T10:45:02.014 INFO:tasks.workunit.client.0.smithi002.stdout:7/507: write d3/d4/d1d/d45/f3a [1276635,44182] 0 2024-03-16T10:45:02.015 INFO:tasks.workunit.client.0.smithi002.stdout:8/663: fdatasync d7/d9/fb5 0 2024-03-16T10:45:02.015 INFO:tasks.workunit.client.0.smithi002.stdout:7/508: stat d3/f91 0 2024-03-16T10:45:02.016 INFO:tasks.workunit.client.0.smithi002.stdout:4/552: creat d2/d33/d1c/d45/faa x:0 0 0 2024-03-16T10:45:02.017 INFO:tasks.workunit.client.0.smithi002.stdout:8/664: stat d7/f8 0 2024-03-16T10:45:02.017 INFO:tasks.workunit.client.0.smithi002.stdout:9/607: symlink d5/lb8 0 2024-03-16T10:45:02.020 INFO:tasks.workunit.client.0.smithi002.stdout:7/509: creat d3/d61/d85/fb6 x:0 0 0 2024-03-16T10:45:02.022 INFO:tasks.workunit.client.0.smithi002.stdout:9/608: rmdir d5/d9/dc/d1e 39 2024-03-16T10:45:02.024 INFO:tasks.workunit.client.0.smithi002.stdout:6/565: dread d15/d2d/d8a/d7c/d87/fb0 [0,4194304] 0 2024-03-16T10:45:02.025 INFO:tasks.workunit.client.0.smithi002.stdout:7/510: mkdir d3/d4/d1d/d87/d96/d5e/db7 0 2024-03-16T10:45:02.026 INFO:tasks.workunit.client.0.smithi002.stdout:6/566: write f8 [2616725,47774] 0 2024-03-16T10:45:02.028 INFO:tasks.workunit.client.0.smithi002.stdout:1/590: write d2/f76 [1200904,27221] 0 2024-03-16T10:45:02.030 INFO:tasks.workunit.client.0.smithi002.stdout:4/553: dwrite d2/d33/d1c/d45/f99 [0,4194304] 0 2024-03-16T10:45:02.030 INFO:tasks.workunit.client.0.smithi002.stdout:2/586: dread d7/d2e/d1d/d2f/f9a [0,4194304] 0 2024-03-16T10:45:02.030 INFO:tasks.workunit.client.0.smithi002.stdout:9/609: rename d5/d9/d22/d3c/d4d/da4/la6 to d5/db1/lb9 0 2024-03-16T10:45:02.031 INFO:tasks.workunit.client.0.smithi002.stdout:8/665: getdents d7/d9/d17/d83 0 2024-03-16T10:45:02.033 INFO:tasks.workunit.client.0.smithi002.stdout:6/567: chown d15/d2d/l4c 0 1 2024-03-16T10:45:02.034 INFO:tasks.workunit.client.0.smithi002.stdout:4/554: creat d2/d33/da/d88/fab x:0 0 0 2024-03-16T10:45:02.037 INFO:tasks.workunit.client.0.smithi002.stdout:6/568: write d15/d2d/f1f [521968,31736] 0 2024-03-16T10:45:02.039 INFO:tasks.workunit.client.0.smithi002.stdout:1/591: unlink d2/df/d12/c62 0 2024-03-16T10:45:02.040 INFO:tasks.workunit.client.0.smithi002.stdout:8/666: mkdir d7/d13/db8/de0 0 2024-03-16T10:45:02.040 INFO:tasks.workunit.client.0.smithi002.stdout:1/592: chown d2/df/d12/d84/d87/d96/db5/dae 24802 1 2024-03-16T10:45:02.041 INFO:tasks.workunit.client.0.smithi002.stdout:4/555: write d2/d33/da/f28 [5181930,75203] 0 2024-03-16T10:45:02.042 INFO:tasks.workunit.client.0.smithi002.stdout:2/587: link d7/d2e/d28/d40/f93 d7/d2e/d28/d44/fb1 0 2024-03-16T10:45:02.044 INFO:tasks.workunit.client.0.smithi002.stdout:7/511: creat d3/d4/d1d/d87/d96/fb8 x:0 0 0 2024-03-16T10:45:02.045 INFO:tasks.workunit.client.0.smithi002.stdout:7/512: fdatasync d3/d4/d1d/d87/d96/d8c/fa4 0 2024-03-16T10:45:02.046 INFO:tasks.workunit.client.0.smithi002.stdout:1/593: stat d2/df/c6f 0 2024-03-16T10:45:02.046 INFO:tasks.workunit.client.0.smithi002.stdout:2/588: read d7/d2e/d28/d44/f76 [3344569,109999] 0 2024-03-16T10:45:02.046 INFO:tasks.workunit.client.0.smithi002.stdout:7/513: fdatasync d3/f15 0 2024-03-16T10:45:02.047 INFO:tasks.workunit.client.0.smithi002.stdout:2/589: fdatasync d7/d2e/d28/d40/f63 0 2024-03-16T10:45:02.049 INFO:tasks.workunit.client.0.smithi002.stdout:6/569: symlink d15/d2d/da8/lc5 0 2024-03-16T10:45:02.051 INFO:tasks.workunit.client.0.smithi002.stdout:6/570: stat d15/d2d/d2c/f46 0 2024-03-16T10:45:02.054 INFO:tasks.workunit.client.0.smithi002.stdout:9/610: rename d5/d9/d22/d3c/d4d/l58 to d5/d9/dc/d1e/d24/d31/lba 0 2024-03-16T10:45:02.054 INFO:tasks.workunit.client.0.smithi002.stdout:6/571: chown d15/d2d/d2c/d8f 19677323 1 2024-03-16T10:45:02.055 INFO:tasks.workunit.client.0.smithi002.stdout:2/590: symlink d7/d2e/d28/d40/d50/lb2 0 2024-03-16T10:45:02.058 INFO:tasks.workunit.client.0.smithi002.stdout:8/667: dwrite d7/d9/d17/d23/d39/fb4 [0,4194304] 0 2024-03-16T10:45:02.058 INFO:tasks.workunit.client.0.smithi002.stdout:4/556: dwrite d2/d33/d1c/f80 [0,4194304] 0 2024-03-16T10:45:02.059 INFO:tasks.workunit.client.0.smithi002.stdout:6/572: write d15/d2d/d2b/d3f/fc0 [969623,7677] 0 2024-03-16T10:45:02.062 INFO:tasks.workunit.client.0.smithi002.stdout:1/594: creat d2/df/d12/d84/d87/d96/db5/d1c/fba x:0 0 0 2024-03-16T10:45:02.066 INFO:tasks.workunit.client.0.smithi002.stdout:2/591: read d7/db/d12/f22 [495556,81399] 0 2024-03-16T10:45:02.068 INFO:tasks.workunit.client.0.smithi002.stdout:1/595: mkdir d2/d58/dbb 0 2024-03-16T10:45:02.068 INFO:tasks.workunit.client.0.smithi002.stdout:2/592: chown d7/d2e/d1d/d2f/c65 1793 1 2024-03-16T10:45:02.069 INFO:tasks.workunit.client.0.smithi002.stdout:6/573: read f11 [3631712,1770] 0 2024-03-16T10:45:02.071 INFO:tasks.workunit.client.0.smithi002.stdout:9/611: creat d5/d9/d25/fbb x:0 0 0 2024-03-16T10:45:02.072 INFO:tasks.workunit.client.0.smithi002.stdout:8/668: dread d7/d9/f51 [0,4194304] 0 2024-03-16T10:45:02.074 INFO:tasks.workunit.client.0.smithi002.stdout:7/514: dwrite d3/d4/d1c/f90 [0,4194304] 0 2024-03-16T10:45:02.075 INFO:tasks.workunit.client.0.smithi002.stdout:4/557: getdents d2/d6/d32/d68 0 2024-03-16T10:45:02.076 INFO:tasks.workunit.client.0.smithi002.stdout:6/574: creat d15/d2d/d2c/fc6 x:0 0 0 2024-03-16T10:45:02.077 INFO:tasks.workunit.client.0.smithi002.stdout:1/596: write d2/df/d12/f37 [6544569,48643] 0 2024-03-16T10:45:02.077 INFO:tasks.workunit.client.0.smithi002.stdout:9/612: truncate d5/d9/d25/d23/f56 2129689 0 2024-03-16T10:45:02.079 INFO:tasks.workunit.client.0.smithi002.stdout:8/669: creat d7/d9/d17/fe1 x:0 0 0 2024-03-16T10:45:02.085 INFO:tasks.workunit.client.0.smithi002.stdout:4/558: creat d2/d33/da/d5e/d93/fac x:0 0 0 2024-03-16T10:45:02.087 INFO:tasks.workunit.client.0.smithi002.stdout:8/670: symlink d7/d9/d17/d23/d2e/le2 0 2024-03-16T10:45:02.090 INFO:tasks.workunit.client.0.smithi002.stdout:0/611: sync 2024-03-16T10:45:02.092 INFO:tasks.workunit.client.0.smithi002.stdout:2/593: dwrite d7/d2e/f25 [4194304,4194304] 0 2024-03-16T10:45:02.093 INFO:tasks.workunit.client.0.smithi002.stdout:6/575: rename d15/d2d/d2b/d3f/d77/fa9 to d15/d2d/d2c/d8f/d96/db5/fc7 0 2024-03-16T10:45:02.095 INFO:tasks.workunit.client.0.smithi002.stdout:8/671: unlink d7/d13/fb1 0 2024-03-16T10:45:02.095 INFO:tasks.workunit.client.0.smithi002.stdout:4/559: rename d2/fe to d2/fad 0 2024-03-16T10:45:02.097 INFO:tasks.workunit.client.0.smithi002.stdout:0/612: unlink d3/d5/f18 0 2024-03-16T10:45:02.098 INFO:tasks.workunit.client.0.smithi002.stdout:0/613: fsync d3/fa5 0 2024-03-16T10:45:02.098 INFO:tasks.workunit.client.0.smithi002.stdout:4/560: mknod d2/d6/cae 0 2024-03-16T10:45:02.100 INFO:tasks.workunit.client.0.smithi002.stdout:7/515: dread d3/d4/d1d/d87/f92 [0,4194304] 0 2024-03-16T10:45:02.102 INFO:tasks.workunit.client.0.smithi002.stdout:6/576: mkdir d15/d2d/d2b/dc8 0 2024-03-16T10:45:02.102 INFO:tasks.workunit.client.0.smithi002.stdout:8/672: write d7/d9/d17/dc3/fc7 [353495,61718] 0 2024-03-16T10:45:02.104 INFO:tasks.workunit.client.0.smithi002.stdout:1/597: dread d2/f68 [0,4194304] 0 2024-03-16T10:45:02.105 INFO:tasks.workunit.client.0.smithi002.stdout:9/613: dread d5/f97 [0,4194304] 0 2024-03-16T10:45:02.107 INFO:tasks.workunit.client.0.smithi002.stdout:7/516: dread - d3/d4/d1d/d87/d5c/d63/d3f/f89 zero size 2024-03-16T10:45:02.111 INFO:tasks.workunit.client.0.smithi002.stdout:9/614: dread - d5/d9/dc/d1e/d94/fa3 zero size 2024-03-16T10:45:02.111 INFO:tasks.workunit.client.0.smithi002.stdout:7/517: mkdir d3/d4/d1d/d87/d96/d5e/db9 0 2024-03-16T10:45:02.112 INFO:tasks.workunit.client.0.smithi002.stdout:1/598: creat d2/d58/dbb/fbc x:0 0 0 2024-03-16T10:45:02.115 INFO:tasks.workunit.client.0.smithi002.stdout:6/577: write f11 [2153170,13929] 0 2024-03-16T10:45:02.119 INFO:tasks.workunit.client.0.smithi002.stdout:6/578: fsync d15/d2d/d73/fa6 0 2024-03-16T10:45:02.119 INFO:tasks.workunit.client.0.smithi002.stdout:0/614: dwrite d3/d28/d2a/d67/f89 [0,4194304] 0 2024-03-16T10:45:02.121 INFO:tasks.workunit.client.0.smithi002.stdout:1/599: stat d2/df/d12/d84/d87/d96/db5/f50 0 2024-03-16T10:45:02.121 INFO:tasks.workunit.client.0.smithi002.stdout:9/615: getdents d5/d9/dc/d28 0 2024-03-16T10:45:02.126 INFO:tasks.workunit.client.0.smithi002.stdout:0/615: chown d3/d28/l38 8867 1 2024-03-16T10:45:02.127 INFO:tasks.workunit.client.0.smithi002.stdout:6/579: creat d15/d2d/da8/fc9 x:0 0 0 2024-03-16T10:45:02.128 INFO:tasks.workunit.client.0.smithi002.stdout:8/673: dwrite d7/d9/d17/d23/d39/f5d [4194304,4194304] 0 2024-03-16T10:45:02.132 INFO:tasks.workunit.client.0.smithi002.stdout:4/561: dwrite d2/d6/d32/d66/d7f/f9f [0,4194304] 0 2024-03-16T10:45:02.136 INFO:tasks.workunit.client.0.smithi002.stdout:6/580: fdatasync d15/d2d/d2b/d3f/d77/d9f/f91 0 2024-03-16T10:45:02.136 INFO:tasks.workunit.client.0.smithi002.stdout:8/674: mkdir d7/de3 0 2024-03-16T10:45:02.137 INFO:tasks.workunit.client.0.smithi002.stdout:1/600: write d2/df/d12/d84/d87/d96/db5/d1c/d5d/f6b [2575378,119358] 0 2024-03-16T10:45:02.142 INFO:tasks.workunit.client.0.smithi002.stdout:7/518: truncate d3/d4/d1d/d87/d5c/d63/f7f 644747 0 2024-03-16T10:45:02.143 INFO:tasks.workunit.client.0.smithi002.stdout:6/581: truncate d15/d2d/f85 277704 0 2024-03-16T10:45:02.149 INFO:tasks.workunit.client.0.smithi002.stdout:6/582: mknod d15/d2d/d2b/dc8/cca 0 2024-03-16T10:45:02.149 INFO:tasks.workunit.client.0.smithi002.stdout:9/616: dread d5/f53 [0,4194304] 0 2024-03-16T10:45:02.149 INFO:tasks.workunit.client.0.smithi002.stdout:1/601: dwrite d2/df/d12/d84/d87/d96/db5/d51/f5a [0,4194304] 0 2024-03-16T10:45:02.151 INFO:tasks.workunit.client.0.smithi002.stdout:0/616: write d3/de/f8f [141309,120187] 0 2024-03-16T10:45:02.153 INFO:tasks.workunit.client.0.smithi002.stdout:4/562: dwrite d2/d33/da/d2e/f47 [0,4194304] 0 2024-03-16T10:45:02.156 INFO:tasks.workunit.client.0.smithi002.stdout:1/602: chown d2/df/d12/d84/d87/d96/da8 58 1 2024-03-16T10:45:02.161 INFO:tasks.workunit.client.0.smithi002.stdout:8/675: symlink d7/d9/d17/dc3/le4 0 2024-03-16T10:45:02.163 INFO:tasks.workunit.client.0.smithi002.stdout:9/617: creat d5/d9/dc/d1e/fbc x:0 0 0 2024-03-16T10:45:02.163 INFO:tasks.workunit.client.0.smithi002.stdout:1/603: read d2/df/d12/d84/d87/d96/db5/f1f [809268,41114] 0 2024-03-16T10:45:02.164 INFO:tasks.workunit.client.0.smithi002.stdout:6/583: getdents d15/d2d/d8a 0 2024-03-16T10:45:02.168 INFO:tasks.workunit.client.0.smithi002.stdout:6/584: readlink d15/d2d/d2b/d3f/d77/lc3 0 2024-03-16T10:45:02.169 INFO:tasks.workunit.client.0.smithi002.stdout:7/519: dwrite d3/d4/d1d/d87/d5c/d63/d34/f6b [0,4194304] 0 2024-03-16T10:45:02.172 INFO:tasks.workunit.client.0.smithi002.stdout:1/604: symlink d2/df/d12/lbd 0 2024-03-16T10:45:02.172 INFO:tasks.workunit.client.0.smithi002.stdout:4/563: dwrite d2/d33/da/d88/fab [0,4194304] 0 2024-03-16T10:45:02.174 INFO:tasks.workunit.client.0.smithi002.stdout:9/618: creat d5/d9/d25/fbd x:0 0 0 2024-03-16T10:45:02.176 INFO:tasks.workunit.client.0.smithi002.stdout:1/605: write d2/f52 [9469740,74602] 0 2024-03-16T10:45:02.179 INFO:tasks.workunit.client.0.smithi002.stdout:6/585: mknod d15/d2d/d2b/d3f/ccb 0 2024-03-16T10:45:02.185 INFO:tasks.workunit.client.0.smithi002.stdout:1/606: mknod d2/df/cbe 0 2024-03-16T10:45:02.186 INFO:tasks.workunit.client.0.smithi002.stdout:7/520: write d3/d4/d1d/d87/d5c/d63/d3f/f41 [3452769,74252] 0 2024-03-16T10:45:02.186 INFO:tasks.workunit.client.0.smithi002.stdout:0/617: dread d3/de/f13 [0,4194304] 0 2024-03-16T10:45:02.186 INFO:tasks.workunit.client.0.smithi002.stdout:3/593: sync 2024-03-16T10:45:02.186 INFO:tasks.workunit.client.0.smithi002.stdout:5/504: sync 2024-03-16T10:45:02.187 INFO:tasks.workunit.client.0.smithi002.stdout:6/586: mkdir d15/d2d/d2c/d8f/d96/dbc/dcc 0 2024-03-16T10:45:02.191 INFO:tasks.workunit.client.0.smithi002.stdout:8/676: dwrite d7/d9/d17/d23/d39/db7/fd8 [4194304,4194304] 0 2024-03-16T10:45:02.193 INFO:tasks.workunit.client.0.smithi002.stdout:4/564: dwrite d2/d33/f1b [0,4194304] 0 2024-03-16T10:45:02.194 INFO:tasks.workunit.client.0.smithi002.stdout:9/619: dwrite d5/d9/d25/d23/d81/fa1 [0,4194304] 0 2024-03-16T10:45:02.195 INFO:tasks.workunit.client.0.smithi002.stdout:6/587: chown d15/d2d/d2b/d3f/ccb 6946024 1 2024-03-16T10:45:02.195 INFO:tasks.workunit.client.0.smithi002.stdout:4/565: chown d2/d33/d1c 371450831 1 2024-03-16T10:45:02.196 INFO:tasks.workunit.client.0.smithi002.stdout:8/677: dread - d7/d9/d17/d23/d39/db7/fd1 zero size 2024-03-16T10:45:02.197 INFO:tasks.workunit.client.0.smithi002.stdout:4/566: write d2/d33/f94 [4828314,128272] 0 2024-03-16T10:45:02.201 INFO:tasks.workunit.client.0.smithi002.stdout:9/620: write d5/d9/d55/f7b [440252,126703] 0 2024-03-16T10:45:02.204 INFO:tasks.workunit.client.0.smithi002.stdout:0/618: write d3/d28/f34 [1345593,7161] 0 2024-03-16T10:45:02.205 INFO:tasks.workunit.client.0.smithi002.stdout:4/567: rmdir d2/d6/d32/d68 39 2024-03-16T10:45:02.206 INFO:tasks.workunit.client.0.smithi002.stdout:9/621: dread - d5/d9/dc/d1e/d91/f84 zero size 2024-03-16T10:45:02.210 INFO:tasks.workunit.client.0.smithi002.stdout:7/521: dread d3/d4/f1e [0,4194304] 0 2024-03-16T10:45:02.210 INFO:tasks.workunit.client.0.smithi002.stdout:4/568: symlink d2/d33/d1c/laf 0 2024-03-16T10:45:02.212 INFO:tasks.workunit.client.0.smithi002.stdout:1/607: rename d2/df/cbe to d2/df/cbf 0 2024-03-16T10:45:02.212 INFO:tasks.workunit.client.0.smithi002.stdout:6/588: dwrite d15/f24 [0,4194304] 0 2024-03-16T10:45:02.212 INFO:tasks.workunit.client.0.smithi002.stdout:0/619: symlink d3/d28/d2a/da1/d9c/lc3 0 2024-03-16T10:45:02.214 INFO:tasks.workunit.client.0.smithi002.stdout:5/505: getdents d0/d16/d1d/d43/d48 0 2024-03-16T10:45:02.215 INFO:tasks.workunit.client.0.smithi002.stdout:4/569: dread - d2/d6/d32/d66/d7f/f96 zero size 2024-03-16T10:45:02.215 INFO:tasks.workunit.client.0.smithi002.stdout:6/589: write fa [3761571,79494] 0 2024-03-16T10:45:02.220 INFO:tasks.workunit.client.0.smithi002.stdout:7/522: mkdir d3/d4/d1d/d87/d5c/d63/d3f/dba 0 2024-03-16T10:45:02.223 INFO:tasks.workunit.client.0.smithi002.stdout:8/678: dwrite d7/d9/fa [0,4194304] 0 2024-03-16T10:45:02.224 INFO:tasks.workunit.client.0.smithi002.stdout:5/506: write d0/d16/d1d/d43/d48/f8e [1969150,25201] 0 2024-03-16T10:45:02.225 INFO:tasks.workunit.client.0.smithi002.stdout:6/590: unlink d15/dab/cad 0 2024-03-16T10:45:02.227 INFO:tasks.workunit.client.0.smithi002.stdout:1/608: mknod d2/df/d12/d84/d87/d96/db5/d51/d56/cc0 0 2024-03-16T10:45:02.232 INFO:tasks.workunit.client.0.smithi002.stdout:4/570: unlink d2/d33/d35/f63 0 2024-03-16T10:45:02.237 INFO:tasks.workunit.client.0.smithi002.stdout:9/622: dread d5/d9/dc/d1e/d24/f4b [0,4194304] 0 2024-03-16T10:45:02.238 INFO:tasks.workunit.client.0.smithi002.stdout:1/609: symlink d2/df/d12/d74/lc1 0 2024-03-16T10:45:02.241 INFO:tasks.workunit.client.0.smithi002.stdout:6/591: symlink d15/d2d/d8a/d7c/lcd 0 2024-03-16T10:45:02.243 INFO:tasks.workunit.client.0.smithi002.stdout:0/620: dwrite d3/d28/d2a/da1/d9c/fb9 [0,4194304] 0 2024-03-16T10:45:02.244 INFO:tasks.workunit.client.0.smithi002.stdout:9/623: symlink d5/d9/d22/d3c/d4d/lbe 0 2024-03-16T10:45:02.245 INFO:tasks.workunit.client.0.smithi002.stdout:7/523: link d3/l20 d3/d4/d1d/d87/d5c/lbb 0 2024-03-16T10:45:02.250 INFO:tasks.workunit.client.0.smithi002.stdout:5/507: dread d0/f18 [0,4194304] 0 2024-03-16T10:45:02.251 INFO:tasks.workunit.client.0.smithi002.stdout:9/624: write d5/d9/dc/d28/f2d [2393578,10641] 0 2024-03-16T10:45:02.251 INFO:tasks.workunit.client.0.smithi002.stdout:0/621: creat d3/d59/fc4 x:0 0 0 2024-03-16T10:45:02.254 INFO:tasks.workunit.client.0.smithi002.stdout:8/679: dread d7/d9/d17/d20/f63 [0,4194304] 0 2024-03-16T10:45:02.259 INFO:tasks.workunit.client.0.smithi002.stdout:9/625: chown d5/d9/d25/d23/f56 17147 1 2024-03-16T10:45:02.259 INFO:tasks.workunit.client.0.smithi002.stdout:4/571: dread d2/d33/f64 [0,4194304] 0 2024-03-16T10:45:02.260 INFO:tasks.workunit.client.0.smithi002.stdout:2/594: sync 2024-03-16T10:45:02.263 INFO:tasks.workunit.client.0.smithi002.stdout:7/524: creat d3/d4/d1d/d45/fbc x:0 0 0 2024-03-16T10:45:02.265 INFO:tasks.workunit.client.0.smithi002.stdout:6/592: write d15/d2d/d2b/d3f/f6d [572047,121212] 0 2024-03-16T10:45:02.266 INFO:tasks.workunit.client.0.smithi002.stdout:0/622: rmdir d3 39 2024-03-16T10:45:02.267 INFO:tasks.workunit.client.1.smithi026.stderr:++ readlink -f fsstress 2024-03-16T10:45:02.267 INFO:tasks.workunit.client.0.smithi002.stdout:9/626: dwrite d5/d9/dc/fa2 [0,4194304] 0 2024-03-16T10:45:02.267 INFO:tasks.workunit.client.0.smithi002.stdout:7/525: write d3/d4/d1d/d87/d5c/d63/d34/f78 [5096023,22260] 0 2024-03-16T10:45:02.268 INFO:tasks.workunit.client.1.smithi026.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-16T10:45:02.269 INFO:tasks.workunit.client.1.smithi026.stderr:+ popd 2024-03-16T10:45:02.269 INFO:tasks.workunit.client.0.smithi002.stdout:9/627: chown d5/d9/dc/d28/f4c 1005342 1 2024-03-16T10:45:02.270 INFO:tasks.workunit.client.1.smithi026.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-16T10:45:02.270 INFO:tasks.workunit.client.1.smithi026.stderr:+ popd 2024-03-16T10:45:02.271 INFO:tasks.workunit.client.1.smithi026.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-16T10:45:02.271 INFO:tasks.workunit.client.1.smithi026.stderr:++ mktemp -d -p . 2024-03-16T10:45:02.274 INFO:tasks.workunit.client.0.smithi002.stdout:5/508: dwrite d0/d16/f5f [0,4194304] 0 2024-03-16T10:45:02.275 INFO:tasks.workunit.client.0.smithi002.stdout:5/509: chown d0/d16/f37 2272 1 2024-03-16T10:45:02.276 INFO:tasks.workunit.client.1.smithi026.stderr:+ T=./tmp.cGmekZRYkU 2024-03-16T10:45:02.276 INFO:tasks.workunit.client.1.smithi026.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.cGmekZRYkU -l 1 -n 1000 -p 10 -v 2024-03-16T10:45:02.281 INFO:tasks.workunit.client.0.smithi002.stdout:4/572: write d2/d6/f13 [6166729,98248] 0 2024-03-16T10:45:02.285 INFO:tasks.workunit.client.1.smithi026.stdout:seed = 1710302334 2024-03-16T10:45:02.287 INFO:tasks.workunit.client.0.smithi002.stdout:7/526: truncate d3/d4/d1d/d87/d5c/d63/d3f/f46 7067 0 2024-03-16T10:45:02.290 INFO:tasks.workunit.client.0.smithi002.stdout:5/510: rmdir d0/d16/d4a/d55 39 2024-03-16T10:45:02.295 INFO:tasks.workunit.client.0.smithi002.stdout:8/680: mkdir d7/d9/d17/d23/d2e/de5 0 2024-03-16T10:45:02.295 INFO:tasks.workunit.client.0.smithi002.stdout:8/681: stat d7/d9/d17/d23/f33 0 2024-03-16T10:45:02.295 INFO:tasks.workunit.client.0.smithi002.stdout:8/682: fsync d7/d9/d17/d23/d39/db7/fd1 0 2024-03-16T10:45:02.295 INFO:tasks.workunit.client.0.smithi002.stdout:4/573: write d2/d33/d1c/f67 [3297847,120552] 0 2024-03-16T10:45:02.295 INFO:tasks.workunit.client.0.smithi002.stdout:4/574: fsync d2/d33/da/f42 0 2024-03-16T10:45:02.295 INFO:tasks.workunit.client.1.smithi026.stdout:1/0: rmdir - no directory 2024-03-16T10:45:02.296 INFO:tasks.workunit.client.0.smithi002.stdout:0/623: symlink d3/d5/d7c/lc5 0 2024-03-16T10:45:02.298 INFO:tasks.workunit.client.0.smithi002.stdout:6/593: creat d15/d2d/d2b/d75/fce x:0 0 0 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/0: chown . 204327 1 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/1: rename - no filename 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/2: link - no file 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/3: dread - no filename 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/4: stat - no entries 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/5: dread - no filename 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/6: fdatasync - no filename 2024-03-16T10:45:02.299 INFO:tasks.workunit.client.1.smithi026.stdout:2/7: chown . 44 1 2024-03-16T10:45:02.300 INFO:tasks.workunit.client.1.smithi026.stdout:2/8: fsync - no filename 2024-03-16T10:45:02.300 INFO:tasks.workunit.client.1.smithi026.stdout:2/9: dwrite - no filename 2024-03-16T10:45:02.300 INFO:tasks.workunit.client.1.smithi026.stdout:2/10: write - no filename 2024-03-16T10:45:02.300 INFO:tasks.workunit.client.1.smithi026.stdout:2/11: write - no filename 2024-03-16T10:45:02.300 INFO:tasks.workunit.client.1.smithi026.stdout:2/12: chown . 7616 1 2024-03-16T10:45:02.300 INFO:tasks.workunit.client.0.smithi002.stdout:4/575: dwrite d2/d33/da/d2e/f8e [0,4194304] 0 2024-03-16T10:45:02.301 INFO:tasks.workunit.client.0.smithi002.stdout:4/576: write d2/d33/d1c/d45/f54 [1022397,32334] 0 2024-03-16T10:45:02.302 INFO:tasks.workunit.client.0.smithi002.stdout:4/577: chown d2/d33/da/d2e/f47 3492123 1 2024-03-16T10:45:02.303 INFO:tasks.workunit.client.0.smithi002.stdout:7/527: unlink d3/d4/d1c/d5b/d64/fa8 0 2024-03-16T10:45:02.304 INFO:tasks.workunit.client.1.smithi026.stdout:0/0: creat f0 x:0 0 0 2024-03-16T10:45:02.305 INFO:tasks.workunit.client.1.smithi026.stdout:0/1: write f0 [839743,37510] 0 2024-03-16T10:45:02.306 INFO:tasks.workunit.client.0.smithi002.stdout:5/511: creat d0/d16/d1d/d43/f99 x:0 0 0 2024-03-16T10:45:02.306 INFO:tasks.workunit.client.1.smithi026.stdout:0/2: stat f0 0 2024-03-16T10:45:02.309 INFO:tasks.workunit.client.1.smithi026.stdout:0/3: read f0 [108865,43879] 0 2024-03-16T10:45:02.309 INFO:tasks.workunit.client.1.smithi026.stdout:1/1: mkdir d0 0 2024-03-16T10:45:02.309 INFO:tasks.workunit.client.1.smithi026.stdout:1/2: dwrite - no filename 2024-03-16T10:45:02.309 INFO:tasks.workunit.client.1.smithi026.stdout:1/3: dread - no filename 2024-03-16T10:45:02.309 INFO:tasks.workunit.client.1.smithi026.stdout:1/4: dwrite - no filename 2024-03-16T10:45:02.310 INFO:tasks.workunit.client.0.smithi002.stdout:8/683: readlink d7/l9a 0 2024-03-16T10:45:02.313 INFO:tasks.workunit.client.1.smithi026.stdout:4/0: unlink - no file 2024-03-16T10:45:02.313 INFO:tasks.workunit.client.0.smithi002.stdout:0/624: rmdir d3/d28/d3d 39 2024-03-16T10:45:02.316 INFO:tasks.workunit.client.0.smithi002.stdout:9/628: link d5/d9/d25/d23/d81/fa1 d5/d9/dc/d1e/d86/fbf 0 2024-03-16T10:45:02.316 INFO:tasks.workunit.client.0.smithi002.stdout:9/629: stat d5/d9/dc/d1e/d24/fa5 0 2024-03-16T10:45:02.320 INFO:tasks.workunit.client.1.smithi026.stdout:6/0: dread - no filename 2024-03-16T10:45:02.320 INFO:tasks.workunit.client.0.smithi002.stdout:4/578: mkdir d2/d33/da/d2e/db0 0 2024-03-16T10:45:02.321 INFO:tasks.workunit.client.1.smithi026.stdout:3/0: symlink l0 0 2024-03-16T10:45:02.321 INFO:tasks.workunit.client.1.smithi026.stdout:3/1: truncate - no filename 2024-03-16T10:45:02.322 INFO:tasks.workunit.client.1.smithi026.stdout:3/2: chown l0 5785223 1 2024-03-16T10:45:02.323 INFO:tasks.workunit.client.0.smithi002.stdout:7/528: creat d3/d4/d1d/d87/d96/d5e/fbd x:0 0 0 2024-03-16T10:45:02.324 INFO:tasks.workunit.client.0.smithi002.stdout:7/529: readlink d3/d4/d1d/d87/d5c/d63/l29 0 2024-03-16T10:45:02.326 INFO:tasks.workunit.client.1.smithi026.stdout:0/4: unlink f0 0 2024-03-16T10:45:02.326 INFO:tasks.workunit.client.1.smithi026.stdout:0/5: dread - no filename 2024-03-16T10:45:02.326 INFO:tasks.workunit.client.1.smithi026.stdout:0/6: stat - no entries 2024-03-16T10:45:02.326 INFO:tasks.workunit.client.1.smithi026.stdout:0/7: dread - no filename 2024-03-16T10:45:02.326 INFO:tasks.workunit.client.1.smithi026.stdout:0/8: rename - no filename 2024-03-16T10:45:02.326 INFO:tasks.workunit.client.1.smithi026.stdout:0/9: rmdir - no directory 2024-03-16T10:45:02.327 INFO:tasks.workunit.client.0.smithi002.stdout:2/595: rename d7/db/f48 to d7/db/fb3 0 2024-03-16T10:45:02.328 INFO:tasks.workunit.client.1.smithi026.stdout:1/5: getdents d0 0 2024-03-16T10:45:02.328 INFO:tasks.workunit.client.1.smithi026.stdout:1/6: link - no file 2024-03-16T10:45:02.328 INFO:tasks.workunit.client.1.smithi026.stdout:1/7: dread - no filename 2024-03-16T10:45:02.329 INFO:tasks.workunit.client.0.smithi002.stdout:8/684: unlink d7/d9/d25/d91/fbe 0 2024-03-16T10:45:02.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/0: dread - no filename 2024-03-16T10:45:02.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/1: fdatasync - no filename 2024-03-16T10:45:02.332 INFO:tasks.workunit.client.0.smithi002.stdout:2/596: dread d7/d27/f56 [0,4194304] 0 2024-03-16T10:45:02.334 INFO:tasks.workunit.client.1.smithi026.stdout:3/3: rename l0 to l1 0 2024-03-16T10:45:02.335 INFO:tasks.workunit.client.1.smithi026.stdout:4/1: creat f0 x:0 0 0 2024-03-16T10:45:02.335 INFO:tasks.workunit.client.0.smithi002.stdout:9/630: fsync d5/f29 0 2024-03-16T10:45:02.336 INFO:tasks.workunit.client.1.smithi026.stdout:0/10: mkdir d1 0 2024-03-16T10:45:02.336 INFO:tasks.workunit.client.1.smithi026.stdout:0/11: write - no filename 2024-03-16T10:45:02.336 INFO:tasks.workunit.client.1.smithi026.stdout:0/12: dread - no filename 2024-03-16T10:45:02.337 INFO:tasks.workunit.client.0.smithi002.stdout:4/579: mknod d2/d6/d32/d66/d7f/cb1 0 2024-03-16T10:45:02.338 INFO:tasks.workunit.client.0.smithi002.stdout:4/580: chown d2/d33/c55 196288 1 2024-03-16T10:45:02.338 INFO:tasks.workunit.client.0.smithi002.stdout:6/594: dwrite fd [0,4194304] 0 2024-03-16T10:45:02.339 INFO:tasks.workunit.client.1.smithi026.stdout:5/0: dwrite - no filename 2024-03-16T10:45:02.340 INFO:tasks.workunit.client.1.smithi026.stdout:6/1: mkdir d0 0 2024-03-16T10:45:02.341 INFO:tasks.workunit.client.0.smithi002.stdout:3/594: sync 2024-03-16T10:45:02.341 INFO:tasks.workunit.client.0.smithi002.stdout:8/685: rmdir d7 39 2024-03-16T10:45:02.343 INFO:tasks.workunit.client.0.smithi002.stdout:0/625: rename d3/d28/d3d/fac to d3/d31/fc6 0 2024-03-16T10:45:02.344 INFO:tasks.workunit.client.0.smithi002.stdout:2/597: mkdir d7/d2e/d28/d40/d50/db4 0 2024-03-16T10:45:02.344 INFO:tasks.workunit.client.1.smithi026.stdout:1/8: symlink d0/l1 0 2024-03-16T10:45:02.345 INFO:tasks.workunit.client.0.smithi002.stdout:0/626: write d3/d5/d7c/f8a [420742,5507] 0 2024-03-16T10:45:02.346 INFO:tasks.workunit.client.0.smithi002.stdout:0/627: chown d3/de/f5c 3138404 1 2024-03-16T10:45:02.348 INFO:tasks.workunit.client.0.smithi002.stdout:4/581: mkdir d2/d33/da/d88/db2 0 2024-03-16T10:45:02.349 INFO:tasks.workunit.client.1.smithi026.stdout:7/2: creat f0 x:0 0 0 2024-03-16T10:45:02.349 INFO:tasks.workunit.client.1.smithi026.stdout:7/3: dread - f0 zero size 2024-03-16T10:45:02.350 INFO:tasks.workunit.client.0.smithi002.stdout:5/512: symlink d0/d16/d1d/d2b/d5d/l9a 0 2024-03-16T10:45:02.352 INFO:tasks.workunit.client.0.smithi002.stdout:5/513: stat d0/d16/d1d/d33/f42 0 2024-03-16T10:45:02.353 INFO:tasks.workunit.client.1.smithi026.stdout:0/13: mkdir d1/d2 0 2024-03-16T10:45:02.353 INFO:tasks.workunit.client.0.smithi002.stdout:2/598: dwrite d7/d27/f80 [0,4194304] 0 2024-03-16T10:45:02.354 INFO:tasks.workunit.client.0.smithi002.stdout:9/631: dwrite d5/d9/d34/f61 [0,4194304] 0 2024-03-16T10:45:02.355 INFO:tasks.workunit.client.1.smithi026.stdout:1/9: rename d0/l1 to d0/l2 0 2024-03-16T10:45:02.355 INFO:tasks.workunit.client.1.smithi026.stdout:1/10: dread - no filename 2024-03-16T10:45:02.359 INFO:tasks.workunit.client.1.smithi026.stdout:7/4: dwrite f0 [0,4194304] 0 2024-03-16T10:45:02.359 INFO:tasks.workunit.client.1.smithi026.stdout:9/0: dread - no filename 2024-03-16T10:45:02.359 INFO:tasks.workunit.client.1.smithi026.stdout:9/1: dread - no filename 2024-03-16T10:45:02.362 INFO:tasks.workunit.client.0.smithi002.stdout:6/595: rename d15/d2d/d2c/fc6 to d15/d2d/d2b/d3f/d77/d7e/fcf 0 2024-03-16T10:45:02.369 INFO:tasks.workunit.client.1.smithi026.stdout:5/1: symlink l0 0 2024-03-16T10:45:02.373 INFO:tasks.workunit.client.1.smithi026.stdout:7/5: dwrite f0 [0,4194304] 0 2024-03-16T10:45:02.373 INFO:tasks.workunit.client.1.smithi026.stdout:6/2: mknod d0/c1 0 2024-03-16T10:45:02.375 INFO:tasks.workunit.client.1.smithi026.stdout:4/2: fsync f0 0 2024-03-16T10:45:02.376 INFO:tasks.workunit.client.0.smithi002.stdout:1/610: sync 2024-03-16T10:45:02.385 INFO:tasks.workunit.client.0.smithi002.stdout:1/611: dwrite d2/d58/dbb/fbc [0,4194304] 0 2024-03-16T10:45:02.389 INFO:tasks.workunit.client.0.smithi002.stdout:8/686: mknod d7/d9/d17/d23/d2e/d3a/ce6 0 2024-03-16T10:45:02.389 INFO:tasks.workunit.client.0.smithi002.stdout:7/530: write d3/d37/f8f [1676633,122226] 0 2024-03-16T10:45:02.390 INFO:tasks.workunit.client.1.smithi026.stdout:8/0: creat f0 x:0 0 0 2024-03-16T10:45:02.390 INFO:tasks.workunit.client.0.smithi002.stdout:8/687: stat d7/d9/d17/d23/d39/c5b 0 2024-03-16T10:45:02.391 INFO:tasks.workunit.client.0.smithi002.stdout:3/595: mkdir d1/db6/d15/dcc 0 2024-03-16T10:45:02.394 INFO:tasks.workunit.client.1.smithi026.stdout:5/2: mkdir d1 0 2024-03-16T10:45:02.394 INFO:tasks.workunit.client.1.smithi026.stdout:5/3: chown l0 63738 1 2024-03-16T10:45:02.394 INFO:tasks.workunit.client.1.smithi026.stdout:5/4: dwrite - no filename 2024-03-16T10:45:02.394 INFO:tasks.workunit.client.1.smithi026.stdout:5/5: fsync - no filename 2024-03-16T10:45:02.395 INFO:tasks.workunit.client.1.smithi026.stdout:5/6: rename d1 to d1/d2 22 2024-03-16T10:45:02.395 INFO:tasks.workunit.client.1.smithi026.stdout:5/7: fdatasync - no filename 2024-03-16T10:45:02.396 INFO:tasks.workunit.client.1.smithi026.stdout:3/4: rename l1 to l2 0 2024-03-16T10:45:02.399 INFO:tasks.workunit.client.0.smithi002.stdout:9/632: rename d5/d9/dc/c27 to d5/d9/d55/cc0 0 2024-03-16T10:45:02.400 INFO:tasks.workunit.client.0.smithi002.stdout:9/633: write d5/d9/d25/d23/f56 [2473420,23474] 0 2024-03-16T10:45:02.403 INFO:tasks.workunit.client.1.smithi026.stdout:6/3: creat d0/f2 x:0 0 0 2024-03-16T10:45:02.403 INFO:tasks.workunit.client.0.smithi002.stdout:3/596: dread d1/dd/d28/d7b/d25/f26 [0,4194304] 0 2024-03-16T10:45:02.404 INFO:tasks.workunit.client.0.smithi002.stdout:0/628: mkdir d3/d28/d2a/dc7 0 2024-03-16T10:45:02.405 INFO:tasks.workunit.client.1.smithi026.stdout:0/14: creat d1/d2/f3 x:0 0 0 2024-03-16T10:45:02.407 INFO:tasks.workunit.client.1.smithi026.stdout:4/3: mkdir d1 0 2024-03-16T10:45:02.407 INFO:tasks.workunit.client.1.smithi026.stdout:4/4: truncate f0 969049 0 2024-03-16T10:45:02.408 INFO:tasks.workunit.client.1.smithi026.stdout:9/2: mkdir d0 0 2024-03-16T10:45:02.408 INFO:tasks.workunit.client.1.smithi026.stdout:9/3: chown d0 111617 1 2024-03-16T10:45:02.409 INFO:tasks.workunit.client.0.smithi002.stdout:5/514: mknod d0/d16/d1d/c9b 0 2024-03-16T10:45:02.411 INFO:tasks.workunit.client.0.smithi002.stdout:2/599: mkdir d7/d2e/d1d/d38/d49/db5 0 2024-03-16T10:45:02.412 INFO:tasks.workunit.client.1.smithi026.stdout:1/11: rename d0/l2 to d0/l3 0 2024-03-16T10:45:02.412 INFO:tasks.workunit.client.1.smithi026.stdout:1/12: write - no filename 2024-03-16T10:45:02.413 INFO:tasks.workunit.client.0.smithi002.stdout:1/612: unlink d2/df/d12/d84/d87/d96/db5/d1c/d5d/l9f 0 2024-03-16T10:45:02.414 INFO:tasks.workunit.client.0.smithi002.stdout:5/515: dwrite d0/d16/d1d/d43/f99 [0,4194304] 0 2024-03-16T10:45:02.415 INFO:tasks.workunit.client.0.smithi002.stdout:5/516: dread - d0/d1f/f77 zero size 2024-03-16T10:45:02.416 INFO:tasks.workunit.client.1.smithi026.stdout:7/6: unlink f0 0 2024-03-16T10:45:02.419 INFO:tasks.workunit.client.1.smithi026.stdout:0/15: symlink d1/d2/l4 0 2024-03-16T10:45:02.423 INFO:tasks.workunit.client.1.smithi026.stdout:5/8: mknod d1/c3 0 2024-03-16T10:45:02.425 INFO:tasks.workunit.client.1.smithi026.stdout:1/13: mkdir d0/d4 0 2024-03-16T10:45:02.425 INFO:tasks.workunit.client.1.smithi026.stdout:1/14: dread - no filename 2024-03-16T10:45:02.429 INFO:tasks.workunit.client.1.smithi026.stdout:6/4: link d0/f2 d0/f3 0 2024-03-16T10:45:02.433 INFO:tasks.workunit.client.1.smithi026.stdout:4/5: creat d1/f2 x:0 0 0 2024-03-16T10:45:02.435 INFO:tasks.workunit.client.1.smithi026.stdout:9/4: symlink d0/l1 0 2024-03-16T10:45:02.437 INFO:tasks.workunit.client.1.smithi026.stdout:0/16: rename d1/d2 to d1/d5 0 2024-03-16T10:45:02.438 INFO:tasks.workunit.client.1.smithi026.stdout:0/17: chown d1/d5/f3 607141 1 2024-03-16T10:45:02.439 INFO:tasks.workunit.client.1.smithi026.stdout:0/18: chown d1/d5/f3 40 1 2024-03-16T10:45:02.439 INFO:tasks.workunit.client.1.smithi026.stdout:0/19: fsync d1/d5/f3 0 2024-03-16T10:45:02.439 INFO:tasks.workunit.client.1.smithi026.stdout:0/20: dread - d1/d5/f3 zero size 2024-03-16T10:45:02.440 INFO:tasks.workunit.client.1.smithi026.stdout:1/15: creat d0/f5 x:0 0 0 2024-03-16T10:45:02.445 INFO:tasks.workunit.client.1.smithi026.stdout:6/5: dread - d0/f3 zero size 2024-03-16T10:45:02.446 INFO:tasks.workunit.client.1.smithi026.stdout:4/6: unlink f0 0 2024-03-16T10:45:02.447 INFO:tasks.workunit.client.1.smithi026.stdout:9/5: unlink d0/l1 0 2024-03-16T10:45:02.447 INFO:tasks.workunit.client.1.smithi026.stdout:9/6: chown d0 51 1 2024-03-16T10:45:02.447 INFO:tasks.workunit.client.1.smithi026.stdout:9/7: truncate - no filename 2024-03-16T10:45:02.447 INFO:tasks.workunit.client.1.smithi026.stdout:9/8: fsync - no filename 2024-03-16T10:45:02.448 INFO:tasks.workunit.client.1.smithi026.stdout:0/21: mkdir d1/d5/d6 0 2024-03-16T10:45:02.451 INFO:tasks.workunit.client.1.smithi026.stdout:6/6: write d0/f2 [615400,102766] 0 2024-03-16T10:45:02.451 INFO:tasks.workunit.client.1.smithi026.stdout:9/9: mknod d0/c2 0 2024-03-16T10:45:02.452 INFO:tasks.workunit.client.1.smithi026.stdout:5/9: link l0 d1/l4 0 2024-03-16T10:45:02.452 INFO:tasks.workunit.client.1.smithi026.stdout:5/10: fsync - no filename 2024-03-16T10:45:02.452 INFO:tasks.workunit.client.1.smithi026.stdout:5/11: chown d1 392 1 2024-03-16T10:45:02.452 INFO:tasks.workunit.client.1.smithi026.stdout:5/12: dwrite - no filename 2024-03-16T10:45:02.453 INFO:tasks.workunit.client.1.smithi026.stdout:5/13: read - no filename 2024-03-16T10:45:02.453 INFO:tasks.workunit.client.1.smithi026.stdout:4/7: rmdir d1 39 2024-03-16T10:45:02.454 INFO:tasks.workunit.client.1.smithi026.stdout:7/7: getdents . 0 2024-03-16T10:45:02.455 INFO:tasks.workunit.client.1.smithi026.stdout:0/22: symlink d1/d5/l7 0 2024-03-16T10:45:02.455 INFO:tasks.workunit.client.1.smithi026.stdout:0/23: stat d1/d5/f3 0 2024-03-16T10:45:02.456 INFO:tasks.workunit.client.1.smithi026.stdout:0/24: chown d1/d5/f3 1 1 2024-03-16T10:45:02.456 INFO:tasks.workunit.client.1.smithi026.stdout:0/25: dread - d1/d5/f3 zero size 2024-03-16T10:45:02.456 INFO:tasks.workunit.client.1.smithi026.stdout:6/7: creat d0/f4 x:0 0 0 2024-03-16T10:45:02.458 INFO:tasks.workunit.client.1.smithi026.stdout:9/10: mknod d0/c3 0 2024-03-16T10:45:02.458 INFO:tasks.workunit.client.1.smithi026.stdout:6/8: write d0/f2 [67433,46460] 0 2024-03-16T10:45:02.458 INFO:tasks.workunit.client.1.smithi026.stdout:5/14: readlink l0 0 2024-03-16T10:45:02.459 INFO:tasks.workunit.client.1.smithi026.stdout:5/15: write - no filename 2024-03-16T10:45:02.459 INFO:tasks.workunit.client.1.smithi026.stdout:5/16: chown d1 7467 1 2024-03-16T10:45:02.460 INFO:tasks.workunit.client.1.smithi026.stdout:7/8: creat f1 x:0 0 0 2024-03-16T10:45:02.460 INFO:tasks.workunit.client.1.smithi026.stdout:7/9: readlink - no filename 2024-03-16T10:45:02.460 INFO:tasks.workunit.client.1.smithi026.stdout:7/10: read - f1 zero size 2024-03-16T10:45:02.461 INFO:tasks.workunit.client.1.smithi026.stdout:7/11: write f1 [888772,129288] 0 2024-03-16T10:45:02.462 INFO:tasks.workunit.client.1.smithi026.stdout:0/26: mknod d1/d5/c8 0 2024-03-16T10:45:02.464 INFO:tasks.workunit.client.1.smithi026.stdout:9/11: creat d0/f4 x:0 0 0 2024-03-16T10:45:02.467 INFO:tasks.workunit.client.1.smithi026.stdout:6/9: unlink d0/f4 0 2024-03-16T10:45:02.468 INFO:tasks.workunit.client.1.smithi026.stdout:5/17: creat d1/f5 x:0 0 0 2024-03-16T10:45:02.469 INFO:tasks.workunit.client.1.smithi026.stdout:7/12: mknod c2 0 2024-03-16T10:45:02.469 INFO:tasks.workunit.client.1.smithi026.stdout:6/10: write d0/f2 [467529,64414] 0 2024-03-16T10:45:02.470 INFO:tasks.workunit.client.1.smithi026.stdout:0/27: dwrite d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.471 INFO:tasks.workunit.client.1.smithi026.stdout:9/12: mknod d0/c5 0 2024-03-16T10:45:02.472 INFO:tasks.workunit.client.1.smithi026.stdout:6/11: fdatasync d0/f3 0 2024-03-16T10:45:02.474 INFO:tasks.workunit.client.0.smithi002.stdout:3/597: mknod d1/dd/d28/ccd 0 2024-03-16T10:45:02.474 INFO:tasks.workunit.client.0.smithi002.stdout:4/582: write d2/d33/da/f49 [218880,54079] 0 2024-03-16T10:45:02.475 INFO:tasks.workunit.client.1.smithi026.stdout:5/18: symlink d1/l6 0 2024-03-16T10:45:02.475 INFO:tasks.workunit.client.0.smithi002.stdout:5/517: dread d0/d16/f19 [0,4194304] 0 2024-03-16T10:45:02.475 INFO:tasks.workunit.client.0.smithi002.stdout:0/629: mkdir d3/d31/d88/dc8 0 2024-03-16T10:45:02.475 INFO:tasks.workunit.client.0.smithi002.stdout:7/531: symlink d3/d4/d1d/lbe 0 2024-03-16T10:45:02.476 INFO:tasks.workunit.client.0.smithi002.stdout:1/613: dread - d2/df/d12/d84/d87/d96/db5/d1c/f7f zero size 2024-03-16T10:45:02.477 INFO:tasks.workunit.client.0.smithi002.stdout:9/634: creat d5/d9/dc/d1e/d91/d82/db4/fc1 x:0 0 0 2024-03-16T10:45:02.478 INFO:tasks.workunit.client.0.smithi002.stdout:3/598: read d1/dd/d28/f31 [730852,31532] 0 2024-03-16T10:45:02.479 INFO:tasks.workunit.client.1.smithi026.stdout:0/28: dwrite d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.479 INFO:tasks.workunit.client.0.smithi002.stdout:6/596: rmdir d15/d2d/d73/d97/d98 0 2024-03-16T10:45:02.480 INFO:tasks.workunit.client.1.smithi026.stdout:0/29: rename d1/d5/d6 to d1/d5/d6/d9 22 2024-03-16T10:45:02.482 INFO:tasks.workunit.client.0.smithi002.stdout:5/518: creat d0/d6b/f9c x:0 0 0 2024-03-16T10:45:02.484 INFO:tasks.workunit.client.1.smithi026.stdout:9/13: rename d0/c3 to d0/c6 0 2024-03-16T10:45:02.484 INFO:tasks.workunit.client.1.smithi026.stdout:9/14: stat d0 0 2024-03-16T10:45:02.486 INFO:tasks.workunit.client.1.smithi026.stdout:5/19: readlink l0 0 2024-03-16T10:45:02.488 INFO:tasks.workunit.client.0.smithi002.stdout:6/597: dwrite d15/d2d/da8/fc9 [0,4194304] 0 2024-03-16T10:45:02.488 INFO:tasks.workunit.client.1.smithi026.stdout:6/12: dread d0/f3 [0,4194304] 0 2024-03-16T10:45:02.490 INFO:tasks.workunit.client.1.smithi026.stdout:0/30: rename d1/d5/c8 to d1/ca 0 2024-03-16T10:45:02.492 INFO:tasks.workunit.client.0.smithi002.stdout:4/583: unlink d2/d33/d1c/f6c 0 2024-03-16T10:45:02.499 INFO:tasks.workunit.client.1.smithi026.stdout:6/13: mkdir d0/d5 0 2024-03-16T10:45:02.499 INFO:tasks.workunit.client.1.smithi026.stdout:9/15: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:02.499 INFO:tasks.workunit.client.0.smithi002.stdout:6/598: dwrite f8 [4194304,4194304] 0 2024-03-16T10:45:02.500 INFO:tasks.workunit.client.1.smithi026.stdout:5/20: dwrite d1/f5 [0,4194304] 0 2024-03-16T10:45:02.500 INFO:tasks.workunit.client.0.smithi002.stdout:9/635: write d5/d9/d22/f6e [490515,130065] 0 2024-03-16T10:45:02.501 INFO:tasks.workunit.client.1.smithi026.stdout:6/14: dread d0/f3 [0,4194304] 0 2024-03-16T10:45:02.504 INFO:tasks.workunit.client.0.smithi002.stdout:7/532: dread d3/d61/d66/f4f [4194304,4194304] 0 2024-03-16T10:45:02.504 INFO:tasks.workunit.client.1.smithi026.stdout:9/16: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:02.505 INFO:tasks.workunit.client.1.smithi026.stdout:9/17: rename d0 to d0/d7 22 2024-03-16T10:45:02.509 INFO:tasks.workunit.client.0.smithi002.stdout:5/519: mkdir d0/d16/d1d/d33/d4b/d9d 0 2024-03-16T10:45:02.511 INFO:tasks.workunit.client.0.smithi002.stdout:1/614: dwrite d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:45:02.513 INFO:tasks.workunit.client.1.smithi026.stdout:5/21: creat d1/f7 x:0 0 0 2024-03-16T10:45:02.514 INFO:tasks.workunit.client.0.smithi002.stdout:8/688: creat d7/d13/db8/fe7 x:0 0 0 2024-03-16T10:45:02.515 INFO:tasks.workunit.client.0.smithi002.stdout:8/689: chown d7/d9/d17/d23/d39/db7/dc2 1148 1 2024-03-16T10:45:02.516 INFO:tasks.workunit.client.0.smithi002.stdout:5/520: dread - d0/d16/d1d/d33/f42 zero size 2024-03-16T10:45:02.517 INFO:tasks.workunit.client.1.smithi026.stdout:6/15: rename d0/f2 to d0/f6 0 2024-03-16T10:45:02.517 INFO:tasks.workunit.client.1.smithi026.stdout:6/16: stat d0/f6 0 2024-03-16T10:45:02.517 INFO:tasks.workunit.client.1.smithi026.stdout:5/22: dread d1/f5 [0,4194304] 0 2024-03-16T10:45:02.520 INFO:tasks.workunit.client.0.smithi002.stdout:0/630: creat d3/d27/d90/dbb/fc9 x:0 0 0 2024-03-16T10:45:02.524 INFO:tasks.workunit.client.1.smithi026.stdout:0/31: truncate d1/d5/f3 3375586 0 2024-03-16T10:45:02.526 INFO:tasks.workunit.client.0.smithi002.stdout:7/533: dread d3/d61/d66/f4f [0,4194304] 0 2024-03-16T10:45:02.526 INFO:tasks.workunit.client.1.smithi026.stdout:5/23: dwrite d1/f7 [0,4194304] 0 2024-03-16T10:45:02.529 INFO:tasks.workunit.client.0.smithi002.stdout:1/615: mknod d2/df/d12/d84/cc2 0 2024-03-16T10:45:02.533 INFO:tasks.workunit.client.0.smithi002.stdout:8/690: chown d7/d9/d17/d23/d2e/l6a 3825257 1 2024-03-16T10:45:02.534 INFO:tasks.workunit.client.0.smithi002.stdout:5/521: rmdir d0/d16/d1d/d33/d4b 39 2024-03-16T10:45:02.535 INFO:tasks.workunit.client.1.smithi026.stdout:6/17: dwrite d0/f3 [0,4194304] 0 2024-03-16T10:45:02.536 INFO:tasks.workunit.client.1.smithi026.stdout:6/18: write d0/f6 [1470179,54272] 0 2024-03-16T10:45:02.536 INFO:tasks.workunit.client.1.smithi026.stdout:0/32: write d1/d5/f3 [104371,92943] 0 2024-03-16T10:45:02.537 INFO:tasks.workunit.client.0.smithi002.stdout:7/534: dwrite d3/d4/d1d/d87/d5c/d63/d34/f98 [0,4194304] 0 2024-03-16T10:45:02.538 INFO:tasks.workunit.client.0.smithi002.stdout:0/631: dread d3/d28/f29 [0,4194304] 0 2024-03-16T10:45:02.539 INFO:tasks.workunit.client.0.smithi002.stdout:3/599: rename d1/dd/d28/d7b/d25/cc1 to d1/dd/d28/d7b/d2c/cce 0 2024-03-16T10:45:02.540 INFO:tasks.workunit.client.0.smithi002.stdout:8/691: read d7/d9/d17/d20/fce [682501,123970] 0 2024-03-16T10:45:02.541 INFO:tasks.workunit.client.0.smithi002.stdout:3/600: dread - d1/dd/d28/d7b/d25/fc9 zero size 2024-03-16T10:45:02.542 INFO:tasks.workunit.client.0.smithi002.stdout:6/599: creat d15/d80/dbe/fd0 x:0 0 0 2024-03-16T10:45:02.542 INFO:tasks.workunit.client.1.smithi026.stdout:6/19: dread d0/f6 [0,4194304] 0 2024-03-16T10:45:02.543 INFO:tasks.workunit.client.1.smithi026.stdout:0/33: dread d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.543 INFO:tasks.workunit.client.0.smithi002.stdout:6/600: readlink d15/d2d/d2c/l3e 0 2024-03-16T10:45:02.546 INFO:tasks.workunit.client.0.smithi002.stdout:6/601: readlink d15/d2d/l4c 0 2024-03-16T10:45:02.546 INFO:tasks.workunit.client.0.smithi002.stdout:6/602: read - d15/d80/fac zero size 2024-03-16T10:45:02.550 INFO:tasks.workunit.client.1.smithi026.stdout:5/24: truncate d1/f7 3500045 0 2024-03-16T10:45:02.553 INFO:tasks.workunit.client.0.smithi002.stdout:1/616: dwrite d2/f68 [0,4194304] 0 2024-03-16T10:45:02.553 INFO:tasks.workunit.client.0.smithi002.stdout:8/692: rmdir d7/d13/db8/d8f 39 2024-03-16T10:45:02.555 INFO:tasks.workunit.client.0.smithi002.stdout:8/693: readlink d7/d13/db8/l66 0 2024-03-16T10:45:02.556 INFO:tasks.workunit.client.0.smithi002.stdout:3/601: creat d1/dd/d28/d7b/d8a/d93/fcf x:0 0 0 2024-03-16T10:45:02.556 INFO:tasks.workunit.client.1.smithi026.stdout:5/25: fdatasync d1/f5 0 2024-03-16T10:45:02.556 INFO:tasks.workunit.client.1.smithi026.stdout:6/20: fdatasync d0/f6 0 2024-03-16T10:45:02.557 INFO:tasks.workunit.client.0.smithi002.stdout:8/694: write d7/d9/d17/d23/d39/f8c [786793,56658] 0 2024-03-16T10:45:02.558 INFO:tasks.workunit.client.0.smithi002.stdout:9/636: creat d5/d9/dc/fc2 x:0 0 0 2024-03-16T10:45:02.558 INFO:tasks.workunit.client.0.smithi002.stdout:3/602: chown d1/dd/d28/d7b/d2c/d63 16324547 1 2024-03-16T10:45:02.559 INFO:tasks.workunit.client.0.smithi002.stdout:6/603: creat d15/d2d/d73/d97/fd1 x:0 0 0 2024-03-16T10:45:02.560 INFO:tasks.workunit.client.1.smithi026.stdout:0/34: truncate d1/d5/f3 548615 0 2024-03-16T10:45:02.561 INFO:tasks.workunit.client.1.smithi026.stdout:5/26: rename d1/l4 to d1/l8 0 2024-03-16T10:45:02.569 INFO:tasks.workunit.client.1.smithi026.stdout:5/27: fdatasync d1/f7 0 2024-03-16T10:45:02.569 INFO:tasks.workunit.client.0.smithi002.stdout:1/617: dread d2/df/d12/f37 [0,4194304] 0 2024-03-16T10:45:02.569 INFO:tasks.workunit.client.0.smithi002.stdout:2/600: sync 2024-03-16T10:45:02.570 INFO:tasks.workunit.client.0.smithi002.stdout:3/603: write d1/f16 [8436580,35551] 0 2024-03-16T10:45:02.571 INFO:tasks.workunit.client.1.smithi026.stdout:5/28: creat d1/f9 x:0 0 0 2024-03-16T10:45:02.572 INFO:tasks.workunit.client.0.smithi002.stdout:9/637: symlink d5/d9/dc/d1e/d86/lc3 0 2024-03-16T10:45:02.573 INFO:tasks.workunit.client.1.smithi026.stdout:6/21: rmdir d0/d5 0 2024-03-16T10:45:02.574 INFO:tasks.workunit.client.0.smithi002.stdout:9/638: chown d5/f7 29576 1 2024-03-16T10:45:02.574 INFO:tasks.workunit.client.0.smithi002.stdout:6/604: mkdir d15/d2d/d2c/d8f/d96/db5/dd2 0 2024-03-16T10:45:02.576 INFO:tasks.workunit.client.0.smithi002.stdout:4/584: rename d2/d33/d1c/f1d to d2/d33/fb3 0 2024-03-16T10:45:02.577 INFO:tasks.workunit.client.1.smithi026.stdout:6/22: dread d0/f3 [0,4194304] 0 2024-03-16T10:45:02.577 INFO:tasks.workunit.client.1.smithi026.stdout:6/23: chown d0 140 1 2024-03-16T10:45:02.578 INFO:tasks.workunit.client.0.smithi002.stdout:9/639: write d5/d9/dc/d1e/f36 [836684,129823] 0 2024-03-16T10:45:02.581 INFO:tasks.workunit.client.0.smithi002.stdout:1/618: dwrite d2/df/d12/d84/d87/d96/db5/d51/fa6 [0,4194304] 0 2024-03-16T10:45:02.583 INFO:tasks.workunit.client.1.smithi026.stdout:6/24: mkdir d0/d7 0 2024-03-16T10:45:02.584 INFO:tasks.workunit.client.1.smithi026.stdout:0/35: dwrite d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.584 INFO:tasks.workunit.client.1.smithi026.stdout:6/25: fdatasync d0/f3 0 2024-03-16T10:45:02.585 INFO:tasks.workunit.client.0.smithi002.stdout:6/605: dwrite f11 [0,4194304] 0 2024-03-16T10:45:02.588 INFO:tasks.workunit.client.0.smithi002.stdout:2/601: dread d7/d2e/d28/d44/fb1 [0,4194304] 0 2024-03-16T10:45:02.588 INFO:tasks.workunit.client.0.smithi002.stdout:6/606: dread d15/d2d/d73/f95 [0,4194304] 0 2024-03-16T10:45:02.589 INFO:tasks.workunit.client.0.smithi002.stdout:5/522: rename d0/d16/d4a to d0/d16/d4a/d9e 22 2024-03-16T10:45:02.593 INFO:tasks.workunit.client.1.smithi026.stdout:6/26: symlink d0/d7/l8 0 2024-03-16T10:45:02.593 INFO:tasks.workunit.client.0.smithi002.stdout:8/695: write d7/d9/d17/d23/d2e/f92 [638556,47747] 0 2024-03-16T10:45:02.594 INFO:tasks.workunit.client.0.smithi002.stdout:4/585: mkdir d2/d33/da/d2e/db4 0 2024-03-16T10:45:02.598 INFO:tasks.workunit.client.0.smithi002.stdout:8/696: write d7/d9/d17/fe1 [285834,83106] 0 2024-03-16T10:45:02.598 INFO:tasks.workunit.client.1.smithi026.stdout:6/27: mknod d0/c9 0 2024-03-16T10:45:02.599 INFO:tasks.workunit.client.0.smithi002.stdout:2/602: dwrite d7/d2e/d1d/d2f/d4b/fa0 [0,4194304] 0 2024-03-16T10:45:02.600 INFO:tasks.workunit.client.1.smithi026.stdout:0/36: link d1/d5/l7 d1/d5/d6/lb 0 2024-03-16T10:45:02.602 INFO:tasks.workunit.client.0.smithi002.stdout:1/619: write d2/df/d12/d84/d87/d96/db5/f39 [777621,128708] 0 2024-03-16T10:45:02.603 INFO:tasks.workunit.client.1.smithi026.stdout:6/28: mknod d0/ca 0 2024-03-16T10:45:02.604 INFO:tasks.workunit.client.1.smithi026.stdout:0/37: dread d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.609 INFO:tasks.workunit.client.0.smithi002.stdout:6/607: unlink d15/d2d/f70 0 2024-03-16T10:45:02.611 INFO:tasks.workunit.client.1.smithi026.stdout:0/38: dwrite d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.611 INFO:tasks.workunit.client.0.smithi002.stdout:5/523: symlink d0/d6b/l9f 0 2024-03-16T10:45:02.612 INFO:tasks.workunit.client.1.smithi026.stdout:6/29: truncate d0/f6 1944496 0 2024-03-16T10:45:02.616 INFO:tasks.workunit.client.0.smithi002.stdout:3/604: creat d1/dd/d28/d7b/d8a/fd0 x:0 0 0 2024-03-16T10:45:02.618 INFO:tasks.workunit.client.0.smithi002.stdout:4/586: creat d2/d6/d32/d66/d7f/fb5 x:0 0 0 2024-03-16T10:45:02.618 INFO:tasks.workunit.client.1.smithi026.stdout:0/39: dwrite d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.623 INFO:tasks.workunit.client.0.smithi002.stdout:0/632: rename d3/d28/d3d/d44 to d3/d59/dca 0 2024-03-16T10:45:02.627 INFO:tasks.workunit.client.0.smithi002.stdout:6/608: fsync d15/d2d/d2c/f56 0 2024-03-16T10:45:02.628 INFO:tasks.workunit.client.1.smithi026.stdout:0/40: symlink d1/d5/d6/lc 0 2024-03-16T10:45:02.630 INFO:tasks.workunit.client.0.smithi002.stdout:6/609: write fa [15260299,105797] 0 2024-03-16T10:45:02.630 INFO:tasks.workunit.client.0.smithi002.stdout:6/610: write d15/f4f [1279396,15214] 0 2024-03-16T10:45:02.631 INFO:tasks.workunit.client.0.smithi002.stdout:5/524: rmdir d0/d16/d1d/d33/d8c 39 2024-03-16T10:45:02.632 INFO:tasks.workunit.client.0.smithi002.stdout:5/525: dread - d0/d16/d4a/d39/f51 zero size 2024-03-16T10:45:02.632 INFO:tasks.workunit.client.0.smithi002.stdout:9/640: creat d5/fc4 x:0 0 0 2024-03-16T10:45:02.633 INFO:tasks.workunit.client.0.smithi002.stdout:9/641: fdatasync d5/d9/dc/fc2 0 2024-03-16T10:45:02.634 INFO:tasks.workunit.client.0.smithi002.stdout:5/526: write d0/d16/d1d/d2b/d5d/f92 [816975,97945] 0 2024-03-16T10:45:02.636 INFO:tasks.workunit.client.0.smithi002.stdout:3/605: creat d1/d5a/fd1 x:0 0 0 2024-03-16T10:45:02.637 INFO:tasks.workunit.client.1.smithi026.stdout:6/30: write d0/f6 [2547385,8421] 0 2024-03-16T10:45:02.637 INFO:tasks.workunit.client.0.smithi002.stdout:3/606: chown d1/dd/d28/d7b/d2c/d97/fa4 7146123 1 2024-03-16T10:45:02.642 INFO:tasks.workunit.client.0.smithi002.stdout:7/535: rename d3/d4/d1d/d87/d5c/d63/l29 to d3/d4/d1d/d87/d96/d5e/lbf 0 2024-03-16T10:45:02.642 INFO:tasks.workunit.client.1.smithi026.stdout:0/41: truncate d1/d5/f3 1933291 0 2024-03-16T10:45:02.649 INFO:tasks.workunit.client.1.smithi026.stdout:6/31: getdents d0 0 2024-03-16T10:45:02.650 INFO:tasks.workunit.client.0.smithi002.stdout:5/527: unlink d0/d1f/f77 0 2024-03-16T10:45:02.652 INFO:tasks.workunit.client.0.smithi002.stdout:9/642: mknod d5/d9/dc/d1e/d91/d83/cc5 0 2024-03-16T10:45:02.652 INFO:tasks.workunit.client.0.smithi002.stdout:5/528: write d0/d16/d1d/d2b/d5d/f64 [2638714,56097] 0 2024-03-16T10:45:02.656 INFO:tasks.workunit.client.1.smithi026.stdout:6/32: link d0/c1 d0/d7/cb 0 2024-03-16T10:45:02.657 INFO:tasks.workunit.client.1.smithi026.stdout:6/33: write d0/f6 [904270,30113] 0 2024-03-16T10:45:02.658 INFO:tasks.workunit.client.0.smithi002.stdout:9/643: dread d5/d9/dc/f1f [0,4194304] 0 2024-03-16T10:45:02.658 INFO:tasks.workunit.client.0.smithi002.stdout:2/603: rename d7/d2e/d28/d44/d6d/ca9 to d7/d2e/d1d/d2f/cb6 0 2024-03-16T10:45:02.659 INFO:tasks.workunit.client.1.smithi026.stdout:0/42: truncate d1/d5/f3 1763182 0 2024-03-16T10:45:02.664 INFO:tasks.workunit.client.1.smithi026.stdout:6/34: symlink d0/d7/lc 0 2024-03-16T10:45:02.665 INFO:tasks.workunit.client.0.smithi002.stdout:2/604: dwrite d7/d2e/d1d/d2f/d4b/f62 [0,4194304] 0 2024-03-16T10:45:02.666 INFO:tasks.workunit.client.0.smithi002.stdout:6/611: creat d15/dab/dc4/fd3 x:0 0 0 2024-03-16T10:45:02.667 INFO:tasks.workunit.client.0.smithi002.stdout:6/612: chown d15/d2d/f1f 1489069 1 2024-03-16T10:45:02.667 INFO:tasks.workunit.client.1.smithi026.stdout:0/43: creat d1/fd x:0 0 0 2024-03-16T10:45:02.672 INFO:tasks.workunit.client.1.smithi026.stdout:6/35: creat d0/d7/fd x:0 0 0 2024-03-16T10:45:02.674 INFO:tasks.workunit.client.0.smithi002.stdout:5/529: mkdir d0/d16/d5b/da0 0 2024-03-16T10:45:02.675 INFO:tasks.workunit.client.1.smithi026.stdout:0/44: stat d1/d5/l7 0 2024-03-16T10:45:02.676 INFO:tasks.workunit.client.1.smithi026.stdout:6/36: dread d0/f3 [0,4194304] 0 2024-03-16T10:45:02.677 INFO:tasks.workunit.client.0.smithi002.stdout:1/620: link d2/df/c6f d2/df/cc3 0 2024-03-16T10:45:02.678 INFO:tasks.workunit.client.0.smithi002.stdout:6/613: dread d15/d2d/f34 [0,4194304] 0 2024-03-16T10:45:02.679 INFO:tasks.workunit.client.0.smithi002.stdout:1/621: read d2/df/d12/d84/d87/d96/db5/f65 [5052854,96377] 0 2024-03-16T10:45:02.679 INFO:tasks.workunit.client.0.smithi002.stdout:3/607: mkdir d1/db6/d15/d87/dd2 0 2024-03-16T10:45:02.679 INFO:tasks.workunit.client.0.smithi002.stdout:1/622: chown d2/d58/dbb 424543740 1 2024-03-16T10:45:02.680 INFO:tasks.workunit.client.0.smithi002.stdout:8/697: link d7/d9/c3e d7/d9/d17/d23/d39/db7/ce8 0 2024-03-16T10:45:02.682 INFO:tasks.workunit.client.1.smithi026.stdout:6/37: creat d0/d7/fe x:0 0 0 2024-03-16T10:45:02.683 INFO:tasks.workunit.client.0.smithi002.stdout:0/633: creat d3/fcb x:0 0 0 2024-03-16T10:45:02.684 INFO:tasks.workunit.client.0.smithi002.stdout:0/634: chown d3/de/f83 3 1 2024-03-16T10:45:02.684 INFO:tasks.workunit.client.0.smithi002.stdout:8/698: dread d7/d9/d17/d23/d39/f8c [0,4194304] 0 2024-03-16T10:45:02.685 INFO:tasks.workunit.client.0.smithi002.stdout:0/635: truncate d3/fcb 729098 0 2024-03-16T10:45:02.687 INFO:tasks.workunit.client.0.smithi002.stdout:9/644: creat d5/d9/dc/d1e/d24/fc6 x:0 0 0 2024-03-16T10:45:02.688 INFO:tasks.workunit.client.0.smithi002.stdout:9/645: dread - d5/d9/dc/fc2 zero size 2024-03-16T10:45:02.689 INFO:tasks.workunit.client.0.smithi002.stdout:9/646: write d5/f29 [1552284,125663] 0 2024-03-16T10:45:02.690 INFO:tasks.workunit.client.0.smithi002.stdout:9/647: chown d5/d9/dc/d1e/d24/f30 8 1 2024-03-16T10:45:02.691 INFO:tasks.workunit.client.1.smithi026.stdout:6/38: rename d0/d7/fe to d0/d7/ff 0 2024-03-16T10:45:02.692 INFO:tasks.workunit.client.0.smithi002.stdout:7/536: symlink d3/d4/d1d/d87/d5c/d63/lc0 0 2024-03-16T10:45:02.703 INFO:tasks.workunit.client.1.smithi026.stdout:6/39: dwrite d0/d7/ff [0,4194304] 0 2024-03-16T10:45:02.703 INFO:tasks.workunit.client.0.smithi002.stdout:6/614: creat d15/d2d/d2c/d5d/fd4 x:0 0 0 2024-03-16T10:45:02.704 INFO:tasks.workunit.client.0.smithi002.stdout:1/623: write d2/df/d12/d84/d87/d96/f9c [836684,75725] 0 2024-03-16T10:45:02.710 INFO:tasks.workunit.client.1.smithi026.stdout:6/40: rename d0/d7 to d0/d10 0 2024-03-16T10:45:02.712 INFO:tasks.workunit.client.0.smithi002.stdout:4/587: rename d2/l48 to d2/d33/da/d2e/db0/lb6 0 2024-03-16T10:45:02.714 INFO:tasks.workunit.client.1.smithi026.stdout:6/41: creat d0/f11 x:0 0 0 2024-03-16T10:45:02.716 INFO:tasks.workunit.client.1.smithi026.stdout:0/45: write d1/d5/f3 [1003918,93942] 0 2024-03-16T10:45:02.717 INFO:tasks.workunit.client.1.smithi026.stdout:0/46: chown d1/d5/l4 23 1 2024-03-16T10:45:02.718 INFO:tasks.workunit.client.1.smithi026.stdout:6/42: creat d0/f12 x:0 0 0 2024-03-16T10:45:02.719 INFO:tasks.workunit.client.1.smithi026.stdout:6/43: truncate d0/f11 964425 0 2024-03-16T10:45:02.719 INFO:tasks.workunit.client.1.smithi026.stdout:0/47: creat d1/d5/d6/fe x:0 0 0 2024-03-16T10:45:02.722 INFO:tasks.workunit.client.1.smithi026.stdout:0/48: dread d1/d5/f3 [0,4194304] 0 2024-03-16T10:45:02.723 INFO:tasks.workunit.client.1.smithi026.stdout:6/44: mknod d0/c13 0 2024-03-16T10:45:02.723 INFO:tasks.workunit.client.1.smithi026.stdout:0/49: dread - d1/fd zero size 2024-03-16T10:45:02.727 INFO:tasks.workunit.client.1.smithi026.stdout:6/45: rename d0/c9 to d0/d10/c14 0 2024-03-16T10:45:02.729 INFO:tasks.workunit.client.1.smithi026.stdout:0/50: mknod d1/d5/d6/cf 0 2024-03-16T10:45:02.730 INFO:tasks.workunit.client.1.smithi026.stdout:6/46: mknod d0/d10/c15 0 2024-03-16T10:45:02.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:02 smithi026 ceph-mon[37041]: pgmap v34: 65 pgs: 65 active+clean; 1.3 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 39 MiB/s rd, 185 MiB/s wr, 502 op/s 2024-03-16T10:45:02.733 INFO:tasks.workunit.client.1.smithi026.stdout:0/51: rename d1/d5/d6/lb to d1/d5/l10 0 2024-03-16T10:45:02.734 INFO:tasks.workunit.client.1.smithi026.stdout:6/47: rename d0/d10/l8 to d0/d10/l16 0 2024-03-16T10:45:02.735 INFO:tasks.workunit.client.1.smithi026.stdout:0/52: mkdir d1/d5/d11 0 2024-03-16T10:45:02.735 INFO:tasks.workunit.client.1.smithi026.stdout:6/48: mkdir d0/d10/d17 0 2024-03-16T10:45:02.736 INFO:tasks.workunit.client.1.smithi026.stdout:6/49: write d0/f11 [1894569,59378] 0 2024-03-16T10:45:02.736 INFO:tasks.workunit.client.1.smithi026.stdout:0/53: mkdir d1/d12 0 2024-03-16T10:45:02.738 INFO:tasks.workunit.client.1.smithi026.stdout:0/54: symlink d1/d12/l13 0 2024-03-16T10:45:02.742 INFO:tasks.workunit.client.0.smithi002.stdout:0/636: mknod d3/d31/d75/db4/ccc 0 2024-03-16T10:45:02.742 INFO:tasks.workunit.client.0.smithi002.stdout:5/530: write d0/d1f/f36 [3171825,56969] 0 2024-03-16T10:45:02.743 INFO:tasks.workunit.client.0.smithi002.stdout:7/537: creat d3/d4/d1c/d5b/d64/fc1 x:0 0 0 2024-03-16T10:45:02.744 INFO:tasks.workunit.client.0.smithi002.stdout:6/615: creat d15/d80/fd5 x:0 0 0 2024-03-16T10:45:02.746 INFO:tasks.workunit.client.0.smithi002.stdout:2/605: dwrite d7/db/d12/f22 [0,4194304] 0 2024-03-16T10:45:02.747 INFO:tasks.workunit.client.1.smithi026.stdout:6/50: fdatasync d0/d10/ff 0 2024-03-16T10:45:02.752 INFO:tasks.workunit.client.1.smithi026.stdout:6/51: creat d0/d10/d17/f18 x:0 0 0 2024-03-16T10:45:02.756 INFO:tasks.workunit.client.0.smithi002.stdout:6/616: rmdir d15 39 2024-03-16T10:45:02.756 INFO:tasks.workunit.client.0.smithi002.stdout:0/637: mknod d3/d59/dca/ccd 0 2024-03-16T10:45:02.758 INFO:tasks.workunit.client.0.smithi002.stdout:2/606: write d7/d2e/d28/d44/f76 [5012153,37273] 0 2024-03-16T10:45:02.758 INFO:tasks.workunit.client.0.smithi002.stdout:0/638: stat d3/de/f13 0 2024-03-16T10:45:02.758 INFO:tasks.workunit.client.0.smithi002.stdout:1/624: dread d2/df/d12/d84/d87/d96/db5/f4b [0,4194304] 0 2024-03-16T10:45:02.758 INFO:tasks.workunit.client.0.smithi002.stdout:7/538: symlink d3/d4/d1d/d45/lc2 0 2024-03-16T10:45:02.760 INFO:tasks.workunit.client.0.smithi002.stdout:1/625: fdatasync d2/df/d12/d84/d87/d96/db5/d1c/d5d/f82 0 2024-03-16T10:45:02.760 INFO:tasks.workunit.client.1.smithi026.stdout:6/52: dwrite d0/f11 [0,4194304] 0 2024-03-16T10:45:02.760 INFO:tasks.workunit.client.1.smithi026.stdout:0/55: truncate d1/d5/f3 219540 0 2024-03-16T10:45:02.761 INFO:tasks.workunit.client.1.smithi026.stdout:0/56: stat d1/d5/d6/cf 0 2024-03-16T10:45:02.767 INFO:tasks.workunit.client.0.smithi002.stdout:5/531: mkdir d0/d1f/da1 0 2024-03-16T10:45:02.769 INFO:tasks.workunit.client.0.smithi002.stdout:0/639: dwrite d3/d28/d3d/f46 [0,4194304] 0 2024-03-16T10:45:02.770 INFO:tasks.workunit.client.0.smithi002.stdout:1/626: dwrite d2/df/d12/d84/d87/d96/db5/f7c [0,4194304] 0 2024-03-16T10:45:02.770 INFO:tasks.workunit.client.0.smithi002.stdout:3/608: rename d1/dd/d28/d7b/d2c/fbb to d1/db6/d15/fd3 0 2024-03-16T10:45:02.772 INFO:tasks.workunit.client.0.smithi002.stdout:7/539: rmdir d3/d37 39 2024-03-16T10:45:02.773 INFO:tasks.workunit.client.1.smithi026.stdout:0/57: mknod d1/d5/d11/c14 0 2024-03-16T10:45:02.779 INFO:tasks.workunit.client.1.smithi026.stdout:0/58: creat d1/f15 x:0 0 0 2024-03-16T10:45:02.781 INFO:tasks.workunit.client.1.smithi026.stdout:0/59: chown d1/d5/f3 99210 1 2024-03-16T10:45:02.781 INFO:tasks.workunit.client.0.smithi002.stdout:8/699: link d7/d13/db8/ld2 d7/d9/d25/d91/le9 0 2024-03-16T10:45:02.784 INFO:tasks.workunit.client.1.smithi026.stdout:6/53: rmdir d0/d10/d17 39 2024-03-16T10:45:02.789 INFO:tasks.workunit.client.0.smithi002.stdout:8/700: dwrite d7/d9/d25/dc9/fcd [0,4194304] 0 2024-03-16T10:45:02.793 INFO:tasks.workunit.client.0.smithi002.stdout:0/640: write d3/d59/fb0 [673718,78788] 0 2024-03-16T10:45:02.797 INFO:tasks.workunit.client.0.smithi002.stdout:1/627: rmdir d2/df/d12/d84/d87/d96/db5/d51/d56 39 2024-03-16T10:45:02.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:02 smithi002 ceph-mon[29248]: pgmap v34: 65 pgs: 65 active+clean; 1.3 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 39 MiB/s rd, 185 MiB/s wr, 502 op/s 2024-03-16T10:45:02.799 INFO:tasks.workunit.client.0.smithi002.stdout:6/617: dread d15/d2d/d2c/f56 [0,4194304] 0 2024-03-16T10:45:02.799 INFO:tasks.workunit.client.0.smithi002.stdout:6/618: chown d15/d2d/d73 130849 1 2024-03-16T10:45:02.800 INFO:tasks.workunit.client.0.smithi002.stdout:2/607: dwrite d7/d2e/d1d/f21 [4194304,4194304] 0 2024-03-16T10:45:02.800 INFO:tasks.workunit.client.1.smithi026.stdout:0/60: creat d1/d5/d11/f16 x:0 0 0 2024-03-16T10:45:02.805 INFO:tasks.workunit.client.1.smithi026.stdout:6/54: dwrite d0/f11 [0,4194304] 0 2024-03-16T10:45:02.806 INFO:tasks.workunit.client.1.smithi026.stdout:6/55: dread - d0/f12 zero size 2024-03-16T10:45:02.806 INFO:tasks.workunit.client.0.smithi002.stdout:1/628: dwrite d2/df/d12/d84/d87/d96/db5/fab [0,4194304] 0 2024-03-16T10:45:02.809 INFO:tasks.workunit.client.0.smithi002.stdout:5/532: mknod d0/d16/d1d/d33/d4b/ca2 0 2024-03-16T10:45:02.812 INFO:tasks.workunit.client.0.smithi002.stdout:8/701: dread d7/d9/f94 [0,4194304] 0 2024-03-16T10:45:02.813 INFO:tasks.workunit.client.0.smithi002.stdout:3/609: creat d1/db6/d15/dcc/fd4 x:0 0 0 2024-03-16T10:45:02.813 INFO:tasks.workunit.client.0.smithi002.stdout:8/702: chown d7/d13/db8/f86 22542299 1 2024-03-16T10:45:02.816 INFO:tasks.workunit.client.0.smithi002.stdout:0/641: rename d3/d28/d65/c78 to d3/d31/daa/cce 0 2024-03-16T10:45:02.818 INFO:tasks.workunit.client.0.smithi002.stdout:6/619: dread d15/f4f [0,4194304] 0 2024-03-16T10:45:02.823 INFO:tasks.workunit.client.0.smithi002.stdout:7/540: mkdir d3/d4/d1d/d87/d5c/dc3 0 2024-03-16T10:45:02.823 INFO:tasks.workunit.client.0.smithi002.stdout:7/541: write d3/f91 [2932661,69551] 0 2024-03-16T10:45:02.825 INFO:tasks.workunit.client.0.smithi002.stdout:7/542: write d3/d61/f94 [535586,7298] 0 2024-03-16T10:45:02.826 INFO:tasks.workunit.client.0.smithi002.stdout:4/588: getdents d2/d33/da/d5e/d93 0 2024-03-16T10:45:02.827 INFO:tasks.workunit.client.0.smithi002.stdout:7/543: read d3/d4/d1d/d87/d5c/d63/f7f [795377,113805] 0 2024-03-16T10:45:02.827 INFO:tasks.workunit.client.0.smithi002.stdout:4/589: write d2/d33/d1c/d45/faa [105487,54936] 0 2024-03-16T10:45:02.829 INFO:tasks.workunit.client.0.smithi002.stdout:1/629: creat d2/d58/dbb/fc4 x:0 0 0 2024-03-16T10:45:02.830 INFO:tasks.workunit.client.0.smithi002.stdout:4/590: dread d2/d33/da/f49 [0,4194304] 0 2024-03-16T10:45:02.833 INFO:tasks.workunit.client.0.smithi002.stdout:8/703: mkdir d7/d9/d17/d23/d2e/d34/dbf/dd9/dea 0 2024-03-16T10:45:02.836 INFO:tasks.workunit.client.0.smithi002.stdout:0/642: mknod d3/d31/ccf 0 2024-03-16T10:45:02.838 INFO:tasks.workunit.client.0.smithi002.stdout:6/620: unlink d15/d2d/d2b/c94 0 2024-03-16T10:45:02.839 INFO:tasks.workunit.client.0.smithi002.stdout:6/621: fdatasync d15/d2d/d73/d97/fbf 0 2024-03-16T10:45:02.840 INFO:tasks.workunit.client.0.smithi002.stdout:6/622: write d15/d2d/d2b/f35 [924567,32041] 0 2024-03-16T10:45:02.841 INFO:tasks.workunit.client.0.smithi002.stdout:2/608: mknod d7/d2e/cb7 0 2024-03-16T10:45:02.844 INFO:tasks.workunit.client.0.smithi002.stdout:7/544: truncate d3/fd 163972 0 2024-03-16T10:45:02.848 INFO:tasks.workunit.client.0.smithi002.stdout:5/533: rename d0/c88 to d0/d16/d4a/d39/ca3 0 2024-03-16T10:45:02.848 INFO:tasks.workunit.client.0.smithi002.stdout:8/704: rename d7/d9 to d7/d9/d17/d23/d39/dd4/deb 22 2024-03-16T10:45:02.849 INFO:tasks.workunit.client.0.smithi002.stdout:0/643: rmdir d3/d31/daa 39 2024-03-16T10:45:02.851 INFO:tasks.workunit.client.0.smithi002.stdout:6/623: unlink fd 0 2024-03-16T10:45:02.852 INFO:tasks.workunit.client.0.smithi002.stdout:0/644: dread d3/d28/f36 [0,4194304] 0 2024-03-16T10:45:02.852 INFO:tasks.workunit.client.0.smithi002.stdout:6/624: dread - d15/d2d/d2c/d8f/fb8 zero size 2024-03-16T10:45:02.853 INFO:tasks.workunit.client.0.smithi002.stdout:2/609: unlink d7/f4c 0 2024-03-16T10:45:02.853 INFO:tasks.workunit.client.0.smithi002.stdout:6/625: fdatasync d15/d2d/d8a/fae 0 2024-03-16T10:45:02.853 INFO:tasks.workunit.client.0.smithi002.stdout:0/645: read d3/d59/fb0 [317299,87939] 0 2024-03-16T10:45:02.855 INFO:tasks.workunit.client.0.smithi002.stdout:7/545: symlink d3/d61/lc4 0 2024-03-16T10:45:02.857 INFO:tasks.workunit.client.0.smithi002.stdout:5/534: chown d0/d24/l40 702 1 2024-03-16T10:45:02.858 INFO:tasks.workunit.client.0.smithi002.stdout:1/630: rename d2/f52 to d2/df/d12/d84/d87/d96/db5/d51/d56/fc5 0 2024-03-16T10:45:02.863 INFO:tasks.workunit.client.0.smithi002.stdout:2/610: stat d7/d27/d3b/c53 0 2024-03-16T10:45:02.866 INFO:tasks.workunit.client.0.smithi002.stdout:7/546: mknod d3/d4/d1c/cc5 0 2024-03-16T10:45:02.868 INFO:tasks.workunit.client.0.smithi002.stdout:5/535: rmdir d0/d16/d4a 39 2024-03-16T10:45:02.874 INFO:tasks.workunit.client.0.smithi002.stdout:1/631: symlink d2/d58/lc6 0 2024-03-16T10:45:02.877 INFO:tasks.workunit.client.0.smithi002.stdout:2/611: dread d7/d2e/d1d/f3e [0,4194304] 0 2024-03-16T10:45:02.878 INFO:tasks.workunit.client.0.smithi002.stdout:5/536: dread d0/d16/d1d/f23 [0,4194304] 0 2024-03-16T10:45:02.885 INFO:tasks.workunit.client.0.smithi002.stdout:7/547: rmdir d3/d61/d85 39 2024-03-16T10:45:02.891 INFO:tasks.workunit.client.0.smithi002.stdout:5/537: dread d0/d16/d1d/d2b/d5d/f64 [0,4194304] 0 2024-03-16T10:45:02.892 INFO:tasks.workunit.client.0.smithi002.stdout:5/538: write d0/d16/d1d/f3b [4595761,69067] 0 2024-03-16T10:45:02.902 INFO:tasks.workunit.client.0.smithi002.stdout:7/548: dwrite d3/d4/d1d/d87/f7e [0,4194304] 0 2024-03-16T10:45:02.907 INFO:tasks.workunit.client.0.smithi002.stdout:5/539: chown d0/d16/d1d/d33/c50 802024531 1 2024-03-16T10:45:02.908 INFO:tasks.workunit.client.0.smithi002.stdout:0/646: getdents d3/d27/d90/dbb 0 2024-03-16T10:45:02.909 INFO:tasks.workunit.client.0.smithi002.stdout:0/647: write d3/d28/d65/f77 [844804,121284] 0 2024-03-16T10:45:02.915 INFO:tasks.workunit.client.0.smithi002.stdout:7/549: chown d3/d61/d85/fb6 2285 1 2024-03-16T10:45:02.916 INFO:tasks.workunit.client.0.smithi002.stdout:2/612: link d7/d2e/d1d/d2f/l81 d7/db/d70/d72/lb8 0 2024-03-16T10:45:02.920 INFO:tasks.workunit.client.0.smithi002.stdout:0/648: mknod d3/da0/cd0 0 2024-03-16T10:45:02.922 INFO:tasks.workunit.client.0.smithi002.stdout:2/613: dread d7/d2e/d1d/d38/f3a [0,4194304] 0 2024-03-16T10:45:02.922 INFO:tasks.workunit.client.0.smithi002.stdout:6/626: dwrite d15/f6b [0,4194304] 0 2024-03-16T10:45:02.922 INFO:tasks.workunit.client.0.smithi002.stdout:0/649: truncate d3/da0/fa3 740291 0 2024-03-16T10:45:02.922 INFO:tasks.workunit.client.0.smithi002.stdout:1/632: link d2/df/d12/d84/d87/d96/db5/l6 d2/df/d12/d84/d87/d96/db5/d1c/lc7 0 2024-03-16T10:45:02.924 INFO:tasks.workunit.client.0.smithi002.stdout:1/633: fdatasync d2/df/d12/d84/d87/d96/db5/d1c/d5d/f82 0 2024-03-16T10:45:02.924 INFO:tasks.workunit.client.0.smithi002.stdout:7/550: rmdir d3/d4/d1d/d87/d5c/d63/d34 39 2024-03-16T10:45:02.928 INFO:tasks.workunit.client.0.smithi002.stdout:5/540: symlink d0/d1f/la4 0 2024-03-16T10:45:02.929 INFO:tasks.workunit.client.0.smithi002.stdout:6/627: dread - d15/d2d/d2c/f7a zero size 2024-03-16T10:45:02.929 INFO:tasks.workunit.client.0.smithi002.stdout:7/551: dread d3/d61/d66/f4f [4194304,4194304] 0 2024-03-16T10:45:02.931 INFO:tasks.workunit.client.0.smithi002.stdout:0/650: mknod d3/d28/d2a/da1/d9c/da9/cd1 0 2024-03-16T10:45:02.935 INFO:tasks.workunit.client.0.smithi002.stdout:1/634: mknod d2/d58/dbb/cc8 0 2024-03-16T10:45:02.938 INFO:tasks.workunit.client.0.smithi002.stdout:6/628: mknod d15/d2d/da8/cd6 0 2024-03-16T10:45:02.939 INFO:tasks.workunit.client.0.smithi002.stdout:6/629: chown d15/d2d/d2b/d3f/f6d 30044 1 2024-03-16T10:45:02.940 INFO:tasks.workunit.client.0.smithi002.stdout:6/630: read d15/d2d/d2c/f46 [1211690,69885] 0 2024-03-16T10:45:02.942 INFO:tasks.workunit.client.0.smithi002.stdout:0/651: creat d3/de/da2/fd2 x:0 0 0 2024-03-16T10:45:02.944 INFO:tasks.workunit.client.0.smithi002.stdout:1/635: mkdir d2/df/d12/d84/d87/d96/db5/dae/dc9 0 2024-03-16T10:45:02.945 INFO:tasks.workunit.client.0.smithi002.stdout:5/541: creat d0/d16/d1d/d33/d8c/fa5 x:0 0 0 2024-03-16T10:45:02.946 INFO:tasks.workunit.client.0.smithi002.stdout:5/542: stat d0/d16/d1d/f23 0 2024-03-16T10:45:02.947 INFO:tasks.workunit.client.0.smithi002.stdout:5/543: dread - d0/d16/d1d/d43/d48/f8f zero size 2024-03-16T10:45:02.950 INFO:tasks.workunit.client.0.smithi002.stdout:7/552: mkdir d3/d4/d1d/d87/d5c/d63/d3f/dba/dc6 0 2024-03-16T10:45:02.952 INFO:tasks.workunit.client.0.smithi002.stdout:6/631: creat d15/dab/dc4/fd7 x:0 0 0 2024-03-16T10:45:02.953 INFO:tasks.workunit.client.0.smithi002.stdout:6/632: chown d15/d2d/d2b/d75/fce 160572182 1 2024-03-16T10:45:02.954 INFO:tasks.workunit.client.0.smithi002.stdout:1/636: dread d2/df/d12/d84/d87/d96/db5/d51/d56/f8c [0,4194304] 0 2024-03-16T10:45:02.959 INFO:tasks.workunit.client.0.smithi002.stdout:6/633: unlink d15/d2d/l4c 0 2024-03-16T10:45:02.960 INFO:tasks.workunit.client.0.smithi002.stdout:1/637: dwrite d2/df/d12/d84/d87/d96/f9c [0,4194304] 0 2024-03-16T10:45:02.961 INFO:tasks.workunit.client.0.smithi002.stdout:0/652: symlink d3/ld3 0 2024-03-16T10:45:02.963 INFO:tasks.workunit.client.0.smithi002.stdout:7/553: mkdir d3/d4/d1d/d87/d5c/dc3/dc7 0 2024-03-16T10:45:02.966 INFO:tasks.workunit.client.0.smithi002.stdout:1/638: stat d2/df/f45 0 2024-03-16T10:45:02.968 INFO:tasks.workunit.client.0.smithi002.stdout:0/653: dwrite d3/d5/d7c/f8a [0,4194304] 0 2024-03-16T10:45:02.973 INFO:tasks.workunit.client.0.smithi002.stdout:5/544: unlink d0/d16/d4a/d39/ca3 0 2024-03-16T10:45:02.976 INFO:tasks.workunit.client.0.smithi002.stdout:6/634: rename d15/d2d/d2c/d86/ca5 to d15/dab/cd8 0 2024-03-16T10:45:02.977 INFO:tasks.workunit.client.0.smithi002.stdout:6/635: chown d15/d2d/d2b/d75/l8b 4139966 1 2024-03-16T10:45:02.981 INFO:tasks.workunit.client.0.smithi002.stdout:1/639: creat d2/df/d12/d29/d5c/fca x:0 0 0 2024-03-16T10:45:02.983 INFO:tasks.workunit.client.0.smithi002.stdout:0/654: rmdir d3/d28/d2a/d67 39 2024-03-16T10:45:02.984 INFO:tasks.workunit.client.0.smithi002.stdout:0/655: truncate d3/de/da2/fd2 290791 0 2024-03-16T10:45:02.986 INFO:tasks.workunit.client.0.smithi002.stdout:5/545: symlink d0/d16/d4a/d39/la6 0 2024-03-16T10:45:02.994 INFO:tasks.workunit.client.0.smithi002.stdout:1/640: creat d2/df/d12/d84/d87/d96/fcb x:0 0 0 2024-03-16T10:45:02.995 INFO:tasks.workunit.client.0.smithi002.stdout:1/641: readlink d2/d58/lc6 0 2024-03-16T10:45:02.998 INFO:tasks.workunit.client.0.smithi002.stdout:5/546: creat d0/d16/d1d/d2b/d5d/fa7 x:0 0 0 2024-03-16T10:45:03.003 INFO:tasks.workunit.client.0.smithi002.stdout:0/656: symlink d3/d28/ld4 0 2024-03-16T10:45:03.011 INFO:tasks.workunit.client.0.smithi002.stdout:6/636: creat d15/d2d/d2b/fd9 x:0 0 0 2024-03-16T10:45:03.012 INFO:tasks.workunit.client.0.smithi002.stdout:5/547: dwrite d0/f18 [4194304,4194304] 0 2024-03-16T10:45:03.015 INFO:tasks.workunit.client.0.smithi002.stdout:0/657: unlink d3/fcb 0 2024-03-16T10:45:03.018 INFO:tasks.workunit.client.0.smithi002.stdout:1/642: rename d2/df/d12/d84/d87/d96/db5/d1c/c43 to d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/ccc 0 2024-03-16T10:45:03.027 INFO:tasks.workunit.client.0.smithi002.stdout:5/548: creat d0/d16/d1d/d33/d8c/fa8 x:0 0 0 2024-03-16T10:45:03.029 INFO:tasks.workunit.client.0.smithi002.stdout:0/658: chown d3/l8 2 1 2024-03-16T10:45:03.035 INFO:tasks.workunit.client.0.smithi002.stdout:5/549: symlink d0/d16/d1d/d2b/d5d/la9 0 2024-03-16T10:45:03.035 INFO:tasks.workunit.client.0.smithi002.stdout:5/550: chown d0/f6f 97 1 2024-03-16T10:45:03.036 INFO:tasks.workunit.client.0.smithi002.stdout:9/648: sync 2024-03-16T10:45:03.037 INFO:tasks.workunit.client.0.smithi002.stdout:9/649: dread - d5/fc4 zero size 2024-03-16T10:45:03.039 INFO:tasks.workunit.client.0.smithi002.stdout:9/650: read - d5/d9/dc/d1e/d91/f84 zero size 2024-03-16T10:45:03.040 INFO:tasks.workunit.client.0.smithi002.stdout:9/651: write d5/d9/dc/fa2 [3803611,67740] 0 2024-03-16T10:45:03.040 INFO:tasks.workunit.client.0.smithi002.stdout:5/551: dwrite d0/d16/f37 [0,4194304] 0 2024-03-16T10:45:03.100 INFO:tasks.workunit.client.0.smithi002.stdout:7/554: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f46 [0,4194304] 0 2024-03-16T10:45:03.102 INFO:tasks.workunit.client.0.smithi002.stdout:4/591: sync 2024-03-16T10:45:03.102 INFO:tasks.workunit.client.0.smithi002.stdout:3/610: sync 2024-03-16T10:45:03.118 INFO:tasks.workunit.client.0.smithi002.stdout:9/652: symlink d5/d9/dc/lc7 0 2024-03-16T10:45:03.127 INFO:tasks.workunit.client.0.smithi002.stdout:3/611: dwrite d1/dd/d28/f6a [0,4194304] 0 2024-03-16T10:45:03.128 INFO:tasks.workunit.client.0.smithi002.stdout:4/592: dread d2/d33/da/f4f [0,4194304] 0 2024-03-16T10:45:03.135 INFO:tasks.workunit.client.0.smithi002.stdout:6/637: dwrite d15/d2d/f20 [0,4194304] 0 2024-03-16T10:45:03.140 INFO:tasks.workunit.client.0.smithi002.stdout:3/612: chown d1/db6/d15/f53 501 1 2024-03-16T10:45:03.150 INFO:tasks.workunit.client.0.smithi002.stdout:7/555: link d3/d37/c9d d3/d4/d1d/d87/d5c/d63/d55/cc8 0 2024-03-16T10:45:03.153 INFO:tasks.workunit.client.0.smithi002.stdout:2/614: sync 2024-03-16T10:45:03.153 INFO:tasks.workunit.client.0.smithi002.stdout:8/705: sync 2024-03-16T10:45:03.157 INFO:tasks.workunit.client.0.smithi002.stdout:7/556: symlink d3/d4/d1c/d5b/d64/d99/lc9 0 2024-03-16T10:45:03.160 INFO:tasks.workunit.client.0.smithi002.stdout:8/706: write d7/fd [1674041,127735] 0 2024-03-16T10:45:03.168 INFO:tasks.workunit.client.0.smithi002.stdout:7/557: creat d3/d4/d1c/d5b/fca x:0 0 0 2024-03-16T10:45:03.174 INFO:tasks.workunit.client.0.smithi002.stdout:6/638: getdents d15/d2d/d2b 0 2024-03-16T10:45:03.176 INFO:tasks.workunit.client.0.smithi002.stdout:8/707: symlink d7/d9/d17/d23/d2e/d34/dde/lec 0 2024-03-16T10:45:03.180 INFO:tasks.workunit.client.0.smithi002.stdout:8/708: creat d7/d9/d17/dc3/fed x:0 0 0 2024-03-16T10:45:03.181 INFO:tasks.workunit.client.0.smithi002.stdout:8/709: stat d7/d9/d17/d20/f63 0 2024-03-16T10:45:03.194 INFO:tasks.workunit.client.0.smithi002.stdout:8/710: getdents d7/d9/d17/d23/d2e/d90 0 2024-03-16T10:45:03.199 INFO:tasks.workunit.client.0.smithi002.stdout:8/711: dread d7/d13/db8/f61 [0,4194304] 0 2024-03-16T10:45:03.199 INFO:tasks.workunit.client.0.smithi002.stdout:4/593: rmdir d2/d6/d32/d66 39 2024-03-16T10:45:03.199 INFO:tasks.workunit.client.0.smithi002.stdout:4/594: readlink d2/d33/d1c/l26 0 2024-03-16T10:45:03.202 INFO:tasks.workunit.client.0.smithi002.stdout:8/712: write d7/d9/d17/d23/d2e/d3a/f47 [1873357,110695] 0 2024-03-16T10:45:03.203 INFO:tasks.workunit.client.0.smithi002.stdout:8/713: write d7/d9/d17/dc3/fed [643529,74348] 0 2024-03-16T10:45:03.204 INFO:tasks.workunit.client.0.smithi002.stdout:8/714: dread - d7/d9/fb5 zero size 2024-03-16T10:45:03.206 INFO:tasks.workunit.client.0.smithi002.stdout:9/653: creat d5/d9/d55/fc8 x:0 0 0 2024-03-16T10:45:03.213 INFO:tasks.workunit.client.0.smithi002.stdout:9/654: dwrite d5/d9/dc/f92 [0,4194304] 0 2024-03-16T10:45:03.222 INFO:tasks.workunit.client.0.smithi002.stdout:1/643: rename d2/df/d12/d84/d87/d96/db5/d1c/c4f to d2/df/d12/d29/ccd 0 2024-03-16T10:45:03.227 INFO:tasks.workunit.client.0.smithi002.stdout:1/644: creat d2/df/d12/d84/d87/d96/db5/d1c/d5d/fce x:0 0 0 2024-03-16T10:45:03.235 INFO:tasks.workunit.client.0.smithi002.stdout:1/645: dwrite d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/faa [0,4194304] 0 2024-03-16T10:45:03.242 INFO:tasks.workunit.client.0.smithi002.stdout:1/646: symlink d2/df/lcf 0 2024-03-16T10:45:03.254 INFO:tasks.workunit.client.0.smithi002.stdout:3/613: write d1/d5a/f75 [1529298,242] 0 2024-03-16T10:45:03.255 INFO:tasks.workunit.client.0.smithi002.stdout:3/614: fdatasync d1/dd/f68 0 2024-03-16T10:45:03.258 INFO:tasks.workunit.client.0.smithi002.stdout:1/647: dread d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f79 [0,4194304] 0 2024-03-16T10:45:03.262 INFO:tasks.workunit.client.0.smithi002.stdout:3/615: chown d1/dd/d28/d37/f8d 66409 1 2024-03-16T10:45:03.263 INFO:tasks.workunit.client.0.smithi002.stdout:1/648: write d2/df/d12/d84/d87/d96/da8/fa9 [1316514,34114] 0 2024-03-16T10:45:03.264 INFO:tasks.workunit.client.0.smithi002.stdout:3/616: dread - d1/db6/d15/d87/d8c/fb3 zero size 2024-03-16T10:45:03.267 INFO:tasks.workunit.client.0.smithi002.stdout:1/649: dread d2/df/d12/f37 [4194304,4194304] 0 2024-03-16T10:45:03.272 INFO:tasks.workunit.client.1.smithi026.stdout:0/61: mkdir d1/d5/d6/d17 0 2024-03-16T10:45:03.278 INFO:tasks.workunit.client.1.smithi026.stdout:0/62: dwrite d1/f15 [0,4194304] 0 2024-03-16T10:45:03.279 INFO:tasks.workunit.client.0.smithi002.stdout:6/639: write d15/d2d/d2c/f50 [734223,80072] 0 2024-03-16T10:45:03.280 INFO:tasks.workunit.client.0.smithi002.stdout:6/640: readlink lf 0 2024-03-16T10:45:03.281 INFO:tasks.workunit.client.0.smithi002.stdout:6/641: chown d15/d2d/d2b/dc1 764 1 2024-03-16T10:45:03.286 INFO:tasks.workunit.client.1.smithi026.stdout:0/63: dwrite d1/f15 [0,4194304] 0 2024-03-16T10:45:03.290 INFO:tasks.workunit.client.1.smithi026.stdout:0/64: chown d1/ca 3 1 2024-03-16T10:45:03.294 INFO:tasks.workunit.client.0.smithi002.stdout:1/650: creat d2/df/d12/d84/d87/d96/da8/fd0 x:0 0 0 2024-03-16T10:45:03.296 INFO:tasks.workunit.client.0.smithi002.stdout:1/651: write d2/df/d12/d84/d87/d96/db5/d1c/d5d/fb9 [427692,120273] 0 2024-03-16T10:45:03.297 INFO:tasks.workunit.client.1.smithi026.stdout:0/65: rename d1/d5/d6/d17 to d1/d18 0 2024-03-16T10:45:03.300 INFO:tasks.workunit.client.0.smithi002.stdout:3/617: mkdir d1/dd/d28/d7b/dd5 0 2024-03-16T10:45:03.300 INFO:tasks.workunit.client.1.smithi026.stdout:0/66: fdatasync d1/d5/f3 0 2024-03-16T10:45:03.302 INFO:tasks.workunit.client.0.smithi002.stdout:6/642: symlink d15/d2d/lda 0 2024-03-16T10:45:03.304 INFO:tasks.workunit.client.0.smithi002.stdout:1/652: dwrite d2/df/d12/d84/d87/d96/db5/fc [4194304,4194304] 0 2024-03-16T10:45:03.304 INFO:tasks.workunit.client.0.smithi002.stdout:3/618: creat d1/dd/d28/d7b/d2c/d97/fd6 x:0 0 0 2024-03-16T10:45:03.305 INFO:tasks.workunit.client.0.smithi002.stdout:6/643: symlink d15/d2d/d2c/d8f/d96/ldb 0 2024-03-16T10:45:03.306 INFO:tasks.workunit.client.0.smithi002.stdout:3/619: chown d1/dd/d28/d7b/d2c/l41 16601 1 2024-03-16T10:45:03.307 INFO:tasks.workunit.client.1.smithi026.stdout:0/67: dwrite d1/d5/d11/f16 [0,4194304] 0 2024-03-16T10:45:03.308 INFO:tasks.workunit.client.1.smithi026.stdout:0/68: dread - d1/fd zero size 2024-03-16T10:45:03.308 INFO:tasks.workunit.client.0.smithi002.stdout:6/644: stat d15/l78 0 2024-03-16T10:45:03.309 INFO:tasks.workunit.client.0.smithi002.stdout:1/653: dread d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:45:03.309 INFO:tasks.workunit.client.0.smithi002.stdout:6/645: write d15/f29 [2979144,29574] 0 2024-03-16T10:45:03.315 INFO:tasks.workunit.client.0.smithi002.stdout:3/620: mkdir d1/dd/d28/d7b/d8a/dd7 0 2024-03-16T10:45:03.320 INFO:tasks.workunit.client.0.smithi002.stdout:3/621: mkdir d1/dd/d74/dd8 0 2024-03-16T10:45:03.325 INFO:tasks.workunit.client.1.smithi026.stdout:0/69: getdents d1/d12 0 2024-03-16T10:45:03.327 INFO:tasks.workunit.client.0.smithi002.stdout:6/646: link c3 d15/d2d/d2b/d75/cdc 0 2024-03-16T10:45:03.329 INFO:tasks.workunit.client.1.smithi026.stdout:0/70: creat d1/d5/d6/f19 x:0 0 0 2024-03-16T10:45:03.329 INFO:tasks.workunit.client.0.smithi002.stdout:1/654: link d2/df/d12/d84/d87/d96/db5/l2e d2/df/d12/d84/ld1 0 2024-03-16T10:45:03.330 INFO:tasks.workunit.client.0.smithi002.stdout:1/655: stat d2/df/d12/f4e 0 2024-03-16T10:45:03.331 INFO:tasks.workunit.client.0.smithi002.stdout:3/622: truncate d1/da5/fc3 797002 0 2024-03-16T10:45:03.332 INFO:tasks.workunit.client.0.smithi002.stdout:3/623: write d1/dd/d28/f55 [327049,68775] 0 2024-03-16T10:45:03.332 INFO:tasks.workunit.client.0.smithi002.stdout:6/647: chown d15/d2d/d2b/d3f/d77/d9f/c9d 0 1 2024-03-16T10:45:03.333 INFO:tasks.workunit.client.0.smithi002.stdout:3/624: write d1/db6/f7f [850375,52406] 0 2024-03-16T10:45:03.335 INFO:tasks.workunit.client.0.smithi002.stdout:1/656: truncate d2/df/f45 891733 0 2024-03-16T10:45:03.336 INFO:tasks.workunit.client.1.smithi026.stdout:0/71: dwrite d1/d5/d11/f16 [0,4194304] 0 2024-03-16T10:45:03.339 INFO:tasks.workunit.client.0.smithi002.stdout:6/648: dwrite d15/d2d/d2b/f35 [0,4194304] 0 2024-03-16T10:45:03.340 INFO:tasks.workunit.client.0.smithi002.stdout:3/625: unlink d1/dd/d28/d7b/d8a/d93/c5c 0 2024-03-16T10:45:03.341 INFO:tasks.workunit.client.0.smithi002.stdout:3/626: chown d1/dd/d28/d7b/d22/l70 6707 1 2024-03-16T10:45:03.342 INFO:tasks.workunit.client.1.smithi026.stdout:0/72: mkdir d1/d18/d1a 0 2024-03-16T10:45:03.343 INFO:tasks.workunit.client.0.smithi002.stdout:0/659: rename d3/ld3 to d3/d31/daa/ld5 0 2024-03-16T10:45:03.344 INFO:tasks.workunit.client.0.smithi002.stdout:6/649: creat d15/d2d/d73/d97/fdd x:0 0 0 2024-03-16T10:45:03.345 INFO:tasks.workunit.client.0.smithi002.stdout:3/627: write d1/dd/d28/d7b/f77 [135421,105185] 0 2024-03-16T10:45:03.347 INFO:tasks.workunit.client.1.smithi026.stdout:0/73: creat d1/d18/f1b x:0 0 0 2024-03-16T10:45:03.348 INFO:tasks.workunit.client.1.smithi026.stdout:0/74: read - d1/fd zero size 2024-03-16T10:45:03.350 INFO:tasks.workunit.client.0.smithi002.stdout:0/660: mkdir d3/da0/dd6 0 2024-03-16T10:45:03.351 INFO:tasks.workunit.client.0.smithi002.stdout:3/628: dwrite d1/dd/f68 [0,4194304] 0 2024-03-16T10:45:03.357 INFO:tasks.workunit.client.0.smithi002.stdout:5/552: rename d0/d6b to d0/d16/d1d/daa 0 2024-03-16T10:45:03.358 INFO:tasks.workunit.client.0.smithi002.stdout:3/629: mkdir d1/dd/d28/d7b/d8a/d93/dd9 0 2024-03-16T10:45:03.359 INFO:tasks.workunit.client.0.smithi002.stdout:6/650: dwrite d15/d2d/d2c/d8f/d96/db5/fc7 [0,4194304] 0 2024-03-16T10:45:03.364 INFO:tasks.workunit.client.1.smithi026.stdout:0/75: link d1/d5/l10 d1/l1c 0 2024-03-16T10:45:03.367 INFO:tasks.workunit.client.0.smithi002.stdout:2/615: rename d7/db/d70/d72/c78 to d7/d2e/d28/d40/cb9 0 2024-03-16T10:45:03.370 INFO:tasks.workunit.client.1.smithi026.stdout:0/76: symlink d1/d5/l1d 0 2024-03-16T10:45:03.370 INFO:tasks.workunit.client.0.smithi002.stdout:2/616: write d7/d2e/d1d/f26 [5390287,124697] 0 2024-03-16T10:45:03.370 INFO:tasks.workunit.client.0.smithi002.stdout:6/651: dwrite d15/d2d/d73/d97/fba [0,4194304] 0 2024-03-16T10:45:03.371 INFO:tasks.workunit.client.1.smithi026.stdout:0/77: truncate d1/d5/d6/fe 964734 0 2024-03-16T10:45:03.372 INFO:tasks.workunit.client.0.smithi002.stdout:3/630: mknod d1/da5/cda 0 2024-03-16T10:45:03.380 INFO:tasks.workunit.client.0.smithi002.stdout:7/558: rename d3/d4/f1e to d3/d61/d66/d88/fcb 0 2024-03-16T10:45:03.381 INFO:tasks.workunit.client.0.smithi002.stdout:7/559: fsync d3/f15 0 2024-03-16T10:45:03.383 INFO:tasks.workunit.client.0.smithi002.stdout:2/617: write d7/db/f35 [2493084,28992] 0 2024-03-16T10:45:03.383 INFO:tasks.workunit.client.0.smithi002.stdout:6/652: creat d15/d80/fde x:0 0 0 2024-03-16T10:45:03.385 INFO:tasks.workunit.client.0.smithi002.stdout:0/661: link d3/d27/cb7 d3/d28/d3d/cd7 0 2024-03-16T10:45:03.390 INFO:tasks.workunit.client.0.smithi002.stdout:7/560: write d3/d4/d1d/d87/d5c/d63/d34/f78 [3274925,66707] 0 2024-03-16T10:45:03.392 INFO:tasks.workunit.client.0.smithi002.stdout:2/618: creat d7/d2e/d28/d40/fba x:0 0 0 2024-03-16T10:45:03.393 INFO:tasks.workunit.client.0.smithi002.stdout:6/653: creat d15/d80/dbe/fdf x:0 0 0 2024-03-16T10:45:03.395 INFO:tasks.workunit.client.0.smithi002.stdout:4/595: rename d2/d33/da/l2a to d2/d33/da/d2e/db0/lb7 0 2024-03-16T10:45:03.395 INFO:tasks.workunit.client.0.smithi002.stdout:4/596: stat d2/d33/da/d88/db2 0 2024-03-16T10:45:03.396 INFO:tasks.workunit.client.0.smithi002.stdout:3/631: creat d1/dd/d28/d7b/d2c/fdb x:0 0 0 2024-03-16T10:45:03.397 INFO:tasks.workunit.client.0.smithi002.stdout:0/662: dread d3/d28/f3b [0,4194304] 0 2024-03-16T10:45:03.398 INFO:tasks.workunit.client.0.smithi002.stdout:2/619: creat d7/d27/d3b/d95/da5/fbb x:0 0 0 2024-03-16T10:45:03.412 INFO:tasks.workunit.client.1.smithi026.stdout:6/56: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:45:03.412 INFO:tasks.workunit.client.1.smithi026.stdout:0/78: rmdir d1/d18 39 2024-03-16T10:45:03.413 INFO:tasks.workunit.client.0.smithi002.stdout:8/715: rename d7/d13/db8/f61 to d7/d9/d17/d23/d39/dd4/fee 0 2024-03-16T10:45:03.413 INFO:tasks.workunit.client.0.smithi002.stdout:4/597: creat d2/d6/fb8 x:0 0 0 2024-03-16T10:45:03.414 INFO:tasks.workunit.client.0.smithi002.stdout:6/654: mkdir d15/d2d/d73/de0 0 2024-03-16T10:45:03.415 INFO:tasks.workunit.client.0.smithi002.stdout:6/655: chown d15/d2d/d2b/d3f/d77/d7e/c90 3672 1 2024-03-16T10:45:03.417 INFO:tasks.workunit.client.0.smithi002.stdout:2/620: read d7/d2e/d28/d44/f4d [162738,112115] 0 2024-03-16T10:45:03.418 INFO:tasks.workunit.client.1.smithi026.stdout:6/57: link d0/f12 d0/f19 0 2024-03-16T10:45:03.430 INFO:tasks.workunit.client.1.smithi026.stdout:6/58: creat d0/d10/f1a x:0 0 0 2024-03-16T10:45:03.431 INFO:tasks.workunit.client.0.smithi002.stdout:3/632: symlink d1/dd/d28/d7b/dd5/ldc 0 2024-03-16T10:45:03.431 INFO:tasks.workunit.client.0.smithi002.stdout:9/655: rename d5/d9/dc/d1e/d86 to d5/d9/d55/dc9 0 2024-03-16T10:45:03.433 INFO:tasks.workunit.client.0.smithi002.stdout:8/716: unlink d7/d9/d17/d23/d2e/l6a 0 2024-03-16T10:45:03.434 INFO:tasks.workunit.client.1.smithi026.stdout:6/59: mknod d0/c1b 0 2024-03-16T10:45:03.434 INFO:tasks.workunit.client.0.smithi002.stdout:6/656: rmdir d15/d2d/d2b/dc8 39 2024-03-16T10:45:03.434 INFO:tasks.workunit.client.1.smithi026.stdout:6/60: chown d0/f12 45940 1 2024-03-16T10:45:03.435 INFO:tasks.workunit.client.0.smithi002.stdout:3/633: chown d1/dd/d28/d7b/d2c/d63/faf 678241237 1 2024-03-16T10:45:03.435 INFO:tasks.workunit.client.0.smithi002.stdout:9/656: read d5/d9/d34/f61 [2120980,8590] 0 2024-03-16T10:45:03.437 INFO:tasks.workunit.client.0.smithi002.stdout:2/621: dwrite d7/db/f58 [0,4194304] 0 2024-03-16T10:45:03.438 INFO:tasks.workunit.client.1.smithi026.stdout:6/61: chown d0/c1 2 1 2024-03-16T10:45:03.442 INFO:tasks.workunit.client.0.smithi002.stdout:5/553: rename d0/d16/d1d to d0/d16/dab 0 2024-03-16T10:45:03.443 INFO:tasks.workunit.client.1.smithi026.stdout:6/62: symlink d0/l1c 0 2024-03-16T10:45:03.449 INFO:tasks.workunit.client.1.smithi026.stdout:6/63: creat d0/f1d x:0 0 0 2024-03-16T10:45:03.449 INFO:tasks.workunit.client.1.smithi026.stdout:6/64: write d0/f11 [71287,98374] 0 2024-03-16T10:45:03.453 INFO:tasks.workunit.client.1.smithi026.stdout:6/65: fdatasync d0/d10/d17/f18 0 2024-03-16T10:45:03.456 INFO:tasks.workunit.client.1.smithi026.stdout:6/66: dread d0/f11 [0,4194304] 0 2024-03-16T10:45:03.457 INFO:tasks.workunit.client.1.smithi026.stdout:6/67: dread - d0/f19 zero size 2024-03-16T10:45:03.466 INFO:tasks.workunit.client.1.smithi026.stdout:6/68: mknod d0/d10/d17/c1e 0 2024-03-16T10:45:03.486 INFO:tasks.workunit.client.1.smithi026.stdout:6/69: getdents d0/d10/d17 0 2024-03-16T10:45:03.487 INFO:tasks.workunit.client.1.smithi026.stdout:6/70: rmdir d0/d10 39 2024-03-16T10:45:03.491 INFO:tasks.workunit.client.1.smithi026.stdout:6/71: truncate d0/d10/ff 1394377 0 2024-03-16T10:45:03.491 INFO:tasks.workunit.client.1.smithi026.stdout:6/72: dread - d0/f12 zero size 2024-03-16T10:45:03.493 INFO:tasks.workunit.client.1.smithi026.stdout:6/73: creat d0/d10/d17/f1f x:0 0 0 2024-03-16T10:45:03.494 INFO:tasks.workunit.client.1.smithi026.stdout:6/74: write d0/f3 [126719,19198] 0 2024-03-16T10:45:03.501 INFO:tasks.workunit.client.1.smithi026.stdout:6/75: dwrite d0/d10/d17/f18 [0,4194304] 0 2024-03-16T10:45:03.570 INFO:tasks.workunit.client.0.smithi002.stdout:2/622: creat d7/d27/d3b/d95/da5/fbc x:0 0 0 2024-03-16T10:45:03.571 INFO:tasks.workunit.client.0.smithi002.stdout:8/717: fdatasync d7/d9/f6f 0 2024-03-16T10:45:03.571 INFO:tasks.workunit.client.0.smithi002.stdout:1/657: rename d2/df/d12/d29/d5c/c64 to d2/df/cd2 0 2024-03-16T10:45:03.572 INFO:tasks.workunit.client.0.smithi002.stdout:3/634: symlink d1/dd/d28/d7b/d8a/d93/dd9/ldd 0 2024-03-16T10:45:03.573 INFO:tasks.workunit.client.0.smithi002.stdout:2/623: write d7/db/d12/f5d [8266672,48720] 0 2024-03-16T10:45:03.575 INFO:tasks.workunit.client.0.smithi002.stdout:0/663: write d3/d28/d3d/f50 [418993,116261] 0 2024-03-16T10:45:03.575 INFO:tasks.workunit.client.0.smithi002.stdout:4/598: write d2/d6/f1e [3698674,127715] 0 2024-03-16T10:45:03.579 INFO:tasks.workunit.client.0.smithi002.stdout:9/657: creat d5/d9/dc/d1e/d24/d31/fca x:0 0 0 2024-03-16T10:45:03.579 INFO:tasks.workunit.client.0.smithi002.stdout:8/718: fsync d7/d9/d17/d23/d39/f8c 0 2024-03-16T10:45:03.579 INFO:tasks.workunit.client.0.smithi002.stdout:2/624: write d7/d2e/d28/d40/fba [686812,13626] 0 2024-03-16T10:45:03.580 INFO:tasks.workunit.client.0.smithi002.stdout:4/599: write d2/d6/f13 [663796,89882] 0 2024-03-16T10:45:03.581 INFO:tasks.workunit.client.0.smithi002.stdout:0/664: fdatasync d3/d5/d7c/fbe 0 2024-03-16T10:45:03.582 INFO:tasks.workunit.client.0.smithi002.stdout:5/554: dwrite d0/f6f [0,4194304] 0 2024-03-16T10:45:03.587 INFO:tasks.workunit.client.0.smithi002.stdout:8/719: read - d7/d9/d25/d91/f96 zero size 2024-03-16T10:45:03.587 INFO:tasks.workunit.client.0.smithi002.stdout:3/635: creat d1/dd/d28/d7b/fde x:0 0 0 2024-03-16T10:45:03.587 INFO:tasks.workunit.client.0.smithi002.stdout:4/600: stat d2/d33/da/d5e/d93/fac 0 2024-03-16T10:45:03.587 INFO:tasks.workunit.client.0.smithi002.stdout:4/601: chown d2/c9d 25867880 1 2024-03-16T10:45:03.587 INFO:tasks.workunit.client.0.smithi002.stdout:1/658: dread d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f79 [0,4194304] 0 2024-03-16T10:45:03.587 INFO:tasks.workunit.client.0.smithi002.stdout:9/658: symlink d5/d9/dc/d28/d8e/db5/lcb 0 2024-03-16T10:45:03.588 INFO:tasks.workunit.client.0.smithi002.stdout:1/659: dread - d2/df/d12/d84/d87/d96/fcb zero size 2024-03-16T10:45:03.589 INFO:tasks.workunit.client.0.smithi002.stdout:8/720: rmdir d7/d9/d17/d23/d39/db7/dc2 39 2024-03-16T10:45:03.590 INFO:tasks.workunit.client.0.smithi002.stdout:4/602: chown d2/d6/d32/d66 48494 1 2024-03-16T10:45:03.592 INFO:tasks.workunit.client.0.smithi002.stdout:0/665: dread d3/de/f8f [0,4194304] 0 2024-03-16T10:45:03.594 INFO:tasks.workunit.client.0.smithi002.stdout:9/659: rename d5/d9/d25/d23/f33 to d5/d9/d25/d23/fcc 0 2024-03-16T10:45:03.595 INFO:tasks.workunit.client.0.smithi002.stdout:3/636: link d1/dd/d74/fa8 d1/db6/d15/d87/d8c/fdf 0 2024-03-16T10:45:03.596 INFO:tasks.workunit.client.0.smithi002.stdout:9/660: read - d5/d9/dc/d1e/d94/fad zero size 2024-03-16T10:45:03.597 INFO:tasks.workunit.client.0.smithi002.stdout:3/637: fsync d1/d5a/f75 0 2024-03-16T10:45:03.598 INFO:tasks.workunit.client.0.smithi002.stdout:1/660: truncate d2/df/d12/d84/d87/d96/db5/f65 3934874 0 2024-03-16T10:45:03.601 INFO:tasks.workunit.client.0.smithi002.stdout:8/721: dwrite d7/d9/d17/d23/d2e/d34/f9e [0,4194304] 0 2024-03-16T10:45:03.602 INFO:tasks.workunit.client.0.smithi002.stdout:9/661: mknod d5/d9/d34/ccd 0 2024-03-16T10:45:03.603 INFO:tasks.workunit.client.0.smithi002.stdout:5/555: truncate d0/d16/dab/f3b 2299979 0 2024-03-16T10:45:03.605 INFO:tasks.workunit.client.0.smithi002.stdout:3/638: rename d1/db6/c3d to d1/dd/d28/d37/ce0 0 2024-03-16T10:45:03.606 INFO:tasks.workunit.client.0.smithi002.stdout:9/662: stat d5/d9/dc/f1f 0 2024-03-16T10:45:03.608 INFO:tasks.workunit.client.0.smithi002.stdout:0/666: dwrite d3/d27/f96 [4194304,4194304] 0 2024-03-16T10:45:03.608 INFO:tasks.workunit.client.0.smithi002.stdout:4/603: write d2/d6/d32/f4e [446634,45166] 0 2024-03-16T10:45:03.611 INFO:tasks.workunit.client.0.smithi002.stdout:2/625: dread d7/db/d12/f5d [4194304,4194304] 0 2024-03-16T10:45:03.611 INFO:tasks.workunit.client.0.smithi002.stdout:5/556: symlink d0/d16/dab/daa/lac 0 2024-03-16T10:45:03.612 INFO:tasks.workunit.client.0.smithi002.stdout:0/667: write d3/d31/f99 [2435713,53438] 0 2024-03-16T10:45:03.613 INFO:tasks.workunit.client.0.smithi002.stdout:9/663: write d5/d9/dc/d28/f2d [2362148,55491] 0 2024-03-16T10:45:03.614 INFO:tasks.workunit.client.0.smithi002.stdout:3/639: chown d1/dd/d28/d7b/d8a/d93/f51 489 1 2024-03-16T10:45:03.617 INFO:tasks.workunit.client.0.smithi002.stdout:4/604: mkdir d2/db9 0 2024-03-16T10:45:03.617 INFO:tasks.workunit.client.0.smithi002.stdout:0/668: unlink d3/fae 0 2024-03-16T10:45:03.619 INFO:tasks.workunit.client.0.smithi002.stdout:8/722: link d7/d9/d17/d23/d2e/f4e d7/d9/d17/fef 0 2024-03-16T10:45:03.620 INFO:tasks.workunit.client.0.smithi002.stdout:4/605: fsync d2/d6/d32/d66/fa4 0 2024-03-16T10:45:03.622 INFO:tasks.workunit.client.0.smithi002.stdout:1/661: link d2/df/d12/d84/d87/d96/db5/d1c/cb6 d2/df/d12/d84/d87/d96/db5/dae/cd3 0 2024-03-16T10:45:03.625 INFO:tasks.workunit.client.0.smithi002.stdout:1/662: stat d2/df/l48 0 2024-03-16T10:45:03.625 INFO:tasks.workunit.client.0.smithi002.stdout:3/640: creat d1/da5/fe1 x:0 0 0 2024-03-16T10:45:03.625 INFO:tasks.workunit.client.0.smithi002.stdout:2/626: dwrite d7/d2e/d28/d40/fa3 [0,4194304] 0 2024-03-16T10:45:03.628 INFO:tasks.workunit.client.0.smithi002.stdout:5/557: dwrite d0/f6f [0,4194304] 0 2024-03-16T10:45:03.630 INFO:tasks.workunit.client.0.smithi002.stdout:4/606: readlink d2/d33/da/d5e/d93/l97 0 2024-03-16T10:45:03.632 INFO:tasks.workunit.client.0.smithi002.stdout:8/723: read d7/d9/d17/d23/d39/f8c [436498,7841] 0 2024-03-16T10:45:03.633 INFO:tasks.workunit.client.0.smithi002.stdout:0/669: fsync d3/d5/f19 0 2024-03-16T10:45:03.635 INFO:tasks.workunit.client.0.smithi002.stdout:8/724: fsync d7/d9/d17/d23/d2e/d34/f6e 0 2024-03-16T10:45:03.638 INFO:tasks.workunit.client.0.smithi002.stdout:5/558: mkdir d0/d24/dad 0 2024-03-16T10:45:03.638 INFO:tasks.workunit.client.0.smithi002.stdout:7/561: sync 2024-03-16T10:45:03.638 INFO:tasks.workunit.client.0.smithi002.stdout:8/725: truncate d7/d9/d17/d23/d2e/d34/fc8 918395 0 2024-03-16T10:45:03.639 INFO:tasks.workunit.client.0.smithi002.stdout:5/559: dread - d0/d16/dab/d33/d8c/f95 zero size 2024-03-16T10:45:03.640 INFO:tasks.workunit.client.0.smithi002.stdout:7/562: read d3/f15 [7395073,105746] 0 2024-03-16T10:45:03.642 INFO:tasks.workunit.client.0.smithi002.stdout:9/664: creat d5/d9/fce x:0 0 0 2024-03-16T10:45:03.642 INFO:tasks.workunit.client.0.smithi002.stdout:1/663: rmdir d2/df/d12/d84/d87/d96/db5/d1c/da0/db1 39 2024-03-16T10:45:03.646 INFO:tasks.workunit.client.0.smithi002.stdout:8/726: dwrite d7/d9/d17/f27 [4194304,4194304] 0 2024-03-16T10:45:03.648 INFO:tasks.workunit.client.0.smithi002.stdout:7/563: mkdir d3/d4/d1d/d87/d5c/d63/d3f/dba/dcc 0 2024-03-16T10:45:03.650 INFO:tasks.workunit.client.0.smithi002.stdout:9/665: dread d5/d9/faf [0,4194304] 0 2024-03-16T10:45:03.651 INFO:tasks.workunit.client.0.smithi002.stdout:2/627: creat d7/db/fbd x:0 0 0 2024-03-16T10:45:03.653 INFO:tasks.workunit.client.0.smithi002.stdout:0/670: dwrite d3/d28/f79 [0,4194304] 0 2024-03-16T10:45:03.655 INFO:tasks.workunit.client.0.smithi002.stdout:0/671: dread d3/de/da2/fd2 [0,4194304] 0 2024-03-16T10:45:03.655 INFO:tasks.workunit.client.0.smithi002.stdout:8/727: mkdir d7/d9/d17/d23/d2e/d5c/df0 0 2024-03-16T10:45:03.657 INFO:tasks.workunit.client.0.smithi002.stdout:8/728: write d7/d9/d17/f3d [692788,33195] 0 2024-03-16T10:45:03.659 INFO:tasks.workunit.client.0.smithi002.stdout:8/729: fdatasync d7/d9/fa 0 2024-03-16T10:45:03.662 INFO:tasks.workunit.client.0.smithi002.stdout:9/666: creat d5/db1/fcf x:0 0 0 2024-03-16T10:45:03.664 INFO:tasks.workunit.client.0.smithi002.stdout:2/628: symlink d7/d27/d3b/d95/da5/lbe 0 2024-03-16T10:45:03.664 INFO:tasks.workunit.client.0.smithi002.stdout:7/564: dread d3/d4/d1d/d87/d5c/d63/d34/f6b [0,4194304] 0 2024-03-16T10:45:03.665 INFO:tasks.workunit.client.0.smithi002.stdout:9/667: truncate d5/f6f 320647 0 2024-03-16T10:45:03.665 INFO:tasks.workunit.client.0.smithi002.stdout:3/641: dwrite d1/db6/f30 [0,4194304] 0 2024-03-16T10:45:03.666 INFO:tasks.workunit.client.0.smithi002.stdout:0/672: symlink d3/d27/ld8 0 2024-03-16T10:45:03.667 INFO:tasks.workunit.client.0.smithi002.stdout:9/668: chown d5/d9/dc/d1e/fbc 9541 1 2024-03-16T10:45:03.668 INFO:tasks.workunit.client.0.smithi002.stdout:0/673: dread - d3/d27/fc2 zero size 2024-03-16T10:45:03.669 INFO:tasks.workunit.client.0.smithi002.stdout:0/674: chown d3/d28/d2a/da1/d9c/da9 7 1 2024-03-16T10:45:03.670 INFO:tasks.workunit.client.0.smithi002.stdout:1/664: creat d2/df/d12/d84/d87/d96/db5/fd4 x:0 0 0 2024-03-16T10:45:03.674 INFO:tasks.workunit.client.0.smithi002.stdout:3/642: readlink d1/db6/d15/l19 0 2024-03-16T10:45:03.675 INFO:tasks.workunit.client.0.smithi002.stdout:2/629: creat d7/d2e/d1d/d2f/d4b/fbf x:0 0 0 2024-03-16T10:45:03.677 INFO:tasks.workunit.client.0.smithi002.stdout:5/560: truncate d0/d1f/f58 467885 0 2024-03-16T10:45:03.678 INFO:tasks.workunit.client.0.smithi002.stdout:9/669: fdatasync d5/f7 0 2024-03-16T10:45:03.679 INFO:tasks.workunit.client.0.smithi002.stdout:9/670: write d5/d9/dc/d1e/f6c [1260258,2830] 0 2024-03-16T10:45:03.679 INFO:tasks.workunit.client.0.smithi002.stdout:4/607: truncate d2/d33/da/f84 1868495 0 2024-03-16T10:45:03.681 INFO:tasks.workunit.client.0.smithi002.stdout:1/665: creat d2/df/d12/d84/fd5 x:0 0 0 2024-03-16T10:45:03.682 INFO:tasks.workunit.client.0.smithi002.stdout:8/730: rename d7/d9/d17/d23/fa9 to d7/d9/d17/d83/ff1 0 2024-03-16T10:45:03.684 INFO:tasks.workunit.client.0.smithi002.stdout:2/630: readlink d7/l9 0 2024-03-16T10:45:03.687 INFO:tasks.workunit.client.0.smithi002.stdout:1/666: dwrite d2/df/d12/d84/d87/d96/db5/d1c/d5d/f6b [0,4194304] 0 2024-03-16T10:45:03.690 INFO:tasks.workunit.client.0.smithi002.stdout:4/608: write d2/f92 [767315,94185] 0 2024-03-16T10:45:03.690 INFO:tasks.workunit.client.0.smithi002.stdout:9/671: mkdir d5/d9/dc/d28/d8e/dd0 0 2024-03-16T10:45:03.698 INFO:tasks.workunit.client.0.smithi002.stdout:3/643: mknod d1/dd/ce2 0 2024-03-16T10:45:03.699 INFO:tasks.workunit.client.0.smithi002.stdout:5/561: creat d0/d24/dad/fae x:0 0 0 2024-03-16T10:45:03.701 INFO:tasks.workunit.client.0.smithi002.stdout:7/565: truncate d3/d4/d1d/d87/d96/d5e/f84 113011 0 2024-03-16T10:45:03.701 INFO:tasks.workunit.client.0.smithi002.stdout:9/672: dwrite d5/d9/d22/f6e [0,4194304] 0 2024-03-16T10:45:03.702 INFO:tasks.workunit.client.0.smithi002.stdout:0/675: creat d3/d5/fd9 x:0 0 0 2024-03-16T10:45:03.702 INFO:tasks.workunit.client.0.smithi002.stdout:4/609: dread d2/d33/d1c/d45/f54 [0,4194304] 0 2024-03-16T10:45:03.704 INFO:tasks.workunit.client.0.smithi002.stdout:1/667: write d2/df/d12/d29/d5c/f6d [2736696,27946] 0 2024-03-16T10:45:03.707 INFO:tasks.workunit.client.0.smithi002.stdout:2/631: symlink d7/d2e/d1d/d38/d49/db5/lc0 0 2024-03-16T10:45:03.708 INFO:tasks.workunit.client.0.smithi002.stdout:2/632: truncate f6 4605650 0 2024-03-16T10:45:03.709 INFO:tasks.workunit.client.0.smithi002.stdout:5/562: creat d0/d16/dab/daa/faf x:0 0 0 2024-03-16T10:45:03.710 INFO:tasks.workunit.client.0.smithi002.stdout:2/633: write d7/d2e/f16 [246255,93937] 0 2024-03-16T10:45:03.712 INFO:tasks.workunit.client.0.smithi002.stdout:6/657: sync 2024-03-16T10:45:03.714 INFO:tasks.workunit.client.0.smithi002.stdout:3/644: dread d1/dd/d28/d7b/d8a/d93/f47 [0,4194304] 0 2024-03-16T10:45:03.714 INFO:tasks.workunit.client.0.smithi002.stdout:8/731: creat d7/d13/db8/ff2 x:0 0 0 2024-03-16T10:45:03.715 INFO:tasks.workunit.client.0.smithi002.stdout:8/732: read - d7/d9/d17/d23/d39/db7/fd1 zero size 2024-03-16T10:45:03.722 INFO:tasks.workunit.client.0.smithi002.stdout:9/673: unlink d5/d9/dc/c15 0 2024-03-16T10:45:03.726 INFO:tasks.workunit.client.0.smithi002.stdout:0/676: write d3/de/f7b [278198,34273] 0 2024-03-16T10:45:03.727 INFO:tasks.workunit.client.0.smithi002.stdout:4/610: dread d2/d33/da/f74 [0,4194304] 0 2024-03-16T10:45:03.728 INFO:tasks.workunit.client.0.smithi002.stdout:3/645: mknod d1/dd/d28/d37/ce3 0 2024-03-16T10:45:03.728 INFO:tasks.workunit.client.0.smithi002.stdout:0/677: write d3/d5/f30 [1419383,128516] 0 2024-03-16T10:45:03.729 INFO:tasks.workunit.client.0.smithi002.stdout:3/646: readlink d1/dd/d28/d7b/d8a/d93/l8e 0 2024-03-16T10:45:03.730 INFO:tasks.workunit.client.0.smithi002.stdout:8/733: creat d7/d9/d17/dc3/ff3 x:0 0 0 2024-03-16T10:45:03.731 INFO:tasks.workunit.client.0.smithi002.stdout:5/563: symlink d0/d16/dab/d33/lb0 0 2024-03-16T10:45:03.732 INFO:tasks.workunit.client.0.smithi002.stdout:5/564: chown d0/d16/d4a/d39/c45 477 1 2024-03-16T10:45:03.732 INFO:tasks.workunit.client.0.smithi002.stdout:9/674: symlink d5/d9/dc/d28/d8e/ld1 0 2024-03-16T10:45:03.734 INFO:tasks.workunit.client.0.smithi002.stdout:7/566: rmdir d3/d4/d1d/d87/d5c/d63/d3f/dba/dc6 0 2024-03-16T10:45:03.737 INFO:tasks.workunit.client.0.smithi002.stdout:6/658: chown d15/d2d/d2b/d75/fce 1842946 1 2024-03-16T10:45:03.738 INFO:tasks.workunit.client.0.smithi002.stdout:1/668: truncate d2/df/d12/d84/d87/d96/db5/d51/d56/fc5 5861669 0 2024-03-16T10:45:03.738 INFO:tasks.workunit.client.0.smithi002.stdout:5/565: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:45:03.739 INFO:tasks.workunit.client.0.smithi002.stdout:4/611: creat d2/d33/d1c/fba x:0 0 0 2024-03-16T10:45:03.741 INFO:tasks.workunit.client.0.smithi002.stdout:6/659: write d15/d2d/fbd [3387455,14573] 0 2024-03-16T10:45:03.743 INFO:tasks.workunit.client.0.smithi002.stdout:2/634: dwrite d7/d2e/d28/d40/d6c/f7a [0,4194304] 0 2024-03-16T10:45:03.743 INFO:tasks.workunit.client.0.smithi002.stdout:7/567: symlink d3/d4/d1c/lcd 0 2024-03-16T10:45:03.744 INFO:tasks.workunit.client.0.smithi002.stdout:1/669: dread - d2/df/d12/d84/d87/d96/db5/d1c/d5d/f82 zero size 2024-03-16T10:45:03.748 INFO:tasks.workunit.client.0.smithi002.stdout:4/612: unlink d2/f92 0 2024-03-16T10:45:03.750 INFO:tasks.workunit.client.0.smithi002.stdout:4/613: write d2/d33/f64 [5188581,98467] 0 2024-03-16T10:45:03.757 INFO:tasks.workunit.client.0.smithi002.stdout:2/635: write d7/d2e/d28/d40/d50/f68 [541375,46961] 0 2024-03-16T10:45:03.760 INFO:tasks.workunit.client.0.smithi002.stdout:3/647: creat d1/dd/d28/d7b/d2c/fe4 x:0 0 0 2024-03-16T10:45:03.763 INFO:tasks.workunit.client.0.smithi002.stdout:8/734: rename d7/d9/d17/d23/d2e/f92 to d7/d9/d17/d23/d2e/ff4 0 2024-03-16T10:45:03.763 INFO:tasks.workunit.client.0.smithi002.stdout:3/648: dread d1/dd/d28/f31 [0,4194304] 0 2024-03-16T10:45:03.764 INFO:tasks.workunit.client.0.smithi002.stdout:9/675: creat d5/d9/fd2 x:0 0 0 2024-03-16T10:45:03.765 INFO:tasks.workunit.client.0.smithi002.stdout:3/649: write d1/dd/d28/d7b/d2c/fe4 [43025,125584] 0 2024-03-16T10:45:03.766 INFO:tasks.workunit.client.0.smithi002.stdout:4/614: creat d2/d33/da/d5e/d93/fbb x:0 0 0 2024-03-16T10:45:03.768 INFO:tasks.workunit.client.0.smithi002.stdout:6/660: mkdir d15/d2d/d2b/d3f/d77/de1 0 2024-03-16T10:45:03.769 INFO:tasks.workunit.client.0.smithi002.stdout:2/636: readlink d7/db/l91 0 2024-03-16T10:45:03.770 INFO:tasks.workunit.client.0.smithi002.stdout:1/670: mkdir d2/df/d12/d84/d87/d96/db5/d1c/da0/dd6 0 2024-03-16T10:45:03.771 INFO:tasks.workunit.client.0.smithi002.stdout:0/678: rename d3/d28/d2a/d67/f9f to d3/de/fda 0 2024-03-16T10:45:03.773 INFO:tasks.workunit.client.0.smithi002.stdout:5/566: truncate d0/d16/f37 1812878 0 2024-03-16T10:45:03.774 INFO:tasks.workunit.client.0.smithi002.stdout:8/735: fsync d7/d9/d17/d20/fce 0 2024-03-16T10:45:03.775 INFO:tasks.workunit.client.0.smithi002.stdout:9/676: symlink d5/d9/dc/d28/ld3 0 2024-03-16T10:45:03.801 INFO:tasks.workunit.client.0.smithi002.stdout:6/661: rmdir d15/d2d/d2b/d3f/d77/d9f 39 2024-03-16T10:45:03.802 INFO:tasks.workunit.client.0.smithi002.stdout:2/637: mknod d7/db/d12/cc1 0 2024-03-16T10:45:03.802 INFO:tasks.workunit.client.0.smithi002.stdout:4/615: chown d2/d33/da/d2e/db0/lb6 123 1 2024-03-16T10:45:03.803 INFO:tasks.workunit.client.0.smithi002.stdout:8/736: mkdir d7/d9/d17/d83/df5 0 2024-03-16T10:45:03.804 INFO:tasks.workunit.client.0.smithi002.stdout:4/616: fdatasync d2/d6/d32/d66/fa4 0 2024-03-16T10:45:03.804 INFO:tasks.workunit.client.0.smithi002.stdout:1/671: mkdir d2/df/d12/dd7 0 2024-03-16T10:45:03.808 INFO:tasks.workunit.client.0.smithi002.stdout:4/617: write d2/d33/da/d2e/f56 [1242067,4756] 0 2024-03-16T10:45:03.808 INFO:tasks.workunit.client.0.smithi002.stdout:6/662: unlink d15/f4f 0 2024-03-16T10:45:03.810 INFO:tasks.workunit.client.0.smithi002.stdout:1/672: dread d2/df/d12/d29/d5c/f6d [0,4194304] 0 2024-03-16T10:45:03.812 INFO:tasks.workunit.client.0.smithi002.stdout:2/638: creat d7/d2e/d1d/fc2 x:0 0 0 2024-03-16T10:45:03.812 INFO:tasks.workunit.client.0.smithi002.stdout:9/677: dread d5/d9/d55/dc9/fbf [0,4194304] 0 2024-03-16T10:45:03.814 INFO:tasks.workunit.client.0.smithi002.stdout:9/678: read d5/d9/d55/f7b [3765341,116316] 0 2024-03-16T10:45:03.816 INFO:tasks.workunit.client.0.smithi002.stdout:0/679: write d3/d59/dca/f53 [74883,32409] 0 2024-03-16T10:45:03.817 INFO:tasks.workunit.client.0.smithi002.stdout:0/680: write d3/de/f7d [673853,97872] 0 2024-03-16T10:45:03.818 INFO:tasks.workunit.client.0.smithi002.stdout:8/737: fdatasync d7/d9/d17/d23/d2e/f8b 0 2024-03-16T10:45:03.818 INFO:tasks.workunit.client.0.smithi002.stdout:0/681: dread - d3/d27/fc2 zero size 2024-03-16T10:45:03.824 INFO:tasks.workunit.client.0.smithi002.stdout:6/663: symlink d15/d2d/d2c/le2 0 2024-03-16T10:45:03.825 INFO:tasks.workunit.client.0.smithi002.stdout:4/618: mknod d2/d6/d32/d66/d7f/cbc 0 2024-03-16T10:45:03.830 INFO:tasks.workunit.client.0.smithi002.stdout:5/567: creat d0/d16/fb1 x:0 0 0 2024-03-16T10:45:03.831 INFO:tasks.workunit.client.0.smithi002.stdout:1/673: dread d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/f95 [0,4194304] 0 2024-03-16T10:45:03.831 INFO:tasks.workunit.client.0.smithi002.stdout:9/679: creat d5/d9/d22/d3c/d4d/d7f/fd4 x:0 0 0 2024-03-16T10:45:03.833 INFO:tasks.workunit.client.0.smithi002.stdout:7/568: rename d3/d4/d1d/d87/d5c/d63/l30 to d3/d4/d1c/d5b/lce 0 2024-03-16T10:45:03.835 INFO:tasks.workunit.client.0.smithi002.stdout:7/569: read d3/d37/d6a/f6e [648788,51640] 0 2024-03-16T10:45:03.837 INFO:tasks.workunit.client.0.smithi002.stdout:9/680: dwrite d5/d9/dc/d1e/d24/f30 [8388608,4194304] 0 2024-03-16T10:45:03.839 INFO:tasks.workunit.client.0.smithi002.stdout:6/664: creat d15/d2d/d8a/d7c/d87/fe3 x:0 0 0 2024-03-16T10:45:03.840 INFO:tasks.workunit.client.0.smithi002.stdout:2/639: symlink d7/d2e/d1d/d38/da8/lc3 0 2024-03-16T10:45:03.843 INFO:tasks.workunit.client.0.smithi002.stdout:0/682: dwrite d3/d31/fc6 [0,4194304] 0 2024-03-16T10:45:03.846 INFO:tasks.workunit.client.0.smithi002.stdout:6/665: dread d15/d2d/d2c/d8f/d96/db5/fc7 [0,4194304] 0 2024-03-16T10:45:03.849 INFO:tasks.workunit.client.0.smithi002.stdout:8/738: symlink d7/d13/db8/d8f/lf6 0 2024-03-16T10:45:03.859 INFO:tasks.workunit.client.0.smithi002.stdout:2/640: dread d7/d2e/d28/d44/f71 [0,4194304] 0 2024-03-16T10:45:03.863 INFO:tasks.workunit.client.0.smithi002.stdout:0/683: mkdir d3/d31/daa/ddb 0 2024-03-16T10:45:03.868 INFO:tasks.workunit.client.0.smithi002.stdout:7/570: symlink d3/lcf 0 2024-03-16T10:45:03.869 INFO:tasks.workunit.client.0.smithi002.stdout:0/684: dwrite d3/d31/d88/fab [0,4194304] 0 2024-03-16T10:45:03.870 INFO:tasks.workunit.client.0.smithi002.stdout:8/739: creat d7/d9/d17/d23/d2e/d34/dbf/ff7 x:0 0 0 2024-03-16T10:45:03.874 INFO:tasks.workunit.client.0.smithi002.stdout:7/571: dread d3/d4/d1c/f90 [0,4194304] 0 2024-03-16T10:45:03.876 INFO:tasks.workunit.client.0.smithi002.stdout:7/572: write d3/d4/d1d/d45/f53 [942722,30357] 0 2024-03-16T10:45:03.883 INFO:tasks.workunit.client.0.smithi002.stdout:7/573: dwrite d3/d4/d1d/d87/d96/fb8 [0,4194304] 0 2024-03-16T10:45:03.884 INFO:tasks.workunit.client.0.smithi002.stdout:8/740: write d7/d13/f84 [327876,121054] 0 2024-03-16T10:45:03.886 INFO:tasks.workunit.client.0.smithi002.stdout:9/681: rename d5/d9/dc/d28/f5a to d5/d9/dc/d1e/d24/d31/fd5 0 2024-03-16T10:45:03.889 INFO:tasks.workunit.client.0.smithi002.stdout:5/568: dwrite d0/d1f/f2c [0,4194304] 0 2024-03-16T10:45:03.902 INFO:tasks.workunit.client.0.smithi002.stdout:0/685: rename d3/d28/d2a/da1/d9c/da9 to d3/d31/d75/db4/ddc 0 2024-03-16T10:45:03.904 INFO:tasks.workunit.client.0.smithi002.stdout:1/674: truncate d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f79 2449214 0 2024-03-16T10:45:03.904 INFO:tasks.workunit.client.0.smithi002.stdout:8/741: symlink d7/d9/d17/d23/d2e/d5c/lf8 0 2024-03-16T10:45:03.905 INFO:tasks.workunit.client.0.smithi002.stdout:8/742: fdatasync d7/d9/f58 0 2024-03-16T10:45:03.906 INFO:tasks.workunit.client.0.smithi002.stdout:9/682: mkdir d5/d9/dc/d28/dd6 0 2024-03-16T10:45:03.908 INFO:tasks.workunit.client.0.smithi002.stdout:7/574: mkdir d3/d4/d1d/d87/d5c/dd0 0 2024-03-16T10:45:03.909 INFO:tasks.workunit.client.0.smithi002.stdout:6/666: write d15/d2d/f22 [312582,47088] 0 2024-03-16T10:45:03.911 INFO:tasks.workunit.client.0.smithi002.stdout:6/667: write f8 [11379878,118018] 0 2024-03-16T10:45:03.912 INFO:tasks.workunit.client.0.smithi002.stdout:6/668: read - d15/d2d/d73/d97/fbf zero size 2024-03-16T10:45:03.917 INFO:tasks.workunit.client.0.smithi002.stdout:0/686: dread d3/d59/dca/f53 [0,4194304] 0 2024-03-16T10:45:03.918 INFO:tasks.workunit.client.0.smithi002.stdout:0/687: readlink d3/d28/d2a/lbd 0 2024-03-16T10:45:03.918 INFO:tasks.workunit.client.0.smithi002.stdout:2/641: write d7/d2e/d1d/d38/f3a [1393984,48305] 0 2024-03-16T10:45:03.919 INFO:tasks.workunit.client.0.smithi002.stdout:9/683: chown d5/d9/d25/d23/fcc 37853314 1 2024-03-16T10:45:03.921 INFO:tasks.workunit.client.0.smithi002.stdout:1/675: dread d2/df/d12/d84/d87/d96/db5/f39 [0,4194304] 0 2024-03-16T10:45:03.927 INFO:tasks.workunit.client.0.smithi002.stdout:1/676: write d2/df/d12/d84/d87/d96/db5/fd4 [420007,79762] 0 2024-03-16T10:45:03.927 INFO:tasks.workunit.client.0.smithi002.stdout:9/684: dwrite d5/fd [4194304,4194304] 0 2024-03-16T10:45:03.927 INFO:tasks.workunit.client.0.smithi002.stdout:9/685: stat d5/d9/d55/dc9/lc3 0 2024-03-16T10:45:03.927 INFO:tasks.workunit.client.0.smithi002.stdout:9/686: chown d5/d7e 0 1 2024-03-16T10:45:03.932 INFO:tasks.workunit.client.0.smithi002.stdout:2/642: creat d7/d2e/d28/d40/fc4 x:0 0 0 2024-03-16T10:45:03.934 INFO:tasks.workunit.client.0.smithi002.stdout:0/688: rename d3/d28/d2a/d67/f70 to d3/d28/d2a/d67/fdd 0 2024-03-16T10:45:03.943 INFO:tasks.workunit.client.0.smithi002.stdout:9/687: unlink d5/db1/fcf 0 2024-03-16T10:45:03.950 INFO:tasks.workunit.client.0.smithi002.stdout:5/569: rename d0/d16/dab/d2b/d5d/c31 to d0/d1f/cb2 0 2024-03-16T10:45:03.961 INFO:tasks.workunit.client.0.smithi002.stdout:1/677: rename d2/df/d12/lb0 to d2/df/d12/dd7/ld8 0 2024-03-16T10:45:03.962 INFO:tasks.workunit.client.0.smithi002.stdout:5/570: mknod d0/d24/dad/cb3 0 2024-03-16T10:45:03.964 INFO:tasks.workunit.client.0.smithi002.stdout:6/669: getdents d15/d80 0 2024-03-16T10:45:03.966 INFO:tasks.workunit.client.0.smithi002.stdout:9/688: mknod d5/d9/cd7 0 2024-03-16T10:45:03.967 INFO:tasks.workunit.client.0.smithi002.stdout:7/575: getdents d3/d4/d1c/d5b/d64/d99 0 2024-03-16T10:45:03.969 INFO:tasks.workunit.client.0.smithi002.stdout:2/643: link d7/d2e/d1d/d2f/l66 d7/d2e/d1d/d38/d49/d8c/lc5 0 2024-03-16T10:45:03.974 INFO:tasks.workunit.client.0.smithi002.stdout:1/678: unlink d2/df/d12/d84/d87/d96/db5/ld 0 2024-03-16T10:45:03.976 INFO:tasks.workunit.client.0.smithi002.stdout:5/571: rmdir d0/d16/d4a/d7d 39 2024-03-16T10:45:03.979 INFO:tasks.workunit.client.0.smithi002.stdout:9/689: rename d5/d9/dc/d1e/f6c to d5/d9/dc/d28/dd6/fd8 0 2024-03-16T10:45:03.981 INFO:tasks.workunit.client.0.smithi002.stdout:1/679: dwrite d2/df/d12/d84/d87/d96/da8/fd0 [0,4194304] 0 2024-03-16T10:45:03.981 INFO:tasks.workunit.client.0.smithi002.stdout:7/576: mknod d3/d4/d1c/d5b/d64/d99/d9c/cd1 0 2024-03-16T10:45:03.984 INFO:tasks.workunit.client.0.smithi002.stdout:2/644: creat d7/d2e/d1d/d2f/d4b/d6e/fc6 x:0 0 0 2024-03-16T10:45:03.984 INFO:tasks.workunit.client.0.smithi002.stdout:1/680: dread - d2/df/d12/d84/d87/d96/db5/f9d zero size 2024-03-16T10:45:03.986 INFO:tasks.workunit.client.0.smithi002.stdout:2/645: fsync d7/d2e/d1d/d2f/d4b/d6e/fab 0 2024-03-16T10:45:03.986 INFO:tasks.workunit.client.0.smithi002.stdout:0/689: getdents d3/d31 0 2024-03-16T10:45:03.987 INFO:tasks.workunit.client.0.smithi002.stdout:2/646: write d7/d2e/d1d/d2f/d4b/d6e/fab [226916,38890] 0 2024-03-16T10:45:03.989 INFO:tasks.workunit.client.0.smithi002.stdout:0/690: dread d3/d28/d2a/d67/f8d [0,4194304] 0 2024-03-16T10:45:03.994 INFO:tasks.workunit.client.0.smithi002.stdout:5/572: mknod d0/d16/dab/d33/d8c/cb4 0 2024-03-16T10:45:03.997 INFO:tasks.workunit.client.0.smithi002.stdout:9/690: unlink d5/d9/d25/d23/f48 0 2024-03-16T10:45:03.999 INFO:tasks.workunit.client.0.smithi002.stdout:1/681: mkdir d2/df/d12/dd7/dd9 0 2024-03-16T10:45:04.000 INFO:tasks.workunit.client.0.smithi002.stdout:4/619: sync 2024-03-16T10:45:04.000 INFO:tasks.workunit.client.0.smithi002.stdout:3/650: sync 2024-03-16T10:45:04.001 INFO:tasks.workunit.client.0.smithi002.stdout:4/620: fdatasync d2/d6/f17 0 2024-03-16T10:45:04.002 INFO:tasks.workunit.client.0.smithi002.stdout:4/621: readlink d2/d6/l16 0 2024-03-16T10:45:04.005 INFO:tasks.workunit.client.0.smithi002.stdout:1/682: dread d2/df/d12/d84/d87/d96/db5/d1c/d5d/f6b [0,4194304] 0 2024-03-16T10:45:04.006 INFO:tasks.workunit.client.0.smithi002.stdout:2/647: read d7/f5e [149578,129871] 0 2024-03-16T10:45:04.006 INFO:tasks.workunit.client.0.smithi002.stdout:0/691: mkdir d3/d31/d75/dde 0 2024-03-16T10:45:04.008 INFO:tasks.workunit.client.0.smithi002.stdout:7/577: mkdir d3/d4/d1d/d87/d5c/dd0/dd2 0 2024-03-16T10:45:04.011 INFO:tasks.workunit.client.0.smithi002.stdout:5/573: dread d0/d16/dab/d2b/d5d/f5a [0,4194304] 0 2024-03-16T10:45:04.013 INFO:tasks.workunit.client.0.smithi002.stdout:6/670: rename d15/d2d/d2b/dc1 to d15/d2d/d2b/de4 0 2024-03-16T10:45:04.015 INFO:tasks.workunit.client.0.smithi002.stdout:2/648: dwrite d7/d2e/d1d/f21 [4194304,4194304] 0 2024-03-16T10:45:04.023 INFO:tasks.workunit.client.0.smithi002.stdout:0/692: creat d3/d59/dca/fdf x:0 0 0 2024-03-16T10:45:04.023 INFO:tasks.workunit.client.0.smithi002.stdout:0/693: chown d3/fc 11 1 2024-03-16T10:45:04.027 INFO:tasks.workunit.client.0.smithi002.stdout:7/578: creat d3/d4/d1c/fd3 x:0 0 0 2024-03-16T10:45:04.029 INFO:tasks.workunit.client.0.smithi002.stdout:0/694: dwrite d3/d5/d7c/fbe [0,4194304] 0 2024-03-16T10:45:04.031 INFO:tasks.workunit.client.0.smithi002.stdout:3/651: mknod d1/dd/ce5 0 2024-03-16T10:45:04.034 INFO:tasks.workunit.client.0.smithi002.stdout:9/691: write d5/d9/dc/d1e/d24/f3d [519741,74733] 0 2024-03-16T10:45:04.034 INFO:tasks.workunit.client.0.smithi002.stdout:9/692: readlink d5/lb8 0 2024-03-16T10:45:04.037 INFO:tasks.workunit.client.0.smithi002.stdout:5/574: mknod d0/d16/dab/daa/cb5 0 2024-03-16T10:45:04.040 INFO:tasks.workunit.client.0.smithi002.stdout:2/649: symlink d7/d27/lc7 0 2024-03-16T10:45:04.042 INFO:tasks.workunit.client.0.smithi002.stdout:4/622: rename d2/d33/d35/l72 to d2/d33/da/d2e/db0/lbd 0 2024-03-16T10:45:04.043 INFO:tasks.workunit.client.0.smithi002.stdout:4/623: dread - d2/fa3 zero size 2024-03-16T10:45:04.043 INFO:tasks.workunit.client.0.smithi002.stdout:4/624: stat d2/d33/d1c/d45/f91 0 2024-03-16T10:45:04.044 INFO:tasks.workunit.client.0.smithi002.stdout:3/652: write d1/dd/d28/d7b/d22/f39 [4963486,4520] 0 2024-03-16T10:45:04.045 INFO:tasks.workunit.client.0.smithi002.stdout:3/653: stat d1/dd/d28/d7b/d25/f26 0 2024-03-16T10:45:04.046 INFO:tasks.workunit.client.0.smithi002.stdout:3/654: fdatasync d1/dd/d28/d7b/d8a/d93/fb7 0 2024-03-16T10:45:04.046 INFO:tasks.workunit.client.0.smithi002.stdout:7/579: mknod d3/d4/d1d/d87/d96/d5e/db7/cd4 0 2024-03-16T10:45:04.047 INFO:tasks.workunit.client.0.smithi002.stdout:7/580: readlink d3/d4/d1c/d5b/d64/l6f 0 2024-03-16T10:45:04.048 INFO:tasks.workunit.client.0.smithi002.stdout:9/693: fsync d5/fa0 0 2024-03-16T10:45:04.049 INFO:tasks.workunit.client.0.smithi002.stdout:9/694: readlink d5/d9/d22/d3c/d4d/lbe 0 2024-03-16T10:45:04.050 INFO:tasks.workunit.client.0.smithi002.stdout:6/671: creat d15/d2d/d73/fe5 x:0 0 0 2024-03-16T10:45:04.052 INFO:tasks.workunit.client.0.smithi002.stdout:2/650: link d7/d2e/f67 d7/d2e/d1d/d2f/d4b/fc8 0 2024-03-16T10:45:04.052 INFO:tasks.workunit.client.0.smithi002.stdout:2/651: read - d7/db/d12/f85 zero size 2024-03-16T10:45:04.053 INFO:tasks.workunit.client.0.smithi002.stdout:3/655: symlink d1/dd/d28/d7b/d73/le6 0 2024-03-16T10:45:04.055 INFO:tasks.workunit.client.0.smithi002.stdout:1/683: rename d2/df/d12/d84/d87/d96/db5/d1c/c25 to d2/df/d12/dd7/cda 0 2024-03-16T10:45:04.056 INFO:tasks.workunit.client.0.smithi002.stdout:7/581: symlink d3/d37/ld5 0 2024-03-16T10:45:04.065 INFO:tasks.workunit.client.0.smithi002.stdout:0/695: rename d3/d27/d90/f9e to d3/d31/d75/dde/fe0 0 2024-03-16T10:45:04.068 INFO:tasks.workunit.client.0.smithi002.stdout:1/684: creat d2/df/d12/d84/d87/d96/db5/d51/d56/fdb x:0 0 0 2024-03-16T10:45:04.069 INFO:tasks.workunit.client.0.smithi002.stdout:3/656: dwrite d1/dd/d28/d7b/d8a/d93/fa2 [0,4194304] 0 2024-03-16T10:45:04.071 INFO:tasks.workunit.client.0.smithi002.stdout:7/582: dread d3/f91 [0,4194304] 0 2024-03-16T10:45:04.071 INFO:tasks.workunit.client.0.smithi002.stdout:1/685: write d2/df/d12/f3f [57963,72793] 0 2024-03-16T10:45:04.071 INFO:tasks.workunit.client.0.smithi002.stdout:6/672: unlink d15/f33 0 2024-03-16T10:45:04.073 INFO:tasks.workunit.client.0.smithi002.stdout:6/673: dread - d15/d80/fac zero size 2024-03-16T10:45:04.073 INFO:tasks.workunit.client.0.smithi002.stdout:6/674: chown d15/dab/dc4 0 1 2024-03-16T10:45:04.075 INFO:tasks.workunit.client.0.smithi002.stdout:4/625: rename d2/d6/l16 to d2/d33/da/d5e/lbe 0 2024-03-16T10:45:04.075 INFO:tasks.workunit.client.0.smithi002.stdout:6/675: fdatasync d15/d2d/d73/d97/fd1 0 2024-03-16T10:45:04.081 INFO:tasks.workunit.client.0.smithi002.stdout:0/696: dread d3/d31/f4b [0,4194304] 0 2024-03-16T10:45:04.082 INFO:tasks.workunit.client.0.smithi002.stdout:9/695: rename d5/d9/dc/d1e/d24/f30 to d5/d7e/fd9 0 2024-03-16T10:45:04.083 INFO:tasks.workunit.client.0.smithi002.stdout:0/697: readlink d3/d28/d2a/d67/l6d 0 2024-03-16T10:45:04.084 INFO:tasks.workunit.client.0.smithi002.stdout:1/686: dwrite d2/df/d12/d84/d87/d96/db5/d51/d56/f8c [4194304,4194304] 0 2024-03-16T10:45:04.086 INFO:tasks.workunit.client.0.smithi002.stdout:1/687: chown d2/df/d12/d84/d87/d96/db5/d1c/cad 7576 1 2024-03-16T10:45:04.087 INFO:tasks.workunit.client.0.smithi002.stdout:7/583: dread d3/d4/d1d/d45/fa1 [0,4194304] 0 2024-03-16T10:45:04.089 INFO:tasks.workunit.client.0.smithi002.stdout:6/676: mknod d15/d2d/d73/ce6 0 2024-03-16T10:45:04.090 INFO:tasks.workunit.client.0.smithi002.stdout:7/584: dread - d3/d4/d1d/d45/fbc zero size 2024-03-16T10:45:04.092 INFO:tasks.workunit.client.0.smithi002.stdout:7/585: read d3/f14 [825906,106229] 0 2024-03-16T10:45:04.093 INFO:tasks.workunit.client.0.smithi002.stdout:9/696: dwrite d5/d9/dc/d1e/d91/d83/f9e [0,4194304] 0 2024-03-16T10:45:04.096 INFO:tasks.workunit.client.0.smithi002.stdout:2/652: rmdir d7 39 2024-03-16T10:45:04.099 INFO:tasks.workunit.client.0.smithi002.stdout:6/677: symlink d15/d2d/d2b/de4/le7 0 2024-03-16T10:45:04.100 INFO:tasks.workunit.client.0.smithi002.stdout:1/688: symlink d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/ldc 0 2024-03-16T10:45:04.102 INFO:tasks.workunit.client.0.smithi002.stdout:3/657: getdents d1/dd/d28/d7b/d25 0 2024-03-16T10:45:04.102 INFO:tasks.workunit.client.0.smithi002.stdout:1/689: chown d2/df/d12/d84/d87/d96/db5/d1c/f7f 2862 1 2024-03-16T10:45:04.109 INFO:tasks.workunit.client.0.smithi002.stdout:7/586: dread d3/d37/f8f [0,4194304] 0 2024-03-16T10:45:04.111 INFO:tasks.workunit.client.0.smithi002.stdout:6/678: dread d15/d2d/d2b/f35 [0,4194304] 0 2024-03-16T10:45:04.112 INFO:tasks.workunit.client.0.smithi002.stdout:2/653: mknod d7/d2e/d28/d44/cc9 0 2024-03-16T10:45:04.115 INFO:tasks.workunit.client.0.smithi002.stdout:0/698: getdents d3/de/da2 0 2024-03-16T10:45:04.116 INFO:tasks.workunit.client.0.smithi002.stdout:4/626: truncate d2/d33/da/f4f 67942 0 2024-03-16T10:45:04.117 INFO:tasks.workunit.client.0.smithi002.stdout:9/697: link d5/d9/dc/d1e/d24/d31/l5b d5/d9/dc/d1e/d24/lda 0 2024-03-16T10:45:04.119 INFO:tasks.workunit.client.0.smithi002.stdout:6/679: mkdir d15/dab/dc4/de8 0 2024-03-16T10:45:04.120 INFO:tasks.workunit.client.0.smithi002.stdout:1/690: creat d2/df/d12/fdd x:0 0 0 2024-03-16T10:45:04.121 INFO:tasks.workunit.client.0.smithi002.stdout:7/587: symlink d3/d61/ld6 0 2024-03-16T10:45:04.123 INFO:tasks.workunit.client.0.smithi002.stdout:4/627: dwrite d2/d33/d1c/d45/faa [0,4194304] 0 2024-03-16T10:45:04.123 INFO:tasks.workunit.client.0.smithi002.stdout:2/654: mknod d7/d2e/d1d/d38/d49/d8c/cca 0 2024-03-16T10:45:04.125 INFO:tasks.workunit.client.0.smithi002.stdout:3/658: link d1/dd/d28/d37/f8d d1/dd/fe7 0 2024-03-16T10:45:04.126 INFO:tasks.workunit.client.0.smithi002.stdout:1/691: read d2/df/d12/d84/d87/d96/db5/fab [1631559,97607] 0 2024-03-16T10:45:04.131 INFO:tasks.workunit.client.0.smithi002.stdout:0/699: creat d3/de/fe1 x:0 0 0 2024-03-16T10:45:04.131 INFO:tasks.workunit.client.0.smithi002.stdout:7/588: write d3/d4/d1d/d87/d5c/d63/d34/f6b [4770762,10302] 0 2024-03-16T10:45:04.131 INFO:tasks.workunit.client.0.smithi002.stdout:2/655: read d7/d2e/d28/d40/f93 [3784335,74967] 0 2024-03-16T10:45:04.132 INFO:tasks.workunit.client.0.smithi002.stdout:4/628: mknod d2/d33/da/d2e/db4/cbf 0 2024-03-16T10:45:04.133 INFO:tasks.workunit.client.0.smithi002.stdout:3/659: creat d1/dd/d28/d7b/d2c/d97/fe8 x:0 0 0 2024-03-16T10:45:04.135 INFO:tasks.workunit.client.0.smithi002.stdout:8/743: sync 2024-03-16T10:45:04.144 INFO:tasks.workunit.client.0.smithi002.stdout:0/700: symlink d3/d31/d75/le2 0 2024-03-16T10:45:04.146 INFO:tasks.workunit.client.0.smithi002.stdout:4/629: stat d2/d33/da/d2e/db4 0 2024-03-16T10:45:04.146 INFO:tasks.workunit.client.0.smithi002.stdout:9/698: dwrite d5/d9/faf [0,4194304] 0 2024-03-16T10:45:04.148 INFO:tasks.workunit.client.0.smithi002.stdout:6/680: dwrite d15/d2d/f34 [0,4194304] 0 2024-03-16T10:45:04.150 INFO:tasks.workunit.client.0.smithi002.stdout:6/681: write f8 [15317528,12911] 0 2024-03-16T10:45:04.150 INFO:tasks.workunit.client.0.smithi002.stdout:2/656: write d7/d2e/d1d/d2f/d4b/fc8 [668928,44747] 0 2024-03-16T10:45:04.150 INFO:tasks.workunit.client.0.smithi002.stdout:7/589: fdatasync d3/d4/d1d/d87/d5c/d63/f4c 0 2024-03-16T10:45:04.150 INFO:tasks.workunit.client.0.smithi002.stdout:6/682: readlink l7 0 2024-03-16T10:45:04.154 INFO:tasks.workunit.client.0.smithi002.stdout:2/657: write f6 [3007231,32408] 0 2024-03-16T10:45:04.159 INFO:tasks.workunit.client.0.smithi002.stdout:3/660: symlink d1/dd/d28/d7b/d8a/dd7/le9 0 2024-03-16T10:45:04.159 INFO:tasks.workunit.client.0.smithi002.stdout:4/630: symlink d2/d33/d35/d59/lc0 0 2024-03-16T10:45:04.162 INFO:tasks.workunit.client.0.smithi002.stdout:6/683: unlink d15/l17 0 2024-03-16T10:45:04.165 INFO:tasks.workunit.client.0.smithi002.stdout:6/684: chown d15/d2d/d73/ce6 52809954 1 2024-03-16T10:45:04.166 INFO:tasks.workunit.client.0.smithi002.stdout:8/744: link d7/d9/d17/d23/d2e/l42 d7/d9/d17/d23/d39/db7/dc2/lf9 0 2024-03-16T10:45:04.172 INFO:tasks.workunit.client.0.smithi002.stdout:3/661: dwrite d1/db6/f30 [0,4194304] 0 2024-03-16T10:45:04.173 INFO:tasks.workunit.client.0.smithi002.stdout:4/631: dwrite d2/d33/d35/f6f [0,4194304] 0 2024-03-16T10:45:04.175 INFO:tasks.workunit.client.0.smithi002.stdout:6/685: rename d15/d2d/d2b/l4d to d15/d2d/d2c/le9 0 2024-03-16T10:45:04.181 INFO:tasks.workunit.client.0.smithi002.stdout:4/632: fdatasync d2/d6/f1a 0 2024-03-16T10:45:04.181 INFO:tasks.workunit.client.0.smithi002.stdout:9/699: dread d5/d9/d25/d23/f3b [0,4194304] 0 2024-03-16T10:45:04.183 INFO:tasks.workunit.client.0.smithi002.stdout:3/662: creat d1/dd/d28/d7b/d2c/d63/fea x:0 0 0 2024-03-16T10:45:04.184 INFO:tasks.workunit.client.0.smithi002.stdout:7/590: dwrite d3/d61/f7d [0,4194304] 0 2024-03-16T10:45:04.185 INFO:tasks.workunit.client.0.smithi002.stdout:2/658: dwrite d7/f5e [0,4194304] 0 2024-03-16T10:45:04.189 INFO:tasks.workunit.client.0.smithi002.stdout:6/686: fsync d15/d2d/d2b/d3f/f6d 0 2024-03-16T10:45:04.190 INFO:tasks.workunit.client.0.smithi002.stdout:3/663: symlink d1/d5a/leb 0 2024-03-16T10:45:04.192 INFO:tasks.workunit.client.0.smithi002.stdout:2/659: rmdir d7/d27 39 2024-03-16T10:45:04.195 INFO:tasks.workunit.client.0.smithi002.stdout:8/745: dwrite d7/d9/d17/d23/d39/fbb [0,4194304] 0 2024-03-16T10:45:04.200 INFO:tasks.workunit.client.0.smithi002.stdout:4/633: dwrite d2/d33/da/d2e/f8e [0,4194304] 0 2024-03-16T10:45:04.200 INFO:tasks.workunit.client.0.smithi002.stdout:7/591: mkdir d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7 0 2024-03-16T10:45:04.200 INFO:tasks.workunit.client.0.smithi002.stdout:9/700: creat d5/d9/d34/fdb x:0 0 0 2024-03-16T10:45:04.201 INFO:tasks.workunit.client.0.smithi002.stdout:2/660: symlink d7/db/d70/d72/lcb 0 2024-03-16T10:45:04.201 INFO:tasks.workunit.client.0.smithi002.stdout:6/687: creat d15/d2d/da8/fea x:0 0 0 2024-03-16T10:45:04.203 INFO:tasks.workunit.client.0.smithi002.stdout:3/664: symlink d1/dd/d28/d7b/d8a/lec 0 2024-03-16T10:45:04.204 INFO:tasks.workunit.client.0.smithi002.stdout:3/665: dread - d1/dd/d28/d7b/d8a/d93/fcf zero size 2024-03-16T10:45:04.210 INFO:tasks.workunit.client.0.smithi002.stdout:9/701: mkdir d5/d9/dc/d1e/d91/ddc 0 2024-03-16T10:45:04.213 INFO:tasks.workunit.client.0.smithi002.stdout:8/746: getdents d7/d13/db8/d8f 0 2024-03-16T10:45:04.215 INFO:tasks.workunit.client.0.smithi002.stdout:9/702: creat d5/d9/d55/fdd x:0 0 0 2024-03-16T10:45:04.215 INFO:tasks.workunit.client.0.smithi002.stdout:8/747: chown d7/d9/d25/d91/cbc 573951090 1 2024-03-16T10:45:04.216 INFO:tasks.workunit.client.0.smithi002.stdout:9/703: write d5/d7e/f8f [1295568,1811] 0 2024-03-16T10:45:04.218 INFO:tasks.workunit.client.0.smithi002.stdout:6/688: write fc [5045233,1074] 0 2024-03-16T10:45:04.218 INFO:tasks.workunit.client.0.smithi002.stdout:8/748: rmdir d7/d9/d17/d23/d39/dd4 39 2024-03-16T10:45:04.219 INFO:tasks.workunit.client.0.smithi002.stdout:2/661: dread d7/db/fb3 [0,4194304] 0 2024-03-16T10:45:04.222 INFO:tasks.workunit.client.0.smithi002.stdout:6/689: read d15/d80/f84 [4021699,94664] 0 2024-03-16T10:45:04.224 INFO:tasks.workunit.client.0.smithi002.stdout:9/704: rename d5/d9/f13 to d5/d9/dc/d1e/d24/fde 0 2024-03-16T10:45:04.225 INFO:tasks.workunit.client.0.smithi002.stdout:2/662: truncate d7/d2e/d1d/d2f/d4b/d6e/fc6 29508 0 2024-03-16T10:45:04.226 INFO:tasks.workunit.client.0.smithi002.stdout:4/634: dread d2/d6/f10 [0,4194304] 0 2024-03-16T10:45:04.231 INFO:tasks.workunit.client.0.smithi002.stdout:7/592: dwrite d3/fd [0,4194304] 0 2024-03-16T10:45:04.232 INFO:tasks.workunit.client.0.smithi002.stdout:4/635: mknod d2/d33/d35/cc1 0 2024-03-16T10:45:04.232 INFO:tasks.workunit.client.0.smithi002.stdout:3/666: dread d1/dd/d28/d7b/d25/f26 [0,4194304] 0 2024-03-16T10:45:04.236 INFO:tasks.workunit.client.0.smithi002.stdout:8/749: dwrite d7/d9/d17/d23/d39/f7a [0,4194304] 0 2024-03-16T10:45:04.242 INFO:tasks.workunit.client.0.smithi002.stdout:9/705: dwrite d5/d9/dc/d1e/d24/f4b [0,4194304] 0 2024-03-16T10:45:04.278 INFO:tasks.workunit.client.0.smithi002.stdout:5/575: sync 2024-03-16T10:45:04.299 INFO:tasks.workunit.client.0.smithi002.stdout:3/667: write d1/dd/d28/d7b/d2c/f6d [1136011,48146] 0 2024-03-16T10:45:04.301 INFO:tasks.workunit.client.0.smithi002.stdout:9/706: fdatasync d5/d9/dc/d1e/d91/f84 0 2024-03-16T10:45:04.302 INFO:tasks.workunit.client.0.smithi002.stdout:5/576: dread - d0/d16/dab/daa/f9c zero size 2024-03-16T10:45:04.303 INFO:tasks.workunit.client.0.smithi002.stdout:3/668: stat d1/db6/d15/dcc/fd4 0 2024-03-16T10:45:04.303 INFO:tasks.workunit.client.0.smithi002.stdout:6/690: dread d15/d2d/d2c/d5d/f72 [0,4194304] 0 2024-03-16T10:45:04.303 INFO:tasks.workunit.client.0.smithi002.stdout:4/636: dwrite d2/d33/d1c/fba [0,4194304] 0 2024-03-16T10:45:04.304 INFO:tasks.workunit.client.0.smithi002.stdout:7/593: dwrite d3/d4/d1c/d5b/d64/d99/fb0 [0,4194304] 0 2024-03-16T10:45:04.306 INFO:tasks.workunit.client.0.smithi002.stdout:9/707: chown d5/d9/dc/d1e/d91/d82/l8a 2012 1 2024-03-16T10:45:04.309 INFO:tasks.workunit.client.0.smithi002.stdout:7/594: write d3/d4/d1d/d87/d5c/d63/d3f/f89 [235225,29109] 0 2024-03-16T10:45:04.310 INFO:tasks.workunit.client.0.smithi002.stdout:9/708: write d5/d9/dc/d1e/d24/fa5 [959053,2838] 0 2024-03-16T10:45:04.316 INFO:tasks.workunit.client.0.smithi002.stdout:3/669: truncate d1/dd/d28/d7b/d8a/d93/f4d 556085 0 2024-03-16T10:45:04.316 INFO:tasks.workunit.client.0.smithi002.stdout:8/750: link d7/d9/d17/d23/d39/f8c d7/d9/ffa 0 2024-03-16T10:45:04.318 INFO:tasks.workunit.client.0.smithi002.stdout:7/595: fsync d3/d4/d1d/d45/fa1 0 2024-03-16T10:45:04.321 INFO:tasks.workunit.client.0.smithi002.stdout:7/596: chown d3/d4/d1d/d87/d5c/dd0 366237 1 2024-03-16T10:45:04.321 INFO:tasks.workunit.client.0.smithi002.stdout:9/709: dwrite d5/d9/dc/d1e/d94/fad [0,4194304] 0 2024-03-16T10:45:04.322 INFO:tasks.workunit.client.0.smithi002.stdout:8/751: write d7/d9/d17/d23/d2e/d34/f79 [5180000,20091] 0 2024-03-16T10:45:04.324 INFO:tasks.workunit.client.0.smithi002.stdout:6/691: dread d15/d2d/da8/fc9 [0,4194304] 0 2024-03-16T10:45:04.325 INFO:tasks.workunit.client.0.smithi002.stdout:5/577: truncate d0/d16/dab/d2b/f4f 596177 0 2024-03-16T10:45:04.326 INFO:tasks.workunit.client.0.smithi002.stdout:9/710: rmdir d5/d9/d25/d23/d81 39 2024-03-16T10:45:04.329 INFO:tasks.workunit.client.0.smithi002.stdout:7/597: dwrite d3/d4/d1d/d87/d5c/d63/d55/f77 [0,4194304] 0 2024-03-16T10:45:04.329 INFO:tasks.workunit.client.0.smithi002.stdout:5/578: mkdir d0/d24/dad/db6 0 2024-03-16T10:45:04.335 INFO:tasks.workunit.client.0.smithi002.stdout:9/711: unlink d5/d9/dc/d28/f2d 0 2024-03-16T10:45:04.335 INFO:tasks.workunit.client.0.smithi002.stdout:8/752: creat d7/d9/d17/d83/df5/ffb x:0 0 0 2024-03-16T10:45:04.335 INFO:tasks.workunit.client.0.smithi002.stdout:6/692: mkdir d15/d2d/d2b/dc8/deb 0 2024-03-16T10:45:04.337 INFO:tasks.workunit.client.0.smithi002.stdout:9/712: write d5/d9/dc/d28/f65 [1924807,80244] 0 2024-03-16T10:45:04.340 INFO:tasks.workunit.client.0.smithi002.stdout:8/753: mknod d7/d9/d17/d23/d2e/d5c/cfc 0 2024-03-16T10:45:04.342 INFO:tasks.workunit.client.0.smithi002.stdout:7/598: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f46 [0,4194304] 0 2024-03-16T10:45:04.343 INFO:tasks.workunit.client.0.smithi002.stdout:5/579: dwrite d0/f57 [0,4194304] 0 2024-03-16T10:45:04.343 INFO:tasks.workunit.client.0.smithi002.stdout:9/713: creat d5/d9/d22/d3c/d4d/fdf x:0 0 0 2024-03-16T10:45:04.347 INFO:tasks.workunit.client.0.smithi002.stdout:9/714: readlink d5/l6 0 2024-03-16T10:45:04.348 INFO:tasks.workunit.client.0.smithi002.stdout:7/599: read - d3/d4/d1d/d87/d96/d5e/fbd zero size 2024-03-16T10:45:04.349 INFO:tasks.workunit.client.0.smithi002.stdout:8/754: rename d7/d9/d25/d67/f8e to d7/d13/db8/d8f/ffd 0 2024-03-16T10:45:04.350 INFO:tasks.workunit.client.0.smithi002.stdout:6/693: dwrite d15/d2d/d2b/d75/fce [0,4194304] 0 2024-03-16T10:45:04.351 INFO:tasks.workunit.client.0.smithi002.stdout:9/715: write d5/d9/d22/f47 [1152952,57844] 0 2024-03-16T10:45:04.352 INFO:tasks.workunit.client.0.smithi002.stdout:8/755: rmdir d7/d13 39 2024-03-16T10:45:04.353 INFO:tasks.workunit.client.0.smithi002.stdout:7/600: chown d3/d4/d1d/d87/d5c/d63/f3c 320 1 2024-03-16T10:45:04.359 INFO:tasks.workunit.client.0.smithi002.stdout:7/601: rename d3/d4/d1d/d87/d5c/d63/d3f to d3/d4/d1d/d87/d5c/d63/d3f/daa/dd8 22 2024-03-16T10:45:04.361 INFO:tasks.workunit.client.0.smithi002.stdout:7/602: dread - d3/d4/d1c/fd3 zero size 2024-03-16T10:45:04.361 INFO:tasks.workunit.client.0.smithi002.stdout:6/694: mkdir d15/d2d/d2b/d3f/d77/de1/dec 0 2024-03-16T10:45:04.366 INFO:tasks.workunit.client.0.smithi002.stdout:8/756: mknod d7/d13/db8/cfe 0 2024-03-16T10:45:04.370 INFO:tasks.workunit.client.0.smithi002.stdout:9/716: dwrite d5/d9/dc/d28/d8e/fae [0,4194304] 0 2024-03-16T10:45:04.375 INFO:tasks.workunit.client.0.smithi002.stdout:4/637: dwrite d2/d6/d32/d68/f4b [0,4194304] 0 2024-03-16T10:45:04.375 INFO:tasks.workunit.client.0.smithi002.stdout:7/603: rename d3/l19 to d3/ld9 0 2024-03-16T10:45:04.375 INFO:tasks.workunit.client.0.smithi002.stdout:5/580: dread d0/d16/dab/d43/d48/f7f [0,4194304] 0 2024-03-16T10:45:04.377 INFO:tasks.workunit.client.0.smithi002.stdout:9/717: rmdir d5/d9/dc/d28/d8e/db5 39 2024-03-16T10:45:04.379 INFO:tasks.workunit.client.0.smithi002.stdout:8/757: link d7/d13/f68 d7/d9/d25/fff 0 2024-03-16T10:45:04.380 INFO:tasks.workunit.client.0.smithi002.stdout:6/695: link d15/d2d/d2c/f7a d15/d2d/fed 0 2024-03-16T10:45:04.385 INFO:tasks.workunit.client.0.smithi002.stdout:7/604: symlink d3/d4/d1d/d87/d96/d5e/lda 0 2024-03-16T10:45:04.387 INFO:tasks.workunit.client.0.smithi002.stdout:9/718: mknod d5/d9/d55/ce0 0 2024-03-16T10:45:04.391 INFO:tasks.workunit.client.0.smithi002.stdout:1/692: sync 2024-03-16T10:45:04.391 INFO:tasks.workunit.client.0.smithi002.stdout:0/701: sync 2024-03-16T10:45:04.393 INFO:tasks.workunit.client.0.smithi002.stdout:4/638: rmdir d2/db9 0 2024-03-16T10:45:04.395 INFO:tasks.workunit.client.0.smithi002.stdout:7/605: mkdir d3/d4/d1d/d87/d5c/ddb 0 2024-03-16T10:45:04.397 INFO:tasks.workunit.client.0.smithi002.stdout:7/606: fsync d3/d4/d1c/d5b/d64/d99/fb0 0 2024-03-16T10:45:04.398 INFO:tasks.workunit.client.0.smithi002.stdout:5/581: rename d0/d1f/l25 to d0/d1f/lb7 0 2024-03-16T10:45:04.399 INFO:tasks.workunit.client.0.smithi002.stdout:7/607: truncate d3/d4/d1d/d45/f53 1630172 0 2024-03-16T10:45:04.400 INFO:tasks.workunit.client.0.smithi002.stdout:4/639: symlink d2/d33/da/d5e/d93/lc2 0 2024-03-16T10:45:04.401 INFO:tasks.workunit.client.0.smithi002.stdout:8/758: getdents d7/d9/d17/dc3 0 2024-03-16T10:45:04.402 INFO:tasks.workunit.client.0.smithi002.stdout:0/702: dread d3/d5/f19 [0,4194304] 0 2024-03-16T10:45:04.403 INFO:tasks.workunit.client.0.smithi002.stdout:6/696: creat d15/d2d/d73/fee x:0 0 0 2024-03-16T10:45:04.405 INFO:tasks.workunit.client.0.smithi002.stdout:7/608: creat d3/d4/d1d/d87/d5c/dc3/fdc x:0 0 0 2024-03-16T10:45:04.407 INFO:tasks.workunit.client.0.smithi002.stdout:4/640: fdatasync d2/d33/da/f74 0 2024-03-16T10:45:04.409 INFO:tasks.workunit.client.0.smithi002.stdout:5/582: mknod d0/d16/dab/d2b/d94/cb8 0 2024-03-16T10:45:04.409 INFO:tasks.workunit.client.0.smithi002.stdout:4/641: stat d2/d6/f89 0 2024-03-16T10:45:04.410 INFO:tasks.workunit.client.0.smithi002.stdout:6/697: rename d15/d2d/d2c/d8f/d96/db5/fc7 to d15/dab/dc4/fef 0 2024-03-16T10:45:04.411 INFO:tasks.workunit.client.0.smithi002.stdout:0/703: dwrite d3/d27/f96 [4194304,4194304] 0 2024-03-16T10:45:04.411 INFO:tasks.workunit.client.0.smithi002.stdout:9/719: dread d5/d9/d22/d3c/d4d/f75 [0,4194304] 0 2024-03-16T10:45:04.415 INFO:tasks.workunit.client.0.smithi002.stdout:4/642: read d2/d33/da/f84 [1246441,98225] 0 2024-03-16T10:45:04.417 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:04 smithi002 ceph-mon[29248]: pgmap v35: 65 pgs: 65 active+clean; 1.3 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 38 MiB/s rd, 162 MiB/s wr, 362 op/s 2024-03-16T10:45:04.472 INFO:tasks.workunit.client.0.smithi002.stdout:8/759: link d7/f31 d7/d9/d25/dc9/f100 0 2024-03-16T10:45:04.473 INFO:tasks.workunit.client.0.smithi002.stdout:1/693: truncate d2/df/d12/d84/d87/d96/db5/f50 916295 0 2024-03-16T10:45:04.475 INFO:tasks.workunit.client.0.smithi002.stdout:7/609: truncate d3/d4/d1d/d87/d5c/d63/d55/f5d 2505912 0 2024-03-16T10:45:04.478 INFO:tasks.workunit.client.0.smithi002.stdout:0/704: dwrite d3/d28/d2a/da1/d9c/fb9 [0,4194304] 0 2024-03-16T10:45:04.478 INFO:tasks.workunit.client.0.smithi002.stdout:8/760: symlink d7/d9/d17/d23/d2e/d34/dbf/dd9/l101 0 2024-03-16T10:45:04.484 INFO:tasks.workunit.client.0.smithi002.stdout:7/610: stat d3/d4/d1d/d87/d96/d5e/lbf 0 2024-03-16T10:45:04.484 INFO:tasks.workunit.client.0.smithi002.stdout:8/761: creat d7/d9/d25/f102 x:0 0 0 2024-03-16T10:45:04.485 INFO:tasks.workunit.client.0.smithi002.stdout:9/720: dwrite d5/d9/dc/d28/dd6/fd8 [0,4194304] 0 2024-03-16T10:45:04.487 INFO:tasks.workunit.client.0.smithi002.stdout:4/643: dread d2/d33/da/f7d [0,4194304] 0 2024-03-16T10:45:04.488 INFO:tasks.workunit.client.0.smithi002.stdout:1/694: dread d2/df/f45 [0,4194304] 0 2024-03-16T10:45:04.489 INFO:tasks.workunit.client.0.smithi002.stdout:6/698: dwrite f11 [0,4194304] 0 2024-03-16T10:45:04.491 INFO:tasks.workunit.client.0.smithi002.stdout:9/721: write d5/fd [6984418,32726] 0 2024-03-16T10:45:04.492 INFO:tasks.workunit.client.0.smithi002.stdout:8/762: creat d7/d13/db8/d8f/f103 x:0 0 0 2024-03-16T10:45:04.495 INFO:tasks.workunit.client.0.smithi002.stdout:4/644: mkdir d2/d33/da/d2e/dc3 0 2024-03-16T10:45:04.498 INFO:tasks.workunit.client.0.smithi002.stdout:6/699: rmdir d15/d2d/d2b/d3f/d77/d7e 39 2024-03-16T10:45:04.499 INFO:tasks.workunit.client.0.smithi002.stdout:6/700: fdatasync d15/d2d/d2b/d75/fce 0 2024-03-16T10:45:04.501 INFO:tasks.workunit.client.0.smithi002.stdout:0/705: getdents d3/d28/d65 0 2024-03-16T10:45:04.501 INFO:tasks.workunit.client.0.smithi002.stdout:7/611: write d3/d37/d6a/f6e [3484906,50093] 0 2024-03-16T10:45:04.501 INFO:tasks.workunit.client.0.smithi002.stdout:8/763: stat d7/d9/d17/d23/la4 0 2024-03-16T10:45:04.502 INFO:tasks.workunit.client.0.smithi002.stdout:7/612: stat d3/d4/d1d/d87/d5c/d63/lc0 0 2024-03-16T10:45:04.503 INFO:tasks.workunit.client.0.smithi002.stdout:9/722: rename d5/d9/d22/d3c/d4d/d7f/c98 to d5/d9/dc/d28/dd6/ce1 0 2024-03-16T10:45:04.504 INFO:tasks.workunit.client.0.smithi002.stdout:9/723: stat d5/fd 0 2024-03-16T10:45:04.504 INFO:tasks.workunit.client.0.smithi002.stdout:9/724: chown d5/d9/d25 393538463 1 2024-03-16T10:45:04.506 INFO:tasks.workunit.client.0.smithi002.stdout:6/701: stat d15/d2d/d8a/l7f 0 2024-03-16T10:45:04.507 INFO:tasks.workunit.client.0.smithi002.stdout:1/695: mkdir d2/df/d12/d84/d87/d96/db5/dde 0 2024-03-16T10:45:04.508 INFO:tasks.workunit.client.0.smithi002.stdout:0/706: rmdir d3/d28/d2a/d67 39 2024-03-16T10:45:04.509 INFO:tasks.workunit.client.0.smithi002.stdout:1/696: chown d2/df/d12/d84/d87/d96/db5/d51/ca1 12668 1 2024-03-16T10:45:04.509 INFO:tasks.workunit.client.0.smithi002.stdout:0/707: stat d3/d59/cba 0 2024-03-16T10:45:04.510 INFO:tasks.workunit.client.0.smithi002.stdout:4/645: symlink d2/d33/lc4 0 2024-03-16T10:45:04.510 INFO:tasks.workunit.client.0.smithi002.stdout:7/613: dwrite d3/d61/d85/fb6 [0,4194304] 0 2024-03-16T10:45:04.513 INFO:tasks.workunit.client.0.smithi002.stdout:6/702: chown d15/d2d/d2b/l57 3438037 1 2024-03-16T10:45:04.513 INFO:tasks.workunit.client.0.smithi002.stdout:9/725: rmdir d5/d9/dc/d1e/d24/d31/db3 0 2024-03-16T10:45:04.515 INFO:tasks.workunit.client.0.smithi002.stdout:4/646: mknod d2/d33/da/d5e/cc5 0 2024-03-16T10:45:04.516 INFO:tasks.workunit.client.0.smithi002.stdout:6/703: chown d15/d2d/d2c/d5d/c62 80 1 2024-03-16T10:45:04.516 INFO:tasks.workunit.client.0.smithi002.stdout:0/708: symlink d3/d28/d2a/dc7/le3 0 2024-03-16T10:45:04.517 INFO:tasks.workunit.client.0.smithi002.stdout:4/647: chown d2/d33/f6a 5 1 2024-03-16T10:45:04.518 INFO:tasks.workunit.client.0.smithi002.stdout:8/764: dwrite d7/f21 [0,4194304] 0 2024-03-16T10:45:04.519 INFO:tasks.workunit.client.0.smithi002.stdout:1/697: creat d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/fdf x:0 0 0 2024-03-16T10:45:04.520 INFO:tasks.workunit.client.0.smithi002.stdout:9/726: creat d5/d9/dc/d1e/d24/fe2 x:0 0 0 2024-03-16T10:45:04.522 INFO:tasks.workunit.client.0.smithi002.stdout:4/648: chown d2/d33/da/d2e/c8a 47 1 2024-03-16T10:45:04.522 INFO:tasks.workunit.client.0.smithi002.stdout:6/704: chown d15/d2d/d8a/l81 24654 1 2024-03-16T10:45:04.527 INFO:tasks.workunit.client.0.smithi002.stdout:0/709: unlink d3/d28/d2a/d67/f89 0 2024-03-16T10:45:04.527 INFO:tasks.workunit.client.0.smithi002.stdout:9/727: write d5/d9/dc/d1e/d94/fa3 [548924,5658] 0 2024-03-16T10:45:04.528 INFO:tasks.workunit.client.0.smithi002.stdout:7/614: dwrite d3/d4/d1d/d87/d5c/d63/d3f/fac [0,4194304] 0 2024-03-16T10:45:04.530 INFO:tasks.workunit.client.0.smithi002.stdout:6/705: truncate d15/d2d/d2c/d5d/f6f 425882 0 2024-03-16T10:45:04.531 INFO:tasks.workunit.client.0.smithi002.stdout:0/710: read d3/d27/f96 [5524126,12326] 0 2024-03-16T10:45:04.531 INFO:tasks.workunit.client.0.smithi002.stdout:8/765: rename d7/d13/db8/cfe to d7/d9/d17/d23/d2e/d5c/df0/c104 0 2024-03-16T10:45:04.533 INFO:tasks.workunit.client.0.smithi002.stdout:9/728: chown d5/d9/dc/d1e/d24/fc6 20497421 1 2024-03-16T10:45:04.534 INFO:tasks.workunit.client.0.smithi002.stdout:8/766: dread - d7/d9/d17/fb0 zero size 2024-03-16T10:45:04.539 INFO:tasks.workunit.client.0.smithi002.stdout:9/729: stat d5/d9/dc/d28/f4c 0 2024-03-16T10:45:04.539 INFO:tasks.workunit.client.0.smithi002.stdout:4/649: dread d2/d33/da/d2e/f8e [0,4194304] 0 2024-03-16T10:45:04.539 INFO:tasks.workunit.client.0.smithi002.stdout:0/711: mkdir d3/d31/d75/de4 0 2024-03-16T10:45:04.541 INFO:tasks.workunit.client.0.smithi002.stdout:8/767: unlink d7/d9/d17/d23/d2e/le2 0 2024-03-16T10:45:04.542 INFO:tasks.workunit.client.0.smithi002.stdout:7/615: dwrite d3/d61/d66/f50 [4194304,4194304] 0 2024-03-16T10:45:04.544 INFO:tasks.workunit.client.0.smithi002.stdout:0/712: rename d3/d59/fb0 to d3/d31/fe5 0 2024-03-16T10:45:04.546 INFO:tasks.workunit.client.0.smithi002.stdout:1/698: truncate d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/faa 2998955 0 2024-03-16T10:45:04.547 INFO:tasks.workunit.client.0.smithi002.stdout:8/768: rmdir d7/d9/d17/d23/d2e/d3a 39 2024-03-16T10:45:04.550 INFO:tasks.workunit.client.0.smithi002.stdout:0/713: symlink d3/le6 0 2024-03-16T10:45:04.552 INFO:tasks.workunit.client.0.smithi002.stdout:7/616: rename d3/d4/d1c/c26 to d3/d4/d1d/d45/cdd 0 2024-03-16T10:45:04.553 INFO:tasks.workunit.client.0.smithi002.stdout:4/650: dwrite d2/d33/d1c/f67 [0,4194304] 0 2024-03-16T10:45:04.556 INFO:tasks.workunit.client.0.smithi002.stdout:0/714: dread - d3/da0/fb1 zero size 2024-03-16T10:45:04.556 INFO:tasks.workunit.client.0.smithi002.stdout:8/769: symlink d7/d9/d17/d20/l105 0 2024-03-16T10:45:04.556 INFO:tasks.workunit.client.0.smithi002.stdout:2/663: sync 2024-03-16T10:45:04.557 INFO:tasks.workunit.client.0.smithi002.stdout:6/706: dwrite d15/f9a [0,4194304] 0 2024-03-16T10:45:04.557 INFO:tasks.workunit.client.0.smithi002.stdout:1/699: truncate d2/df/f45 1120761 0 2024-03-16T10:45:04.558 INFO:tasks.workunit.client.0.smithi002.stdout:9/730: dwrite d5/d9/d55/dc9/fbf [0,4194304] 0 2024-03-16T10:45:04.565 INFO:tasks.workunit.client.0.smithi002.stdout:9/731: read - d5/d9/fce zero size 2024-03-16T10:45:04.567 INFO:tasks.workunit.client.0.smithi002.stdout:7/617: mkdir d3/d4/dde 0 2024-03-16T10:45:04.571 INFO:tasks.workunit.client.0.smithi002.stdout:8/770: dread d7/d13/db8/f98 [0,4194304] 0 2024-03-16T10:45:04.571 INFO:tasks.workunit.client.0.smithi002.stdout:1/700: mkdir d2/de0 0 2024-03-16T10:45:04.572 INFO:tasks.workunit.client.0.smithi002.stdout:6/707: symlink d15/d2d/lf0 0 2024-03-16T10:45:04.574 INFO:tasks.workunit.client.0.smithi002.stdout:7/618: readlink d3/l20 0 2024-03-16T10:45:04.575 INFO:tasks.workunit.client.0.smithi002.stdout:0/715: symlink d3/d31/d75/db4/le7 0 2024-03-16T10:45:04.575 INFO:tasks.workunit.client.0.smithi002.stdout:7/619: chown d3/d4/d1d/d87/d96/d5e 99643223 1 2024-03-16T10:45:04.576 INFO:tasks.workunit.client.0.smithi002.stdout:0/716: stat d3/d59/dca/faf 0 2024-03-16T10:45:04.576 INFO:tasks.workunit.client.0.smithi002.stdout:9/732: rename d5/d9/d25/d23/fcc to d5/d9/d25/d23/d81/fe3 0 2024-03-16T10:45:04.577 INFO:tasks.workunit.client.0.smithi002.stdout:4/651: rmdir d2/d33/da/d2e/dc3 0 2024-03-16T10:45:04.581 INFO:tasks.workunit.client.0.smithi002.stdout:6/708: write d15/d2d/da8/fc9 [2423418,108083] 0 2024-03-16T10:45:04.581 INFO:tasks.workunit.client.0.smithi002.stdout:2/664: creat d7/d2e/d1d/d38/fcc x:0 0 0 2024-03-16T10:45:04.583 INFO:tasks.workunit.client.0.smithi002.stdout:8/771: fsync d7/f31 0 2024-03-16T10:45:04.584 INFO:tasks.workunit.client.0.smithi002.stdout:4/652: mknod d2/d6/d32/d68/cc6 0 2024-03-16T10:45:04.586 INFO:tasks.workunit.client.0.smithi002.stdout:1/701: creat d2/fe1 x:0 0 0 2024-03-16T10:45:04.589 INFO:tasks.workunit.client.0.smithi002.stdout:9/733: dwrite d5/d9/dc/f92 [0,4194304] 0 2024-03-16T10:45:04.589 INFO:tasks.workunit.client.0.smithi002.stdout:0/717: dwrite d3/d28/d65/f77 [0,4194304] 0 2024-03-16T10:45:04.589 INFO:tasks.workunit.client.0.smithi002.stdout:6/709: unlink d15/d2d/d2c/f30 0 2024-03-16T10:45:04.591 INFO:tasks.workunit.client.0.smithi002.stdout:3/670: sync 2024-03-16T10:45:04.597 INFO:tasks.workunit.client.0.smithi002.stdout:4/653: write d2/f79 [274068,119448] 0 2024-03-16T10:45:04.600 INFO:tasks.workunit.client.0.smithi002.stdout:0/718: write d3/fa5 [56445,127562] 0 2024-03-16T10:45:04.601 INFO:tasks.workunit.client.0.smithi002.stdout:8/772: rename d7/d9/d17/d23/d39/db7/dc2 to d7/d9/d17/d23/d39/db7/dc2/d106 22 2024-03-16T10:45:04.603 INFO:tasks.workunit.client.0.smithi002.stdout:7/620: dwrite d3/f91 [4194304,4194304] 0 2024-03-16T10:45:04.603 INFO:tasks.workunit.client.0.smithi002.stdout:1/702: mkdir d2/df/d12/d84/d87/d96/db5/de2 0 2024-03-16T10:45:04.604 INFO:tasks.workunit.client.0.smithi002.stdout:0/719: fsync d3/d28/f8e 0 2024-03-16T10:45:04.607 INFO:tasks.workunit.client.0.smithi002.stdout:9/734: fsync d5/d9/dc/d28/d90/fa7 0 2024-03-16T10:45:04.609 INFO:tasks.workunit.client.0.smithi002.stdout:8/773: symlink d7/d9/d17/d23/d2e/d5c/l107 0 2024-03-16T10:45:04.610 INFO:tasks.workunit.client.0.smithi002.stdout:2/665: dwrite d7/d2e/d1d/d2f/d4b/d6e/fc6 [0,4194304] 0 2024-03-16T10:45:04.611 INFO:tasks.workunit.client.0.smithi002.stdout:6/710: write d15/d2d/d2c/f54 [1038211,29529] 0 2024-03-16T10:45:04.612 INFO:tasks.workunit.client.0.smithi002.stdout:8/774: chown d7/d9 224 1 2024-03-16T10:45:04.614 INFO:tasks.workunit.client.0.smithi002.stdout:9/735: chown d5/d9/d22/caa 3461 1 2024-03-16T10:45:04.614 INFO:tasks.workunit.client.0.smithi002.stdout:1/703: readlink d2/df/d12/d84/d87/d96/db5/d1c/lc7 0 2024-03-16T10:45:04.616 INFO:tasks.workunit.client.0.smithi002.stdout:0/720: mknod d3/d28/d2a/dc7/ce8 0 2024-03-16T10:45:04.617 INFO:tasks.workunit.client.0.smithi002.stdout:2/666: unlink d7/d2e/d1d/d38/d49/l4e 0 2024-03-16T10:45:04.618 INFO:tasks.workunit.client.0.smithi002.stdout:6/711: chown d15/d2d/d2c/l37 417440 1 2024-03-16T10:45:04.618 INFO:tasks.workunit.client.0.smithi002.stdout:8/775: truncate d7/f31 375890 0 2024-03-16T10:45:04.619 INFO:tasks.workunit.client.0.smithi002.stdout:0/721: symlink d3/d31/d75/dde/le9 0 2024-03-16T10:45:04.619 INFO:tasks.workunit.client.0.smithi002.stdout:9/736: write d5/d9/dc/d28/f65 [686528,120599] 0 2024-03-16T10:45:04.621 INFO:tasks.workunit.client.0.smithi002.stdout:1/704: symlink d2/df/d12/d84/d87/d96/db5/d51/d56/le3 0 2024-03-16T10:45:04.622 INFO:tasks.workunit.client.0.smithi002.stdout:1/705: write d2/df/d12/f4e [1140352,19732] 0 2024-03-16T10:45:04.623 INFO:tasks.workunit.client.0.smithi002.stdout:3/671: truncate d1/dd/d28/d7b/d2c/d63/f79 681315 0 2024-03-16T10:45:04.625 INFO:tasks.workunit.client.0.smithi002.stdout:1/706: chown d2/df/d12/d84/d87/d96/db5/f4b 3546 1 2024-03-16T10:45:04.625 INFO:tasks.workunit.client.0.smithi002.stdout:0/722: mkdir d3/d31/d75/dea 0 2024-03-16T10:45:04.627 INFO:tasks.workunit.client.0.smithi002.stdout:7/621: dwrite d3/d37/f48 [0,4194304] 0 2024-03-16T10:45:04.628 INFO:tasks.workunit.client.0.smithi002.stdout:2/667: mkdir d7/d2e/d28/d40/d50/db4/dcd 0 2024-03-16T10:45:04.632 INFO:tasks.workunit.client.0.smithi002.stdout:6/712: creat d15/d2d/d2b/d3f/d77/de1/dec/ff1 x:0 0 0 2024-03-16T10:45:04.633 INFO:tasks.workunit.client.0.smithi002.stdout:0/723: fdatasync d3/d28/f54 0 2024-03-16T10:45:04.635 INFO:tasks.workunit.client.0.smithi002.stdout:9/737: mkdir d5/d9/dc/de4 0 2024-03-16T10:45:04.636 INFO:tasks.workunit.client.0.smithi002.stdout:7/622: mkdir d3/d4/d1c/d5b/d64/ddf 0 2024-03-16T10:45:04.637 INFO:tasks.workunit.client.0.smithi002.stdout:2/668: symlink d7/db/d12/lce 0 2024-03-16T10:45:04.638 INFO:tasks.workunit.client.0.smithi002.stdout:2/669: write d7/d2e/d1d/fc2 [832368,82546] 0 2024-03-16T10:45:04.640 INFO:tasks.workunit.client.0.smithi002.stdout:6/713: creat d15/d2d/d2c/d8f/d96/dbc/ff2 x:0 0 0 2024-03-16T10:45:04.640 INFO:tasks.workunit.client.0.smithi002.stdout:2/670: fdatasync d7/d2e/d1d/f9e 0 2024-03-16T10:45:04.642 INFO:tasks.workunit.client.0.smithi002.stdout:3/672: truncate d1/dd/fe7 4493794 0 2024-03-16T10:45:04.642 INFO:tasks.workunit.client.0.smithi002.stdout:2/671: fdatasync d7/d2e/d28/d40/fc4 0 2024-03-16T10:45:04.643 INFO:tasks.workunit.client.0.smithi002.stdout:2/672: write d7/d2e/d1d/d2f/d4b/f62 [2427241,58092] 0 2024-03-16T10:45:04.644 INFO:tasks.workunit.client.0.smithi002.stdout:1/707: dread d2/df/d12/f60 [0,4194304] 0 2024-03-16T10:45:04.645 INFO:tasks.workunit.client.0.smithi002.stdout:2/673: write d7/d2e/d1d/f21 [4288404,113773] 0 2024-03-16T10:45:04.647 INFO:tasks.workunit.client.0.smithi002.stdout:0/724: unlink d3/d28/f3b 0 2024-03-16T10:45:04.648 INFO:tasks.workunit.client.0.smithi002.stdout:0/725: chown d3/d28/d3d/f46 2 1 2024-03-16T10:45:04.648 INFO:tasks.workunit.client.0.smithi002.stdout:5/583: sync 2024-03-16T10:45:04.648 INFO:tasks.workunit.client.0.smithi002.stdout:9/738: stat d5/d9/d25/l21 0 2024-03-16T10:45:04.649 INFO:tasks.workunit.client.0.smithi002.stdout:9/739: read - d5/d9/d25/fbb zero size 2024-03-16T10:45:04.650 INFO:tasks.workunit.client.0.smithi002.stdout:3/673: chown d1/db6/d15/l21 14567 1 2024-03-16T10:45:04.652 INFO:tasks.workunit.client.0.smithi002.stdout:3/674: truncate d1/dd/d28/d7b/d2c/d63/fea 636600 0 2024-03-16T10:45:04.652 INFO:tasks.workunit.client.0.smithi002.stdout:8/776: link d7/d9/d17/d23/d2e/d3a/f5f d7/f108 0 2024-03-16T10:45:04.653 INFO:tasks.workunit.client.0.smithi002.stdout:3/675: fdatasync d1/dd/d28/d7b/d8a/fc8 0 2024-03-16T10:45:04.653 INFO:tasks.workunit.client.0.smithi002.stdout:8/777: chown d7/d13/db8/f7b 22 1 2024-03-16T10:45:04.654 INFO:tasks.workunit.client.0.smithi002.stdout:1/708: creat d2/df/d12/dd7/fe4 x:0 0 0 2024-03-16T10:45:04.655 INFO:tasks.workunit.client.0.smithi002.stdout:2/674: mknod d7/d2e/d1d/d2f/ccf 0 2024-03-16T10:45:04.656 INFO:tasks.workunit.client.0.smithi002.stdout:0/726: creat d3/d59/feb x:0 0 0 2024-03-16T10:45:04.657 INFO:tasks.workunit.client.0.smithi002.stdout:7/623: mknod d3/d61/ce0 0 2024-03-16T10:45:04.657 INFO:tasks.workunit.client.0.smithi002.stdout:6/714: symlink d15/dab/dc4/de8/lf3 0 2024-03-16T10:45:04.661 INFO:tasks.workunit.client.0.smithi002.stdout:0/727: dread d3/d28/d65/f7a [0,4194304] 0 2024-03-16T10:45:04.662 INFO:tasks.workunit.client.0.smithi002.stdout:2/675: dwrite d7/d2e/d1d/d2f/f7d [0,4194304] 0 2024-03-16T10:45:04.665 INFO:tasks.workunit.client.0.smithi002.stdout:7/624: rmdir d3/d4/d1d/d87/d5c/dd0 39 2024-03-16T10:45:04.668 INFO:tasks.workunit.client.0.smithi002.stdout:8/778: truncate d7/fc 1959491 0 2024-03-16T10:45:04.670 INFO:tasks.workunit.client.0.smithi002.stdout:1/709: symlink d2/de0/le5 0 2024-03-16T10:45:04.670 INFO:tasks.workunit.client.0.smithi002.stdout:1/710: readlink d2/df/l91 0 2024-03-16T10:45:04.672 INFO:tasks.workunit.client.0.smithi002.stdout:2/676: creat d7/d2e/d28/d44/d6d/fd0 x:0 0 0 2024-03-16T10:45:04.672 INFO:tasks.workunit.client.0.smithi002.stdout:3/676: dread d1/dd/d28/d7b/f24 [0,4194304] 0 2024-03-16T10:45:04.673 INFO:tasks.workunit.client.0.smithi002.stdout:2/677: dread - d7/d2e/d1d/f9e zero size 2024-03-16T10:45:04.673 INFO:tasks.workunit.client.0.smithi002.stdout:9/740: write d5/d9/dc/f1f [1509403,48545] 0 2024-03-16T10:45:04.675 INFO:tasks.workunit.client.0.smithi002.stdout:0/728: rename d3/d31/d75/db4/ddc/lb8 to d3/d28/d2a/d67/lec 0 2024-03-16T10:45:04.677 INFO:tasks.workunit.client.0.smithi002.stdout:7/625: symlink d3/d4/d1c/d5b/d64/le1 0 2024-03-16T10:45:04.677 INFO:tasks.workunit.client.0.smithi002.stdout:7/626: chown d3/d61 387734 1 2024-03-16T10:45:04.682 INFO:tasks.workunit.client.0.smithi002.stdout:6/715: creat d15/d2d/d2b/ff4 x:0 0 0 2024-03-16T10:45:04.686 INFO:tasks.workunit.client.0.smithi002.stdout:2/678: symlink d7/d27/ld1 0 2024-03-16T10:45:04.688 INFO:tasks.workunit.client.0.smithi002.stdout:3/677: write d1/fb [1442163,21641] 0 2024-03-16T10:45:04.689 INFO:tasks.workunit.client.0.smithi002.stdout:7/627: symlink d3/d61/le2 0 2024-03-16T10:45:04.691 INFO:tasks.workunit.client.0.smithi002.stdout:8/779: truncate d7/d9/d17/d20/f63 1011019 0 2024-03-16T10:45:04.691 INFO:tasks.workunit.client.0.smithi002.stdout:0/729: dread d3/de/f51 [0,4194304] 0 2024-03-16T10:45:04.692 INFO:tasks.workunit.client.0.smithi002.stdout:6/716: fsync d15/d2d/f6c 0 2024-03-16T10:45:04.692 INFO:tasks.workunit.client.0.smithi002.stdout:3/678: dread - d1/dd/d28/f9d zero size 2024-03-16T10:45:04.693 INFO:tasks.workunit.client.0.smithi002.stdout:1/711: getdents d2/df 0 2024-03-16T10:45:04.695 INFO:tasks.workunit.client.0.smithi002.stdout:3/679: chown d1/dd/d74 0 1 2024-03-16T10:45:04.695 INFO:tasks.workunit.client.0.smithi002.stdout:1/712: chown d2/df/d12/d29 37667927 1 2024-03-16T10:45:04.696 INFO:tasks.workunit.client.0.smithi002.stdout:8/780: mkdir d7/d9/d17/d23/d2e/d34/dbf/d109 0 2024-03-16T10:45:04.697 INFO:tasks.workunit.client.0.smithi002.stdout:1/713: read d2/f68 [3886854,45935] 0 2024-03-16T10:45:04.699 INFO:tasks.workunit.client.0.smithi002.stdout:6/717: mkdir d15/d80/df5 0 2024-03-16T10:45:04.699 INFO:tasks.workunit.client.0.smithi002.stdout:2/679: dwrite d7/db/d12/f5d [0,4194304] 0 2024-03-16T10:45:04.699 INFO:tasks.workunit.client.0.smithi002.stdout:1/714: write d2/f76 [925118,15812] 0 2024-03-16T10:45:04.701 INFO:tasks.workunit.client.0.smithi002.stdout:9/741: link d5/d9/dc/d1e/d24/d31/l66 d5/d9/le5 0 2024-03-16T10:45:04.702 INFO:tasks.workunit.client.0.smithi002.stdout:0/730: mknod d3/d31/ced 0 2024-03-16T10:45:04.703 INFO:tasks.workunit.client.0.smithi002.stdout:3/680: fsync d1/f1d 0 2024-03-16T10:45:04.707 INFO:tasks.workunit.client.0.smithi002.stdout:7/628: write d3/d4/d1d/d87/d5c/d63/d55/f5d [621423,37937] 0 2024-03-16T10:45:04.709 INFO:tasks.workunit.client.0.smithi002.stdout:6/718: creat d15/d2d/d2c/d5d/ff6 x:0 0 0 2024-03-16T10:45:04.710 INFO:tasks.workunit.client.0.smithi002.stdout:1/715: dread d2/df/d12/f4e [0,4194304] 0 2024-03-16T10:45:04.713 INFO:tasks.workunit.client.0.smithi002.stdout:9/742: creat d5/d9/dc/d28/fe6 x:0 0 0 2024-03-16T10:45:04.713 INFO:tasks.workunit.client.0.smithi002.stdout:2/680: dread d7/d2e/d1d/f4f [0,4194304] 0 2024-03-16T10:45:04.714 INFO:tasks.workunit.client.0.smithi002.stdout:9/743: chown d5/d9/d22/d3c/d4d/l5e 2 1 2024-03-16T10:45:04.715 INFO:tasks.workunit.client.0.smithi002.stdout:3/681: symlink d1/dd/d28/d7b/d2c/d97/led 0 2024-03-16T10:45:04.719 INFO:tasks.workunit.client.0.smithi002.stdout:1/716: rename d2/df/d12/d84/d87/d96/db5/d1c/d5d/c5f to d2/d58/dbb/ce6 0 2024-03-16T10:45:04.721 INFO:tasks.workunit.client.0.smithi002.stdout:2/681: mkdir d7/d27/d3b/d95/dd2 0 2024-03-16T10:45:04.721 INFO:tasks.workunit.client.0.smithi002.stdout:8/781: link d7/d9/d17/d20/f5e d7/de3/f10a 0 2024-03-16T10:45:04.722 INFO:tasks.workunit.client.0.smithi002.stdout:0/731: write d3/d28/d65/f7a [623803,63599] 0 2024-03-16T10:45:04.722 INFO:tasks.workunit.client.0.smithi002.stdout:8/782: dread - d7/d13/db8/ff2 zero size 2024-03-16T10:45:04.723 INFO:tasks.workunit.client.0.smithi002.stdout:3/682: mkdir d1/dd/d28/d7b/dd5/dee 0 2024-03-16T10:45:04.723 INFO:tasks.workunit.client.0.smithi002.stdout:7/629: truncate d3/d4/d1c/d5b/f6d 1498221 0 2024-03-16T10:45:04.724 INFO:tasks.workunit.client.0.smithi002.stdout:6/719: truncate d15/d2d/fed 405750 0 2024-03-16T10:45:04.725 INFO:tasks.workunit.client.0.smithi002.stdout:1/717: symlink d2/df/d12/d74/le7 0 2024-03-16T10:45:04.726 INFO:tasks.workunit.client.0.smithi002.stdout:2/682: creat d7/d2e/d1d/fd3 x:0 0 0 2024-03-16T10:45:04.727 INFO:tasks.workunit.client.0.smithi002.stdout:0/732: chown d3/d31/c41 29010320 1 2024-03-16T10:45:04.727 INFO:tasks.workunit.client.0.smithi002.stdout:2/683: fsync d7/d27/f56 0 2024-03-16T10:45:04.728 INFO:tasks.workunit.client.0.smithi002.stdout:1/718: mkdir d2/d58/de8 0 2024-03-16T10:45:04.729 INFO:tasks.workunit.client.0.smithi002.stdout:9/744: rename d5/l6 to d5/d9/d25/le7 0 2024-03-16T10:45:04.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:04 smithi026 ceph-mon[37041]: pgmap v35: 65 pgs: 65 active+clean; 1.3 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 38 MiB/s rd, 162 MiB/s wr, 362 op/s 2024-03-16T10:45:04.732 INFO:tasks.workunit.client.0.smithi002.stdout:9/745: write d5/d9/dc/f11 [4880123,29828] 0 2024-03-16T10:45:04.732 INFO:tasks.workunit.client.0.smithi002.stdout:7/630: dwrite d3/d4/d1c/d5b/fca [0,4194304] 0 2024-03-16T10:45:04.732 INFO:tasks.workunit.client.0.smithi002.stdout:0/733: creat d3/d59/dca/fee x:0 0 0 2024-03-16T10:45:04.735 INFO:tasks.workunit.client.0.smithi002.stdout:8/783: link d7/d9/d17/f27 d7/d9/d17/f10b 0 2024-03-16T10:45:04.736 INFO:tasks.workunit.client.0.smithi002.stdout:0/734: write d3/d28/f33 [3244058,68172] 0 2024-03-16T10:45:04.736 INFO:tasks.workunit.client.0.smithi002.stdout:6/720: rename d15/d2d/d2c/d5d/f6f to d15/d2d/d2b/d3f/ff7 0 2024-03-16T10:45:04.738 INFO:tasks.workunit.client.0.smithi002.stdout:8/784: write d7/d13/db8/fe7 [529420,17593] 0 2024-03-16T10:45:04.742 INFO:tasks.workunit.client.0.smithi002.stdout:1/719: creat d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/fe9 x:0 0 0 2024-03-16T10:45:04.743 INFO:tasks.workunit.client.0.smithi002.stdout:9/746: symlink d5/d9/d55/le8 0 2024-03-16T10:45:04.746 INFO:tasks.workunit.client.0.smithi002.stdout:0/735: truncate d3/f49 611796 0 2024-03-16T10:45:04.747 INFO:tasks.workunit.client.0.smithi002.stdout:8/785: rmdir d7/d9/d17/d23/d2e/d5c/df0 39 2024-03-16T10:45:04.749 INFO:tasks.workunit.client.0.smithi002.stdout:3/683: dwrite d1/db6/f4a [0,4194304] 0 2024-03-16T10:45:04.749 INFO:tasks.workunit.client.0.smithi002.stdout:9/747: dwrite d5/d9/fce [0,4194304] 0 2024-03-16T10:45:04.752 INFO:tasks.workunit.client.0.smithi002.stdout:6/721: mknod d15/d2d/d2b/d3f/d77/d9f/cf8 0 2024-03-16T10:45:04.754 INFO:tasks.workunit.client.0.smithi002.stdout:9/748: truncate d5/d9/dc/d1e/fbc 691782 0 2024-03-16T10:45:04.755 INFO:tasks.workunit.client.0.smithi002.stdout:9/749: chown d5/d9/d55/dc9/lc3 0 1 2024-03-16T10:45:04.757 INFO:tasks.workunit.client.0.smithi002.stdout:3/684: creat d1/dd/d28/d7b/d2c/d97/fef x:0 0 0 2024-03-16T10:45:04.758 INFO:tasks.workunit.client.0.smithi002.stdout:9/750: dread d5/d9/dc/d1e/fbc [0,4194304] 0 2024-03-16T10:45:04.759 INFO:tasks.workunit.client.0.smithi002.stdout:1/720: creat d2/df/d12/fea x:0 0 0 2024-03-16T10:45:04.761 INFO:tasks.workunit.client.0.smithi002.stdout:6/722: mkdir d15/d2d/d8a/d7c/d87/df9 0 2024-03-16T10:45:04.762 INFO:tasks.workunit.client.0.smithi002.stdout:7/631: link d3/d4/d1d/d87/d5c/d63/d34/c4b d3/d4/d1d/d87/d5c/dd0/dd2/ce3 0 2024-03-16T10:45:04.766 INFO:tasks.workunit.client.0.smithi002.stdout:3/685: dwrite d1/dd/d28/d7b/d8a/fd0 [0,4194304] 0 2024-03-16T10:45:04.768 INFO:tasks.workunit.client.0.smithi002.stdout:1/721: mkdir d2/df/d12/dd7/dd9/deb 0 2024-03-16T10:45:04.770 INFO:tasks.workunit.client.0.smithi002.stdout:1/722: chown d2/df/d12/d84/d87/d96/fcb 120052117 1 2024-03-16T10:45:04.771 INFO:tasks.workunit.client.0.smithi002.stdout:7/632: symlink d3/d4/d1d/d87/d5c/dc3/le4 0 2024-03-16T10:45:04.771 INFO:tasks.workunit.client.0.smithi002.stdout:8/786: write d7/fc [74292,37302] 0 2024-03-16T10:45:04.772 INFO:tasks.workunit.client.0.smithi002.stdout:7/633: readlink d3/l20 0 2024-03-16T10:45:04.775 INFO:tasks.workunit.client.0.smithi002.stdout:9/751: creat d5/d9/d34/fe9 x:0 0 0 2024-03-16T10:45:04.779 INFO:tasks.workunit.client.0.smithi002.stdout:3/686: getdents d1/db6/d15 0 2024-03-16T10:45:04.779 INFO:tasks.workunit.client.0.smithi002.stdout:7/634: link d3/d4/d1d/d87/d5c/d63/f7f d3/d4/d1d/d87/d5c/d63/d3f/fe5 0 2024-03-16T10:45:04.780 INFO:tasks.workunit.client.0.smithi002.stdout:9/752: mknod d5/d9/dc/de4/cea 0 2024-03-16T10:45:04.782 INFO:tasks.workunit.client.0.smithi002.stdout:1/723: dwrite d2/df/d12/f72 [0,4194304] 0 2024-03-16T10:45:04.782 INFO:tasks.workunit.client.0.smithi002.stdout:7/635: chown d3/d4/d1d/d45/fa1 7728206 1 2024-03-16T10:45:04.782 INFO:tasks.workunit.client.0.smithi002.stdout:8/787: link d7/f8 d7/d9/d17/d83/df5/f10c 0 2024-03-16T10:45:04.783 INFO:tasks.workunit.client.0.smithi002.stdout:6/723: dread d15/d2d/d2b/f61 [0,4194304] 0 2024-03-16T10:45:04.784 INFO:tasks.workunit.client.0.smithi002.stdout:9/753: chown d5/d9/dc/d1e/d24/d31/f32 129 1 2024-03-16T10:45:04.786 INFO:tasks.workunit.client.0.smithi002.stdout:6/724: truncate d15/f67 244796 0 2024-03-16T10:45:04.788 INFO:tasks.workunit.client.0.smithi002.stdout:9/754: creat d5/d9/dc/d28/dd6/feb x:0 0 0 2024-03-16T10:45:04.791 INFO:tasks.workunit.client.0.smithi002.stdout:8/788: creat d7/d9/d17/d23/d39/db7/f10d x:0 0 0 2024-03-16T10:45:04.792 INFO:tasks.workunit.client.0.smithi002.stdout:6/725: read d15/d2d/fbd [3930924,76664] 0 2024-03-16T10:45:04.794 INFO:tasks.workunit.client.0.smithi002.stdout:8/789: stat d7/d13/f84 0 2024-03-16T10:45:04.795 INFO:tasks.workunit.client.0.smithi002.stdout:3/687: truncate d1/dd/f68 1051628 0 2024-03-16T10:45:04.795 INFO:tasks.workunit.client.0.smithi002.stdout:1/724: dwrite d2/df/d12/d84/d87/d96/db5/f7c [4194304,4194304] 0 2024-03-16T10:45:04.797 INFO:tasks.workunit.client.0.smithi002.stdout:7/636: dwrite d3/d4/d1d/d87/d96/d5e/fbd [0,4194304] 0 2024-03-16T10:45:04.800 INFO:tasks.workunit.client.0.smithi002.stdout:9/755: rename d5/d9/dc/de4 to d5/db1/dec 0 2024-03-16T10:45:04.800 INFO:tasks.workunit.client.0.smithi002.stdout:3/688: dread - d1/dd/d28/d7b/d2c/d97/fef zero size 2024-03-16T10:45:04.802 INFO:tasks.workunit.client.0.smithi002.stdout:7/637: dread - d3/d4/d1d/d45/f82 zero size 2024-03-16T10:45:04.803 INFO:tasks.workunit.client.0.smithi002.stdout:8/790: rename d7/d9/d17/fb0 to d7/d9/d17/dc3/f10e 0 2024-03-16T10:45:04.805 INFO:tasks.workunit.client.0.smithi002.stdout:3/689: write d1/dd/d28/d7b/d8a/d93/fbf [276812,82051] 0 2024-03-16T10:45:04.809 INFO:tasks.workunit.client.0.smithi002.stdout:9/756: rename d5/d9/d22/d3c/d4d to d5/d9/dc/d1e/d24/ded 0 2024-03-16T10:45:04.811 INFO:tasks.workunit.client.0.smithi002.stdout:6/726: dwrite d15/d2d/d2b/f63 [0,4194304] 0 2024-03-16T10:45:04.814 INFO:tasks.workunit.client.0.smithi002.stdout:6/727: dread - d15/d80/fd5 zero size 2024-03-16T10:45:04.814 INFO:tasks.workunit.client.0.smithi002.stdout:1/725: dread d2/df/d12/d84/d87/d96/db5/d51/d56/f8b [0,4194304] 0 2024-03-16T10:45:04.815 INFO:tasks.workunit.client.0.smithi002.stdout:7/638: mkdir d3/d4/d1d/d87/d96/d5e/db7/de6 0 2024-03-16T10:45:04.822 INFO:tasks.workunit.client.0.smithi002.stdout:9/757: rename d5/d9/c6a to d5/db1/cee 0 2024-03-16T10:45:04.822 INFO:tasks.workunit.client.0.smithi002.stdout:3/690: dwrite d1/db6/d15/fd3 [4194304,4194304] 0 2024-03-16T10:45:04.824 INFO:tasks.workunit.client.0.smithi002.stdout:6/728: symlink d15/dab/dc4/lfa 0 2024-03-16T10:45:04.825 INFO:tasks.workunit.client.0.smithi002.stdout:7/639: symlink d3/d4/d1d/d87/d5c/d63/d3f/d75/le7 0 2024-03-16T10:45:04.827 INFO:tasks.workunit.client.0.smithi002.stdout:1/726: mknod d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/cec 0 2024-03-16T10:45:04.828 INFO:tasks.workunit.client.0.smithi002.stdout:8/791: truncate d7/d9/d17/d23/d2e/d5c/f8a 7197922 0 2024-03-16T10:45:04.832 INFO:tasks.workunit.client.0.smithi002.stdout:6/729: creat d15/dab/ffb x:0 0 0 2024-03-16T10:45:04.833 INFO:tasks.workunit.client.0.smithi002.stdout:6/730: chown d15/d2d/d8a/d7c/d87/fe3 243383449 1 2024-03-16T10:45:04.834 INFO:tasks.workunit.client.0.smithi002.stdout:6/731: write d15/d80/dbe/fdf [177018,75874] 0 2024-03-16T10:45:04.836 INFO:tasks.workunit.client.0.smithi002.stdout:1/727: mkdir d2/de0/ded 0 2024-03-16T10:45:04.837 INFO:tasks.workunit.client.0.smithi002.stdout:1/728: read - d2/df/d12/fea zero size 2024-03-16T10:45:04.837 INFO:tasks.workunit.client.0.smithi002.stdout:8/792: dread - d7/d9/d17/f74 zero size 2024-03-16T10:45:04.840 INFO:tasks.workunit.client.0.smithi002.stdout:9/758: mkdir d5/d9/dc/d1e/def 0 2024-03-16T10:45:04.840 INFO:tasks.workunit.client.0.smithi002.stdout:9/759: fdatasync d5/d9/dc/fa2 0 2024-03-16T10:45:04.841 INFO:tasks.workunit.client.0.smithi002.stdout:3/691: dread d1/d5a/f95 [4194304,4194304] 0 2024-03-16T10:45:04.841 INFO:tasks.workunit.client.0.smithi002.stdout:7/640: creat d3/d4/d1c/d5b/d64/ddf/fe8 x:0 0 0 2024-03-16T10:45:04.841 INFO:tasks.workunit.client.0.smithi002.stdout:9/760: fsync d5/d9/dc/f11 0 2024-03-16T10:45:04.842 INFO:tasks.workunit.client.0.smithi002.stdout:3/692: truncate d1/f16 8510749 0 2024-03-16T10:45:04.845 INFO:tasks.workunit.client.0.smithi002.stdout:1/729: unlink d2/d58/dbb/fc4 0 2024-03-16T10:45:04.850 INFO:tasks.workunit.client.0.smithi002.stdout:7/641: dread - d3/f57 zero size 2024-03-16T10:45:04.851 INFO:tasks.workunit.client.0.smithi002.stdout:7/642: chown d3/d4/d1d/d87/d5c/d63/lc0 0 1 2024-03-16T10:45:04.855 INFO:tasks.workunit.client.0.smithi002.stdout:3/693: read d1/dd/d28/d7b/d8a/d93/f4d [58977,80891] 0 2024-03-16T10:45:04.859 INFO:tasks.workunit.client.0.smithi002.stdout:9/761: mkdir d5/d9/dc/d1e/d91/ddc/df0 0 2024-03-16T10:45:04.862 INFO:tasks.workunit.client.0.smithi002.stdout:6/732: write d15/d80/f84 [4583600,93565] 0 2024-03-16T10:45:04.864 INFO:tasks.workunit.client.0.smithi002.stdout:1/730: fdatasync d2/df/d12/d84/d87/d96/db5/f50 0 2024-03-16T10:45:04.869 INFO:tasks.workunit.client.0.smithi002.stdout:3/694: symlink d1/dd/d28/d7b/d8a/lf0 0 2024-03-16T10:45:04.871 INFO:tasks.workunit.client.0.smithi002.stdout:6/733: mkdir d15/d2d/d2b/de4/dfc 0 2024-03-16T10:45:04.874 INFO:tasks.workunit.client.0.smithi002.stdout:9/762: mkdir d5/d9/dc/d1e/d91/df1 0 2024-03-16T10:45:04.876 INFO:tasks.workunit.client.0.smithi002.stdout:3/695: dwrite d1/dd/d28/d7b/d2c/fdb [0,4194304] 0 2024-03-16T10:45:04.876 INFO:tasks.workunit.client.0.smithi002.stdout:9/763: read - d5/d9/dc/d1e/d24/ded/d7f/fd4 zero size 2024-03-16T10:45:04.877 INFO:tasks.workunit.client.0.smithi002.stdout:1/731: dread d2/df/d12/d84/d87/d96/db5/d51/d56/f8c [0,4194304] 0 2024-03-16T10:45:04.878 INFO:tasks.workunit.client.0.smithi002.stdout:7/643: link d3/d4/d1c/f33 d3/d4/d1d/d87/d5c/dd0/dd2/fe9 0 2024-03-16T10:45:04.878 INFO:tasks.workunit.client.0.smithi002.stdout:9/764: write d5/d9/d55/fdd [716142,129229] 0 2024-03-16T10:45:04.881 INFO:tasks.workunit.client.0.smithi002.stdout:6/734: unlink d15/d2d/d2c/l3e 0 2024-03-16T10:45:04.884 INFO:tasks.workunit.client.0.smithi002.stdout:7/644: chown d3/d4/d1c/d5b/lce 14405 1 2024-03-16T10:45:04.884 INFO:tasks.workunit.client.0.smithi002.stdout:3/696: getdents d1/db6/d15/d87/dd2 0 2024-03-16T10:45:04.885 INFO:tasks.workunit.client.0.smithi002.stdout:1/732: mkdir d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/dee 0 2024-03-16T10:45:04.885 INFO:tasks.workunit.client.0.smithi002.stdout:3/697: stat d1/cbe 0 2024-03-16T10:45:04.886 INFO:tasks.workunit.client.0.smithi002.stdout:7/645: chown d3/d4/d1d/d87/d96/fb8 799 1 2024-03-16T10:45:04.957 INFO:tasks.workunit.client.0.smithi002.stdout:1/733: dread - d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/fdf zero size 2024-03-16T10:45:04.958 INFO:tasks.workunit.client.0.smithi002.stdout:6/735: rename d15/d2d/d2c/d8f to d15/dfd 0 2024-03-16T10:45:04.960 INFO:tasks.workunit.client.0.smithi002.stdout:3/698: mkdir d1/dd/d28/d7b/d8a/dd7/df1 0 2024-03-16T10:45:04.960 INFO:tasks.workunit.client.0.smithi002.stdout:6/736: write d15/d2d/d8a/fae [454318,73130] 0 2024-03-16T10:45:04.964 INFO:tasks.workunit.client.0.smithi002.stdout:3/699: symlink d1/db6/d15/dcc/lf2 0 2024-03-16T10:45:04.965 INFO:tasks.workunit.client.0.smithi002.stdout:3/700: write d1/f8f [1414034,101502] 0 2024-03-16T10:45:04.966 INFO:tasks.workunit.client.0.smithi002.stdout:9/765: dread d5/d7e/fd9 [4194304,4194304] 0 2024-03-16T10:45:04.966 INFO:tasks.workunit.client.0.smithi002.stdout:3/701: readlink d1/dd/d28/d7b/d73/l92 0 2024-03-16T10:45:04.967 INFO:tasks.workunit.client.0.smithi002.stdout:9/766: dread - d5/d9/dc/d1e/d24/fe2 zero size 2024-03-16T10:45:04.973 INFO:tasks.workunit.client.0.smithi002.stdout:7/646: rename d3/d61/d66/d88/fcb to d3/d4/fea 0 2024-03-16T10:45:04.974 INFO:tasks.workunit.client.0.smithi002.stdout:3/702: rmdir d1/dd/d28/d7b/d8a/d93/dd9 39 2024-03-16T10:45:04.977 INFO:tasks.workunit.client.0.smithi002.stdout:9/767: dread d5/d9/d25/d23/d81/fe3 [0,4194304] 0 2024-03-16T10:45:04.978 INFO:tasks.workunit.client.0.smithi002.stdout:6/737: rename d15/l7d to d15/dab/lfe 0 2024-03-16T10:45:04.980 INFO:tasks.workunit.client.0.smithi002.stdout:7/647: symlink d3/d4/d1d/d87/d5c/dc3/leb 0 2024-03-16T10:45:04.981 INFO:tasks.workunit.client.0.smithi002.stdout:3/703: truncate d1/dd/d28/d37/f59 123775 0 2024-03-16T10:45:04.982 INFO:tasks.workunit.client.0.smithi002.stdout:3/704: chown d1/dd/d28/d7b/d22/l46 78 1 2024-03-16T10:45:04.983 INFO:tasks.workunit.client.0.smithi002.stdout:9/768: symlink d5/d9/dc/d28/dd6/lf2 0 2024-03-16T10:45:04.983 INFO:tasks.workunit.client.0.smithi002.stdout:7/648: dread d3/d61/d66/f4d [0,4194304] 0 2024-03-16T10:45:04.984 INFO:tasks.workunit.client.0.smithi002.stdout:7/649: fdatasync d3/d37/d6a/f72 0 2024-03-16T10:45:04.984 INFO:tasks.workunit.client.0.smithi002.stdout:6/738: symlink d15/d2d/d2c/lff 0 2024-03-16T10:45:04.985 INFO:tasks.workunit.client.0.smithi002.stdout:7/650: stat d3/d4/d1d/d87/d5c/dc3 0 2024-03-16T10:45:04.985 INFO:tasks.workunit.client.0.smithi002.stdout:6/739: dread - d15/d2d/d2b/ff4 zero size 2024-03-16T10:45:04.987 INFO:tasks.workunit.client.0.smithi002.stdout:9/769: mknod d5/d9/dc/d1e/d91/d83/cf3 0 2024-03-16T10:45:04.988 INFO:tasks.workunit.client.0.smithi002.stdout:9/770: truncate d5/d9/dc/d1e/d94/fa3 682449 0 2024-03-16T10:45:04.990 INFO:tasks.workunit.client.0.smithi002.stdout:3/705: rename d1/db6/d15/f7d to d1/dd/d28/d7b/d2c/d97/ff3 0 2024-03-16T10:45:04.991 INFO:tasks.workunit.client.0.smithi002.stdout:6/740: mknod d15/dfd/d96/c100 0 2024-03-16T10:45:04.993 INFO:tasks.workunit.client.0.smithi002.stdout:6/741: dread - d15/d2d/da8/fea zero size 2024-03-16T10:45:04.994 INFO:tasks.workunit.client.0.smithi002.stdout:9/771: dwrite d5/d9/d34/fe9 [0,4194304] 0 2024-03-16T10:45:04.995 INFO:tasks.workunit.client.0.smithi002.stdout:9/772: truncate d5/d9/dc/d28/fe6 876172 0 2024-03-16T10:45:04.997 INFO:tasks.workunit.client.0.smithi002.stdout:7/651: mknod d3/d4/d1d/d87/d5c/ddb/cec 0 2024-03-16T10:45:04.998 INFO:tasks.workunit.client.0.smithi002.stdout:6/742: fsync d15/d2d/f6c 0 2024-03-16T10:45:04.999 INFO:tasks.workunit.client.0.smithi002.stdout:9/773: truncate d5/d9/dc/f43 4105102 0 2024-03-16T10:45:05.000 INFO:tasks.workunit.client.0.smithi002.stdout:6/743: write d15/f69 [4257200,92441] 0 2024-03-16T10:45:05.003 INFO:tasks.workunit.client.0.smithi002.stdout:9/774: creat d5/d9/dc/d1e/def/ff4 x:0 0 0 2024-03-16T10:45:05.005 INFO:tasks.workunit.client.0.smithi002.stdout:9/775: stat d5/d9/dc/d1e/f6b 0 2024-03-16T10:45:05.006 INFO:tasks.workunit.client.0.smithi002.stdout:7/652: dwrite d3/d61/d66/fb4 [4194304,4194304] 0 2024-03-16T10:45:05.010 INFO:tasks.workunit.client.0.smithi002.stdout:7/653: symlink d3/d4/d1d/d87/d5c/dd0/led 0 2024-03-16T10:45:05.011 INFO:tasks.workunit.client.0.smithi002.stdout:9/776: dread d5/d9/dc/d28/f4c [0,4194304] 0 2024-03-16T10:45:05.012 INFO:tasks.workunit.client.0.smithi002.stdout:7/654: creat d3/d4/d1c/fee x:0 0 0 2024-03-16T10:45:05.015 INFO:tasks.workunit.client.0.smithi002.stdout:7/655: mkdir d3/d4/d1d/d87/d5c/ddb/def 0 2024-03-16T10:45:05.018 INFO:tasks.workunit.client.0.smithi002.stdout:9/777: dwrite d5/d9/dc/fc2 [0,4194304] 0 2024-03-16T10:45:05.022 INFO:tasks.workunit.client.0.smithi002.stdout:9/778: mknod d5/d9/d25/d23/cf5 0 2024-03-16T10:45:05.024 INFO:tasks.workunit.client.0.smithi002.stdout:9/779: dread - d5/d9/dc/d1e/d91/d82/db4/fc1 zero size 2024-03-16T10:45:05.024 INFO:tasks.workunit.client.0.smithi002.stdout:9/780: write d5/fc4 [593446,31506] 0 2024-03-16T10:45:05.027 INFO:tasks.workunit.client.0.smithi002.stdout:6/744: dread d15/d2d/d2c/f55 [0,4194304] 0 2024-03-16T10:45:05.028 INFO:tasks.workunit.client.0.smithi002.stdout:6/745: chown d15/d2d/d73/fa6 0 1 2024-03-16T10:45:05.028 INFO:tasks.workunit.client.0.smithi002.stdout:9/781: creat d5/d9/d25/ff6 x:0 0 0 2024-03-16T10:45:05.029 INFO:tasks.workunit.client.0.smithi002.stdout:6/746: mknod d15/d2d/d2b/d75/c101 0 2024-03-16T10:45:05.030 INFO:tasks.workunit.client.0.smithi002.stdout:6/747: chown d15/d2d/d2b/d3f/c5b 1 1 2024-03-16T10:45:05.031 INFO:tasks.workunit.client.0.smithi002.stdout:6/748: fsync d15/d2d/d73/d97/fd1 0 2024-03-16T10:45:05.031 INFO:tasks.workunit.client.0.smithi002.stdout:9/782: creat d5/d9/d34/ff7 x:0 0 0 2024-03-16T10:45:05.032 INFO:tasks.workunit.client.0.smithi002.stdout:9/783: chown d5/d9/dc 6386593 1 2024-03-16T10:45:05.035 INFO:tasks.workunit.client.0.smithi002.stdout:9/784: creat d5/d9/dc/d28/d8e/db5/ff8 x:0 0 0 2024-03-16T10:45:05.036 INFO:tasks.workunit.client.0.smithi002.stdout:6/749: dwrite d15/d2d/d2c/f3d [0,4194304] 0 2024-03-16T10:45:05.036 INFO:tasks.workunit.client.0.smithi002.stdout:9/785: dread - d5/d9/dc/d1e/d24/fc6 zero size 2024-03-16T10:45:05.037 INFO:tasks.workunit.client.0.smithi002.stdout:6/750: write d15/f69 [4313394,87592] 0 2024-03-16T10:45:05.038 INFO:tasks.workunit.client.0.smithi002.stdout:9/786: dread d5/f6f [0,4194304] 0 2024-03-16T10:45:05.041 INFO:tasks.workunit.client.0.smithi002.stdout:6/751: fsync d15/d2d/d73/f95 0 2024-03-16T10:45:05.043 INFO:tasks.workunit.client.0.smithi002.stdout:9/787: mkdir d5/d9/dc/d1e/d94/df9 0 2024-03-16T10:45:05.047 INFO:tasks.workunit.client.0.smithi002.stdout:6/752: creat d15/d2d/f102 x:0 0 0 2024-03-16T10:45:05.049 INFO:tasks.workunit.client.0.smithi002.stdout:9/788: dread d5/d9/d25/d23/f56 [0,4194304] 0 2024-03-16T10:45:05.049 INFO:tasks.workunit.client.0.smithi002.stdout:6/753: write d15/d2d/f28 [952640,119515] 0 2024-03-16T10:45:05.052 INFO:tasks.workunit.client.0.smithi002.stdout:6/754: chown d15/d2d/f34 230 1 2024-03-16T10:45:05.053 INFO:tasks.workunit.client.0.smithi002.stdout:6/755: fdatasync d15/dab/dc4/fd7 0 2024-03-16T10:45:05.056 INFO:tasks.workunit.client.0.smithi002.stdout:6/756: symlink d15/d2d/d73/de0/l103 0 2024-03-16T10:45:05.059 INFO:tasks.workunit.client.0.smithi002.stdout:1/734: truncate d2/df/d12/d84/d87/d96/db5/d51/fa6 1070448 0 2024-03-16T10:45:05.063 INFO:tasks.workunit.client.0.smithi002.stdout:1/735: link d2/df/d12/d84/d87/d96/db5/fe d2/df/d12/d84/d87/d96/db5/d51/d56/fef 0 2024-03-16T10:45:05.067 INFO:tasks.workunit.client.0.smithi002.stdout:1/736: rename d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/fdf to d2/df/d12/d84/d87/d96/db5/dae/ff0 0 2024-03-16T10:45:05.071 INFO:tasks.workunit.client.0.smithi002.stdout:1/737: dread d2/df/d12/d84/d87/d96/db5/d1c/d5d/fb9 [0,4194304] 0 2024-03-16T10:45:05.075 INFO:tasks.workunit.client.0.smithi002.stdout:1/738: link d2/df/d12/d29/d5c/f98 d2/de0/ded/ff1 0 2024-03-16T10:45:05.076 INFO:tasks.workunit.client.0.smithi002.stdout:7/656: getdents d3/d4/d1d/d87/d5c/dd0 0 2024-03-16T10:45:05.076 INFO:tasks.workunit.client.0.smithi002.stdout:1/739: chown d2/df/d12/d84/d87/d96/db5/d51/d56/f8b 2887856 1 2024-03-16T10:45:05.077 INFO:tasks.workunit.client.0.smithi002.stdout:7/657: chown d3/d4/d1d/d87/d5c/d63/l80 2 1 2024-03-16T10:45:05.078 INFO:tasks.workunit.client.0.smithi002.stdout:7/658: chown d3/d4/d1d/d45/lb2 2685879 1 2024-03-16T10:45:05.079 INFO:tasks.workunit.client.0.smithi002.stdout:4/654: sync 2024-03-16T10:45:05.079 INFO:tasks.workunit.client.0.smithi002.stdout:5/584: sync 2024-03-16T10:45:05.079 INFO:tasks.workunit.client.0.smithi002.stdout:2/684: sync 2024-03-16T10:45:05.079 INFO:tasks.workunit.client.0.smithi002.stdout:4/655: stat d2/d33/d35/d59/l7a 0 2024-03-16T10:45:05.080 INFO:tasks.workunit.client.0.smithi002.stdout:4/656: write d2/fa3 [892941,123238] 0 2024-03-16T10:45:05.082 INFO:tasks.workunit.client.0.smithi002.stdout:4/657: write d2/d6/d32/fa6 [691380,6783] 0 2024-03-16T10:45:05.083 INFO:tasks.workunit.client.0.smithi002.stdout:5/585: chown d0/d24/c6d 2 1 2024-03-16T10:45:05.083 INFO:tasks.workunit.client.0.smithi002.stdout:9/789: write d5/d9/dc/f16 [14971,39514] 0 2024-03-16T10:45:05.084 INFO:tasks.workunit.client.0.smithi002.stdout:5/586: truncate d0/d16/f5f 9389691 0 2024-03-16T10:45:05.085 INFO:tasks.workunit.client.0.smithi002.stdout:1/740: rename d2/df/d12/d29/d5c/fca to d2/df/d12/d84/d87/d96/db5/d1c/da0/ff2 0 2024-03-16T10:45:05.087 INFO:tasks.workunit.client.0.smithi002.stdout:4/658: creat d2/d33/da/d5e/fc7 x:0 0 0 2024-03-16T10:45:05.088 INFO:tasks.workunit.client.0.smithi002.stdout:1/741: dread d2/df/f45 [0,4194304] 0 2024-03-16T10:45:05.089 INFO:tasks.workunit.client.0.smithi002.stdout:9/790: symlink d5/d9/dc/d28/d90/lfa 0 2024-03-16T10:45:05.090 INFO:tasks.workunit.client.0.smithi002.stdout:5/587: dread d0/d16/dab/d2b/d5d/f92 [0,4194304] 0 2024-03-16T10:45:05.091 INFO:tasks.workunit.client.0.smithi002.stdout:9/791: write d5/d9/dc/d28/d90/f9b [767484,65298] 0 2024-03-16T10:45:05.091 INFO:tasks.workunit.client.0.smithi002.stdout:9/792: fsync d5/d9/d34/fe9 0 2024-03-16T10:45:05.096 INFO:tasks.workunit.client.0.smithi002.stdout:7/659: write d3/d4/d1d/d87/d96/d8c/f95 [189264,124660] 0 2024-03-16T10:45:05.097 INFO:tasks.workunit.client.0.smithi002.stdout:5/588: mkdir d0/d24/dad/db6/db9 0 2024-03-16T10:45:05.098 INFO:tasks.workunit.client.0.smithi002.stdout:7/660: fsync d3/d61/d66/fb4 0 2024-03-16T10:45:05.099 INFO:tasks.workunit.client.0.smithi002.stdout:9/793: dwrite d5/d9/dc/d1e/f6b [0,4194304] 0 2024-03-16T10:45:05.100 INFO:tasks.workunit.client.0.smithi002.stdout:9/794: read d5/d9/dc/d1e/f6b [3610262,10807] 0 2024-03-16T10:45:05.102 INFO:tasks.workunit.client.0.smithi002.stdout:7/661: fsync d3/d4/d1c/f90 0 2024-03-16T10:45:05.104 INFO:tasks.workunit.client.0.smithi002.stdout:5/589: symlink d0/d16/dab/d2b/d5d/lba 0 2024-03-16T10:45:05.104 INFO:tasks.workunit.client.0.smithi002.stdout:1/742: getdents d2/df/d12/dd7/dd9 0 2024-03-16T10:45:05.110 INFO:tasks.workunit.client.0.smithi002.stdout:9/795: rename d5/d9/dc/d1e/d24/l38 to d5/db1/dec/lfb 0 2024-03-16T10:45:05.112 INFO:tasks.workunit.client.0.smithi002.stdout:7/662: symlink d3/d4/d1c/d5b/d64/ddf/lf0 0 2024-03-16T10:45:05.116 INFO:tasks.workunit.client.0.smithi002.stdout:5/590: creat d0/d16/d4a/d55/fbb x:0 0 0 2024-03-16T10:45:05.122 INFO:tasks.workunit.client.0.smithi002.stdout:7/663: dwrite d3/f11 [8388608,4194304] 0 2024-03-16T10:45:05.123 INFO:tasks.workunit.client.0.smithi002.stdout:1/743: write d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/f95 [150944,126553] 0 2024-03-16T10:45:05.125 INFO:tasks.workunit.client.0.smithi002.stdout:7/664: chown d3/d37/l67 17028534 1 2024-03-16T10:45:05.130 INFO:tasks.workunit.client.0.smithi002.stdout:5/591: dwrite d0/d16/dab/d2b/d5d/f64 [0,4194304] 0 2024-03-16T10:45:05.131 INFO:tasks.workunit.client.0.smithi002.stdout:9/796: dwrite d5/d9/dc/d1e/d24/fde [0,4194304] 0 2024-03-16T10:45:05.131 INFO:tasks.workunit.client.0.smithi002.stdout:7/665: unlink d3/d4/d1d/d87/d5c/d63/d55/f77 0 2024-03-16T10:45:05.135 INFO:tasks.workunit.client.0.smithi002.stdout:9/797: rmdir d5/d9/dc/d1e 39 2024-03-16T10:45:05.137 INFO:tasks.workunit.client.0.smithi002.stdout:9/798: fsync d5/d9/dc/d28/f65 0 2024-03-16T10:45:05.140 INFO:tasks.workunit.client.0.smithi002.stdout:7/666: link d3/d4/d1c/d5b/d64/d99/lc9 d3/d4/lf1 0 2024-03-16T10:45:05.141 INFO:tasks.workunit.client.0.smithi002.stdout:5/592: dwrite d0/d16/dab/d2b/d5d/f5a [0,4194304] 0 2024-03-16T10:45:05.144 INFO:tasks.workunit.client.0.smithi002.stdout:7/667: chown d3/d4/d1d/d87/d5c/dc3/leb 88836 1 2024-03-16T10:45:05.145 INFO:tasks.workunit.client.0.smithi002.stdout:5/593: getdents d0/d24/dad/db6/db9 0 2024-03-16T10:45:05.148 INFO:tasks.workunit.client.0.smithi002.stdout:9/799: dwrite d5/f53 [0,4194304] 0 2024-03-16T10:45:05.151 INFO:tasks.workunit.client.0.smithi002.stdout:5/594: rename d0/d16/fb1 to d0/d16/dab/d33/d8c/fbc 0 2024-03-16T10:45:05.152 INFO:tasks.workunit.client.0.smithi002.stdout:9/800: creat d5/db1/dec/ffc x:0 0 0 2024-03-16T10:45:05.152 INFO:tasks.workunit.client.0.smithi002.stdout:5/595: dread - d0/d16/dab/f85 zero size 2024-03-16T10:45:05.154 INFO:tasks.workunit.client.0.smithi002.stdout:5/596: fsync d0/d16/f19 0 2024-03-16T10:45:05.155 INFO:tasks.workunit.client.0.smithi002.stdout:9/801: truncate d5/d9/dc/d28/f65 2875781 0 2024-03-16T10:45:05.157 INFO:tasks.workunit.client.0.smithi002.stdout:9/802: readlink d5/db1/dec/lfb 0 2024-03-16T10:45:05.159 INFO:tasks.workunit.client.0.smithi002.stdout:9/803: write d5/f29 [3452945,13410] 0 2024-03-16T10:45:05.160 INFO:tasks.workunit.client.0.smithi002.stdout:9/804: chown d5/d9/le5 13794 1 2024-03-16T10:45:05.162 INFO:tasks.workunit.client.0.smithi002.stdout:5/597: dwrite d0/d16/dab/d43/f99 [0,4194304] 0 2024-03-16T10:45:05.162 INFO:tasks.workunit.client.0.smithi002.stdout:9/805: fdatasync d5/d9/dc/d28/fa8 0 2024-03-16T10:45:05.167 INFO:tasks.workunit.client.0.smithi002.stdout:9/806: link d5/d9/dc/d28/fa8 d5/d9/dc/d28/dd6/ffd 0 2024-03-16T10:45:05.170 INFO:tasks.workunit.client.0.smithi002.stdout:9/807: write d5/d9/d55/f69 [820119,62104] 0 2024-03-16T10:45:05.172 INFO:tasks.workunit.client.0.smithi002.stdout:5/598: dwrite d0/d24/dad/fae [0,4194304] 0 2024-03-16T10:45:05.174 INFO:tasks.workunit.client.0.smithi002.stdout:9/808: creat d5/d9/d55/dc9/ffe x:0 0 0 2024-03-16T10:45:05.177 INFO:tasks.workunit.client.0.smithi002.stdout:5/599: rename d0/d16/d4a/d39/f7c to d0/d1f/fbd 0 2024-03-16T10:45:05.178 INFO:tasks.workunit.client.0.smithi002.stdout:9/809: rmdir d5/d9/dc/d1e/d91/d82/db4 39 2024-03-16T10:45:05.180 INFO:tasks.workunit.client.0.smithi002.stdout:9/810: chown d5/d9/d25/c4e 253997 1 2024-03-16T10:45:05.182 INFO:tasks.workunit.client.0.smithi002.stdout:9/811: write d5/d9/fd2 [826718,34265] 0 2024-03-16T10:45:05.182 INFO:tasks.workunit.client.0.smithi002.stdout:0/736: sync 2024-03-16T10:45:05.182 INFO:tasks.workunit.client.0.smithi002.stdout:8/793: sync 2024-03-16T10:45:05.184 INFO:tasks.workunit.client.0.smithi002.stdout:9/812: write d5/f53 [2415878,94885] 0 2024-03-16T10:45:05.185 INFO:tasks.workunit.client.0.smithi002.stdout:5/600: dwrite d0/d16/dab/daa/f9c [0,4194304] 0 2024-03-16T10:45:05.186 INFO:tasks.workunit.client.0.smithi002.stdout:5/601: chown d0/d16/dab/d33/d4b 54473 1 2024-03-16T10:45:05.188 INFO:tasks.workunit.client.0.smithi002.stdout:8/794: chown d7/d9/d17/f27 1988 1 2024-03-16T10:45:05.189 INFO:tasks.workunit.client.0.smithi002.stdout:0/737: symlink d3/da0/lef 0 2024-03-16T10:45:05.190 INFO:tasks.workunit.client.0.smithi002.stdout:5/602: mknod d0/d16/dab/d2b/d94/cbe 0 2024-03-16T10:45:05.194 INFO:tasks.workunit.client.0.smithi002.stdout:8/795: mkdir d7/d9/d17/d23/d39/db7/dc2/d10f 0 2024-03-16T10:45:05.196 INFO:tasks.workunit.client.0.smithi002.stdout:5/603: creat d0/d16/dab/d2b/d94/fbf x:0 0 0 2024-03-16T10:45:05.198 INFO:tasks.workunit.client.0.smithi002.stdout:9/813: rename d5/d9/dc/c35 to d5/d9/d25/d23/d81/cff 0 2024-03-16T10:45:05.199 INFO:tasks.workunit.client.0.smithi002.stdout:9/814: chown d5/d9/dc/d28 2111 1 2024-03-16T10:45:05.199 INFO:tasks.workunit.client.0.smithi002.stdout:9/815: chown d5/d9/dc/d28/d8e 7490954 1 2024-03-16T10:45:05.199 INFO:tasks.workunit.client.0.smithi002.stdout:0/738: mknod d3/d31/d75/dea/cf0 0 2024-03-16T10:45:05.200 INFO:tasks.workunit.client.0.smithi002.stdout:5/604: symlink d0/d16/dab/d43/d48/lc0 0 2024-03-16T10:45:05.200 INFO:tasks.workunit.client.0.smithi002.stdout:9/816: readlink d5/d9/dc/d28/d90/lfa 0 2024-03-16T10:45:05.201 INFO:tasks.workunit.client.0.smithi002.stdout:5/605: dread - d0/f86 zero size 2024-03-16T10:45:05.201 INFO:tasks.workunit.client.0.smithi002.stdout:8/796: link d7/f8 d7/d9/d17/d23/d2e/d34/dbf/f110 0 2024-03-16T10:45:05.202 INFO:tasks.workunit.client.0.smithi002.stdout:0/739: dread - d3/d31/d88/f8b zero size 2024-03-16T10:45:05.203 INFO:tasks.workunit.client.0.smithi002.stdout:0/740: chown d3/de/fe1 383814115 1 2024-03-16T10:45:05.204 INFO:tasks.workunit.client.0.smithi002.stdout:0/741: fsync d3/d5/f30 0 2024-03-16T10:45:05.210 INFO:tasks.workunit.client.0.smithi002.stdout:5/606: write d0/d16/f19 [421391,827] 0 2024-03-16T10:45:05.212 INFO:tasks.workunit.client.0.smithi002.stdout:0/742: dwrite d3/d28/d3d/f46 [0,4194304] 0 2024-03-16T10:45:05.216 INFO:tasks.workunit.client.0.smithi002.stdout:0/743: chown d3/d27/d90 153485 1 2024-03-16T10:45:05.218 INFO:tasks.workunit.client.0.smithi002.stdout:9/817: dread d5/fd [4194304,4194304] 0 2024-03-16T10:45:05.218 INFO:tasks.workunit.client.0.smithi002.stdout:5/607: dread d0/d24/dad/fae [0,4194304] 0 2024-03-16T10:45:05.219 INFO:tasks.workunit.client.0.smithi002.stdout:8/797: dread d7/d9/d17/d23/d2e/d3a/fac [0,4194304] 0 2024-03-16T10:45:05.219 INFO:tasks.workunit.client.0.smithi002.stdout:0/744: symlink d3/d27/lf1 0 2024-03-16T10:45:05.220 INFO:tasks.workunit.client.0.smithi002.stdout:0/745: readlink d3/d28/d65/l76 0 2024-03-16T10:45:05.223 INFO:tasks.workunit.client.0.smithi002.stdout:5/608: creat d0/d16/d4a/d7d/fc1 x:0 0 0 2024-03-16T10:45:05.223 INFO:tasks.workunit.client.0.smithi002.stdout:9/818: mknod d5/d9/d25/dac/c100 0 2024-03-16T10:45:05.227 INFO:tasks.workunit.client.0.smithi002.stdout:8/798: creat d7/d13/db8/f111 x:0 0 0 2024-03-16T10:45:05.228 INFO:tasks.workunit.client.0.smithi002.stdout:5/609: creat d0/d24/dad/db6/db9/fc2 x:0 0 0 2024-03-16T10:45:05.229 INFO:tasks.workunit.client.0.smithi002.stdout:5/610: write d0/d16/dab/d2b/d5d/f5a [3870986,99782] 0 2024-03-16T10:45:05.235 INFO:tasks.workunit.client.0.smithi002.stdout:0/746: link d3/d28/d3d/cd7 d3/d28/d3d/cf2 0 2024-03-16T10:45:05.237 INFO:tasks.workunit.client.0.smithi002.stdout:5/611: fsync d0/d16/f37 0 2024-03-16T10:45:05.238 INFO:tasks.workunit.client.0.smithi002.stdout:5/612: dread - d0/d16/dab/d33/d8c/f98 zero size 2024-03-16T10:45:05.238 INFO:tasks.workunit.client.0.smithi002.stdout:9/819: rename d5/db1/lb9 to d5/d9/dc/d1e/d24/d31/l101 0 2024-03-16T10:45:05.239 INFO:tasks.workunit.client.0.smithi002.stdout:9/820: fsync d5/d9/d55/fc8 0 2024-03-16T10:45:05.239 INFO:tasks.workunit.client.0.smithi002.stdout:5/613: dread - d0/d16/dab/d33/d8c/f95 zero size 2024-03-16T10:45:05.245 INFO:tasks.workunit.client.0.smithi002.stdout:9/821: link d5/d9/d34/fe9 d5/d9/d25/d23/d81/f102 0 2024-03-16T10:45:05.245 INFO:tasks.workunit.client.0.smithi002.stdout:8/799: write d7/f8 [1446057,108213] 0 2024-03-16T10:45:05.246 INFO:tasks.workunit.client.0.smithi002.stdout:5/614: getdents d0/d24/dad/db6 0 2024-03-16T10:45:05.248 INFO:tasks.workunit.client.0.smithi002.stdout:9/822: creat d5/d9/dc/d1e/def/f103 x:0 0 0 2024-03-16T10:45:05.249 INFO:tasks.workunit.client.0.smithi002.stdout:5/615: fsync d0/d16/dab/d2b/d5d/f92 0 2024-03-16T10:45:05.249 INFO:tasks.workunit.client.0.smithi002.stdout:0/747: dread d3/fa5 [0,4194304] 0 2024-03-16T10:45:05.250 INFO:tasks.workunit.client.0.smithi002.stdout:0/748: rmdir d3/de/da2 39 2024-03-16T10:45:05.252 INFO:tasks.workunit.client.0.smithi002.stdout:0/749: write d3/d28/f79 [5220523,107858] 0 2024-03-16T10:45:05.274 INFO:tasks.workunit.client.0.smithi002.stdout:6/757: sync 2024-03-16T10:45:05.275 INFO:tasks.workunit.client.0.smithi002.stdout:3/706: sync 2024-03-16T10:45:05.288 INFO:tasks.workunit.client.0.smithi002.stdout:5/616: rename d0/d16/d4a/d39/f51 to d0/fc3 0 2024-03-16T10:45:05.288 INFO:tasks.workunit.client.0.smithi002.stdout:8/800: dwrite d7/d9/d17/d23/d2e/ff4 [0,4194304] 0 2024-03-16T10:45:05.290 INFO:tasks.workunit.client.0.smithi002.stdout:3/707: symlink d1/da5/lf4 0 2024-03-16T10:45:05.293 INFO:tasks.workunit.client.0.smithi002.stdout:6/758: rename d15/l78 to d15/d2d/d8a/d7c/d87/l104 0 2024-03-16T10:45:05.293 INFO:tasks.workunit.client.0.smithi002.stdout:5/617: mkdir d0/d16/d5b/da0/dc4 0 2024-03-16T10:45:05.295 INFO:tasks.workunit.client.0.smithi002.stdout:0/750: truncate d3/d28/d2a/d67/fdd 272995 0 2024-03-16T10:45:05.303 INFO:tasks.workunit.client.0.smithi002.stdout:5/618: creat d0/d16/d4a/d7d/fc5 x:0 0 0 2024-03-16T10:45:05.304 INFO:tasks.workunit.client.0.smithi002.stdout:0/751: mknod d3/d28/d2a/dc7/cf3 0 2024-03-16T10:45:05.304 INFO:tasks.workunit.client.0.smithi002.stdout:6/759: fdatasync d15/f67 0 2024-03-16T10:45:05.305 INFO:tasks.workunit.client.0.smithi002.stdout:6/760: write fa [8815777,57557] 0 2024-03-16T10:45:05.309 INFO:tasks.workunit.client.0.smithi002.stdout:3/708: dwrite d1/dd/d28/d7b/d8a/d93/f47 [0,4194304] 0 2024-03-16T10:45:05.330 INFO:tasks.workunit.client.0.smithi002.stdout:8/801: rmdir d7/d9/d17/d23/d39/db7/dc2/d10f 0 2024-03-16T10:45:05.331 INFO:tasks.workunit.client.0.smithi002.stdout:0/752: mkdir d3/d31/d75/df4 0 2024-03-16T10:45:05.336 INFO:tasks.workunit.client.0.smithi002.stdout:8/802: creat d7/de3/f112 x:0 0 0 2024-03-16T10:45:05.337 INFO:tasks.workunit.client.0.smithi002.stdout:5/619: creat d0/d16/dab/d2b/fc6 x:0 0 0 2024-03-16T10:45:05.340 INFO:tasks.workunit.client.0.smithi002.stdout:5/620: truncate d0/d16/dab/d33/d8c/fbc 680391 0 2024-03-16T10:45:05.341 INFO:tasks.workunit.client.0.smithi002.stdout:8/803: fdatasync d7/d9/d17/d23/d2e/f4e 0 2024-03-16T10:45:05.341 INFO:tasks.workunit.client.0.smithi002.stdout:8/804: chown d7/d9/d25/dc9/f100 563828 1 2024-03-16T10:45:05.342 INFO:tasks.workunit.client.0.smithi002.stdout:5/621: chown d0/d16/dab/d2b/fc6 864 1 2024-03-16T10:45:05.343 INFO:tasks.workunit.client.0.smithi002.stdout:8/805: chown d7/d9/d17/d23/d2e/d3a 90217 1 2024-03-16T10:45:05.343 INFO:tasks.workunit.client.0.smithi002.stdout:0/753: dread d3/d28/f8e [0,4194304] 0 2024-03-16T10:45:05.344 INFO:tasks.workunit.client.0.smithi002.stdout:0/754: dread - d3/da0/fb1 zero size 2024-03-16T10:45:05.347 INFO:tasks.workunit.client.0.smithi002.stdout:6/761: write d15/d2d/d2c/f7a [1073573,44088] 0 2024-03-16T10:45:05.348 INFO:tasks.workunit.client.0.smithi002.stdout:5/622: rename d0/ca to d0/d16/dab/cc7 0 2024-03-16T10:45:05.349 INFO:tasks.workunit.client.0.smithi002.stdout:5/623: chown d0/f57 54504 1 2024-03-16T10:45:05.350 INFO:tasks.workunit.client.0.smithi002.stdout:5/624: write d0/d16/f5f [4468133,8710] 0 2024-03-16T10:45:05.351 INFO:tasks.workunit.client.0.smithi002.stdout:6/762: creat d15/dfd/d96/dbc/f105 x:0 0 0 2024-03-16T10:45:05.351 INFO:tasks.workunit.client.0.smithi002.stdout:3/709: write d1/db6/d15/fa7 [697763,7711] 0 2024-03-16T10:45:05.353 INFO:tasks.workunit.client.0.smithi002.stdout:8/806: write d7/d9/d25/d67/f81 [789857,40613] 0 2024-03-16T10:45:05.353 INFO:tasks.workunit.client.0.smithi002.stdout:0/755: write d3/d28/f36 [1148229,22131] 0 2024-03-16T10:45:05.356 INFO:tasks.workunit.client.0.smithi002.stdout:3/710: chown d1/db6/d15/d87/d8c/cc7 77 1 2024-03-16T10:45:05.357 INFO:tasks.workunit.client.0.smithi002.stdout:6/763: truncate d15/d2d/d2b/d3f/d77/d7e/fcf 44536 0 2024-03-16T10:45:05.357 INFO:tasks.workunit.client.0.smithi002.stdout:5/625: link d0/d16/dab/d43/d48/f8e d0/d16/fc8 0 2024-03-16T10:45:05.357 INFO:tasks.workunit.client.0.smithi002.stdout:6/764: stat d15/f2f 0 2024-03-16T10:45:05.358 INFO:tasks.workunit.client.0.smithi002.stdout:8/807: creat d7/d9/d17/d23/d2e/d34/dbf/dd9/f113 x:0 0 0 2024-03-16T10:45:05.360 INFO:tasks.workunit.client.0.smithi002.stdout:5/626: chown d0/d16/dab/d2b/f78 0 1 2024-03-16T10:45:05.360 INFO:tasks.workunit.client.0.smithi002.stdout:8/808: stat d7/d9/d17/d23/d2e/d34/dde/lec 0 2024-03-16T10:45:05.362 INFO:tasks.workunit.client.0.smithi002.stdout:6/765: mknod d15/d2d/d2b/dc8/deb/c106 0 2024-03-16T10:45:05.362 INFO:tasks.workunit.client.0.smithi002.stdout:5/627: mknod d0/d16/dab/d33/d8c/cc9 0 2024-03-16T10:45:05.363 INFO:tasks.workunit.client.0.smithi002.stdout:6/766: readlink d15/d2d/d8a/l7f 0 2024-03-16T10:45:05.365 INFO:tasks.workunit.client.0.smithi002.stdout:2/685: sync 2024-03-16T10:45:05.365 INFO:tasks.workunit.client.0.smithi002.stdout:4/659: sync 2024-03-16T10:45:05.365 INFO:tasks.workunit.client.0.smithi002.stdout:5/628: rmdir d0/d16/d4a/d7d 39 2024-03-16T10:45:05.366 INFO:tasks.workunit.client.0.smithi002.stdout:6/767: unlink d15/f1a 0 2024-03-16T10:45:05.367 INFO:tasks.workunit.client.0.smithi002.stdout:5/629: dread - d0/d16/d4a/d55/fbb zero size 2024-03-16T10:45:05.368 INFO:tasks.workunit.client.0.smithi002.stdout:2/686: mkdir d7/d27/d3b/dd4 0 2024-03-16T10:45:05.369 INFO:tasks.workunit.client.0.smithi002.stdout:0/756: write d3/d31/fe5 [886608,89423] 0 2024-03-16T10:45:05.371 INFO:tasks.workunit.client.0.smithi002.stdout:2/687: write d7/d2e/d1d/d2f/d4b/fc8 [2003019,61524] 0 2024-03-16T10:45:05.372 INFO:tasks.workunit.client.0.smithi002.stdout:5/630: fsync d0/d16/dab/d33/d8c/f95 0 2024-03-16T10:45:05.373 INFO:tasks.workunit.client.0.smithi002.stdout:6/768: mkdir d15/dfd/d96/dbb/d107 0 2024-03-16T10:45:05.378 INFO:tasks.workunit.client.0.smithi002.stdout:8/809: truncate d7/d9/d17/d83/df5/f10c 904873 0 2024-03-16T10:45:05.379 INFO:tasks.workunit.client.0.smithi002.stdout:2/688: symlink d7/d2e/d1d/d38/d49/db5/ld5 0 2024-03-16T10:45:05.382 INFO:tasks.workunit.client.0.smithi002.stdout:3/711: dwrite d1/dd/d28/d37/f59 [0,4194304] 0 2024-03-16T10:45:05.383 INFO:tasks.workunit.client.0.smithi002.stdout:6/769: unlink d15/d2d/d2c/le9 0 2024-03-16T10:45:05.384 INFO:tasks.workunit.client.0.smithi002.stdout:2/689: write d7/d2e/d28/d44/f76 [1780669,38921] 0 2024-03-16T10:45:05.386 INFO:tasks.workunit.client.0.smithi002.stdout:0/757: link d3/d5/fd d3/ff5 0 2024-03-16T10:45:05.388 INFO:tasks.workunit.client.0.smithi002.stdout:4/660: dread d2/d6/d32/d66/d7f/f9f [0,4194304] 0 2024-03-16T10:45:05.389 INFO:tasks.workunit.client.0.smithi002.stdout:4/661: dread - d2/d33/da/d5e/d93/fbb zero size 2024-03-16T10:45:05.390 INFO:tasks.workunit.client.0.smithi002.stdout:3/712: mkdir d1/dd/d28/d7b/d2c/d97/df5 0 2024-03-16T10:45:05.391 INFO:tasks.workunit.client.0.smithi002.stdout:4/662: stat d2/d33/da/d2e/db4/cbf 0 2024-03-16T10:45:05.391 INFO:tasks.workunit.client.0.smithi002.stdout:3/713: dread - d1/da5/fe1 zero size 2024-03-16T10:45:05.392 INFO:tasks.workunit.client.0.smithi002.stdout:2/690: creat d7/db/d12/fd6 x:0 0 0 2024-03-16T10:45:05.392 INFO:tasks.workunit.client.0.smithi002.stdout:5/631: write d0/d1f/f58 [1232306,98878] 0 2024-03-16T10:45:05.392 INFO:tasks.workunit.client.0.smithi002.stdout:4/663: fdatasync d2/d6/f7e 0 2024-03-16T10:45:05.394 INFO:tasks.workunit.client.0.smithi002.stdout:3/714: dread - d1/dd/d28/d7b/d2c/d97/ff3 zero size 2024-03-16T10:45:05.395 INFO:tasks.workunit.client.0.smithi002.stdout:6/770: creat d15/dfd/d96/dbb/d107/f108 x:0 0 0 2024-03-16T10:45:05.396 INFO:tasks.workunit.client.0.smithi002.stdout:8/810: rename d7/d9/d17/d20/lae to d7/d9/d17/d23/d2e/d34/dbf/l114 0 2024-03-16T10:45:05.397 INFO:tasks.workunit.client.0.smithi002.stdout:5/632: write d0/d16/dab/daa/f9c [2863183,112287] 0 2024-03-16T10:45:05.400 INFO:tasks.workunit.client.0.smithi002.stdout:8/811: write d7/d13/db8/f7c [186298,88156] 0 2024-03-16T10:45:05.403 INFO:tasks.workunit.client.0.smithi002.stdout:4/664: rename d2/d33/da/f42 to d2/d33/d1c/fc8 0 2024-03-16T10:45:05.404 INFO:tasks.workunit.client.0.smithi002.stdout:6/771: mknod d15/d2d/d2b/c109 0 2024-03-16T10:45:05.405 INFO:tasks.workunit.client.0.smithi002.stdout:2/691: dwrite d7/db/d12/f85 [0,4194304] 0 2024-03-16T10:45:05.407 INFO:tasks.workunit.client.0.smithi002.stdout:0/758: write d3/f2d [279475,14224] 0 2024-03-16T10:45:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:4/665: mknod d2/d6/cc9 0 2024-03-16T10:45:05.410 INFO:tasks.workunit.client.0.smithi002.stdout:0/759: dread - d3/d31/d88/f8b zero size 2024-03-16T10:45:05.411 INFO:tasks.workunit.client.0.smithi002.stdout:5/633: dwrite d0/d16/f66 [4194304,4194304] 0 2024-03-16T10:45:05.413 INFO:tasks.workunit.client.0.smithi002.stdout:3/715: creat d1/dd/d28/d7b/d8a/d93/ff6 x:0 0 0 2024-03-16T10:45:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:4/666: write d2/d33/d1c/f80 [4388644,81779] 0 2024-03-16T10:45:05.414 INFO:tasks.workunit.client.0.smithi002.stdout:6/772: chown d15/d2d/d2b/l51 308556 1 2024-03-16T10:45:05.415 INFO:tasks.workunit.client.0.smithi002.stdout:3/716: dread - d1/dd/d28/d7b/d2c/d97/fef zero size 2024-03-16T10:45:05.417 INFO:tasks.workunit.client.0.smithi002.stdout:5/634: symlink d0/d16/d5b/lca 0 2024-03-16T10:45:05.418 INFO:tasks.workunit.client.0.smithi002.stdout:3/717: chown d1/db6/d15/d87/d8c/la6 61 1 2024-03-16T10:45:05.419 INFO:tasks.workunit.client.0.smithi002.stdout:5/635: dread - d0/d16/dab/d33/d8c/f95 zero size 2024-03-16T10:45:05.420 INFO:tasks.workunit.client.0.smithi002.stdout:4/667: rmdir d2/d33/da/d2e 39 2024-03-16T10:45:05.420 INFO:tasks.workunit.client.0.smithi002.stdout:5/636: dread - d0/d16/dab/d43/d48/f8f zero size 2024-03-16T10:45:05.421 INFO:tasks.workunit.client.0.smithi002.stdout:4/668: readlink d2/d6/l9e 0 2024-03-16T10:45:05.423 INFO:tasks.workunit.client.0.smithi002.stdout:4/669: truncate d2/d6/d32/d66/d7f/f9f 4217119 0 2024-03-16T10:45:05.424 INFO:tasks.workunit.client.0.smithi002.stdout:6/773: symlink d15/d2d/d2b/d3f/d77/d7e/l10a 0 2024-03-16T10:45:05.427 INFO:tasks.workunit.client.0.smithi002.stdout:0/760: rename d3/d28/d3d/cd7 to d3/d28/d2a/da1/cf6 0 2024-03-16T10:45:05.427 INFO:tasks.workunit.client.0.smithi002.stdout:2/692: getdents d7/d27/d3b/d95/da5 0 2024-03-16T10:45:05.430 INFO:tasks.workunit.client.0.smithi002.stdout:6/774: mknod d15/d2d/d2b/dc8/deb/c10b 0 2024-03-16T10:45:05.431 INFO:tasks.workunit.client.0.smithi002.stdout:0/761: write d3/de/f51 [4271151,58373] 0 2024-03-16T10:45:05.432 INFO:tasks.workunit.client.0.smithi002.stdout:3/718: mknod d1/dd/d28/d7b/d8a/d93/cf7 0 2024-03-16T10:45:05.433 INFO:tasks.workunit.client.0.smithi002.stdout:5/637: mknod d0/d16/d4a/d7d/ccb 0 2024-03-16T10:45:05.435 INFO:tasks.workunit.client.0.smithi002.stdout:4/670: unlink d2/d33/d1c/f80 0 2024-03-16T10:45:05.435 INFO:tasks.workunit.client.0.smithi002.stdout:3/719: write d1/dd/d28/d7b/d2c/d63/fea [1633013,58038] 0 2024-03-16T10:45:05.438 INFO:tasks.workunit.client.0.smithi002.stdout:6/775: symlink d15/d80/df5/l10c 0 2024-03-16T10:45:05.438 INFO:tasks.workunit.client.0.smithi002.stdout:5/638: mkdir d0/d16/dab/d43/dcc 0 2024-03-16T10:45:05.440 INFO:tasks.workunit.client.0.smithi002.stdout:3/720: creat d1/dd/d28/ff8 x:0 0 0 2024-03-16T10:45:05.440 INFO:tasks.workunit.client.0.smithi002.stdout:0/762: dwrite d3/d5/d7c/fbe [0,4194304] 0 2024-03-16T10:45:05.441 INFO:tasks.workunit.client.0.smithi002.stdout:4/671: rename d2/d33/da/f74 to d2/d6/fca 0 2024-03-16T10:45:05.443 INFO:tasks.workunit.client.0.smithi002.stdout:4/672: chown d2/d6/l9e 236313504 1 2024-03-16T10:45:05.445 INFO:tasks.workunit.client.0.smithi002.stdout:5/639: write d0/d16/dab/d43/f99 [2069924,126356] 0 2024-03-16T10:45:05.448 INFO:tasks.workunit.client.0.smithi002.stdout:2/693: dwrite d7/db/f7c [0,4194304] 0 2024-03-16T10:45:05.448 INFO:tasks.workunit.client.0.smithi002.stdout:6/776: rename d15/d2d/d73/f95 to d15/d2d/d2b/d3f/d77/d7e/f10d 0 2024-03-16T10:45:05.448 INFO:tasks.workunit.client.0.smithi002.stdout:2/694: chown d7/d2e/d1d 331146 1 2024-03-16T10:45:05.451 INFO:tasks.workunit.client.0.smithi002.stdout:6/777: creat d15/dfd/d96/db5/f10e x:0 0 0 2024-03-16T10:45:05.453 INFO:tasks.workunit.client.0.smithi002.stdout:4/673: rename d2/d33/la0 to d2/d33/d1c/lcb 0 2024-03-16T10:45:05.455 INFO:tasks.workunit.client.0.smithi002.stdout:6/778: mknod d15/d80/dbe/c10f 0 2024-03-16T10:45:05.455 INFO:tasks.workunit.client.0.smithi002.stdout:2/695: dwrite d7/db/f58 [0,4194304] 0 2024-03-16T10:45:05.458 INFO:tasks.workunit.client.0.smithi002.stdout:4/674: write d2/d6/d32/d66/d7f/f9f [2317816,1266] 0 2024-03-16T10:45:05.459 INFO:tasks.workunit.client.0.smithi002.stdout:6/779: dread - d15/d2d/d73/d97/fd1 zero size 2024-03-16T10:45:05.461 INFO:tasks.workunit.client.0.smithi002.stdout:3/721: write d1/dd/d74/fb5 [997014,44749] 0 2024-03-16T10:45:05.461 INFO:tasks.workunit.client.0.smithi002.stdout:0/763: write d3/f49 [1180262,82642] 0 2024-03-16T10:45:05.463 INFO:tasks.workunit.client.0.smithi002.stdout:2/696: creat d7/d2e/d28/fd7 x:0 0 0 2024-03-16T10:45:05.464 INFO:tasks.workunit.client.0.smithi002.stdout:4/675: creat d2/d6/fcc x:0 0 0 2024-03-16T10:45:05.464 INFO:tasks.workunit.client.0.smithi002.stdout:3/722: mkdir d1/dd/d28/d37/df9 0 2024-03-16T10:45:05.465 INFO:tasks.workunit.client.0.smithi002.stdout:6/780: mkdir d15/d2d/d110 0 2024-03-16T10:45:05.467 INFO:tasks.workunit.client.0.smithi002.stdout:2/697: mkdir d7/d2e/d28/d40/dd8 0 2024-03-16T10:45:05.469 INFO:tasks.workunit.client.0.smithi002.stdout:0/764: getdents d3 0 2024-03-16T10:45:05.471 INFO:tasks.workunit.client.0.smithi002.stdout:4/676: creat d2/d33/fcd x:0 0 0 2024-03-16T10:45:05.472 INFO:tasks.workunit.client.0.smithi002.stdout:3/723: mknod d1/dd/d28/d7b/d2c/d97/df5/cfa 0 2024-03-16T10:45:05.475 INFO:tasks.workunit.client.0.smithi002.stdout:2/698: dwrite d7/d2e/d1d/fc2 [0,4194304] 0 2024-03-16T10:45:05.477 INFO:tasks.workunit.client.0.smithi002.stdout:4/677: symlink d2/d6/d32/d66/lce 0 2024-03-16T10:45:05.478 INFO:tasks.workunit.client.0.smithi002.stdout:3/724: link d1/dd/l71 d1/dd/d74/dd8/lfb 0 2024-03-16T10:45:05.478 INFO:tasks.workunit.client.0.smithi002.stdout:6/781: truncate d15/d2d/d2c/f36 4010590 0 2024-03-16T10:45:05.479 INFO:tasks.workunit.client.0.smithi002.stdout:2/699: rmdir d7/d2e/d1d/d2f/d4b 39 2024-03-16T10:45:05.480 INFO:tasks.workunit.client.0.smithi002.stdout:0/765: creat d3/d31/daa/ddb/ff7 x:0 0 0 2024-03-16T10:45:05.481 INFO:tasks.workunit.client.0.smithi002.stdout:3/725: rmdir d1/dd/d28/d7b/dd5 39 2024-03-16T10:45:05.482 INFO:tasks.workunit.client.0.smithi002.stdout:2/700: creat d7/d27/d3b/fd9 x:0 0 0 2024-03-16T10:45:05.482 INFO:tasks.workunit.client.0.smithi002.stdout:3/726: chown d1/dd/d28/d7b/d2c/d63/faf 0 1 2024-03-16T10:45:05.483 INFO:tasks.workunit.client.0.smithi002.stdout:6/782: rename d15/l21 to d15/d2d/d2b/d3f/l111 0 2024-03-16T10:45:05.483 INFO:tasks.workunit.client.0.smithi002.stdout:4/678: rmdir d2/d33/da/d2e 39 2024-03-16T10:45:05.484 INFO:tasks.workunit.client.0.smithi002.stdout:0/766: mkdir d3/df8 0 2024-03-16T10:45:05.485 INFO:tasks.workunit.client.0.smithi002.stdout:6/783: fsync d15/d2d/d2c/d5d/f76 0 2024-03-16T10:45:05.487 INFO:tasks.workunit.client.0.smithi002.stdout:4/679: fdatasync d2/d6/f10 0 2024-03-16T10:45:05.491 INFO:tasks.workunit.client.0.smithi002.stdout:3/727: creat d1/db6/d15/ffc x:0 0 0 2024-03-16T10:45:05.493 INFO:tasks.workunit.client.0.smithi002.stdout:6/784: dwrite d15/d2d/d2c/f7a [0,4194304] 0 2024-03-16T10:45:05.495 INFO:tasks.workunit.client.0.smithi002.stdout:2/701: link d7/d2e/d28/l61 d7/d2e/d28/d40/lda 0 2024-03-16T10:45:05.497 INFO:tasks.workunit.client.0.smithi002.stdout:3/728: unlink d1/dd/d28/c76 0 2024-03-16T10:45:05.498 INFO:tasks.workunit.client.0.smithi002.stdout:6/785: fdatasync d15/d2d/fbd 0 2024-03-16T10:45:05.500 INFO:tasks.workunit.client.0.smithi002.stdout:7/668: sync 2024-03-16T10:45:05.500 INFO:tasks.workunit.client.0.smithi002.stdout:9/823: sync 2024-03-16T10:45:05.500 INFO:tasks.workunit.client.0.smithi002.stdout:1/744: sync 2024-03-16T10:45:05.501 INFO:tasks.workunit.client.0.smithi002.stdout:9/824: stat d5/d9/d25/dac 0 2024-03-16T10:45:05.501 INFO:tasks.workunit.client.0.smithi002.stdout:3/729: mkdir d1/db6/d15/d87/d8c/dfd 0 2024-03-16T10:45:05.502 INFO:tasks.workunit.client.0.smithi002.stdout:1/745: read - d2/df/d12/d84/d87/d96/fcb zero size 2024-03-16T10:45:05.503 INFO:tasks.workunit.client.0.smithi002.stdout:4/680: creat d2/d6/d32/d66/fcf x:0 0 0 2024-03-16T10:45:05.504 INFO:tasks.workunit.client.0.smithi002.stdout:2/702: symlink d7/d2e/d28/d40/dd8/ldb 0 2024-03-16T10:45:05.506 INFO:tasks.workunit.client.0.smithi002.stdout:2/703: chown d7/d2e/d28/d40/fc4 179647782 1 2024-03-16T10:45:05.509 INFO:tasks.workunit.client.0.smithi002.stdout:6/786: unlink d15/d2d/d2b/l38 0 2024-03-16T10:45:05.511 INFO:tasks.workunit.client.0.smithi002.stdout:0/767: mknod d3/d28/cf9 0 2024-03-16T10:45:05.512 INFO:tasks.workunit.client.0.smithi002.stdout:6/787: chown d15/d2d/d2b/d3f/d77/d7e/cb1 48700 1 2024-03-16T10:45:05.513 INFO:tasks.workunit.client.0.smithi002.stdout:1/746: symlink d2/df/d12/d84/d87/d96/db5/de2/lf3 0 2024-03-16T10:45:05.514 INFO:tasks.workunit.client.0.smithi002.stdout:2/704: dread d7/db/f58 [0,4194304] 0 2024-03-16T10:45:05.514 INFO:tasks.workunit.client.0.smithi002.stdout:7/669: symlink d3/d61/lf2 0 2024-03-16T10:45:05.517 INFO:tasks.workunit.client.0.smithi002.stdout:7/670: readlink d3/d4/d1d/d87/d5c/d63/l5a 0 2024-03-16T10:45:05.517 INFO:tasks.workunit.client.0.smithi002.stdout:4/681: write d2/d33/d35/f85 [673160,61791] 0 2024-03-16T10:45:05.518 INFO:tasks.workunit.client.0.smithi002.stdout:2/705: rename d7/d2e/d1d/f26 to d7/d2e/d46/fdc 0 2024-03-16T10:45:05.520 INFO:tasks.workunit.client.0.smithi002.stdout:9/825: dread d5/d9/dc/d28/dd6/fd8 [0,4194304] 0 2024-03-16T10:45:05.524 INFO:tasks.workunit.client.0.smithi002.stdout:7/671: unlink d3/d4/d1d/d45/lc2 0 2024-03-16T10:45:05.524 INFO:tasks.workunit.client.0.smithi002.stdout:3/730: dwrite d1/dd/d28/d7b/d22/f44 [0,4194304] 0 2024-03-16T10:45:05.524 INFO:tasks.workunit.client.0.smithi002.stdout:0/768: dread d3/fc [4194304,4194304] 0 2024-03-16T10:45:05.528 INFO:tasks.workunit.client.0.smithi002.stdout:2/706: symlink d7/db/d70/ldd 0 2024-03-16T10:45:05.529 INFO:tasks.workunit.client.0.smithi002.stdout:1/747: truncate d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f79 1642566 0 2024-03-16T10:45:05.533 INFO:tasks.workunit.client.0.smithi002.stdout:2/707: mknod d7/db/d12/cde 0 2024-03-16T10:45:05.533 INFO:tasks.workunit.client.0.smithi002.stdout:9/826: link d5/d9/dc/d28/c51 d5/d9/c104 0 2024-03-16T10:45:05.534 INFO:tasks.workunit.client.0.smithi002.stdout:0/769: write d3/de/fe1 [547420,76391] 0 2024-03-16T10:45:05.535 INFO:tasks.workunit.client.0.smithi002.stdout:3/731: rename d1/db6/c42 to d1/dd/d28/d7b/d8a/d93/cfe 0 2024-03-16T10:45:05.537 INFO:tasks.workunit.client.0.smithi002.stdout:6/788: dwrite d15/dab/dc4/fef [0,4194304] 0 2024-03-16T10:45:05.539 INFO:tasks.workunit.client.0.smithi002.stdout:6/789: dread - d15/d2d/d73/fee zero size 2024-03-16T10:45:05.540 INFO:tasks.workunit.client.0.smithi002.stdout:7/672: link d3/d4/d1c/f81 d3/d4/d1d/d87/d5c/ddb/def/ff3 0 2024-03-16T10:45:05.542 INFO:tasks.workunit.client.0.smithi002.stdout:3/732: unlink d1/dd/d74/cc5 0 2024-03-16T10:45:05.543 INFO:tasks.workunit.client.0.smithi002.stdout:4/682: dread d2/d33/da/f5f [0,4194304] 0 2024-03-16T10:45:05.544 INFO:tasks.workunit.client.0.smithi002.stdout:1/748: truncate d2/df/d12/d84/d87/d96/db5/d51/d56/fef 1994979 0 2024-03-16T10:45:05.546 INFO:tasks.workunit.client.0.smithi002.stdout:0/770: mknod d3/d31/d75/df4/cfa 0 2024-03-16T10:45:05.546 INFO:tasks.workunit.client.0.smithi002.stdout:7/673: rmdir d3/d4/d1d/d87/d5c/dd0 39 2024-03-16T10:45:05.548 INFO:tasks.workunit.client.0.smithi002.stdout:6/790: mkdir d15/d2d/d2b/d3f/d77/d112 0 2024-03-16T10:45:05.548 INFO:tasks.workunit.client.0.smithi002.stdout:2/708: link d7/db/l91 d7/d2e/d1d/d38/ldf 0 2024-03-16T10:45:05.549 INFO:tasks.workunit.client.0.smithi002.stdout:1/749: read - d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f8f zero size 2024-03-16T10:45:05.551 INFO:tasks.workunit.client.0.smithi002.stdout:6/791: rmdir d15/d2d/d8a 39 2024-03-16T10:45:05.551 INFO:tasks.workunit.client.0.smithi002.stdout:3/733: creat d1/dd/d28/d7b/dd5/dee/fff x:0 0 0 2024-03-16T10:45:05.553 INFO:tasks.workunit.client.0.smithi002.stdout:7/674: creat d3/d4/ff4 x:0 0 0 2024-03-16T10:45:05.555 INFO:tasks.workunit.client.0.smithi002.stdout:4/683: dwrite d2/d6/d32/fa6 [0,4194304] 0 2024-03-16T10:45:05.557 INFO:tasks.workunit.client.0.smithi002.stdout:1/750: mknod d2/df/d12/d84/d87/cf4 0 2024-03-16T10:45:05.558 INFO:tasks.workunit.client.0.smithi002.stdout:6/792: symlink d15/d2d/d73/l113 0 2024-03-16T10:45:05.559 INFO:tasks.workunit.client.0.smithi002.stdout:0/771: dwrite d3/d28/d65/f77 [0,4194304] 0 2024-03-16T10:45:05.560 INFO:tasks.workunit.client.0.smithi002.stdout:7/675: unlink d3/lcf 0 2024-03-16T10:45:05.567 INFO:tasks.workunit.client.0.smithi002.stdout:3/734: rename d1/dd/ce2 to d1/dd/d28/d7b/d2c/d97/c100 0 2024-03-16T10:45:05.571 INFO:tasks.workunit.client.0.smithi002.stdout:4/684: dwrite d2/d33/fb3 [0,4194304] 0 2024-03-16T10:45:05.572 INFO:tasks.workunit.client.0.smithi002.stdout:0/772: fdatasync d3/d59/dca/f53 0 2024-03-16T10:45:05.572 INFO:tasks.workunit.client.0.smithi002.stdout:6/793: creat d15/dfd/f114 x:0 0 0 2024-03-16T10:45:05.573 INFO:tasks.workunit.client.0.smithi002.stdout:3/735: fdatasync d1/dd/d28/d7b/d2c/d97/fef 0 2024-03-16T10:45:05.573 INFO:tasks.workunit.client.0.smithi002.stdout:0/773: chown d3/d28/d2a 2 1 2024-03-16T10:45:05.575 INFO:tasks.workunit.client.0.smithi002.stdout:7/676: symlink d3/d61/lf5 0 2024-03-16T10:45:05.576 INFO:tasks.workunit.client.0.smithi002.stdout:4/685: dread - d2/d33/da/d5e/f9a zero size 2024-03-16T10:45:05.578 INFO:tasks.workunit.client.0.smithi002.stdout:0/774: chown d3/d31/fc6 708 1 2024-03-16T10:45:05.579 INFO:tasks.workunit.client.0.smithi002.stdout:6/794: mkdir d15/d2d/d73/d115 0 2024-03-16T10:45:05.581 INFO:tasks.workunit.client.0.smithi002.stdout:3/736: mknod d1/dd/d28/d7b/d2c/d63/c101 0 2024-03-16T10:45:05.582 INFO:tasks.workunit.client.0.smithi002.stdout:7/677: write d3/d4/d1d/d87/d5c/d63/d3f/f46 [256607,75856] 0 2024-03-16T10:45:05.584 INFO:tasks.workunit.client.0.smithi002.stdout:6/795: unlink d15/d2d/d73/ce6 0 2024-03-16T10:45:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:1/751: dread d2/df/d12/d29/d5c/f6e [0,4194304] 0 2024-03-16T10:45:05.589 INFO:tasks.workunit.client.0.smithi002.stdout:3/737: unlink d1/dd/f9e 0 2024-03-16T10:45:05.590 INFO:tasks.workunit.client.0.smithi002.stdout:6/796: creat d15/d2d/d2b/d3f/f116 x:0 0 0 2024-03-16T10:45:05.591 INFO:tasks.workunit.client.0.smithi002.stdout:1/752: chown d2/df/d12/d84/d87/d96/db5/f9d 11482458 1 2024-03-16T10:45:05.593 INFO:tasks.workunit.client.0.smithi002.stdout:7/678: getdents d3/d4/d1c/d5b/d64/ddf 0 2024-03-16T10:45:05.599 INFO:tasks.workunit.client.0.smithi002.stdout:4/686: dread d2/d33/da/d5e/d93/fa2 [0,4194304] 0 2024-03-16T10:45:05.600 INFO:tasks.workunit.client.0.smithi002.stdout:1/753: getdents d2/df/d12/d84/d87/d96/db5/d1c/d5d 0 2024-03-16T10:45:05.601 INFO:tasks.workunit.client.0.smithi002.stdout:0/775: dread d3/d31/f99 [0,4194304] 0 2024-03-16T10:45:05.605 INFO:tasks.workunit.client.0.smithi002.stdout:6/797: write f4 [2599687,86699] 0 2024-03-16T10:45:05.608 INFO:tasks.workunit.client.0.smithi002.stdout:4/687: truncate d2/d33/fb3 1246210 0 2024-03-16T10:45:05.608 INFO:tasks.workunit.client.0.smithi002.stdout:3/738: dread d1/dd/d28/f33 [0,4194304] 0 2024-03-16T10:45:05.609 INFO:tasks.workunit.client.0.smithi002.stdout:0/776: write d3/d28/f54 [92944,95162] 0 2024-03-16T10:45:05.611 INFO:tasks.workunit.client.0.smithi002.stdout:6/798: mkdir d15/d2d/d2b/de4/d117 0 2024-03-16T10:45:05.614 INFO:tasks.workunit.client.0.smithi002.stdout:3/739: rmdir d1/dd/d28/d7b 39 2024-03-16T10:45:05.616 INFO:tasks.workunit.client.0.smithi002.stdout:0/777: mknod d3/d31/d75/dea/cfb 0 2024-03-16T10:45:05.618 INFO:tasks.workunit.client.0.smithi002.stdout:7/679: dwrite d3/d4/d1d/d87/d5c/d63/f93 [0,4194304] 0 2024-03-16T10:45:05.621 INFO:tasks.workunit.client.0.smithi002.stdout:1/754: dread d2/df/d12/d84/d87/d96/db5/d51/d56/fef [0,4194304] 0 2024-03-16T10:45:05.624 INFO:tasks.workunit.client.0.smithi002.stdout:3/740: mknod d1/dd/d28/d37/df9/c102 0 2024-03-16T10:45:05.625 INFO:tasks.workunit.client.0.smithi002.stdout:6/799: truncate fa 5838697 0 2024-03-16T10:45:05.625 INFO:tasks.workunit.client.0.smithi002.stdout:0/778: truncate d3/d28/f4d 3010869 0 2024-03-16T10:45:05.626 INFO:tasks.workunit.client.0.smithi002.stdout:4/688: dwrite d2/d6/d32/d66/d7f/f9f [0,4194304] 0 2024-03-16T10:45:05.628 INFO:tasks.workunit.client.0.smithi002.stdout:6/800: fsync d15/d2d/d2b/fd9 0 2024-03-16T10:45:05.631 INFO:tasks.workunit.client.0.smithi002.stdout:0/779: unlink d3/d28/c37 0 2024-03-16T10:45:05.632 INFO:tasks.workunit.client.0.smithi002.stdout:1/755: rmdir d2/df/d12/d84/d87/d96/db5/dae/dc9 0 2024-03-16T10:45:05.633 INFO:tasks.workunit.client.0.smithi002.stdout:3/741: creat d1/dd/f103 x:0 0 0 2024-03-16T10:45:05.636 INFO:tasks.workunit.client.0.smithi002.stdout:4/689: dwrite d2/d6/d32/d66/d7f/fb5 [0,4194304] 0 2024-03-16T10:45:05.637 INFO:tasks.workunit.client.0.smithi002.stdout:0/780: mknod d3/d31/d88/dc8/cfc 0 2024-03-16T10:45:05.646 INFO:tasks.workunit.client.0.smithi002.stdout:4/690: creat d2/d33/da/d88/db2/fd0 x:0 0 0 2024-03-16T10:45:05.647 INFO:tasks.workunit.client.0.smithi002.stdout:0/781: creat d3/d27/d90/ffd x:0 0 0 2024-03-16T10:45:05.650 INFO:tasks.workunit.client.0.smithi002.stdout:4/691: write d2/d33/da/d88/f9b [164097,3972] 0 2024-03-16T10:45:05.651 INFO:tasks.workunit.client.0.smithi002.stdout:0/782: symlink d3/d31/d75/dde/lfe 0 2024-03-16T10:45:05.651 INFO:tasks.workunit.client.0.smithi002.stdout:1/756: write d2/f68 [4177730,45964] 0 2024-03-16T10:45:05.652 INFO:tasks.workunit.client.0.smithi002.stdout:0/783: chown d3/d27/d90/ffd 43409471 1 2024-03-16T10:45:05.652 INFO:tasks.workunit.client.0.smithi002.stdout:4/692: creat d2/d33/da/d88/db2/fd1 x:0 0 0 2024-03-16T10:45:05.653 INFO:tasks.workunit.client.0.smithi002.stdout:1/757: fdatasync d2/df/d12/dd7/fe4 0 2024-03-16T10:45:05.654 INFO:tasks.workunit.client.0.smithi002.stdout:3/742: write d1/dd/fe7 [1414301,53040] 0 2024-03-16T10:45:05.658 INFO:tasks.workunit.client.0.smithi002.stdout:4/693: getdents d2/d6 0 2024-03-16T10:45:05.660 INFO:tasks.workunit.client.0.smithi002.stdout:4/694: fdatasync d2/d6/d32/d68/f4b 0 2024-03-16T10:45:05.666 INFO:tasks.workunit.client.0.smithi002.stdout:4/695: rename d2/d33/da/f25 to d2/d33/da/d2e/db4/fd2 0 2024-03-16T10:45:05.668 INFO:tasks.workunit.client.0.smithi002.stdout:1/758: dwrite d2/df/d12/d84/d87/d96/db5/d51/fa6 [0,4194304] 0 2024-03-16T10:45:05.669 INFO:tasks.workunit.client.0.smithi002.stdout:1/759: chown d2/c59 17447 1 2024-03-16T10:45:05.670 INFO:tasks.workunit.client.0.smithi002.stdout:4/696: rmdir d2/d33/d1c 39 2024-03-16T10:45:05.672 INFO:tasks.workunit.client.0.smithi002.stdout:1/760: read d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f79 [1156691,71531] 0 2024-03-16T10:45:05.672 INFO:tasks.workunit.client.0.smithi002.stdout:3/743: dread d1/dd/d28/d7b/d8a/d93/fa2 [0,4194304] 0 2024-03-16T10:45:05.674 INFO:tasks.workunit.client.0.smithi002.stdout:3/744: fdatasync d1/dd/d28/d7b/d2c/d63/fea 0 2024-03-16T10:45:05.679 INFO:tasks.workunit.client.0.smithi002.stdout:4/697: getdents d2/d33/da/d2e/db4 0 2024-03-16T10:45:05.679 INFO:tasks.workunit.client.0.smithi002.stdout:3/745: creat d1/db6/d15/d87/dd2/f104 x:0 0 0 2024-03-16T10:45:05.681 INFO:tasks.workunit.client.0.smithi002.stdout:4/698: fdatasync d2/d33/fa9 0 2024-03-16T10:45:05.681 INFO:tasks.workunit.client.0.smithi002.stdout:1/761: write d2/df/d12/d84/d87/d96/db5/f55 [1124409,110049] 0 2024-03-16T10:45:05.683 INFO:tasks.workunit.client.0.smithi002.stdout:3/746: truncate d1/dd/f68 1678911 0 2024-03-16T10:45:05.684 INFO:tasks.workunit.client.0.smithi002.stdout:4/699: creat d2/d6/d32/fd3 x:0 0 0 2024-03-16T10:45:05.690 INFO:tasks.workunit.client.0.smithi002.stdout:1/762: dwrite d2/df/d12/d84/d87/d96/db5/fc [0,4194304] 0 2024-03-16T10:45:05.692 INFO:tasks.workunit.client.0.smithi002.stdout:4/700: dread d2/d33/da/d5e/f75 [0,4194304] 0 2024-03-16T10:45:05.695 INFO:tasks.workunit.client.0.smithi002.stdout:1/763: rmdir d2/df/d12/d84/d87/d96/db5/d1c/da0/db1 39 2024-03-16T10:45:05.699 INFO:tasks.workunit.client.0.smithi002.stdout:4/701: dread d2/d6/d32/d66/d7f/f9f [0,4194304] 0 2024-03-16T10:45:05.702 INFO:tasks.workunit.client.0.smithi002.stdout:4/702: mknod d2/d6/d32/d66/cd4 0 2024-03-16T10:45:05.704 INFO:tasks.workunit.client.0.smithi002.stdout:3/747: dread d1/dd/d28/d7b/d8a/d93/f91 [0,4194304] 0 2024-03-16T10:45:05.705 INFO:tasks.workunit.client.0.smithi002.stdout:4/703: unlink d2/d33/lc4 0 2024-03-16T10:45:05.706 INFO:tasks.workunit.client.0.smithi002.stdout:3/748: write d1/da5/fe1 [683421,103540] 0 2024-03-16T10:45:05.709 INFO:tasks.workunit.client.0.smithi002.stdout:3/749: dread - d1/db6/d15/f67 zero size 2024-03-16T10:45:05.709 INFO:tasks.workunit.client.0.smithi002.stdout:1/764: dwrite d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/faa [0,4194304] 0 2024-03-16T10:45:05.711 INFO:tasks.workunit.client.0.smithi002.stdout:3/750: dread - d1/fb0 zero size 2024-03-16T10:45:05.712 INFO:tasks.workunit.client.0.smithi002.stdout:1/765: symlink d2/df/d12/dd7/dd9/lf5 0 2024-03-16T10:45:05.715 INFO:tasks.workunit.client.0.smithi002.stdout:4/704: getdents d2/d33 0 2024-03-16T10:45:05.715 INFO:tasks.workunit.client.0.smithi002.stdout:3/751: mkdir d1/dd/d28/d7b/d2c/d63/d105 0 2024-03-16T10:45:05.716 INFO:tasks.workunit.client.0.smithi002.stdout:4/705: readlink d2/d33/d1c/l9c 0 2024-03-16T10:45:05.718 INFO:tasks.workunit.client.0.smithi002.stdout:1/766: truncate d2/df/d12/d84/d87/d96/db5/f1f 3285406 0 2024-03-16T10:45:05.719 INFO:tasks.workunit.client.0.smithi002.stdout:4/706: rmdir d2/d33/da/d2e/db4 39 2024-03-16T10:45:05.720 INFO:tasks.workunit.client.0.smithi002.stdout:4/707: truncate d2/d6/f89 317859 0 2024-03-16T10:45:05.720 INFO:tasks.workunit.client.0.smithi002.stdout:3/752: symlink d1/db6/d15/dcc/l106 0 2024-03-16T10:45:05.722 INFO:tasks.workunit.client.0.smithi002.stdout:4/708: chown d2/d33/da/f4f 185336 1 2024-03-16T10:45:05.723 INFO:tasks.workunit.client.0.smithi002.stdout:3/753: creat d1/dd/f107 x:0 0 0 2024-03-16T10:45:05.724 INFO:tasks.workunit.client.0.smithi002.stdout:4/709: dread - d2/d6/d32/d66/fcf zero size 2024-03-16T10:45:05.725 INFO:tasks.workunit.client.0.smithi002.stdout:1/767: dwrite d2/df/d12/d84/d87/d96/f9c [0,4194304] 0 2024-03-16T10:45:05.725 INFO:tasks.workunit.client.0.smithi002.stdout:4/710: read d2/d33/fb3 [570302,30573] 0 2024-03-16T10:45:05.728 INFO:tasks.workunit.client.0.smithi002.stdout:3/754: creat d1/db6/d15/f108 x:0 0 0 2024-03-16T10:45:05.731 INFO:tasks.workunit.client.0.smithi002.stdout:1/768: rename d2/de0 to d2/df/d12/df6 0 2024-03-16T10:45:05.731 INFO:tasks.workunit.client.0.smithi002.stdout:3/755: dread d1/dd/d28/d7b/d2c/fe4 [0,4194304] 0 2024-03-16T10:45:05.731 INFO:tasks.workunit.client.0.smithi002.stdout:1/769: chown d2/df/d12/c18 38 1 2024-03-16T10:45:05.732 INFO:tasks.workunit.client.0.smithi002.stdout:1/770: stat d2/d58/lc6 0 2024-03-16T10:45:05.732 INFO:tasks.workunit.client.0.smithi002.stdout:4/711: link d2/d33/da/d5e/c71 d2/d33/d35/d59/cd5 0 2024-03-16T10:45:05.733 INFO:tasks.workunit.client.0.smithi002.stdout:3/756: rmdir d1/dd/d74/dd8 39 2024-03-16T10:45:05.734 INFO:tasks.workunit.client.0.smithi002.stdout:3/757: dread - d1/dd/d28/d7b/d8a/d93/fc2 zero size 2024-03-16T10:45:05.735 INFO:tasks.workunit.client.0.smithi002.stdout:4/712: truncate d2/d33/da/ff 5187049 0 2024-03-16T10:45:05.736 INFO:tasks.workunit.client.0.smithi002.stdout:1/771: creat d2/df/d12/dd7/dd9/deb/ff7 x:0 0 0 2024-03-16T10:45:05.737 INFO:tasks.workunit.client.0.smithi002.stdout:4/713: creat d2/d6/fd6 x:0 0 0 2024-03-16T10:45:05.737 INFO:tasks.workunit.client.0.smithi002.stdout:1/772: dread d2/df/d12/df6/ded/ff1 [0,4194304] 0 2024-03-16T10:45:05.738 INFO:tasks.workunit.client.0.smithi002.stdout:4/714: read d2/d33/da/f5f [2201660,105732] 0 2024-03-16T10:45:05.739 INFO:tasks.workunit.client.0.smithi002.stdout:3/758: rename d1/dd/d74/fa8 to d1/dd/f109 0 2024-03-16T10:45:05.739 INFO:tasks.workunit.client.0.smithi002.stdout:4/715: readlink d2/d6/d32/d66/l6b 0 2024-03-16T10:45:05.739 INFO:tasks.workunit.client.0.smithi002.stdout:3/759: chown d1/dd/d28/d7b/d22 863118 1 2024-03-16T10:45:05.741 INFO:tasks.workunit.client.0.smithi002.stdout:1/773: read d2/df/d12/d84/d87/d96/db5/d51/d56/fc5 [915458,28213] 0 2024-03-16T10:45:05.741 INFO:tasks.workunit.client.0.smithi002.stdout:4/716: creat d2/d33/fd7 x:0 0 0 2024-03-16T10:45:05.744 INFO:tasks.workunit.client.0.smithi002.stdout:1/774: creat d2/df/d12/d84/d87/d96/db5/d51/d56/ff8 x:0 0 0 2024-03-16T10:45:05.746 INFO:tasks.workunit.client.0.smithi002.stdout:1/775: fsync d2/df/d12/d29/d5c/f6e 0 2024-03-16T10:45:05.747 INFO:tasks.workunit.client.0.smithi002.stdout:1/776: chown d2/df/d12/d84/d87/d96/db5/d1c/lc7 9568730 1 2024-03-16T10:45:05.749 INFO:tasks.workunit.client.0.smithi002.stdout:8/812: sync 2024-03-16T10:45:05.753 INFO:tasks.workunit.client.0.smithi002.stdout:1/777: getdents d2/df/d12/d84/d87/d96/db5/d1c/d5d 0 2024-03-16T10:45:05.755 INFO:tasks.workunit.client.0.smithi002.stdout:4/717: write d2/d33/fb3 [1612799,34831] 0 2024-03-16T10:45:05.757 INFO:tasks.workunit.client.0.smithi002.stdout:3/760: dread d1/db6/d15/fd3 [0,4194304] 0 2024-03-16T10:45:05.757 INFO:tasks.workunit.client.0.smithi002.stdout:1/778: chown d2/df/d12/d29/d5c/l7a 0 1 2024-03-16T10:45:05.762 INFO:tasks.workunit.client.0.smithi002.stdout:4/718: stat d2/d33/d35/f6f 0 2024-03-16T10:45:05.762 INFO:tasks.workunit.client.0.smithi002.stdout:1/779: creat d2/df/d12/d74/ff9 x:0 0 0 2024-03-16T10:45:05.762 INFO:tasks.workunit.client.0.smithi002.stdout:8/813: dwrite d7/d9/d17/d23/fd3 [0,4194304] 0 2024-03-16T10:45:05.762 INFO:tasks.workunit.client.0.smithi002.stdout:3/761: mknod d1/dd/d28/d7b/d2c/d63/d105/c10a 0 2024-03-16T10:45:05.764 INFO:tasks.workunit.client.0.smithi002.stdout:1/780: write d2/df/d12/d84/d87/d96/da8/fa9 [390474,115179] 0 2024-03-16T10:45:05.766 INFO:tasks.workunit.client.0.smithi002.stdout:3/762: chown d1/dd/d28/d7b/d8a/d93/la0 34928564 1 2024-03-16T10:45:05.766 INFO:tasks.workunit.client.0.smithi002.stdout:8/814: creat d7/d9/d17/d83/df5/f115 x:0 0 0 2024-03-16T10:45:05.767 INFO:tasks.workunit.client.0.smithi002.stdout:1/781: creat d2/df/d12/d74/ffa x:0 0 0 2024-03-16T10:45:05.769 INFO:tasks.workunit.client.0.smithi002.stdout:1/782: write d2/df/d12/d84/d87/d96/f9c [3676840,116196] 0 2024-03-16T10:45:05.769 INFO:tasks.workunit.client.0.smithi002.stdout:3/763: creat d1/db6/d15/d87/dd2/f10b x:0 0 0 2024-03-16T10:45:05.770 INFO:tasks.workunit.client.0.smithi002.stdout:8/815: creat d7/d13/db8/f116 x:0 0 0 2024-03-16T10:45:05.771 INFO:tasks.workunit.client.0.smithi002.stdout:8/816: chown d7/d13/f4c 3762 1 2024-03-16T10:45:05.772 INFO:tasks.workunit.client.0.smithi002.stdout:3/764: mkdir d1/dd/d28/d7b/d8a/dd7/df1/d10c 0 2024-03-16T10:45:05.777 INFO:tasks.workunit.client.0.smithi002.stdout:3/765: unlink d1/db6/d15/l9f 0 2024-03-16T10:45:05.778 INFO:tasks.workunit.client.0.smithi002.stdout:1/783: write d2/df/f45 [1150643,46096] 0 2024-03-16T10:45:05.778 INFO:tasks.workunit.client.0.smithi002.stdout:8/817: write d7/d9/d17/d20/f5e [530420,97211] 0 2024-03-16T10:45:05.779 INFO:tasks.workunit.client.0.smithi002.stdout:3/766: fdatasync d1/dd/d28/d7b/f77 0 2024-03-16T10:45:05.780 INFO:tasks.workunit.client.0.smithi002.stdout:3/767: chown d1/dd/d28/d7b/dd5/dee 3892161 1 2024-03-16T10:45:05.782 INFO:tasks.workunit.client.0.smithi002.stdout:8/818: truncate d7/d9/d17/f10b 7843846 0 2024-03-16T10:45:05.782 INFO:tasks.workunit.client.0.smithi002.stdout:1/784: truncate d2/df/d12/d84/d87/d96/db5/d51/fa6 5064121 0 2024-03-16T10:45:05.782 INFO:tasks.workunit.client.0.smithi002.stdout:3/768: fsync d1/dd/d28/d7b/d2c/fe4 0 2024-03-16T10:45:05.784 INFO:tasks.workunit.client.0.smithi002.stdout:3/769: write d1/dd/d28/d7b/dd5/dee/fff [260444,18709] 0 2024-03-16T10:45:05.785 INFO:tasks.workunit.client.0.smithi002.stdout:1/785: mknod d2/df/d12/d84/d87/cfb 0 2024-03-16T10:45:05.789 INFO:tasks.workunit.client.0.smithi002.stdout:3/770: truncate d1/dd/d28/d7b/d2c/fdb 4844930 0 2024-03-16T10:45:05.789 INFO:tasks.workunit.client.0.smithi002.stdout:5/640: sync 2024-03-16T10:45:05.790 INFO:tasks.workunit.client.0.smithi002.stdout:1/786: symlink d2/df/d12/d84/d87/d96/db5/dae/lfc 0 2024-03-16T10:45:05.790 INFO:tasks.workunit.client.0.smithi002.stdout:8/819: dwrite d7/d9/d17/d23/d2e/d34/dbf/ff7 [0,4194304] 0 2024-03-16T10:45:05.793 INFO:tasks.workunit.client.0.smithi002.stdout:3/771: rmdir d1/db6/d15/dcc 39 2024-03-16T10:45:05.797 INFO:tasks.workunit.client.0.smithi002.stdout:1/787: mknod d2/d58/de8/cfd 0 2024-03-16T10:45:05.797 INFO:tasks.workunit.client.0.smithi002.stdout:3/772: symlink d1/dd/d28/d37/l10d 0 2024-03-16T10:45:05.799 INFO:tasks.workunit.client.0.smithi002.stdout:8/820: write d7/d13/db8/f98 [593057,118163] 0 2024-03-16T10:45:05.800 INFO:tasks.workunit.client.0.smithi002.stdout:3/773: chown d1/dd/d28/d7b/d2c/c6b 0 1 2024-03-16T10:45:05.802 INFO:tasks.workunit.client.0.smithi002.stdout:5/641: dwrite d0/d16/d4a/d7d/fc1 [0,4194304] 0 2024-03-16T10:45:05.804 INFO:tasks.workunit.client.0.smithi002.stdout:8/821: read d7/d9/d17/d23/d2e/d34/f9e [2561740,120822] 0 2024-03-16T10:45:05.804 INFO:tasks.workunit.client.1.smithi026.stdout:2/13: sync 2024-03-16T10:45:05.804 INFO:tasks.workunit.client.1.smithi026.stdout:2/14: dwrite - no filename 2024-03-16T10:45:05.804 INFO:tasks.workunit.client.1.smithi026.stdout:2/15: dwrite - no filename 2024-03-16T10:45:05.804 INFO:tasks.workunit.client.1.smithi026.stdout:2/16: write - no filename 2024-03-16T10:45:05.805 INFO:tasks.workunit.client.1.smithi026.stdout:8/1: sync 2024-03-16T10:45:05.805 INFO:tasks.workunit.client.1.smithi026.stdout:2/17: chown . 6 1 2024-03-16T10:45:05.805 INFO:tasks.workunit.client.0.smithi002.stdout:8/822: truncate d7/d9/d25/f102 108993 0 2024-03-16T10:45:05.805 INFO:tasks.workunit.client.0.smithi002.stdout:1/788: rename d2/df/d12/d84/d87/d96/db5/fe to d2/df/d12/d84/d87/d96/db5/d51/d56/ffe 0 2024-03-16T10:45:05.808 INFO:tasks.workunit.client.1.smithi026.stdout:8/2: unlink f0 0 2024-03-16T10:45:05.808 INFO:tasks.workunit.client.1.smithi026.stdout:8/3: link - no file 2024-03-16T10:45:05.808 INFO:tasks.workunit.client.1.smithi026.stdout:8/4: dread - no filename 2024-03-16T10:45:05.808 INFO:tasks.workunit.client.1.smithi026.stdout:8/5: dwrite - no filename 2024-03-16T10:45:05.808 INFO:tasks.workunit.client.1.smithi026.stdout:8/6: rename - no filename 2024-03-16T10:45:05.809 INFO:tasks.workunit.client.1.smithi026.stdout:8/7: write - no filename 2024-03-16T10:45:05.809 INFO:tasks.workunit.client.1.smithi026.stdout:8/8: fdatasync - no filename 2024-03-16T10:45:05.810 INFO:tasks.workunit.client.0.smithi002.stdout:5/642: rmdir d0/d24/dad/db6 39 2024-03-16T10:45:05.811 INFO:tasks.workunit.client.0.smithi002.stdout:5/643: fsync d0/d1f/f2c 0 2024-03-16T10:45:05.812 INFO:tasks.workunit.client.0.smithi002.stdout:5/644: dread - d0/d16/dab/d33/f42 zero size 2024-03-16T10:45:05.812 INFO:tasks.workunit.client.1.smithi026.stdout:8/9: creat f1 x:0 0 0 2024-03-16T10:45:05.812 INFO:tasks.workunit.client.1.smithi026.stdout:8/10: chown f1 0 1 2024-03-16T10:45:05.813 INFO:tasks.workunit.client.0.smithi002.stdout:3/774: mknod d1/db6/d15/c10e 0 2024-03-16T10:45:05.813 INFO:tasks.workunit.client.0.smithi002.stdout:5/645: dread - d0/d16/dab/d2b/fc6 zero size 2024-03-16T10:45:05.813 INFO:tasks.workunit.client.1.smithi026.stdout:8/11: chown f1 33080 1 2024-03-16T10:45:05.813 INFO:tasks.workunit.client.1.smithi026.stdout:2/18: symlink l0 0 2024-03-16T10:45:05.814 INFO:tasks.workunit.client.0.smithi002.stdout:5/646: readlink d0/d16/dab/d2b/d5d/la9 0 2024-03-16T10:45:05.814 INFO:tasks.workunit.client.0.smithi002.stdout:5/647: chown d0/d16/l1b 8 1 2024-03-16T10:45:05.818 INFO:tasks.workunit.client.0.smithi002.stdout:1/789: mknod d2/df/d12/d84/d87/d96/db5/cff 0 2024-03-16T10:45:05.819 INFO:tasks.workunit.client.0.smithi002.stdout:5/648: write d0/fc3 [171182,21038] 0 2024-03-16T10:45:05.819 INFO:tasks.workunit.client.0.smithi002.stdout:8/823: creat d7/d9/f117 x:0 0 0 2024-03-16T10:45:05.820 INFO:tasks.workunit.client.0.smithi002.stdout:5/649: readlink d0/d16/d5b/l62 0 2024-03-16T10:45:05.822 INFO:tasks.workunit.client.0.smithi002.stdout:5/650: truncate d0/d16/dab/f85 327523 0 2024-03-16T10:45:05.826 INFO:tasks.workunit.client.0.smithi002.stdout:5/651: fdatasync d0/d16/dab/d2b/d94/f7a 0 2024-03-16T10:45:05.827 INFO:tasks.workunit.client.0.smithi002.stdout:3/775: dread d1/dd/d28/f6a [0,4194304] 0 2024-03-16T10:45:05.829 INFO:tasks.workunit.client.0.smithi002.stdout:8/824: dwrite d7/d13/db8/d8f/fdb [0,4194304] 0 2024-03-16T10:45:05.831 INFO:tasks.workunit.client.0.smithi002.stdout:8/825: chown d7/d13/db8/f98 3568477 1 2024-03-16T10:45:05.834 INFO:tasks.workunit.client.0.smithi002.stdout:3/776: rename d1/db6/d15/d87/d8c/fdf to d1/dd/d28/d37/f10f 0 2024-03-16T10:45:05.837 INFO:tasks.workunit.client.0.smithi002.stdout:1/790: dread d2/df/d12/d29/f34 [0,4194304] 0 2024-03-16T10:45:05.840 INFO:tasks.workunit.client.0.smithi002.stdout:8/826: dwrite d7/d9/d17/d23/d39/fdc [0,4194304] 0 2024-03-16T10:45:05.844 INFO:tasks.workunit.client.0.smithi002.stdout:1/791: fsync d2/df/d12/d84/d87/d96/db5/dae/ff0 0 2024-03-16T10:45:05.845 INFO:tasks.workunit.client.0.smithi002.stdout:3/777: dwrite d1/dd/d28/ff8 [0,4194304] 0 2024-03-16T10:45:05.851 INFO:tasks.workunit.client.0.smithi002.stdout:8/827: mkdir d7/d9/d17/d118 0 2024-03-16T10:45:05.862 INFO:tasks.workunit.client.0.smithi002.stdout:1/792: dwrite d2/df/d12/d84/d87/d96/db5/d51/d56/f8c [0,4194304] 0 2024-03-16T10:45:05.863 INFO:tasks.workunit.client.0.smithi002.stdout:1/793: stat d2/df/d12/d84/d87/d96/db5/d51/f5a 0 2024-03-16T10:45:05.869 INFO:tasks.workunit.client.0.smithi002.stdout:1/794: unlink d2/df/d12/d84/d87/d96/db5/d51/d56/ff8 0 2024-03-16T10:45:05.870 INFO:tasks.workunit.client.0.smithi002.stdout:3/778: write d1/f35 [7496305,15030] 0 2024-03-16T10:45:05.872 INFO:tasks.workunit.client.0.smithi002.stdout:1/795: truncate d2/df/d12/d74/ffa 248174 0 2024-03-16T10:45:05.872 INFO:tasks.workunit.client.0.smithi002.stdout:8/828: dread d7/d9/d17/d23/d39/f72 [0,4194304] 0 2024-03-16T10:45:05.873 INFO:tasks.workunit.client.0.smithi002.stdout:3/779: mkdir d1/dd/d74/dd8/d110 0 2024-03-16T10:45:05.875 INFO:tasks.workunit.client.0.smithi002.stdout:3/780: write d1/dd/d28/d37/f8d [2747956,64431] 0 2024-03-16T10:45:05.879 INFO:tasks.workunit.client.0.smithi002.stdout:3/781: mknod d1/dd/d28/d7b/d2c/d63/c111 0 2024-03-16T10:45:05.880 INFO:tasks.workunit.client.0.smithi002.stdout:8/829: creat d7/d9/d17/d23/d2e/f119 x:0 0 0 2024-03-16T10:45:05.884 INFO:tasks.workunit.client.0.smithi002.stdout:8/830: chown d7/d9/d17/d23/d39/f8c 50614 1 2024-03-16T10:45:05.885 INFO:tasks.workunit.client.0.smithi002.stdout:3/782: symlink d1/db6/d15/d87/d8c/dfd/l112 0 2024-03-16T10:45:05.887 INFO:tasks.workunit.client.0.smithi002.stdout:3/783: read d1/dd/d28/d7b/dd5/dee/fff [33353,107491] 0 2024-03-16T10:45:05.897 INFO:tasks.workunit.client.0.smithi002.stdout:8/831: dread d7/d13/db8/d8f/ffd [0,4194304] 0 2024-03-16T10:45:05.897 INFO:tasks.workunit.client.0.smithi002.stdout:3/784: rename d1/db6/f30 to d1/db6/d15/d87/f113 0 2024-03-16T10:45:05.899 INFO:tasks.workunit.client.0.smithi002.stdout:8/832: rmdir d7/d9/d17/d83/df5 39 2024-03-16T10:45:05.904 INFO:tasks.workunit.client.0.smithi002.stdout:8/833: dread d7/d9/d17/d23/d2e/ff4 [0,4194304] 0 2024-03-16T10:45:05.905 INFO:tasks.workunit.client.0.smithi002.stdout:3/785: link d1/dd/d28/d7b/d25/fc9 d1/dd/d28/d37/df9/f114 0 2024-03-16T10:45:05.907 INFO:tasks.workunit.client.0.smithi002.stdout:8/834: mknod d7/d9/d17/dc3/c11a 0 2024-03-16T10:45:05.911 INFO:tasks.workunit.client.0.smithi002.stdout:1/796: truncate d2/df/d12/d84/d87/d96/db5/f7c 2741943 0 2024-03-16T10:45:05.915 INFO:tasks.workunit.client.0.smithi002.stdout:8/835: dwrite d7/fd [0,4194304] 0 2024-03-16T10:45:05.925 INFO:tasks.workunit.client.0.smithi002.stdout:8/836: creat d7/d9/d17/d23/d2e/d5c/f11b x:0 0 0 2024-03-16T10:45:05.928 INFO:tasks.workunit.client.0.smithi002.stdout:8/837: chown d7/f60 2160 1 2024-03-16T10:45:05.928 INFO:tasks.workunit.client.0.smithi002.stdout:1/797: dread d2/df/d12/d84/d87/d96/db5/f65 [0,4194304] 0 2024-03-16T10:45:05.929 INFO:tasks.workunit.client.0.smithi002.stdout:9/827: sync 2024-03-16T10:45:05.930 INFO:tasks.workunit.client.0.smithi002.stdout:8/838: truncate d7/d9/d17/d23/d2e/d90/fba 507859 0 2024-03-16T10:45:05.931 INFO:tasks.workunit.client.0.smithi002.stdout:8/839: dread - d7/d9/d17/dc3/ff3 zero size 2024-03-16T10:45:05.931 INFO:tasks.workunit.client.0.smithi002.stdout:8/840: readlink d7/d13/db8/l95 0 2024-03-16T10:45:05.932 INFO:tasks.workunit.client.0.smithi002.stdout:1/798: unlink d2/f76 0 2024-03-16T10:45:05.937 INFO:tasks.workunit.client.0.smithi002.stdout:8/841: creat d7/d9/d17/d23/d2e/d34/dbf/dd9/f11c x:0 0 0 2024-03-16T10:45:05.938 INFO:tasks.workunit.client.0.smithi002.stdout:1/799: mkdir d2/df/d12/d84/d100 0 2024-03-16T10:45:05.940 INFO:tasks.workunit.client.0.smithi002.stdout:8/842: truncate d7/d9/d25/d67/fc0 1293450 0 2024-03-16T10:45:05.941 INFO:tasks.workunit.client.0.smithi002.stdout:9/828: creat d5/d9/d34/f105 x:0 0 0 2024-03-16T10:45:05.943 INFO:tasks.workunit.client.0.smithi002.stdout:1/800: fdatasync d2/df/d12/fa5 0 2024-03-16T10:45:05.944 INFO:tasks.workunit.client.0.smithi002.stdout:3/786: dwrite d1/dd/d28/d7b/f7c [0,4194304] 0 2024-03-16T10:45:05.944 INFO:tasks.workunit.client.0.smithi002.stdout:8/843: unlink d7/d9/d25/d91/f96 0 2024-03-16T10:45:05.946 INFO:tasks.workunit.client.0.smithi002.stdout:1/801: dread - d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/fe9 zero size 2024-03-16T10:45:05.950 INFO:tasks.workunit.client.0.smithi002.stdout:9/829: dread - d5/d9/dc/d1e/d91/d82/db4/fc1 zero size 2024-03-16T10:45:05.952 INFO:tasks.workunit.client.1.smithi026.stdout:3/5: sync 2024-03-16T10:45:05.952 INFO:tasks.workunit.client.1.smithi026.stdout:1/16: sync 2024-03-16T10:45:05.952 INFO:tasks.workunit.client.0.smithi002.stdout:8/844: mknod d7/d9/d17/d83/da2/c11d 0 2024-03-16T10:45:05.955 INFO:tasks.workunit.client.0.smithi002.stdout:3/787: mkdir d1/dd/d28/d7b/d2c/d63/d115 0 2024-03-16T10:45:05.955 INFO:tasks.workunit.client.0.smithi002.stdout:3/788: readlink d1/la 0 2024-03-16T10:45:05.958 INFO:tasks.workunit.client.0.smithi002.stdout:8/845: dwrite d7/d13/db8/f86 [0,4194304] 0 2024-03-16T10:45:05.959 INFO:tasks.workunit.client.0.smithi002.stdout:9/830: mknod d5/d9/dc/d1e/d24/c106 0 2024-03-16T10:45:05.962 INFO:tasks.workunit.client.0.smithi002.stdout:1/802: link d2/df/d12/d74/ff9 d2/df/d12/dd7/dd9/f101 0 2024-03-16T10:45:05.966 INFO:tasks.workunit.client.0.smithi002.stdout:8/846: symlink d7/l11e 0 2024-03-16T10:45:05.968 INFO:tasks.workunit.client.0.smithi002.stdout:8/847: write d7/d13/db8/f116 [390450,94950] 0 2024-03-16T10:45:05.968 INFO:tasks.workunit.client.0.smithi002.stdout:1/803: mkdir d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/d102 0 2024-03-16T10:45:05.969 INFO:tasks.workunit.client.0.smithi002.stdout:8/848: readlink d7/d9/d17/d23/d39/l3f 0 2024-03-16T10:45:05.969 INFO:tasks.workunit.client.0.smithi002.stdout:1/804: write d2/f68 [1997859,122044] 0 2024-03-16T10:45:05.970 INFO:tasks.workunit.client.0.smithi002.stdout:9/831: mkdir d5/d9/dc/d1e/d91/d107 0 2024-03-16T10:45:05.975 INFO:tasks.workunit.client.0.smithi002.stdout:8/849: rename d7/d9/d25/fff to d7/d9/d17/d23/d2e/de5/f11f 0 2024-03-16T10:45:05.975 INFO:tasks.workunit.client.0.smithi002.stdout:1/805: dread d2/df/d12/d84/d87/d96/db5/d51/d56/f8c [0,4194304] 0 2024-03-16T10:45:05.978 INFO:tasks.workunit.client.0.smithi002.stdout:8/850: creat d7/d9/d17/d83/da2/f120 x:0 0 0 2024-03-16T10:45:05.979 INFO:tasks.workunit.client.0.smithi002.stdout:8/851: fdatasync d7/d9/fa 0 2024-03-16T10:45:05.985 INFO:tasks.workunit.client.0.smithi002.stdout:1/806: creat d2/df/d12/f103 x:0 0 0 2024-03-16T10:45:05.988 INFO:tasks.workunit.client.0.smithi002.stdout:8/852: mknod d7/d9/d17/d23/c121 0 2024-03-16T10:45:05.989 INFO:tasks.workunit.client.0.smithi002.stdout:8/853: chown d7/d9/d17/d23/d2e/d34/dbf/d109 1697 1 2024-03-16T10:45:05.989 INFO:tasks.workunit.client.0.smithi002.stdout:1/807: mknod d2/df/d12/d84/d87/d96/db5/d51/d56/c104 0 2024-03-16T10:45:05.990 INFO:tasks.workunit.client.0.smithi002.stdout:1/808: dread - d2/df/d12/dd7/fe4 zero size 2024-03-16T10:45:05.992 INFO:tasks.workunit.client.0.smithi002.stdout:8/854: getdents d7/d9/d17/d23/d2e/d34/dbf/d109 0 2024-03-16T10:45:05.994 INFO:tasks.workunit.client.0.smithi002.stdout:1/809: fdatasync d2/df/d12/d84/d87/d96/db5/d51/f5a 0 2024-03-16T10:45:05.998 INFO:tasks.workunit.client.0.smithi002.stdout:8/855: dwrite d7/d13/db8/f111 [0,4194304] 0 2024-03-16T10:45:06.002 INFO:tasks.workunit.client.0.smithi002.stdout:8/856: rmdir d7/d9/d17/d23/d2e/d34/dbf/dd9 39 2024-03-16T10:45:06.004 INFO:tasks.workunit.client.0.smithi002.stdout:8/857: write d7/d13/db8/f7c [852760,95694] 0 2024-03-16T10:45:06.008 INFO:tasks.workunit.client.0.smithi002.stdout:1/810: getdents d2/df/d12/df6 0 2024-03-16T10:45:06.009 INFO:tasks.workunit.client.1.smithi026.stdout:8/12: getdents . 0 2024-03-16T10:45:06.009 INFO:tasks.workunit.client.0.smithi002.stdout:1/811: chown d2/df/d12/d84/d87/d96/db5/d1c/c40 69384476 1 2024-03-16T10:45:06.010 INFO:tasks.workunit.client.0.smithi002.stdout:8/858: creat d7/d9/d17/d23/d2e/de5/f122 x:0 0 0 2024-03-16T10:45:06.011 INFO:tasks.workunit.client.1.smithi026.stdout:3/6: chown l2 1718 1 2024-03-16T10:45:06.012 INFO:tasks.workunit.client.0.smithi002.stdout:1/812: mkdir d2/df/d12/d84/d87/d96/d105 0 2024-03-16T10:45:06.014 INFO:tasks.workunit.client.0.smithi002.stdout:8/859: mkdir d7/d9/d25/d91/d123 0 2024-03-16T10:45:06.015 INFO:tasks.workunit.client.1.smithi026.stdout:8/13: symlink l2 0 2024-03-16T10:45:06.015 INFO:tasks.workunit.client.0.smithi002.stdout:8/860: write d7/d9/d17/d20/f5e [164461,63770] 0 2024-03-16T10:45:06.019 INFO:tasks.workunit.client.1.smithi026.stdout:1/17: rename d0/l3 to d0/l6 0 2024-03-16T10:45:06.019 INFO:tasks.workunit.client.1.smithi026.stdout:1/18: fsync d0/f5 0 2024-03-16T10:45:06.020 INFO:tasks.workunit.client.0.smithi002.stdout:1/813: mkdir d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/d102/d106 0 2024-03-16T10:45:06.020 INFO:tasks.workunit.client.1.smithi026.stdout:1/19: truncate d0/f5 489160 0 2024-03-16T10:45:06.021 INFO:tasks.workunit.client.1.smithi026.stdout:8/14: mknod c3 0 2024-03-16T10:45:06.021 INFO:tasks.workunit.client.1.smithi026.stdout:1/20: write d0/f5 [1272466,62347] 0 2024-03-16T10:45:06.022 INFO:tasks.workunit.client.0.smithi002.stdout:8/861: creat d7/d9/d25/d91/f124 x:0 0 0 2024-03-16T10:45:06.023 INFO:tasks.workunit.client.1.smithi026.stdout:1/21: fsync d0/f5 0 2024-03-16T10:45:06.024 INFO:tasks.workunit.client.0.smithi002.stdout:1/814: unlink d2/df/d12/d84/d87/d96/db5/d1c/f4a 0 2024-03-16T10:45:06.024 INFO:tasks.workunit.client.1.smithi026.stdout:1/22: write d0/f5 [70708,52909] 0 2024-03-16T10:45:06.028 INFO:tasks.workunit.client.0.smithi002.stdout:8/862: dwrite d7/d9/d25/d91/f124 [0,4194304] 0 2024-03-16T10:45:06.028 INFO:tasks.workunit.client.0.smithi002.stdout:1/815: fdatasync d2/df/d12/d84/d87/d96/db5/f7e 0 2024-03-16T10:45:06.031 INFO:tasks.workunit.client.1.smithi026.stdout:8/15: link f1 f4 0 2024-03-16T10:45:06.032 INFO:tasks.workunit.client.0.smithi002.stdout:1/816: write d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/fe9 [855633,54499] 0 2024-03-16T10:45:06.033 INFO:tasks.workunit.client.1.smithi026.stdout:3/7: getdents . 0 2024-03-16T10:45:06.033 INFO:tasks.workunit.client.1.smithi026.stdout:3/8: truncate - no filename 2024-03-16T10:45:06.033 INFO:tasks.workunit.client.1.smithi026.stdout:3/9: dwrite - no filename 2024-03-16T10:45:06.033 INFO:tasks.workunit.client.1.smithi026.stdout:3/10: dwrite - no filename 2024-03-16T10:45:06.033 INFO:tasks.workunit.client.1.smithi026.stdout:3/11: dread - no filename 2024-03-16T10:45:06.033 INFO:tasks.workunit.client.1.smithi026.stdout:3/12: rmdir - no directory 2024-03-16T10:45:06.034 INFO:tasks.workunit.client.1.smithi026.stdout:4/8: sync 2024-03-16T10:45:06.034 INFO:tasks.workunit.client.1.smithi026.stdout:6/76: sync 2024-03-16T10:45:06.034 INFO:tasks.workunit.client.1.smithi026.stdout:0/79: sync 2024-03-16T10:45:06.034 INFO:tasks.workunit.client.1.smithi026.stdout:5/29: sync 2024-03-16T10:45:06.034 INFO:tasks.workunit.client.1.smithi026.stdout:7/13: sync 2024-03-16T10:45:06.034 INFO:tasks.workunit.client.1.smithi026.stdout:9/18: sync 2024-03-16T10:45:06.034 INFO:tasks.workunit.client.1.smithi026.stdout:6/77: fdatasync d0/f1d 0 2024-03-16T10:45:06.035 INFO:tasks.workunit.client.1.smithi026.stdout:5/30: read d1/f7 [1535953,76399] 0 2024-03-16T10:45:06.035 INFO:tasks.workunit.client.1.smithi026.stdout:1/23: creat d0/d4/f7 x:0 0 0 2024-03-16T10:45:06.037 INFO:tasks.workunit.client.1.smithi026.stdout:8/16: mkdir d5 0 2024-03-16T10:45:06.038 INFO:tasks.workunit.client.0.smithi002.stdout:8/863: dwrite d7/d9/d25/d67/f81 [0,4194304] 0 2024-03-16T10:45:06.038 INFO:tasks.workunit.client.0.smithi002.stdout:1/817: mknod d2/df/d12/d84/d87/d96/da8/c107 0 2024-03-16T10:45:06.038 INFO:tasks.workunit.client.1.smithi026.stdout:6/78: write d0/d10/fd [44991,72778] 0 2024-03-16T10:45:06.040 INFO:tasks.workunit.client.0.smithi002.stdout:7/680: sync 2024-03-16T10:45:06.040 INFO:tasks.workunit.client.0.smithi002.stdout:2/709: sync 2024-03-16T10:45:06.040 INFO:tasks.workunit.client.0.smithi002.stdout:6/801: sync 2024-03-16T10:45:06.040 INFO:tasks.workunit.client.1.smithi026.stdout:3/13: mknod c3 0 2024-03-16T10:45:06.040 INFO:tasks.workunit.client.1.smithi026.stdout:3/14: dread - no filename 2024-03-16T10:45:06.041 INFO:tasks.workunit.client.0.smithi002.stdout:6/802: write d15/d2d/d2b/f63 [6307727,48089] 0 2024-03-16T10:45:06.043 INFO:tasks.workunit.client.0.smithi002.stdout:7/681: write d3/d4/d1d/d87/d5c/d63/d34/f6b [5638219,55340] 0 2024-03-16T10:45:06.044 INFO:tasks.workunit.client.0.smithi002.stdout:6/803: dread - d15/d2d/d2c/d5d/fd4 zero size 2024-03-16T10:45:06.044 INFO:tasks.workunit.client.0.smithi002.stdout:7/682: fdatasync d3/f52 0 2024-03-16T10:45:06.044 INFO:tasks.workunit.client.0.smithi002.stdout:1/818: creat d2/df/f108 x:0 0 0 2024-03-16T10:45:06.045 INFO:tasks.workunit.client.1.smithi026.stdout:7/14: symlink l3 0 2024-03-16T10:45:06.046 INFO:tasks.workunit.client.0.smithi002.stdout:7/683: rename d3/d4/d1d to d3/d4/d1d/d87/d5c/d63/d3f/daa/df6 22 2024-03-16T10:45:06.048 INFO:tasks.workunit.client.1.smithi026.stdout:9/19: unlink d0/c2 0 2024-03-16T10:45:06.049 INFO:tasks.workunit.client.1.smithi026.stdout:0/80: dwrite d1/d5/d6/f19 [0,4194304] 0 2024-03-16T10:45:06.050 INFO:tasks.workunit.client.1.smithi026.stdout:9/20: chown d0/f4 7088339 1 2024-03-16T10:45:06.051 INFO:tasks.workunit.client.1.smithi026.stdout:5/31: dwrite d1/f7 [0,4194304] 0 2024-03-16T10:45:06.051 INFO:tasks.workunit.client.1.smithi026.stdout:0/81: readlink d1/d5/d6/lc 0 2024-03-16T10:45:06.052 INFO:tasks.workunit.client.0.smithi002.stdout:8/864: dwrite d7/d9/d17/d23/d2e/d34/fc8 [0,4194304] 0 2024-03-16T10:45:06.053 INFO:tasks.workunit.client.1.smithi026.stdout:5/32: write d1/f5 [1257659,58892] 0 2024-03-16T10:45:06.053 INFO:tasks.workunit.client.1.smithi026.stdout:3/15: mkdir d4 0 2024-03-16T10:45:06.055 INFO:tasks.workunit.client.1.smithi026.stdout:5/33: write d1/f7 [5079290,8620] 0 2024-03-16T10:45:06.062 INFO:tasks.workunit.client.1.smithi026.stdout:5/34: dread d1/f5 [0,4194304] 0 2024-03-16T10:45:06.065 INFO:tasks.workunit.client.1.smithi026.stdout:4/9: creat d1/f3 x:0 0 0 2024-03-16T10:45:06.069 INFO:tasks.workunit.client.0.smithi002.stdout:7/684: mkdir d3/d4/d1d/d87/d5c/dc3/df7 0 2024-03-16T10:45:06.070 INFO:tasks.workunit.client.1.smithi026.stdout:7/15: unlink c2 0 2024-03-16T10:45:06.071 INFO:tasks.workunit.client.1.smithi026.stdout:7/16: write f1 [660478,36947] 0 2024-03-16T10:45:06.073 INFO:tasks.workunit.client.1.smithi026.stdout:8/17: creat d5/f6 x:0 0 0 2024-03-16T10:45:06.074 INFO:tasks.workunit.client.1.smithi026.stdout:8/18: fdatasync d5/f6 0 2024-03-16T10:45:06.075 INFO:tasks.workunit.client.1.smithi026.stdout:9/21: chown d0/c6 23 1 2024-03-16T10:45:06.077 INFO:tasks.workunit.client.0.smithi002.stdout:6/804: unlink d15/d2d/d8a/fae 0 2024-03-16T10:45:06.078 INFO:tasks.workunit.client.0.smithi002.stdout:6/805: truncate d15/dfd/d96/dbc/f105 777504 0 2024-03-16T10:45:06.079 INFO:tasks.workunit.client.0.smithi002.stdout:6/806: fdatasync d15/d2d/d73/d97/fbf 0 2024-03-16T10:45:06.082 INFO:tasks.workunit.client.0.smithi002.stdout:7/685: truncate d3/d61/f94 816328 0 2024-03-16T10:45:06.083 INFO:tasks.workunit.client.0.smithi002.stdout:6/807: creat d15/dfd/d96/db5/f118 x:0 0 0 2024-03-16T10:45:06.086 INFO:tasks.workunit.client.0.smithi002.stdout:8/865: dread d7/d9/d17/d20/f6b [0,4194304] 0 2024-03-16T10:45:06.089 INFO:tasks.workunit.client.1.smithi026.stdout:4/10: mknod d1/c4 0 2024-03-16T10:45:06.094 INFO:tasks.workunit.client.1.smithi026.stdout:7/17: symlink l4 0 2024-03-16T10:45:06.096 INFO:tasks.workunit.client.1.smithi026.stdout:8/19: mkdir d5/d7 0 2024-03-16T10:45:06.097 INFO:tasks.workunit.client.0.smithi002.stdout:8/866: fsync d7/f31 0 2024-03-16T10:45:06.098 INFO:tasks.workunit.client.1.smithi026.stdout:9/22: write d0/f4 [4215419,42314] 0 2024-03-16T10:45:06.101 INFO:tasks.workunit.client.0.smithi002.stdout:1/819: symlink d2/df/d12/dd7/l109 0 2024-03-16T10:45:06.101 INFO:tasks.workunit.client.0.smithi002.stdout:8/867: dread d7/d9/d17/d23/d2e/f4e [0,4194304] 0 2024-03-16T10:45:06.102 INFO:tasks.workunit.client.1.smithi026.stdout:3/16: creat d4/f5 x:0 0 0 2024-03-16T10:45:06.102 INFO:tasks.workunit.client.1.smithi026.stdout:9/23: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:06.104 INFO:tasks.workunit.client.0.smithi002.stdout:6/808: write d15/f2f [5120937,82117] 0 2024-03-16T10:45:06.105 INFO:tasks.workunit.client.0.smithi002.stdout:1/820: mkdir d2/df/d12/d84/d87/d96/db5/de2/d10a 0 2024-03-16T10:45:06.106 INFO:tasks.workunit.client.1.smithi026.stdout:9/24: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:06.107 INFO:tasks.workunit.client.1.smithi026.stdout:6/79: rename d0/d10/c15 to d0/c20 0 2024-03-16T10:45:06.108 INFO:tasks.workunit.client.1.smithi026.stdout:6/80: chown d0/f3 58 1 2024-03-16T10:45:06.108 INFO:tasks.workunit.client.0.smithi002.stdout:7/686: truncate d3/d37/f48 942878 0 2024-03-16T10:45:06.110 INFO:tasks.workunit.client.1.smithi026.stdout:9/25: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:06.111 INFO:tasks.workunit.client.0.smithi002.stdout:6/809: creat d15/d2d/d73/de0/f119 x:0 0 0 2024-03-16T10:45:06.112 INFO:tasks.workunit.client.0.smithi002.stdout:6/810: dread - d15/d2d/d2b/ff4 zero size 2024-03-16T10:45:06.112 INFO:tasks.workunit.client.1.smithi026.stdout:4/11: symlink d1/l5 0 2024-03-16T10:45:06.113 INFO:tasks.workunit.client.1.smithi026.stdout:4/12: chown d1/f2 2289 1 2024-03-16T10:45:06.113 INFO:tasks.workunit.client.0.smithi002.stdout:1/821: mknod d2/df/d12/d84/d87/d96/db5/dae/c10b 0 2024-03-16T10:45:06.113 INFO:tasks.workunit.client.1.smithi026.stdout:9/26: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:06.114 INFO:tasks.workunit.client.1.smithi026.stdout:7/18: symlink l5 0 2024-03-16T10:45:06.115 INFO:tasks.workunit.client.1.smithi026.stdout:7/19: write f1 [403524,115428] 0 2024-03-16T10:45:06.115 INFO:tasks.workunit.client.0.smithi002.stdout:7/687: chown d3/d4/d1d/d87/d5c/d63/d3f/l47 258627585 1 2024-03-16T10:45:06.116 INFO:tasks.workunit.client.1.smithi026.stdout:8/20: creat d5/f8 x:0 0 0 2024-03-16T10:45:06.118 INFO:tasks.workunit.client.0.smithi002.stdout:6/811: dwrite d15/d2d/d73/de0/f119 [0,4194304] 0 2024-03-16T10:45:06.119 INFO:tasks.workunit.client.1.smithi026.stdout:0/82: rename d1/fd to d1/f1e 0 2024-03-16T10:45:06.120 INFO:tasks.workunit.client.0.smithi002.stdout:8/868: mkdir d7/d9/d17/d23/d2e/d34/dbf/dd9/dea/d125 0 2024-03-16T10:45:06.121 INFO:tasks.workunit.client.1.smithi026.stdout:3/17: creat d4/f6 x:0 0 0 2024-03-16T10:45:06.122 INFO:tasks.workunit.client.0.smithi002.stdout:5/652: sync 2024-03-16T10:45:06.122 INFO:tasks.workunit.client.0.smithi002.stdout:4/719: sync 2024-03-16T10:45:06.122 INFO:tasks.workunit.client.0.smithi002.stdout:0/784: sync 2024-03-16T10:45:06.123 INFO:tasks.workunit.client.1.smithi026.stdout:4/13: dwrite d1/f3 [0,4194304] 0 2024-03-16T10:45:06.123 INFO:tasks.workunit.client.0.smithi002.stdout:7/688: dwrite d3/f11 [4194304,4194304] 0 2024-03-16T10:45:06.124 INFO:tasks.workunit.client.1.smithi026.stdout:3/18: dread - d4/f6 zero size 2024-03-16T10:45:06.125 INFO:tasks.workunit.client.1.smithi026.stdout:9/27: mkdir d0/d8 0 2024-03-16T10:45:06.125 INFO:tasks.workunit.client.1.smithi026.stdout:9/28: readlink - no filename 2024-03-16T10:45:06.126 INFO:tasks.workunit.client.0.smithi002.stdout:0/785: chown d3/d31/f99 1 1 2024-03-16T10:45:06.126 INFO:tasks.workunit.client.1.smithi026.stdout:4/14: dread - d1/f2 zero size 2024-03-16T10:45:06.127 INFO:tasks.workunit.client.0.smithi002.stdout:0/786: fsync d3/de/fe1 0 2024-03-16T10:45:06.127 INFO:tasks.workunit.client.1.smithi026.stdout:9/29: chown d0/d8 536 1 2024-03-16T10:45:06.128 INFO:tasks.workunit.client.0.smithi002.stdout:1/822: creat d2/df/d12/d84/d87/d96/db5/d51/f10c x:0 0 0 2024-03-16T10:45:06.130 INFO:tasks.workunit.client.1.smithi026.stdout:4/15: chown d1/f2 0 1 2024-03-16T10:45:06.130 INFO:tasks.workunit.client.1.smithi026.stdout:8/21: dwrite d5/f8 [0,4194304] 0 2024-03-16T10:45:06.130 INFO:tasks.workunit.client.1.smithi026.stdout:7/20: unlink l5 0 2024-03-16T10:45:06.130 INFO:tasks.workunit.client.0.smithi002.stdout:1/823: write d2/df/d12/f3f [1798582,31252] 0 2024-03-16T10:45:06.132 INFO:tasks.workunit.client.1.smithi026.stdout:4/16: write d1/f3 [13697,74072] 0 2024-03-16T10:45:06.135 INFO:tasks.workunit.client.0.smithi002.stdout:1/824: dread - d2/df/d12/f103 zero size 2024-03-16T10:45:06.139 INFO:tasks.workunit.client.0.smithi002.stdout:7/689: rmdir d3/d4/d1c 39 2024-03-16T10:45:06.141 INFO:tasks.workunit.client.1.smithi026.stdout:0/83: mknod d1/d5/d11/c1f 0 2024-03-16T10:45:06.142 INFO:tasks.workunit.client.1.smithi026.stdout:0/84: truncate d1/f1e 46281 0 2024-03-16T10:45:06.143 INFO:tasks.workunit.client.1.smithi026.stdout:0/85: write d1/d5/d11/f16 [2699652,120597] 0 2024-03-16T10:45:06.144 INFO:tasks.workunit.client.0.smithi002.stdout:5/653: unlink d0/d16/d4a/d7d/f90 0 2024-03-16T10:45:06.144 INFO:tasks.workunit.client.1.smithi026.stdout:0/86: truncate d1/f15 4915593 0 2024-03-16T10:45:06.145 INFO:tasks.workunit.client.1.smithi026.stdout:6/81: mknod d0/d10/c21 0 2024-03-16T10:45:06.146 INFO:tasks.workunit.client.0.smithi002.stdout:8/869: mknod d7/d9/d17/d23/d39/dd4/c126 0 2024-03-16T10:45:06.147 INFO:tasks.workunit.client.1.smithi026.stdout:5/35: getdents d1 0 2024-03-16T10:45:06.148 INFO:tasks.workunit.client.1.smithi026.stdout:5/36: fdatasync d1/f7 0 2024-03-16T10:45:06.152 INFO:tasks.workunit.client.1.smithi026.stdout:3/19: symlink d4/l7 0 2024-03-16T10:45:06.153 INFO:tasks.workunit.client.0.smithi002.stdout:3/789: sync 2024-03-16T10:45:06.153 INFO:tasks.workunit.client.0.smithi002.stdout:9/832: sync 2024-03-16T10:45:06.153 INFO:tasks.workunit.client.0.smithi002.stdout:2/710: sync 2024-03-16T10:45:06.154 INFO:tasks.workunit.client.1.smithi026.stdout:9/30: fdatasync d0/f4 0 2024-03-16T10:45:06.155 INFO:tasks.workunit.client.0.smithi002.stdout:4/720: symlink d2/d33/d1c/d45/ld8 0 2024-03-16T10:45:06.157 INFO:tasks.workunit.client.1.smithi026.stdout:7/21: mkdir d6 0 2024-03-16T10:45:06.157 INFO:tasks.workunit.client.0.smithi002.stdout:4/721: truncate d2/d33/d1c/d45/faa 5131423 0 2024-03-16T10:45:06.158 INFO:tasks.workunit.client.0.smithi002.stdout:6/812: rmdir d15/d2d/d110 0 2024-03-16T10:45:06.160 INFO:tasks.workunit.client.0.smithi002.stdout:2/711: dread d7/d2e/d28/d44/d6d/f9f [0,4194304] 0 2024-03-16T10:45:06.160 INFO:tasks.workunit.client.0.smithi002.stdout:1/825: creat d2/df/d12/d84/d87/d96/da8/f10d x:0 0 0 2024-03-16T10:45:06.161 INFO:tasks.workunit.client.1.smithi026.stdout:4/17: creat d1/f6 x:0 0 0 2024-03-16T10:45:06.161 INFO:tasks.workunit.client.1.smithi026.stdout:4/18: stat d1/l5 0 2024-03-16T10:45:06.162 INFO:tasks.workunit.client.1.smithi026.stdout:4/19: write d1/f3 [1334006,63350] 0 2024-03-16T10:45:06.163 INFO:tasks.workunit.client.1.smithi026.stdout:4/20: truncate d1/f6 349651 0 2024-03-16T10:45:06.165 INFO:tasks.workunit.client.1.smithi026.stdout:0/87: mknod d1/d5/c20 0 2024-03-16T10:45:06.166 INFO:tasks.workunit.client.0.smithi002.stdout:5/654: fdatasync d0/f86 0 2024-03-16T10:45:06.166 INFO:tasks.workunit.client.1.smithi026.stdout:0/88: write d1/d5/d11/f16 [3359005,71079] 0 2024-03-16T10:45:06.169 INFO:tasks.workunit.client.0.smithi002.stdout:5/655: dread d0/d16/dab/f85 [0,4194304] 0 2024-03-16T10:45:06.169 INFO:tasks.workunit.client.1.smithi026.stdout:0/89: write d1/d5/d11/f16 [4270143,110712] 0 2024-03-16T10:45:06.170 INFO:tasks.workunit.client.1.smithi026.stdout:5/37: creat d1/fa x:0 0 0 2024-03-16T10:45:06.173 INFO:tasks.workunit.client.0.smithi002.stdout:3/790: mkdir d1/db6/d15/d87/d8c/d116 0 2024-03-16T10:45:06.174 INFO:tasks.workunit.client.1.smithi026.stdout:0/90: dwrite d1/f15 [0,4194304] 0 2024-03-16T10:45:06.180 INFO:tasks.workunit.client.0.smithi002.stdout:6/813: mknod d15/d2d/d2b/d3f/d77/d9f/c11a 0 2024-03-16T10:45:06.181 INFO:tasks.workunit.client.0.smithi002.stdout:6/814: fdatasync d15/d2d/d2c/f50 0 2024-03-16T10:45:06.181 INFO:tasks.workunit.client.0.smithi002.stdout:6/815: readlink d15/d2d/d2c/d5d/l6a 0 2024-03-16T10:45:06.182 INFO:tasks.workunit.client.0.smithi002.stdout:2/712: creat d7/d2e/d1d/d38/d49/fe0 x:0 0 0 2024-03-16T10:45:06.182 INFO:tasks.workunit.client.0.smithi002.stdout:6/816: readlink l0 0 2024-03-16T10:45:06.183 INFO:tasks.workunit.client.0.smithi002.stdout:6/817: fdatasync d15/f9a 0 2024-03-16T10:45:06.183 INFO:tasks.workunit.client.0.smithi002.stdout:1/826: mknod d2/d58/dbb/c10e 0 2024-03-16T10:45:06.184 INFO:tasks.workunit.client.1.smithi026.stdout:5/38: mkdir d1/db 0 2024-03-16T10:45:06.184 INFO:tasks.workunit.client.0.smithi002.stdout:1/827: chown d2/df/d12/d84/d87/d96/db5/lb 22 1 2024-03-16T10:45:06.187 INFO:tasks.workunit.client.0.smithi002.stdout:5/656: mknod d0/d16/dab/daa/ccd 0 2024-03-16T10:45:06.189 INFO:tasks.workunit.client.1.smithi026.stdout:4/21: link d1/f2 d1/f7 0 2024-03-16T10:45:06.190 INFO:tasks.workunit.client.0.smithi002.stdout:3/791: creat d1/dd/d28/d7b/d2c/d63/f117 x:0 0 0 2024-03-16T10:45:06.191 INFO:tasks.workunit.client.1.smithi026.stdout:5/39: fdatasync d1/f5 0 2024-03-16T10:45:06.191 INFO:tasks.workunit.client.1.smithi026.stdout:2/19: sync 2024-03-16T10:45:06.192 INFO:tasks.workunit.client.0.smithi002.stdout:9/833: creat d5/d9/dc/d1e/d94/df9/f108 x:0 0 0 2024-03-16T10:45:06.193 INFO:tasks.workunit.client.1.smithi026.stdout:9/31: rmdir d0/d8 0 2024-03-16T10:45:06.198 INFO:tasks.workunit.client.1.smithi026.stdout:4/22: creat d1/f8 x:0 0 0 2024-03-16T10:45:06.199 INFO:tasks.workunit.client.1.smithi026.stdout:9/32: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:06.199 INFO:tasks.workunit.client.1.smithi026.stdout:9/33: chown d0 7164346 1 2024-03-16T10:45:06.204 INFO:tasks.workunit.client.0.smithi002.stdout:6/818: mknod d15/d2d/d8a/c11b 0 2024-03-16T10:45:06.204 INFO:tasks.workunit.client.1.smithi026.stdout:9/34: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:06.204 INFO:tasks.workunit.client.1.smithi026.stdout:9/35: chown d0 71 1 2024-03-16T10:45:06.205 INFO:tasks.workunit.client.1.smithi026.stdout:2/20: mknod c1 0 2024-03-16T10:45:06.205 INFO:tasks.workunit.client.1.smithi026.stdout:2/21: dread - no filename 2024-03-16T10:45:06.207 INFO:tasks.workunit.client.1.smithi026.stdout:4/23: mknod d1/c9 0 2024-03-16T10:45:06.209 INFO:tasks.workunit.client.1.smithi026.stdout:7/22: truncate f1 757663 0 2024-03-16T10:45:06.210 INFO:tasks.workunit.client.1.smithi026.stdout:7/23: chown l3 46435916 1 2024-03-16T10:45:06.211 INFO:tasks.workunit.client.0.smithi002.stdout:8/870: link d7/d9/d17/d23/d2e/d90/l93 d7/d13/db8/de0/l127 0 2024-03-16T10:45:06.213 INFO:tasks.workunit.client.1.smithi026.stdout:5/40: unlink d1/l8 0 2024-03-16T10:45:06.213 INFO:tasks.workunit.client.1.smithi026.stdout:4/24: dwrite d1/f8 [0,4194304] 0 2024-03-16T10:45:06.214 INFO:tasks.workunit.client.0.smithi002.stdout:0/787: rmdir d3/da0/dd6 0 2024-03-16T10:45:06.216 INFO:tasks.workunit.client.1.smithi026.stdout:9/36: creat d0/f9 x:0 0 0 2024-03-16T10:45:06.217 INFO:tasks.workunit.client.1.smithi026.stdout:9/37: dread - d0/f9 zero size 2024-03-16T10:45:06.217 INFO:tasks.workunit.client.0.smithi002.stdout:5/657: mknod d0/d16/d4a/d39/cce 0 2024-03-16T10:45:06.218 INFO:tasks.workunit.client.0.smithi002.stdout:5/658: dread - d0/d16/dab/d33/d8c/fa5 zero size 2024-03-16T10:45:06.218 INFO:tasks.workunit.client.1.smithi026.stdout:2/22: mkdir d2 0 2024-03-16T10:45:06.218 INFO:tasks.workunit.client.1.smithi026.stdout:2/23: dwrite - no filename 2024-03-16T10:45:06.219 INFO:tasks.workunit.client.1.smithi026.stdout:2/24: dwrite - no filename 2024-03-16T10:45:06.219 INFO:tasks.workunit.client.1.smithi026.stdout:2/25: write - no filename 2024-03-16T10:45:06.219 INFO:tasks.workunit.client.1.smithi026.stdout:4/25: dread d1/f3 [0,4194304] 0 2024-03-16T10:45:06.219 INFO:tasks.workunit.client.1.smithi026.stdout:4/26: write d1/f6 [906265,28703] 0 2024-03-16T10:45:06.220 INFO:tasks.workunit.client.1.smithi026.stdout:4/27: write d1/f3 [4629804,38876] 0 2024-03-16T10:45:06.221 INFO:tasks.workunit.client.0.smithi002.stdout:3/792: unlink d1/dd/d28/f55 0 2024-03-16T10:45:06.221 INFO:tasks.workunit.client.1.smithi026.stdout:0/91: truncate d1/d5/d6/f19 3031094 0 2024-03-16T10:45:06.223 INFO:tasks.workunit.client.1.smithi026.stdout:0/92: read d1/f1e [29698,115496] 0 2024-03-16T10:45:06.223 INFO:tasks.workunit.client.1.smithi026.stdout:0/93: readlink d1/d5/d6/lc 0 2024-03-16T10:45:06.225 INFO:tasks.workunit.client.0.smithi002.stdout:9/834: unlink d5/f7 0 2024-03-16T10:45:06.229 INFO:tasks.workunit.client.0.smithi002.stdout:2/713: mkdir d7/d27/d3b/dd4/de1 0 2024-03-16T10:45:06.231 INFO:tasks.workunit.client.1.smithi026.stdout:0/94: dwrite d1/f15 [0,4194304] 0 2024-03-16T10:45:06.232 INFO:tasks.workunit.client.0.smithi002.stdout:7/690: rename d3/d61/f7d to d3/ff8 0 2024-03-16T10:45:06.236 INFO:tasks.workunit.client.0.smithi002.stdout:6/819: mkdir d15/dab/d11c 0 2024-03-16T10:45:06.239 INFO:tasks.workunit.client.0.smithi002.stdout:1/828: dwrite d2/df/d12/d84/d87/d96/db5/d1c/f7f [0,4194304] 0 2024-03-16T10:45:06.243 INFO:tasks.workunit.client.1.smithi026.stdout:9/38: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:06.245 INFO:tasks.workunit.client.0.smithi002.stdout:0/788: mkdir d3/d31/daa/dff 0 2024-03-16T10:45:06.245 INFO:tasks.workunit.client.0.smithi002.stdout:0/789: chown d3/de/f51 5982251 1 2024-03-16T10:45:06.255 INFO:tasks.workunit.client.1.smithi026.stdout:5/41: creat d1/db/fc x:0 0 0 2024-03-16T10:45:06.256 INFO:tasks.workunit.client.0.smithi002.stdout:9/835: symlink d5/d9/dc/d1e/d91/ddc/l109 0 2024-03-16T10:45:06.258 INFO:tasks.workunit.client.0.smithi002.stdout:4/722: creat d2/d33/da/fd9 x:0 0 0 2024-03-16T10:45:06.258 INFO:tasks.workunit.client.1.smithi026.stdout:2/26: creat d2/f3 x:0 0 0 2024-03-16T10:45:06.259 INFO:tasks.workunit.client.1.smithi026.stdout:4/28: write d1/f7 [133325,129509] 0 2024-03-16T10:45:06.260 INFO:tasks.workunit.client.0.smithi002.stdout:2/714: mkdir d7/d27/d3b/dd4/de2 0 2024-03-16T10:45:06.262 INFO:tasks.workunit.client.1.smithi026.stdout:9/39: mkdir d0/da 0 2024-03-16T10:45:06.266 INFO:tasks.workunit.client.0.smithi002.stdout:6/820: creat d15/d2d/d73/de0/f11d x:0 0 0 2024-03-16T10:45:06.266 INFO:tasks.workunit.client.1.smithi026.stdout:2/27: dwrite d2/f3 [0,4194304] 0 2024-03-16T10:45:06.268 INFO:tasks.workunit.client.1.smithi026.stdout:5/42: rmdir d1/db 39 2024-03-16T10:45:06.269 INFO:tasks.workunit.client.1.smithi026.stdout:5/43: chown d1/l6 15155 1 2024-03-16T10:45:06.271 INFO:tasks.workunit.client.1.smithi026.stdout:4/29: creat d1/fa x:0 0 0 2024-03-16T10:45:06.272 INFO:tasks.workunit.client.0.smithi002.stdout:8/871: creat d7/d13/db8/de0/f128 x:0 0 0 2024-03-16T10:45:06.273 INFO:tasks.workunit.client.1.smithi026.stdout:7/24: dwrite f1 [0,4194304] 0 2024-03-16T10:45:06.273 INFO:tasks.workunit.client.0.smithi002.stdout:9/836: dread d5/d9/dc/d1e/d94/fad [0,4194304] 0 2024-03-16T10:45:06.281 INFO:tasks.workunit.client.1.smithi026.stdout:2/28: mknod d2/c4 0 2024-03-16T10:45:06.282 INFO:tasks.workunit.client.1.smithi026.stdout:2/29: chown d2 51 1 2024-03-16T10:45:06.283 INFO:tasks.workunit.client.0.smithi002.stdout:3/793: creat d1/dd/d74/dd8/d110/f118 x:0 0 0 2024-03-16T10:45:06.285 INFO:tasks.workunit.client.1.smithi026.stdout:2/30: dread d2/f3 [0,4194304] 0 2024-03-16T10:45:06.288 INFO:tasks.workunit.client.0.smithi002.stdout:4/723: mknod d2/d33/da/d88/db2/cda 0 2024-03-16T10:45:06.288 INFO:tasks.workunit.client.0.smithi002.stdout:4/724: dread - d2/d6/fcc zero size 2024-03-16T10:45:06.289 INFO:tasks.workunit.client.1.smithi026.stdout:4/30: write d1/f3 [2474640,114629] 0 2024-03-16T10:45:06.289 INFO:tasks.workunit.client.1.smithi026.stdout:2/31: dread d2/f3 [0,4194304] 0 2024-03-16T10:45:06.289 INFO:tasks.workunit.client.0.smithi002.stdout:2/715: write d7/d2e/d1d/d2f/d4b/fc8 [2359630,94289] 0 2024-03-16T10:45:06.293 INFO:tasks.workunit.client.0.smithi002.stdout:6/821: creat d15/dfd/d96/dbb/f11e x:0 0 0 2024-03-16T10:45:06.296 INFO:tasks.workunit.client.1.smithi026.stdout:4/31: dwrite d1/f2 [0,4194304] 0 2024-03-16T10:45:06.297 INFO:tasks.workunit.client.0.smithi002.stdout:9/837: creat d5/d9/dc/d1e/def/f10a x:0 0 0 2024-03-16T10:45:06.298 INFO:tasks.workunit.client.1.smithi026.stdout:7/25: mknod d6/c7 0 2024-03-16T10:45:06.300 INFO:tasks.workunit.client.0.smithi002.stdout:8/872: write d7/d9/d17/d23/d2e/f8b [1020756,2388] 0 2024-03-16T10:45:06.302 INFO:tasks.workunit.client.1.smithi026.stdout:7/26: dread f1 [0,4194304] 0 2024-03-16T10:45:06.305 INFO:tasks.workunit.client.1.smithi026.stdout:5/44: creat d1/fd x:0 0 0 2024-03-16T10:45:06.307 INFO:tasks.workunit.client.1.smithi026.stdout:4/32: mkdir d1/db 0 2024-03-16T10:45:06.308 INFO:tasks.workunit.client.1.smithi026.stdout:4/33: readlink d1/l5 0 2024-03-16T10:45:06.309 INFO:tasks.workunit.client.0.smithi002.stdout:4/725: mkdir d2/d6/ddb 0 2024-03-16T10:45:06.309 INFO:tasks.workunit.client.0.smithi002.stdout:7/691: creat d3/ff9 x:0 0 0 2024-03-16T10:45:06.310 INFO:tasks.workunit.client.1.smithi026.stdout:7/27: dwrite f1 [0,4194304] 0 2024-03-16T10:45:06.310 INFO:tasks.workunit.client.1.smithi026.stdout:2/32: link d2/f3 d2/f5 0 2024-03-16T10:45:06.312 INFO:tasks.workunit.client.1.smithi026.stdout:9/40: getdents d0 0 2024-03-16T10:45:06.315 INFO:tasks.workunit.client.0.smithi002.stdout:1/829: creat d2/df/d12/d84/d87/d96/db5/d1c/da0/f10f x:0 0 0 2024-03-16T10:45:06.315 INFO:tasks.workunit.client.0.smithi002.stdout:2/716: write d7/d2e/f20 [1863428,41100] 0 2024-03-16T10:45:06.318 INFO:tasks.workunit.client.1.smithi026.stdout:4/34: rename d1/f2 to d1/db/fc 0 2024-03-16T10:45:06.320 INFO:tasks.workunit.client.1.smithi026.stdout:2/33: creat d2/f6 x:0 0 0 2024-03-16T10:45:06.321 INFO:tasks.workunit.client.1.smithi026.stdout:2/34: write d2/f6 [622358,599] 0 2024-03-16T10:45:06.321 INFO:tasks.workunit.client.1.smithi026.stdout:9/41: mkdir d0/db 0 2024-03-16T10:45:06.322 INFO:tasks.workunit.client.1.smithi026.stdout:9/42: chown d0/c6 8 1 2024-03-16T10:45:06.323 INFO:tasks.workunit.client.0.smithi002.stdout:7/692: dread d3/d4/d1d/d87/d5c/d63/d34/f6b [0,4194304] 0 2024-03-16T10:45:06.324 INFO:tasks.workunit.client.1.smithi026.stdout:5/45: mkdir d1/de 0 2024-03-16T10:45:06.329 INFO:tasks.workunit.client.0.smithi002.stdout:0/790: truncate d3/f49 382922 0 2024-03-16T10:45:06.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/28: rename l4 to d6/l8 0 2024-03-16T10:45:06.334 INFO:tasks.workunit.client.0.smithi002.stdout:0/791: dwrite d3/d31/daa/fad [0,4194304] 0 2024-03-16T10:45:06.335 INFO:tasks.workunit.client.0.smithi002.stdout:0/792: dread - d3/d59/fc4 zero size 2024-03-16T10:45:06.336 INFO:tasks.workunit.client.1.smithi026.stdout:2/35: symlink d2/l7 0 2024-03-16T10:45:06.337 INFO:tasks.workunit.client.1.smithi026.stdout:5/46: rename d1 to d1/df 22 2024-03-16T10:45:06.340 INFO:tasks.workunit.client.1.smithi026.stdout:7/29: mknod d6/c9 0 2024-03-16T10:45:06.340 INFO:tasks.workunit.client.1.smithi026.stdout:4/35: dwrite d1/db/fc [0,4194304] 0 2024-03-16T10:45:06.344 INFO:tasks.workunit.client.1.smithi026.stdout:2/36: dread d2/f6 [0,4194304] 0 2024-03-16T10:45:06.346 INFO:tasks.workunit.client.1.smithi026.stdout:1/24: sync 2024-03-16T10:45:06.347 INFO:tasks.workunit.client.1.smithi026.stdout:4/36: creat d1/db/fd x:0 0 0 2024-03-16T10:45:06.351 INFO:tasks.workunit.client.1.smithi026.stdout:5/47: symlink d1/db/l10 0 2024-03-16T10:45:06.353 INFO:tasks.workunit.client.1.smithi026.stdout:7/30: readlink d6/l8 0 2024-03-16T10:45:06.354 INFO:tasks.workunit.client.1.smithi026.stdout:1/25: dwrite d0/f5 [0,4194304] 0 2024-03-16T10:45:06.355 INFO:tasks.workunit.client.1.smithi026.stdout:1/26: dread - d0/d4/f7 zero size 2024-03-16T10:45:06.355 INFO:tasks.workunit.client.1.smithi026.stdout:2/37: mknod d2/c8 0 2024-03-16T10:45:06.355 INFO:tasks.workunit.client.1.smithi026.stdout:1/27: chown d0/d4 0 1 2024-03-16T10:45:06.358 INFO:tasks.workunit.client.1.smithi026.stdout:4/37: mkdir d1/db/de 0 2024-03-16T10:45:06.358 INFO:tasks.workunit.client.1.smithi026.stdout:4/38: dread - d1/fa zero size 2024-03-16T10:45:06.359 INFO:tasks.workunit.client.1.smithi026.stdout:4/39: dread - d1/db/fd zero size 2024-03-16T10:45:06.360 INFO:tasks.workunit.client.1.smithi026.stdout:5/48: rmdir d1/db 39 2024-03-16T10:45:06.360 INFO:tasks.workunit.client.1.smithi026.stdout:5/49: dread - d1/fa zero size 2024-03-16T10:45:06.362 INFO:tasks.workunit.client.1.smithi026.stdout:5/50: chown d1/f7 8 1 2024-03-16T10:45:06.376 INFO:tasks.workunit.client.0.smithi002.stdout:2/717: mknod d7/d2e/d1d/d38/d49/ce3 0 2024-03-16T10:45:06.377 INFO:tasks.workunit.client.0.smithi002.stdout:5/659: link d0/d16/dab/d2b/c75 d0/d16/ccf 0 2024-03-16T10:45:06.380 INFO:tasks.workunit.client.0.smithi002.stdout:6/822: creat d15/d2d/d8a/d7c/f11f x:0 0 0 2024-03-16T10:45:06.382 INFO:tasks.workunit.client.0.smithi002.stdout:1/830: link d2/df/d12/d84/d87/d96/db5/fab d2/df/d12/d84/d87/d96/db5/d1c/d5d/f110 0 2024-03-16T10:45:06.382 INFO:tasks.workunit.client.0.smithi002.stdout:6/823: write d15/d2d/d2c/f54 [1915620,86639] 0 2024-03-16T10:45:06.382 INFO:tasks.workunit.client.0.smithi002.stdout:8/873: creat d7/d9/d17/d23/d2e/d34/dbf/f129 x:0 0 0 2024-03-16T10:45:06.383 INFO:tasks.workunit.client.0.smithi002.stdout:5/660: creat d0/d16/d4a/d55/fd0 x:0 0 0 2024-03-16T10:45:06.384 INFO:tasks.workunit.client.0.smithi002.stdout:0/793: link d3/da0/fb1 d3/df8/f100 0 2024-03-16T10:45:06.386 INFO:tasks.workunit.client.0.smithi002.stdout:4/726: rename d2/d33/da/d2e/f47 to d2/d33/da/d2e/fdc 0 2024-03-16T10:45:06.388 INFO:tasks.workunit.client.0.smithi002.stdout:9/838: getdents d5/d9/d22/d3c 0 2024-03-16T10:45:06.388 INFO:tasks.workunit.client.0.smithi002.stdout:3/794: sync 2024-03-16T10:45:06.390 INFO:tasks.workunit.client.0.smithi002.stdout:1/831: write d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/faa [2596183,75542] 0 2024-03-16T10:45:06.392 INFO:tasks.workunit.client.0.smithi002.stdout:4/727: write d2/d33/da/d5e/d93/fac [858592,39016] 0 2024-03-16T10:45:06.393 INFO:tasks.workunit.client.0.smithi002.stdout:6/824: mknod d15/d2d/d73/de0/c120 0 2024-03-16T10:45:06.394 INFO:tasks.workunit.client.0.smithi002.stdout:9/839: read - d5/d9/d25/fbb zero size 2024-03-16T10:45:06.395 INFO:tasks.workunit.client.0.smithi002.stdout:1/832: fdatasync d2/df/d12/d84/d87/d96/db5/fab 0 2024-03-16T10:45:06.397 INFO:tasks.workunit.client.1.smithi026.stdout:2/38: rename d2/f3 to d2/f9 0 2024-03-16T10:45:06.398 INFO:tasks.workunit.client.0.smithi002.stdout:4/728: mknod d2/d33/da/d5e/d93/cdd 0 2024-03-16T10:45:06.398 INFO:tasks.workunit.client.0.smithi002.stdout:2/718: dread d7/d2e/f7e [0,4194304] 0 2024-03-16T10:45:06.399 INFO:tasks.workunit.client.1.smithi026.stdout:1/28: creat d0/f8 x:0 0 0 2024-03-16T10:45:06.401 INFO:tasks.workunit.client.1.smithi026.stdout:1/29: dread - d0/f8 zero size 2024-03-16T10:45:06.401 INFO:tasks.workunit.client.0.smithi002.stdout:5/661: getdents d0/d24/dad 0 2024-03-16T10:45:06.401 INFO:tasks.workunit.client.1.smithi026.stdout:4/40: dwrite d1/db/fc [4194304,4194304] 0 2024-03-16T10:45:06.404 INFO:tasks.workunit.client.0.smithi002.stdout:6/825: unlink d15/d2d/d73/fa6 0 2024-03-16T10:45:06.405 INFO:tasks.workunit.client.0.smithi002.stdout:6/826: fdatasync d15/d80/fd5 0 2024-03-16T10:45:06.405 INFO:tasks.workunit.client.1.smithi026.stdout:1/30: dread d0/f5 [0,4194304] 0 2024-03-16T10:45:06.407 INFO:tasks.workunit.client.1.smithi026.stdout:2/39: symlink d2/la 0 2024-03-16T10:45:06.407 INFO:tasks.workunit.client.1.smithi026.stdout:2/40: stat c1 0 2024-03-16T10:45:06.412 INFO:tasks.workunit.client.0.smithi002.stdout:5/662: dread d0/d16/dab/d2b/d5d/f92 [0,4194304] 0 2024-03-16T10:45:06.413 INFO:tasks.workunit.client.1.smithi026.stdout:7/31: getdents d6 0 2024-03-16T10:45:06.414 INFO:tasks.workunit.client.0.smithi002.stdout:5/663: chown d0/d16/dab/d2b/d5d 308 1 2024-03-16T10:45:06.414 INFO:tasks.workunit.client.0.smithi002.stdout:6/827: readlink d15/d2d/d2b/l3a 0 2024-03-16T10:45:06.415 INFO:tasks.workunit.client.0.smithi002.stdout:5/664: write d0/d16/f5f [6433218,36472] 0 2024-03-16T10:45:06.416 INFO:tasks.workunit.client.1.smithi026.stdout:1/31: unlink d0/d4/f7 0 2024-03-16T10:45:06.419 INFO:tasks.workunit.client.0.smithi002.stdout:0/794: dread d3/d28/f34 [0,4194304] 0 2024-03-16T10:45:06.420 INFO:tasks.workunit.client.0.smithi002.stdout:1/833: creat d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/dee/f111 x:0 0 0 2024-03-16T10:45:06.420 INFO:tasks.workunit.client.1.smithi026.stdout:5/51: rmdir d1/de 0 2024-03-16T10:45:06.421 INFO:tasks.workunit.client.0.smithi002.stdout:1/834: dread - d2/df/f108 zero size 2024-03-16T10:45:06.423 INFO:tasks.workunit.client.1.smithi026.stdout:4/41: symlink d1/db/de/lf 0 2024-03-16T10:45:06.424 INFO:tasks.workunit.client.0.smithi002.stdout:2/719: dread d7/d2e/d1d/d2f/d4b/fa0 [0,4194304] 0 2024-03-16T10:45:06.425 INFO:tasks.workunit.client.1.smithi026.stdout:7/32: mkdir d6/da 0 2024-03-16T10:45:06.426 INFO:tasks.workunit.client.1.smithi026.stdout:7/33: fsync f1 0 2024-03-16T10:45:06.426 INFO:tasks.workunit.client.1.smithi026.stdout:7/34: write f1 [2434803,17109] 0 2024-03-16T10:45:06.427 INFO:tasks.workunit.client.0.smithi002.stdout:6/828: unlink d15/dfd/d96/c100 0 2024-03-16T10:45:06.430 INFO:tasks.workunit.client.1.smithi026.stdout:7/35: dread f1 [0,4194304] 0 2024-03-16T10:45:06.430 INFO:tasks.workunit.client.0.smithi002.stdout:4/729: creat d2/fde x:0 0 0 2024-03-16T10:45:06.431 INFO:tasks.workunit.client.1.smithi026.stdout:1/32: symlink d0/d4/l9 0 2024-03-16T10:45:06.435 INFO:tasks.workunit.client.0.smithi002.stdout:9/840: rename d5/d9/dc/d1e/d91/ddc to d5/d9/dc/d1e/d91/d82/d10b 0 2024-03-16T10:45:06.437 INFO:tasks.workunit.client.1.smithi026.stdout:5/52: chown d1/db 3653996 1 2024-03-16T10:45:06.438 INFO:tasks.workunit.client.1.smithi026.stdout:5/53: fdatasync d1/f7 0 2024-03-16T10:45:06.438 INFO:tasks.workunit.client.0.smithi002.stdout:1/835: creat d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/d102/f112 x:0 0 0 2024-03-16T10:45:06.439 INFO:tasks.workunit.client.1.smithi026.stdout:1/33: dwrite d0/f8 [0,4194304] 0 2024-03-16T10:45:06.439 INFO:tasks.workunit.client.0.smithi002.stdout:2/720: dread d7/d2e/f25 [4194304,4194304] 0 2024-03-16T10:45:06.440 INFO:tasks.workunit.client.0.smithi002.stdout:2/721: dread - d7/d2e/d28/d40/fc4 zero size 2024-03-16T10:45:06.441 INFO:tasks.workunit.client.1.smithi026.stdout:4/42: rename d1/db/de/lf to d1/db/de/l10 0 2024-03-16T10:45:06.451 INFO:tasks.workunit.client.0.smithi002.stdout:8/874: write d7/f108 [341494,109520] 0 2024-03-16T10:45:06.451 INFO:tasks.workunit.client.0.smithi002.stdout:7/693: dwrite d3/d4/d1c/f90 [0,4194304] 0 2024-03-16T10:45:06.452 INFO:tasks.workunit.client.0.smithi002.stdout:0/795: creat d3/d31/daa/dff/f101 x:0 0 0 2024-03-16T10:45:06.455 INFO:tasks.workunit.client.0.smithi002.stdout:9/841: symlink d5/db1/l10c 0 2024-03-16T10:45:06.455 INFO:tasks.workunit.client.1.smithi026.stdout:4/43: creat d1/db/f11 x:0 0 0 2024-03-16T10:45:06.456 INFO:tasks.workunit.client.0.smithi002.stdout:9/842: fdatasync d5/d9/d25/d23/d81/fa1 0 2024-03-16T10:45:06.459 INFO:tasks.workunit.client.1.smithi026.stdout:1/34: dwrite d0/f5 [0,4194304] 0 2024-03-16T10:45:06.462 INFO:tasks.workunit.client.1.smithi026.stdout:2/41: write d2/f5 [1866976,8027] 0 2024-03-16T10:45:06.463 INFO:tasks.workunit.client.0.smithi002.stdout:8/875: dwrite d7/d9/d17/d23/d2e/f8b [0,4194304] 0 2024-03-16T10:45:06.464 INFO:tasks.workunit.client.1.smithi026.stdout:5/54: symlink d1/l11 0 2024-03-16T10:45:06.465 INFO:tasks.workunit.client.1.smithi026.stdout:4/44: dwrite d1/f7 [8388608,4194304] 0 2024-03-16T10:45:06.470 INFO:tasks.workunit.client.1.smithi026.stdout:1/35: symlink d0/la 0 2024-03-16T10:45:06.471 INFO:tasks.workunit.client.1.smithi026.stdout:4/45: dread d1/f3 [0,4194304] 0 2024-03-16T10:45:06.472 INFO:tasks.workunit.client.1.smithi026.stdout:4/46: write d1/f6 [313890,35871] 0 2024-03-16T10:45:06.473 INFO:tasks.workunit.client.0.smithi002.stdout:0/796: write d3/d31/d75/dde/fe0 [1136811,113415] 0 2024-03-16T10:45:06.473 INFO:tasks.workunit.client.1.smithi026.stdout:2/42: mkdir d2/db 0 2024-03-16T10:45:06.474 INFO:tasks.workunit.client.1.smithi026.stdout:4/47: fdatasync d1/db/fd 0 2024-03-16T10:45:06.474 INFO:tasks.workunit.client.1.smithi026.stdout:2/43: readlink d2/l7 0 2024-03-16T10:45:06.475 INFO:tasks.workunit.client.1.smithi026.stdout:4/48: read d1/f3 [1297389,56186] 0 2024-03-16T10:45:06.476 INFO:tasks.workunit.client.1.smithi026.stdout:4/49: read d1/f7 [3593721,100182] 0 2024-03-16T10:45:06.477 INFO:tasks.workunit.client.1.smithi026.stdout:5/55: fdatasync d1/f5 0 2024-03-16T10:45:06.481 INFO:tasks.workunit.client.1.smithi026.stdout:1/36: mknod d0/d4/cb 0 2024-03-16T10:45:06.483 INFO:tasks.workunit.client.0.smithi002.stdout:9/843: creat d5/d9/d25/dac/f10d x:0 0 0 2024-03-16T10:45:06.485 INFO:tasks.workunit.client.0.smithi002.stdout:6/829: creat d15/dfd/f121 x:0 0 0 2024-03-16T10:45:06.486 INFO:tasks.workunit.client.1.smithi026.stdout:2/44: creat d2/fc x:0 0 0 2024-03-16T10:45:06.489 INFO:tasks.workunit.client.1.smithi026.stdout:4/50: mknod d1/db/c12 0 2024-03-16T10:45:06.491 INFO:tasks.workunit.client.0.smithi002.stdout:9/844: dwrite d5/d9/dc/d28/fe6 [0,4194304] 0 2024-03-16T10:45:06.492 INFO:tasks.workunit.client.1.smithi026.stdout:7/36: truncate f1 513992 0 2024-03-16T10:45:06.493 INFO:tasks.workunit.client.0.smithi002.stdout:9/845: write d5/d9/d34/ff7 [312564,15308] 0 2024-03-16T10:45:06.493 INFO:tasks.workunit.client.0.smithi002.stdout:4/730: write d2/d33/da/d5e/d93/fa7 [4170907,126202] 0 2024-03-16T10:45:06.494 INFO:tasks.workunit.client.0.smithi002.stdout:2/722: link d7/d27/d3b/c9b d7/d2e/d28/d44/ce4 0 2024-03-16T10:45:06.496 INFO:tasks.workunit.client.0.smithi002.stdout:4/731: chown d2/d33/d35/f6f 240 1 2024-03-16T10:45:06.497 INFO:tasks.workunit.client.1.smithi026.stdout:2/45: mkdir d2/dd 0 2024-03-16T10:45:06.502 INFO:tasks.workunit.client.1.smithi026.stdout:4/51: readlink d1/db/de/l10 0 2024-03-16T10:45:06.503 INFO:tasks.workunit.client.1.smithi026.stdout:4/52: dread - d1/db/fd zero size 2024-03-16T10:45:06.504 INFO:tasks.workunit.client.1.smithi026.stdout:4/53: read d1/f8 [2016473,118894] 0 2024-03-16T10:45:06.504 INFO:tasks.workunit.client.0.smithi002.stdout:3/795: rename d1/dd/d28/d37/ce0 to d1/dd/d28/d7b/d2c/c119 0 2024-03-16T10:45:06.505 INFO:tasks.workunit.client.1.smithi026.stdout:4/54: fsync d1/f8 0 2024-03-16T10:45:06.505 INFO:tasks.workunit.client.0.smithi002.stdout:0/797: dread d3/d31/f4b [0,4194304] 0 2024-03-16T10:45:06.510 INFO:tasks.workunit.client.1.smithi026.stdout:2/46: unlink d2/f6 0 2024-03-16T10:45:06.511 INFO:tasks.workunit.client.0.smithi002.stdout:3/796: dread d1/dd/d28/d7b/d2c/d97/fa4 [0,4194304] 0 2024-03-16T10:45:06.511 INFO:tasks.workunit.client.0.smithi002.stdout:2/723: read - d7/d2e/d28/f6a zero size 2024-03-16T10:45:06.512 INFO:tasks.workunit.client.0.smithi002.stdout:2/724: readlink d7/db/d12/lce 0 2024-03-16T10:45:06.513 INFO:tasks.workunit.client.0.smithi002.stdout:2/725: readlink d7/d2e/d1d/d38/d49/db5/ld5 0 2024-03-16T10:45:06.513 INFO:tasks.workunit.client.0.smithi002.stdout:9/846: creat d5/d9/dc/d28/f10e x:0 0 0 2024-03-16T10:45:06.514 INFO:tasks.workunit.client.1.smithi026.stdout:2/47: dread d2/f5 [0,4194304] 0 2024-03-16T10:45:06.515 INFO:tasks.workunit.client.1.smithi026.stdout:2/48: write d2/f9 [1533007,104196] 0 2024-03-16T10:45:06.515 INFO:tasks.workunit.client.1.smithi026.stdout:4/55: creat d1/f13 x:0 0 0 2024-03-16T10:45:06.516 INFO:tasks.workunit.client.0.smithi002.stdout:9/847: write d5/d9/d55/dc9/fbf [1593720,88401] 0 2024-03-16T10:45:06.516 INFO:tasks.workunit.client.1.smithi026.stdout:4/56: fsync d1/db/f11 0 2024-03-16T10:45:06.517 INFO:tasks.workunit.client.0.smithi002.stdout:7/694: creat d3/d4/d1d/d87/d5c/d63/ffa x:0 0 0 2024-03-16T10:45:06.519 INFO:tasks.workunit.client.0.smithi002.stdout:9/848: write d5/d9/dc/d1e/def/f103 [29291,80972] 0 2024-03-16T10:45:06.521 INFO:tasks.workunit.client.0.smithi002.stdout:9/849: dread - d5/d9/dc/d1e/d24/ded/d7f/fd4 zero size 2024-03-16T10:45:06.521 INFO:tasks.workunit.client.0.smithi002.stdout:9/850: fdatasync d5/d9/dc/d1e/d94/fa3 0 2024-03-16T10:45:06.523 INFO:tasks.workunit.client.0.smithi002.stdout:5/665: rename d0/d16/d4a/d55/l69 to d0/d16/dab/daa/ld1 0 2024-03-16T10:45:06.523 INFO:tasks.workunit.client.1.smithi026.stdout:1/37: truncate d0/f8 195494 0 2024-03-16T10:45:06.524 INFO:tasks.workunit.client.0.smithi002.stdout:5/666: stat d0/d16/d4a 0 2024-03-16T10:45:06.524 INFO:tasks.workunit.client.1.smithi026.stdout:1/38: chown d0/d4 43920 1 2024-03-16T10:45:06.524 INFO:tasks.workunit.client.1.smithi026.stdout:1/39: stat d0/d4/l9 0 2024-03-16T10:45:06.526 INFO:tasks.workunit.client.0.smithi002.stdout:0/798: creat d3/d31/daa/f102 x:0 0 0 2024-03-16T10:45:06.527 INFO:tasks.workunit.client.1.smithi026.stdout:2/49: rename d2/fc to d2/db/fe 0 2024-03-16T10:45:06.527 INFO:tasks.workunit.client.1.smithi026.stdout:2/50: rename d2/dd to d2/dd/df 22 2024-03-16T10:45:06.528 INFO:tasks.workunit.client.1.smithi026.stdout:4/57: mknod d1/db/c14 0 2024-03-16T10:45:06.530 INFO:tasks.workunit.client.0.smithi002.stdout:8/876: rmdir d7/d9/d17/d23/d2e/d34/dbf/d109 0 2024-03-16T10:45:06.530 INFO:tasks.workunit.client.0.smithi002.stdout:5/667: dwrite d0/d16/dab/d2b/d94/fbf [0,4194304] 0 2024-03-16T10:45:06.530 INFO:tasks.workunit.client.1.smithi026.stdout:7/37: rmdir d6/da 0 2024-03-16T10:45:06.531 INFO:tasks.workunit.client.0.smithi002.stdout:1/836: sync 2024-03-16T10:45:06.532 INFO:tasks.workunit.client.0.smithi002.stdout:2/726: creat d7/d2e/d1d/d38/d49/fe5 x:0 0 0 2024-03-16T10:45:06.534 INFO:tasks.workunit.client.1.smithi026.stdout:1/40: unlink d0/f5 0 2024-03-16T10:45:06.535 INFO:tasks.workunit.client.0.smithi002.stdout:2/727: read d7/d2e/d1d/d2f/d4b/f62 [4027686,45316] 0 2024-03-16T10:45:06.536 INFO:tasks.workunit.client.0.smithi002.stdout:7/695: creat d3/d4/d1c/d5b/d64/d99/d9c/ffb x:0 0 0 2024-03-16T10:45:06.537 INFO:tasks.workunit.client.0.smithi002.stdout:6/830: link d15/d2d/d73/d97/ca7 d15/dab/dc4/de8/c122 0 2024-03-16T10:45:06.537 INFO:tasks.workunit.client.0.smithi002.stdout:7/696: fsync d3/f15 0 2024-03-16T10:45:06.539 INFO:tasks.workunit.client.1.smithi026.stdout:2/51: rename d2/f5 to d2/db/f10 0 2024-03-16T10:45:06.541 INFO:tasks.workunit.client.0.smithi002.stdout:0/799: dread d3/de/f51 [0,4194304] 0 2024-03-16T10:45:06.541 INFO:tasks.workunit.client.0.smithi002.stdout:9/851: rmdir d5/d9/dc/d28/d8e 39 2024-03-16T10:45:06.545 INFO:tasks.workunit.client.1.smithi026.stdout:1/41: stat d0/l6 0 2024-03-16T10:45:06.545 INFO:tasks.workunit.client.0.smithi002.stdout:1/837: dread d2/f68 [0,4194304] 0 2024-03-16T10:45:06.547 INFO:tasks.workunit.client.1.smithi026.stdout:4/58: rename d1/db/de/l10 to d1/db/l15 0 2024-03-16T10:45:06.549 INFO:tasks.workunit.client.1.smithi026.stdout:2/52: mkdir d2/d11 0 2024-03-16T10:45:06.550 INFO:tasks.workunit.client.0.smithi002.stdout:5/668: dread d0/d16/dab/d2b/d5d/f64 [0,4194304] 0 2024-03-16T10:45:06.552 INFO:tasks.workunit.client.0.smithi002.stdout:5/669: read - d0/d16/dab/d33/f42 zero size 2024-03-16T10:45:06.552 INFO:tasks.workunit.client.1.smithi026.stdout:1/42: creat d0/d4/fc x:0 0 0 2024-03-16T10:45:06.553 INFO:tasks.workunit.client.0.smithi002.stdout:1/838: dwrite d2/df/d12/d84/d87/d96/da8/fd0 [0,4194304] 0 2024-03-16T10:45:06.554 INFO:tasks.workunit.client.1.smithi026.stdout:4/59: creat d1/db/f16 x:0 0 0 2024-03-16T10:45:06.555 INFO:tasks.workunit.client.0.smithi002.stdout:5/670: truncate d0/d16/d4a/d55/fd0 612278 0 2024-03-16T10:45:06.557 INFO:tasks.workunit.client.0.smithi002.stdout:6/831: dread d15/d2d/d2b/f63 [0,4194304] 0 2024-03-16T10:45:06.557 INFO:tasks.workunit.client.0.smithi002.stdout:1/839: write d2/df/d12/d84/d87/d96/db5/dae/ff0 [299046,39161] 0 2024-03-16T10:45:06.557 INFO:tasks.workunit.client.0.smithi002.stdout:5/671: stat d0/d16/dab/d43/d48 0 2024-03-16T10:45:06.558 INFO:tasks.workunit.client.0.smithi002.stdout:5/672: chown d0/f57 25 1 2024-03-16T10:45:06.559 INFO:tasks.workunit.client.0.smithi002.stdout:6/832: write d15/d80/fde [76512,30021] 0 2024-03-16T10:45:06.561 INFO:tasks.workunit.client.0.smithi002.stdout:6/833: stat d15/dfd/d96/dbc/dcc 0 2024-03-16T10:45:06.562 INFO:tasks.workunit.client.0.smithi002.stdout:6/834: chown d15/d2d/d2c/c4b 69 1 2024-03-16T10:45:06.562 INFO:tasks.workunit.client.0.smithi002.stdout:4/732: creat d2/d33/d35/fdf x:0 0 0 2024-03-16T10:45:06.563 INFO:tasks.workunit.client.1.smithi026.stdout:4/60: read d1/f3 [3261074,26656] 0 2024-03-16T10:45:06.564 INFO:tasks.workunit.client.0.smithi002.stdout:5/673: read d0/d16/dab/daa/f9c [211472,28281] 0 2024-03-16T10:45:06.565 INFO:tasks.workunit.client.0.smithi002.stdout:2/728: mknod d7/d2e/d28/d44/d6d/ce6 0 2024-03-16T10:45:06.567 INFO:tasks.workunit.client.1.smithi026.stdout:2/53: truncate d2/db/f10 486334 0 2024-03-16T10:45:06.568 INFO:tasks.workunit.client.0.smithi002.stdout:6/835: dread d15/d2d/d2c/f50 [0,4194304] 0 2024-03-16T10:45:06.579 INFO:tasks.workunit.client.0.smithi002.stdout:8/877: creat d7/d9/d17/d23/d2e/d5c/df0/f12a x:0 0 0 2024-03-16T10:45:06.585 INFO:tasks.workunit.client.0.smithi002.stdout:4/733: creat d2/d33/da/d88/fe0 x:0 0 0 2024-03-16T10:45:06.586 INFO:tasks.workunit.client.0.smithi002.stdout:5/674: write d0/d24/dad/fae [977728,16474] 0 2024-03-16T10:45:06.588 INFO:tasks.workunit.client.1.smithi026.stdout:1/43: rename d0/f8 to d0/fd 0 2024-03-16T10:45:06.588 INFO:tasks.workunit.client.1.smithi026.stdout:2/54: rmdir d2 39 2024-03-16T10:45:06.591 INFO:tasks.workunit.client.1.smithi026.stdout:4/61: truncate d1/f7 329430 0 2024-03-16T10:45:06.592 INFO:tasks.workunit.client.1.smithi026.stdout:7/38: write f1 [1087633,28609] 0 2024-03-16T10:45:06.595 INFO:tasks.workunit.client.0.smithi002.stdout:3/797: getdents d1/dd/d74 0 2024-03-16T10:45:06.596 INFO:tasks.workunit.client.1.smithi026.stdout:4/62: symlink d1/db/l17 0 2024-03-16T10:45:06.598 INFO:tasks.workunit.client.1.smithi026.stdout:7/39: unlink f1 0 2024-03-16T10:45:06.598 INFO:tasks.workunit.client.1.smithi026.stdout:7/40: truncate - no filename 2024-03-16T10:45:06.598 INFO:tasks.workunit.client.1.smithi026.stdout:7/41: rename d6 to d6/db 22 2024-03-16T10:45:06.598 INFO:tasks.workunit.client.1.smithi026.stdout:7/42: truncate - no filename 2024-03-16T10:45:06.599 INFO:tasks.workunit.client.0.smithi002.stdout:1/840: write d2/df/d12/d84/d87/d96/db5/f8 [1442631,46635] 0 2024-03-16T10:45:06.600 INFO:tasks.workunit.client.1.smithi026.stdout:2/55: dread d2/db/f10 [0,4194304] 0 2024-03-16T10:45:06.600 INFO:tasks.workunit.client.0.smithi002.stdout:4/734: readlink d2/d33/da/d2e/db0/lb7 0 2024-03-16T10:45:06.600 INFO:tasks.workunit.client.0.smithi002.stdout:4/735: stat d2 0 2024-03-16T10:45:06.601 INFO:tasks.workunit.client.0.smithi002.stdout:1/841: read d2/df/d12/d84/d87/d96/db5/d51/fa6 [4924100,84412] 0 2024-03-16T10:45:06.602 INFO:tasks.workunit.client.1.smithi026.stdout:4/63: unlink d1/db/c12 0 2024-03-16T10:45:06.602 INFO:tasks.workunit.client.0.smithi002.stdout:4/736: truncate d2/d33/da/d5e/d93/fbb 129994 0 2024-03-16T10:45:06.604 INFO:tasks.workunit.client.0.smithi002.stdout:2/729: write d7/db/f2a [770419,57818] 0 2024-03-16T10:45:06.607 INFO:tasks.workunit.client.1.smithi026.stdout:7/43: creat d6/fc x:0 0 0 2024-03-16T10:45:06.608 INFO:tasks.workunit.client.1.smithi026.stdout:7/44: fdatasync d6/fc 0 2024-03-16T10:45:06.609 INFO:tasks.workunit.client.0.smithi002.stdout:7/697: fdatasync d3/d4/d1d/d87/d5c/dd0/dd2/fe9 0 2024-03-16T10:45:06.612 INFO:tasks.workunit.client.0.smithi002.stdout:3/798: rename d1/f1a to d1/dd/d28/d7b/d2c/d97/df5/f11a 0 2024-03-16T10:45:06.612 INFO:tasks.workunit.client.1.smithi026.stdout:4/64: unlink d1/db/fd 0 2024-03-16T10:45:06.613 INFO:tasks.workunit.client.0.smithi002.stdout:0/800: dwrite d3/d28/d2a/d67/fdd [0,4194304] 0 2024-03-16T10:45:06.615 INFO:tasks.workunit.client.1.smithi026.stdout:1/44: fdatasync d0/fd 0 2024-03-16T10:45:06.618 INFO:tasks.workunit.client.1.smithi026.stdout:7/45: mknod d6/cd 0 2024-03-16T10:45:06.620 INFO:tasks.workunit.client.0.smithi002.stdout:4/737: creat d2/d6/fe1 x:0 0 0 2024-03-16T10:45:06.621 INFO:tasks.workunit.client.0.smithi002.stdout:4/738: truncate d2/d33/da/fd9 724750 0 2024-03-16T10:45:06.622 INFO:tasks.workunit.client.0.smithi002.stdout:4/739: read - d2/d33/da/d5e/f9a zero size 2024-03-16T10:45:06.624 INFO:tasks.workunit.client.0.smithi002.stdout:2/730: symlink d7/d2e/d28/d44/le7 0 2024-03-16T10:45:06.625 INFO:tasks.workunit.client.1.smithi026.stdout:4/65: unlink d1/f13 0 2024-03-16T10:45:06.626 INFO:tasks.workunit.client.1.smithi026.stdout:4/66: read d1/f3 [2668883,13529] 0 2024-03-16T10:45:06.628 INFO:tasks.workunit.client.1.smithi026.stdout:4/67: truncate d1/fa 975017 0 2024-03-16T10:45:06.630 INFO:tasks.workunit.client.1.smithi026.stdout:1/45: mknod d0/ce 0 2024-03-16T10:45:06.630 INFO:tasks.workunit.client.1.smithi026.stdout:2/56: dwrite d2/f9 [0,4194304] 0 2024-03-16T10:45:06.631 INFO:tasks.workunit.client.1.smithi026.stdout:2/57: chown l0 72 1 2024-03-16T10:45:06.631 INFO:tasks.workunit.client.1.smithi026.stdout:1/46: dread d0/fd [0,4194304] 0 2024-03-16T10:45:06.633 INFO:tasks.workunit.client.1.smithi026.stdout:2/58: truncate d2/db/fe 524034 0 2024-03-16T10:45:06.633 INFO:tasks.workunit.client.1.smithi026.stdout:1/47: chown d0/d4/fc 5 1 2024-03-16T10:45:06.633 INFO:tasks.workunit.client.0.smithi002.stdout:7/698: rename d3/d4/d1d/d45/f3a to d3/d4/d1d/d87/d5c/ddb/def/ffc 0 2024-03-16T10:45:06.634 INFO:tasks.workunit.client.1.smithi026.stdout:7/46: mkdir d6/de 0 2024-03-16T10:45:06.637 INFO:tasks.workunit.client.0.smithi002.stdout:0/801: mknod d3/d31/d75/dde/c103 0 2024-03-16T10:45:06.639 INFO:tasks.workunit.client.1.smithi026.stdout:1/48: dwrite d0/d4/fc [0,4194304] 0 2024-03-16T10:45:06.642 INFO:tasks.workunit.client.0.smithi002.stdout:1/842: unlink d2/df/lcf 0 2024-03-16T10:45:06.646 INFO:tasks.workunit.client.1.smithi026.stdout:2/59: mknod d2/c12 0 2024-03-16T10:45:06.652 INFO:tasks.workunit.client.1.smithi026.stdout:1/49: mknod d0/d4/cf 0 2024-03-16T10:45:06.655 INFO:tasks.workunit.client.1.smithi026.stdout:4/68: link d1/f6 d1/f18 0 2024-03-16T10:45:06.656 INFO:tasks.workunit.client.1.smithi026.stdout:2/60: write d2/db/f10 [4574282,15022] 0 2024-03-16T10:45:06.656 INFO:tasks.workunit.client.1.smithi026.stdout:2/61: stat d2/f9 0 2024-03-16T10:45:06.658 INFO:tasks.workunit.client.1.smithi026.stdout:7/47: mknod d6/de/cf 0 2024-03-16T10:45:06.658 INFO:tasks.workunit.client.1.smithi026.stdout:7/48: stat d6/fc 0 2024-03-16T10:45:06.658 INFO:tasks.workunit.client.1.smithi026.stdout:7/49: dread - d6/fc zero size 2024-03-16T10:45:06.659 INFO:tasks.workunit.client.1.smithi026.stdout:7/50: write d6/fc [667711,85191] 0 2024-03-16T10:45:06.662 INFO:tasks.workunit.client.0.smithi002.stdout:7/699: mknod d3/d4/d1c/cfd 0 2024-03-16T10:45:06.665 INFO:tasks.workunit.client.1.smithi026.stdout:1/50: dwrite d0/d4/fc [4194304,4194304] 0 2024-03-16T10:45:06.665 INFO:tasks.workunit.client.0.smithi002.stdout:3/799: rmdir d1/dd/d28/d7b/d8a/d93/dd9 39 2024-03-16T10:45:06.666 INFO:tasks.workunit.client.1.smithi026.stdout:1/51: read d0/fd [4862,62638] 0 2024-03-16T10:45:06.668 INFO:tasks.workunit.client.0.smithi002.stdout:7/700: dwrite d3/d4/d1d/d87/f7e [0,4194304] 0 2024-03-16T10:45:06.670 INFO:tasks.workunit.client.1.smithi026.stdout:4/69: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:06.671 INFO:tasks.workunit.client.0.smithi002.stdout:5/675: link d0/d1f/c44 d0/d24/dad/db6/db9/cd2 0 2024-03-16T10:45:06.672 INFO:tasks.workunit.client.0.smithi002.stdout:5/676: dread - d0/d16/d4a/d7d/fc5 zero size 2024-03-16T10:45:06.672 INFO:tasks.workunit.client.1.smithi026.stdout:2/62: creat d2/f13 x:0 0 0 2024-03-16T10:45:06.675 INFO:tasks.workunit.client.0.smithi002.stdout:4/740: link d2/d33/d1c/d45/faa d2/d6/ddb/fe2 0 2024-03-16T10:45:06.676 INFO:tasks.workunit.client.1.smithi026.stdout:7/51: rmdir d6 39 2024-03-16T10:45:06.678 INFO:tasks.workunit.client.1.smithi026.stdout:1/52: truncate d0/fd 470183 0 2024-03-16T10:45:06.679 INFO:tasks.workunit.client.0.smithi002.stdout:5/677: dwrite d0/d16/dab/d33/d8c/f98 [0,4194304] 0 2024-03-16T10:45:06.681 INFO:tasks.workunit.client.1.smithi026.stdout:4/70: mkdir d1/db/d19 0 2024-03-16T10:45:06.682 INFO:tasks.workunit.client.1.smithi026.stdout:4/71: write d1/db/f16 [935972,58552] 0 2024-03-16T10:45:06.683 INFO:tasks.workunit.client.1.smithi026.stdout:2/63: rename d2/db to d2/d14 0 2024-03-16T10:45:06.684 INFO:tasks.workunit.client.0.smithi002.stdout:3/800: write d1/db6/d15/fd3 [9426474,54299] 0 2024-03-16T10:45:06.686 INFO:tasks.workunit.client.1.smithi026.stdout:7/52: truncate d6/fc 779011 0 2024-03-16T10:45:06.688 INFO:tasks.workunit.client.1.smithi026.stdout:1/53: dwrite d0/d4/fc [4194304,4194304] 0 2024-03-16T10:45:06.688 INFO:tasks.workunit.client.0.smithi002.stdout:3/801: write d1/dd/d28/d7b/d8a/fd0 [2005273,48349] 0 2024-03-16T10:45:06.688 INFO:tasks.workunit.client.1.smithi026.stdout:6/82: sync 2024-03-16T10:45:06.689 INFO:tasks.workunit.client.1.smithi026.stdout:8/22: sync 2024-03-16T10:45:06.689 INFO:tasks.workunit.client.0.smithi002.stdout:5/678: dwrite d0/d16/dab/d33/d8c/fa8 [0,4194304] 0 2024-03-16T10:45:06.693 INFO:tasks.workunit.client.1.smithi026.stdout:2/64: write d2/d14/fe [510826,21207] 0 2024-03-16T10:45:06.693 INFO:tasks.workunit.client.0.smithi002.stdout:7/701: mknod d3/d4/d1d/d87/d5c/dd0/cfe 0 2024-03-16T10:45:06.694 INFO:tasks.workunit.client.0.smithi002.stdout:7/702: chown d3/d4/d1d/d87/d96/d8c 107651096 1 2024-03-16T10:45:06.695 INFO:tasks.workunit.client.1.smithi026.stdout:6/83: dread d0/d10/d17/f18 [0,4194304] 0 2024-03-16T10:45:06.695 INFO:tasks.workunit.client.0.smithi002.stdout:4/741: unlink d2/d6/c8 0 2024-03-16T10:45:06.697 INFO:tasks.workunit.client.1.smithi026.stdout:7/53: mknod d6/c10 0 2024-03-16T10:45:06.702 INFO:tasks.workunit.client.1.smithi026.stdout:8/23: creat d5/f9 x:0 0 0 2024-03-16T10:45:06.702 INFO:tasks.workunit.client.1.smithi026.stdout:8/24: stat d5/f6 0 2024-03-16T10:45:06.703 INFO:tasks.workunit.client.0.smithi002.stdout:0/802: creat d3/d28/d2a/f104 x:0 0 0 2024-03-16T10:45:06.704 INFO:tasks.workunit.client.1.smithi026.stdout:8/25: dread - f1 zero size 2024-03-16T10:45:06.705 INFO:tasks.workunit.client.1.smithi026.stdout:6/84: dwrite d0/d10/d17/f1f [0,4194304] 0 2024-03-16T10:45:06.706 INFO:tasks.workunit.client.1.smithi026.stdout:7/54: dread d6/fc [0,4194304] 0 2024-03-16T10:45:06.706 INFO:tasks.workunit.client.1.smithi026.stdout:6/85: fdatasync d0/f3 0 2024-03-16T10:45:06.707 INFO:tasks.workunit.client.1.smithi026.stdout:7/55: stat d6/de 0 2024-03-16T10:45:06.709 INFO:tasks.workunit.client.1.smithi026.stdout:7/56: dread d6/fc [0,4194304] 0 2024-03-16T10:45:06.711 INFO:tasks.workunit.client.0.smithi002.stdout:3/802: readlink d1/db6/d15/l5b 0 2024-03-16T10:45:06.715 INFO:tasks.workunit.client.0.smithi002.stdout:5/679: dread d0/d16/d4a/d39/f93 [0,4194304] 0 2024-03-16T10:45:06.716 INFO:tasks.workunit.client.0.smithi002.stdout:5/680: chown d0/d16/c7b 551 1 2024-03-16T10:45:06.717 INFO:tasks.workunit.client.1.smithi026.stdout:4/72: write d1/f8 [777007,111956] 0 2024-03-16T10:45:06.719 INFO:tasks.workunit.client.0.smithi002.stdout:2/731: link d7/d2e/d1d/d2f/l81 d7/d27/d3b/d95/dd2/le8 0 2024-03-16T10:45:06.721 INFO:tasks.workunit.client.0.smithi002.stdout:0/803: symlink d3/d31/d75/dea/l105 0 2024-03-16T10:45:06.722 INFO:tasks.workunit.client.0.smithi002.stdout:0/804: write d3/d28/f8e [4574275,92165] 0 2024-03-16T10:45:06.723 INFO:tasks.workunit.client.1.smithi026.stdout:2/65: creat d2/dd/f15 x:0 0 0 2024-03-16T10:45:06.724 INFO:tasks.workunit.client.1.smithi026.stdout:2/66: stat d2 0 2024-03-16T10:45:06.729 INFO:tasks.workunit.client.0.smithi002.stdout:5/681: creat d0/d16/dab/d33/d4b/fd3 x:0 0 0 2024-03-16T10:45:06.729 INFO:tasks.workunit.client.1.smithi026.stdout:2/67: dwrite d2/f9 [0,4194304] 0 2024-03-16T10:45:06.730 INFO:tasks.workunit.client.0.smithi002.stdout:1/843: dwrite d2/df/d12/d84/d87/d96/db5/d51/d56/ffe [0,4194304] 0 2024-03-16T10:45:06.730 INFO:tasks.workunit.client.0.smithi002.stdout:5/682: fsync d0/f6f 0 2024-03-16T10:45:06.730 INFO:tasks.workunit.client.1.smithi026.stdout:1/54: truncate d0/d4/fc 2414990 0 2024-03-16T10:45:06.730 INFO:tasks.workunit.client.1.smithi026.stdout:2/68: chown d2/l7 1864384120 1 2024-03-16T10:45:06.730 INFO:tasks.workunit.client.1.smithi026.stdout:1/55: readlink d0/d4/l9 0 2024-03-16T10:45:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:06 smithi026 ceph-mon[37041]: pgmap v36: 65 pgs: 65 active+clean; 1.3 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 38 MiB/s rd, 162 MiB/s wr, 362 op/s 2024-03-16T10:45:06.733 INFO:tasks.workunit.client.0.smithi002.stdout:2/732: stat d7/d2e/d28/l61 0 2024-03-16T10:45:06.734 INFO:tasks.workunit.client.1.smithi026.stdout:7/57: creat d6/de/f11 x:0 0 0 2024-03-16T10:45:06.736 INFO:tasks.workunit.client.1.smithi026.stdout:8/26: mknod d5/d7/ca 0 2024-03-16T10:45:06.738 INFO:tasks.workunit.client.0.smithi002.stdout:4/742: rename d2/d6/f1a to d2/d33/fe3 0 2024-03-16T10:45:06.739 INFO:tasks.workunit.client.1.smithi026.stdout:6/86: mknod d0/c22 0 2024-03-16T10:45:06.740 INFO:tasks.workunit.client.1.smithi026.stdout:2/69: mknod d2/dd/c16 0 2024-03-16T10:45:06.741 INFO:tasks.workunit.client.0.smithi002.stdout:3/803: dwrite d1/dd/fb8 [0,4194304] 0 2024-03-16T10:45:06.743 INFO:tasks.workunit.client.0.smithi002.stdout:3/804: chown d1/db6/d15/d87/d8c/cc7 126 1 2024-03-16T10:45:06.745 INFO:tasks.workunit.client.0.smithi002.stdout:5/683: creat d0/d24/dad/db6/db9/fd4 x:0 0 0 2024-03-16T10:45:06.746 INFO:tasks.workunit.client.0.smithi002.stdout:5/684: stat d0/d16/d4a/d39/f53 0 2024-03-16T10:45:06.747 INFO:tasks.workunit.client.0.smithi002.stdout:5/685: fdatasync d0/d16/dab/d33/d8c/fa8 0 2024-03-16T10:45:06.748 INFO:tasks.workunit.client.0.smithi002.stdout:7/703: link d3/d4/d1d/d45/la3 d3/d4/d1d/d87/d5c/d63/d34/lff 0 2024-03-16T10:45:06.749 INFO:tasks.workunit.client.1.smithi026.stdout:1/56: dread d0/d4/fc [0,4194304] 0 2024-03-16T10:45:06.749 INFO:tasks.workunit.client.0.smithi002.stdout:6/836: sync 2024-03-16T10:45:06.749 INFO:tasks.workunit.client.0.smithi002.stdout:9/852: sync 2024-03-16T10:45:06.749 INFO:tasks.workunit.client.0.smithi002.stdout:8/878: sync 2024-03-16T10:45:06.750 INFO:tasks.workunit.client.1.smithi026.stdout:2/70: dwrite d2/dd/f15 [0,4194304] 0 2024-03-16T10:45:06.753 INFO:tasks.workunit.client.0.smithi002.stdout:9/853: truncate d5/d9/dc/d1e/f36 5284995 0 2024-03-16T10:45:06.754 INFO:tasks.workunit.client.0.smithi002.stdout:2/733: unlink d7/d2e/d1d/d2f/f83 0 2024-03-16T10:45:06.755 INFO:tasks.workunit.client.0.smithi002.stdout:0/805: dread d3/d28/d65/f6a [0,4194304] 0 2024-03-16T10:45:06.755 INFO:tasks.workunit.client.0.smithi002.stdout:9/854: chown d5/d9/dc/d1e/d94/df9/f108 81 1 2024-03-16T10:45:06.755 INFO:tasks.workunit.client.0.smithi002.stdout:2/734: write d7/d2e/f16 [1224180,84509] 0 2024-03-16T10:45:06.757 INFO:tasks.workunit.client.0.smithi002.stdout:2/735: chown d7/d27/d3f/fac 828163 1 2024-03-16T10:45:06.758 INFO:tasks.workunit.client.0.smithi002.stdout:0/806: write d3/de/f7b [1204548,28013] 0 2024-03-16T10:45:06.758 INFO:tasks.workunit.client.1.smithi026.stdout:6/87: chown d0/c20 52275 1 2024-03-16T10:45:06.758 INFO:tasks.workunit.client.1.smithi026.stdout:7/58: truncate d6/fc 795836 0 2024-03-16T10:45:06.760 INFO:tasks.workunit.client.0.smithi002.stdout:4/743: mknod d2/d33/d1c/d45/ce4 0 2024-03-16T10:45:06.761 INFO:tasks.workunit.client.0.smithi002.stdout:4/744: chown d2/d33/da/d5e/d93/fa2 1605 1 2024-03-16T10:45:06.764 INFO:tasks.workunit.client.0.smithi002.stdout:3/805: symlink d1/da5/l11b 0 2024-03-16T10:45:06.764 INFO:tasks.workunit.client.0.smithi002.stdout:4/745: write d2/d33/da/d88/fab [2767130,98574] 0 2024-03-16T10:45:06.765 INFO:tasks.workunit.client.1.smithi026.stdout:1/57: read d0/fd [324923,6532] 0 2024-03-16T10:45:06.766 INFO:tasks.workunit.client.0.smithi002.stdout:3/806: write d1/da5/fe1 [746378,33761] 0 2024-03-16T10:45:06.767 INFO:tasks.workunit.client.0.smithi002.stdout:5/686: mkdir d0/d16/d5b/da0/dd5 0 2024-03-16T10:45:06.769 INFO:tasks.workunit.client.0.smithi002.stdout:7/704: fdatasync d3/d4/d1c/f2b 0 2024-03-16T10:45:06.772 INFO:tasks.workunit.client.0.smithi002.stdout:8/879: rename d7/d9/d17/d23/d2e/d34/dbf/f129 to d7/d9/d25/d67/f12b 0 2024-03-16T10:45:06.773 INFO:tasks.workunit.client.1.smithi026.stdout:4/73: getdents d1/db 0 2024-03-16T10:45:06.773 INFO:tasks.workunit.client.0.smithi002.stdout:1/844: dwrite d2/df/d12/fa5 [0,4194304] 0 2024-03-16T10:45:06.776 INFO:tasks.workunit.client.1.smithi026.stdout:8/27: dwrite f1 [0,4194304] 0 2024-03-16T10:45:06.777 INFO:tasks.workunit.client.1.smithi026.stdout:1/58: symlink d0/l10 0 2024-03-16T10:45:06.780 INFO:tasks.workunit.client.0.smithi002.stdout:1/845: dwrite d2/df/d12/d84/d87/d96/db5/f55 [0,4194304] 0 2024-03-16T10:45:06.780 INFO:tasks.workunit.client.0.smithi002.stdout:1/846: readlink d2/d58/lc6 0 2024-03-16T10:45:06.782 INFO:tasks.workunit.client.0.smithi002.stdout:0/807: symlink d3/d28/d2a/dc7/l106 0 2024-03-16T10:45:06.788 INFO:tasks.workunit.client.1.smithi026.stdout:4/74: symlink d1/db/l1a 0 2024-03-16T10:45:06.789 INFO:tasks.workunit.client.0.smithi002.stdout:6/837: mknod d15/d2d/c123 0 2024-03-16T10:45:06.790 INFO:tasks.workunit.client.1.smithi026.stdout:4/75: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:06.793 INFO:tasks.workunit.client.0.smithi002.stdout:3/807: chown d1/dd/c2a 10950612 1 2024-03-16T10:45:06.793 INFO:tasks.workunit.client.1.smithi026.stdout:1/59: creat d0/f11 x:0 0 0 2024-03-16T10:45:06.795 INFO:tasks.workunit.client.1.smithi026.stdout:1/60: read d0/fd [324223,124041] 0 2024-03-16T10:45:06.796 INFO:tasks.workunit.client.1.smithi026.stdout:6/88: creat d0/d10/f23 x:0 0 0 2024-03-16T10:45:06.797 INFO:tasks.workunit.client.1.smithi026.stdout:4/76: dwrite d1/db/f11 [0,4194304] 0 2024-03-16T10:45:06.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:06 smithi002 ceph-mon[29248]: pgmap v36: 65 pgs: 65 active+clean; 1.3 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 38 MiB/s rd, 162 MiB/s wr, 362 op/s 2024-03-16T10:45:06.799 INFO:tasks.workunit.client.1.smithi026.stdout:6/89: write d0/f1d [357585,91104] 0 2024-03-16T10:45:06.799 INFO:tasks.workunit.client.0.smithi002.stdout:5/687: symlink d0/d16/dab/d2b/d94/ld6 0 2024-03-16T10:45:06.800 INFO:tasks.workunit.client.1.smithi026.stdout:2/71: getdents d2/dd 0 2024-03-16T10:45:06.801 INFO:tasks.workunit.client.1.smithi026.stdout:2/72: chown d2/d14/fe 391465 1 2024-03-16T10:45:06.801 INFO:tasks.workunit.client.1.smithi026.stdout:2/73: stat d2 0 2024-03-16T10:45:06.802 INFO:tasks.workunit.client.1.smithi026.stdout:7/59: write d6/fc [674489,129101] 0 2024-03-16T10:45:06.803 INFO:tasks.workunit.client.0.smithi002.stdout:8/880: truncate d7/d9/d25/dc9/f100 1374348 0 2024-03-16T10:45:06.805 INFO:tasks.workunit.client.0.smithi002.stdout:8/881: dread - d7/d13/db8/d8f/f103 zero size 2024-03-16T10:45:06.807 INFO:tasks.workunit.client.1.smithi026.stdout:1/61: symlink d0/l12 0 2024-03-16T10:45:06.809 INFO:tasks.workunit.client.1.smithi026.stdout:2/74: dwrite d2/d14/f10 [4194304,4194304] 0 2024-03-16T10:45:06.810 INFO:tasks.workunit.client.1.smithi026.stdout:2/75: dread - d2/f13 zero size 2024-03-16T10:45:06.815 INFO:tasks.workunit.client.1.smithi026.stdout:7/60: symlink d6/de/l12 0 2024-03-16T10:45:06.816 INFO:tasks.workunit.client.0.smithi002.stdout:3/808: mknod d1/dd/d28/d7b/d25/c11c 0 2024-03-16T10:45:06.817 INFO:tasks.workunit.client.0.smithi002.stdout:7/705: dread d3/d4/d1c/f33 [0,4194304] 0 2024-03-16T10:45:06.818 INFO:tasks.workunit.client.1.smithi026.stdout:1/62: rename d0/l12 to d0/l13 0 2024-03-16T10:45:06.818 INFO:tasks.workunit.client.0.smithi002.stdout:0/808: dwrite d3/f2d [0,4194304] 0 2024-03-16T10:45:06.819 INFO:tasks.workunit.client.1.smithi026.stdout:1/63: readlink d0/l10 0 2024-03-16T10:45:06.821 INFO:tasks.workunit.client.1.smithi026.stdout:2/76: mkdir d2/d14/d17 0 2024-03-16T10:45:06.824 INFO:tasks.workunit.client.1.smithi026.stdout:7/61: dwrite d6/fc [0,4194304] 0 2024-03-16T10:45:06.824 INFO:tasks.workunit.client.0.smithi002.stdout:8/882: rmdir d7/d9/d25/dc9 39 2024-03-16T10:45:06.825 INFO:tasks.workunit.client.1.smithi026.stdout:1/64: dread d0/d4/fc [0,4194304] 0 2024-03-16T10:45:06.826 INFO:tasks.workunit.client.0.smithi002.stdout:8/883: write d7/d9/d17/d23/d2e/d34/f6e [586761,54360] 0 2024-03-16T10:45:06.831 INFO:tasks.workunit.client.0.smithi002.stdout:9/855: link d5/d9/dc/d28/dd6/ffd d5/d9/dc/d1e/d94/f10f 0 2024-03-16T10:45:06.833 INFO:tasks.workunit.client.0.smithi002.stdout:4/746: write d2/d33/d35/f40 [987181,4063] 0 2024-03-16T10:45:06.835 INFO:tasks.workunit.client.1.smithi026.stdout:8/28: getdents d5 0 2024-03-16T10:45:06.836 INFO:tasks.workunit.client.0.smithi002.stdout:1/847: mkdir d2/d113 0 2024-03-16T10:45:06.837 INFO:tasks.workunit.client.1.smithi026.stdout:2/77: dwrite d2/dd/f15 [0,4194304] 0 2024-03-16T10:45:06.837 INFO:tasks.workunit.client.1.smithi026.stdout:7/62: dwrite d6/fc [0,4194304] 0 2024-03-16T10:45:06.838 INFO:tasks.workunit.client.0.smithi002.stdout:1/848: read - d2/df/d12/fdd zero size 2024-03-16T10:45:06.838 INFO:tasks.workunit.client.0.smithi002.stdout:9/856: dwrite d5/d9/d25/dac/f10d [0,4194304] 0 2024-03-16T10:45:06.840 INFO:tasks.workunit.client.0.smithi002.stdout:1/849: write d2/df/d12/d84/fd5 [864557,39056] 0 2024-03-16T10:45:06.841 INFO:tasks.workunit.client.0.smithi002.stdout:1/850: readlink d2/df/d12/l53 0 2024-03-16T10:45:06.845 INFO:tasks.workunit.client.0.smithi002.stdout:6/838: symlink d15/dfd/l124 0 2024-03-16T10:45:06.846 INFO:tasks.workunit.client.1.smithi026.stdout:2/78: dwrite d2/d14/f10 [0,4194304] 0 2024-03-16T10:45:06.849 INFO:tasks.workunit.client.1.smithi026.stdout:4/77: rmdir d1/db/d19 0 2024-03-16T10:45:06.854 INFO:tasks.workunit.client.1.smithi026.stdout:1/65: rename d0/la to d0/d4/l14 0 2024-03-16T10:45:06.858 INFO:tasks.workunit.client.1.smithi026.stdout:1/66: dread d0/d4/fc [0,4194304] 0 2024-03-16T10:45:06.858 INFO:tasks.workunit.client.0.smithi002.stdout:7/706: rename d3/d4/d1d/d45 to d3/d4/d1d/d87/d5c/d63/d3f/dba/d100 0 2024-03-16T10:45:06.867 INFO:tasks.workunit.client.0.smithi002.stdout:4/747: mknod d2/d33/da/d5e/d93/ce5 0 2024-03-16T10:45:06.868 INFO:tasks.workunit.client.1.smithi026.stdout:7/63: mknod d6/c13 0 2024-03-16T10:45:06.868 INFO:tasks.workunit.client.1.smithi026.stdout:2/79: creat d2/dd/f18 x:0 0 0 2024-03-16T10:45:06.868 INFO:tasks.workunit.client.1.smithi026.stdout:3/20: sync 2024-03-16T10:45:06.869 INFO:tasks.workunit.client.0.smithi002.stdout:8/884: dwrite d7/d9/d17/d23/d39/dd4/fee [0,4194304] 0 2024-03-16T10:45:06.872 INFO:tasks.workunit.client.0.smithi002.stdout:1/851: unlink d2/df/d12/d29/d5c/l77 0 2024-03-16T10:45:06.874 INFO:tasks.workunit.client.0.smithi002.stdout:3/809: rmdir d1/db6/d15/dcc 39 2024-03-16T10:45:06.875 INFO:tasks.workunit.client.0.smithi002.stdout:5/688: link d0/d16/dab/d2b/d5d/l9a d0/d16/d4a/d39/ld7 0 2024-03-16T10:45:06.875 INFO:tasks.workunit.client.0.smithi002.stdout:1/852: write d2/df/d12/d84/d87/d96/db5/d51/d56/ffe [4802737,119960] 0 2024-03-16T10:45:06.877 INFO:tasks.workunit.client.0.smithi002.stdout:5/689: chown d0/d16/dab/f85 1163 1 2024-03-16T10:45:06.881 INFO:tasks.workunit.client.0.smithi002.stdout:6/839: dread d15/f1c [0,4194304] 0 2024-03-16T10:45:06.885 INFO:tasks.workunit.client.0.smithi002.stdout:4/748: dread d2/d6/f73 [0,4194304] 0 2024-03-16T10:45:06.889 INFO:tasks.workunit.client.0.smithi002.stdout:6/840: write d15/d2d/d2b/d3f/d77/d9f/fa1 [4160405,27197] 0 2024-03-16T10:45:06.892 INFO:tasks.workunit.client.0.smithi002.stdout:5/690: dread d0/d16/dab/d2b/d94/fbf [0,4194304] 0 2024-03-16T10:45:06.893 INFO:tasks.workunit.client.0.smithi002.stdout:9/857: dwrite d5/d9/d25/d23/d81/fe3 [0,4194304] 0 2024-03-16T10:45:06.893 INFO:tasks.workunit.client.0.smithi002.stdout:1/853: dwrite d2/df/d12/d84/d87/d96/db5/d51/d56/fef [4194304,4194304] 0 2024-03-16T10:45:06.897 INFO:tasks.workunit.client.0.smithi002.stdout:5/691: fsync d0/d16/dab/d33/d8c/f98 0 2024-03-16T10:45:06.899 INFO:tasks.workunit.client.0.smithi002.stdout:9/858: rename d5/d9/d25 to d5/d9/d25/d23/d81/d110 22 2024-03-16T10:45:06.902 INFO:tasks.workunit.client.0.smithi002.stdout:4/749: dwrite d2/d33/da/d88/fab [0,4194304] 0 2024-03-16T10:45:06.902 INFO:tasks.workunit.client.0.smithi002.stdout:1/854: chown d2/df/d12/d84/d87/d96/db5/l8a 10 1 2024-03-16T10:45:06.904 INFO:tasks.workunit.client.0.smithi002.stdout:4/750: read - d2/d6/fd6 zero size 2024-03-16T10:45:06.904 INFO:tasks.workunit.client.0.smithi002.stdout:8/885: link d7/d9/d25/d91/ld7 d7/d9/d25/dc9/l12c 0 2024-03-16T10:45:06.908 INFO:tasks.workunit.client.0.smithi002.stdout:1/855: symlink d2/d58/l114 0 2024-03-16T10:45:06.910 INFO:tasks.workunit.client.0.smithi002.stdout:9/859: mkdir d5/d9/d25/d23/d111 0 2024-03-16T10:45:06.911 INFO:tasks.workunit.client.0.smithi002.stdout:1/856: stat d2/df/d12/d84/d87/d96 0 2024-03-16T10:45:06.913 INFO:tasks.workunit.client.0.smithi002.stdout:3/810: dread d1/f1d [0,4194304] 0 2024-03-16T10:45:06.914 INFO:tasks.workunit.client.0.smithi002.stdout:8/886: dread d7/fd [0,4194304] 0 2024-03-16T10:45:06.915 INFO:tasks.workunit.client.0.smithi002.stdout:1/857: write d2/df/d12/d84/d87/d96/db5/fab [1294415,97102] 0 2024-03-16T10:45:06.918 INFO:tasks.workunit.client.0.smithi002.stdout:8/887: write d7/d9/d17/d83/da2/f120 [32637,46457] 0 2024-03-16T10:45:06.919 INFO:tasks.workunit.client.0.smithi002.stdout:3/811: rmdir d1/dd/d28/d7b/d2c/d63 39 2024-03-16T10:45:06.920 INFO:tasks.workunit.client.0.smithi002.stdout:9/860: dwrite d5/d9/d25/f62 [0,4194304] 0 2024-03-16T10:45:06.921 INFO:tasks.workunit.client.0.smithi002.stdout:1/858: rmdir d2/d58 39 2024-03-16T10:45:06.922 INFO:tasks.workunit.client.0.smithi002.stdout:8/888: stat d7/d9/d25/d91/ld7 0 2024-03-16T10:45:06.924 INFO:tasks.workunit.client.0.smithi002.stdout:3/812: chown d1/dd/d28/d7b/d2c/d63/l72 2217 1 2024-03-16T10:45:06.928 INFO:tasks.workunit.client.0.smithi002.stdout:9/861: rename d5/db1/dec/cea to d5/d9/d25/d23/d81/c112 0 2024-03-16T10:45:06.929 INFO:tasks.workunit.client.0.smithi002.stdout:9/862: dread - d5/d9/d25/fbb zero size 2024-03-16T10:45:06.929 INFO:tasks.workunit.client.0.smithi002.stdout:9/863: chown d5/d9/d25/dac 474382 1 2024-03-16T10:45:06.929 INFO:tasks.workunit.client.0.smithi002.stdout:3/813: symlink d1/db6/d15/d87/d8c/l11d 0 2024-03-16T10:45:06.931 INFO:tasks.workunit.client.0.smithi002.stdout:1/859: mkdir d2/d58/d115 0 2024-03-16T10:45:06.932 INFO:tasks.workunit.client.0.smithi002.stdout:1/860: chown d2/df/d12/d84/d87/d96/db5/d51/d56 414525355 1 2024-03-16T10:45:06.933 INFO:tasks.workunit.client.0.smithi002.stdout:1/861: chown d2/df/d12/df6/le5 711640432 1 2024-03-16T10:45:06.933 INFO:tasks.workunit.client.0.smithi002.stdout:8/889: rename d7/d9/d17/d23/d2e/d34/f9e to d7/d13/db8/d8f/f12d 0 2024-03-16T10:45:06.936 INFO:tasks.workunit.client.0.smithi002.stdout:9/864: creat d5/d9/dc/d1e/d91/d83/f113 x:0 0 0 2024-03-16T10:45:06.939 INFO:tasks.workunit.client.0.smithi002.stdout:1/862: dwrite d2/df/d12/d84/d87/d96/fcb [0,4194304] 0 2024-03-16T10:45:06.940 INFO:tasks.workunit.client.0.smithi002.stdout:8/890: mknod d7/d13/db8/de0/c12e 0 2024-03-16T10:45:06.944 INFO:tasks.workunit.client.0.smithi002.stdout:4/751: truncate d2/d33/da/d5e/f75 445173 0 2024-03-16T10:45:06.952 INFO:tasks.workunit.client.0.smithi002.stdout:3/814: rename d1/db6/d15/d87/d8c/d116 to d1/db6/d15/d87/d11e 0 2024-03-16T10:45:06.953 INFO:tasks.workunit.client.0.smithi002.stdout:4/752: mknod d2/d6/ddb/ce6 0 2024-03-16T10:45:06.954 INFO:tasks.workunit.client.0.smithi002.stdout:3/815: read d1/dd/d28/d37/f10f [417223,36343] 0 2024-03-16T10:45:06.954 INFO:tasks.workunit.client.0.smithi002.stdout:3/816: readlink d1/l7 0 2024-03-16T10:45:06.956 INFO:tasks.workunit.client.0.smithi002.stdout:8/891: rename d7/d9/d17/d23/d39/f5d to d7/d9/d25/d91/d123/f12f 0 2024-03-16T10:45:06.959 INFO:tasks.workunit.client.0.smithi002.stdout:4/753: readlink d2/d33/d1c/lcb 0 2024-03-16T10:45:06.961 INFO:tasks.workunit.client.0.smithi002.stdout:4/754: write d2/d33/d35/f40 [1627804,125918] 0 2024-03-16T10:45:06.964 INFO:tasks.workunit.client.0.smithi002.stdout:3/817: unlink d1/dd/f107 0 2024-03-16T10:45:06.974 INFO:tasks.workunit.client.0.smithi002.stdout:3/818: dread d1/db6/d15/fd3 [4194304,4194304] 0 2024-03-16T10:45:06.974 INFO:tasks.workunit.client.0.smithi002.stdout:4/755: rename d2/d33/da to d2/d6/d32/d66/d7f/de7 0 2024-03-16T10:45:06.979 INFO:tasks.workunit.client.0.smithi002.stdout:8/892: symlink d7/d9/d17/d23/d2e/d5c/l130 0 2024-03-16T10:45:06.982 INFO:tasks.workunit.client.0.smithi002.stdout:3/819: creat d1/db6/d15/d87/d11e/f11f x:0 0 0 2024-03-16T10:45:06.985 INFO:tasks.workunit.client.0.smithi002.stdout:3/820: dread d1/dd/fb8 [0,4194304] 0 2024-03-16T10:45:06.991 INFO:tasks.workunit.client.0.smithi002.stdout:3/821: unlink d1/db6/d15/ffc 0 2024-03-16T10:45:06.992 INFO:tasks.workunit.client.0.smithi002.stdout:3/822: chown d1/db6/l18 6853872 1 2024-03-16T10:45:07.002 INFO:tasks.workunit.client.1.smithi026.stdout:0/95: sync 2024-03-16T10:45:07.002 INFO:tasks.workunit.client.1.smithi026.stdout:9/43: sync 2024-03-16T10:45:07.003 INFO:tasks.workunit.client.1.smithi026.stdout:9/44: write d0/f9 [895008,8939] 0 2024-03-16T10:45:07.003 INFO:tasks.workunit.client.0.smithi002.stdout:3/823: symlink d1/dd/d28/d7b/d8a/d93/dd9/l120 0 2024-03-16T10:45:07.004 INFO:tasks.workunit.client.1.smithi026.stdout:9/45: chown d0/c6 10 1 2024-03-16T10:45:07.005 INFO:tasks.workunit.client.1.smithi026.stdout:9/46: write d0/f4 [2998867,64320] 0 2024-03-16T10:45:07.009 INFO:tasks.workunit.client.0.smithi002.stdout:4/756: write d2/f34 [672726,99063] 0 2024-03-16T10:45:07.010 INFO:tasks.workunit.client.0.smithi002.stdout:4/757: chown d2/d6/d32/d66/d7f/de7/d88/fab 146415009 1 2024-03-16T10:45:07.031 INFO:tasks.workunit.client.0.smithi002.stdout:3/824: truncate d1/db6/d15/f1b 5313917 0 2024-03-16T10:45:07.034 INFO:tasks.workunit.client.0.smithi002.stdout:3/825: mknod d1/db6/d15/d87/d11e/c121 0 2024-03-16T10:45:07.037 INFO:tasks.workunit.client.0.smithi002.stdout:3/826: symlink d1/dd/d28/d7b/dd5/dee/l122 0 2024-03-16T10:45:07.037 INFO:tasks.workunit.client.0.smithi002.stdout:3/827: chown d1/db6/d15/d87/d8c/dfd/l112 128940819 1 2024-03-16T10:45:07.041 INFO:tasks.workunit.client.0.smithi002.stdout:3/828: rename d1/dd/d28/d7b/d8a/d93/dd9 to d1/dd/d28/d7b/dd5/d123 0 2024-03-16T10:45:07.041 INFO:tasks.workunit.client.0.smithi002.stdout:3/829: stat d1/dd/d28/d7b/d2c/d63/d115 0 2024-03-16T10:45:07.047 INFO:tasks.workunit.client.0.smithi002.stdout:3/830: dwrite d1/dd/d28/d7b/d2c/d63/f117 [0,4194304] 0 2024-03-16T10:45:07.052 INFO:tasks.workunit.client.0.smithi002.stdout:3/831: creat d1/dd/d28/d7b/dd5/d123/f124 x:0 0 0 2024-03-16T10:45:07.052 INFO:tasks.workunit.client.0.smithi002.stdout:3/832: fdatasync d1/dd/d28/d7b/fde 0 2024-03-16T10:45:07.054 INFO:tasks.workunit.client.0.smithi002.stdout:3/833: write d1/db6/d15/d87/d8c/fb3 [370295,51593] 0 2024-03-16T10:45:07.058 INFO:tasks.workunit.client.0.smithi002.stdout:3/834: rename d1/dd/d74/fb5 to d1/dd/d28/d7b/d2c/d63/f125 0 2024-03-16T10:45:07.059 INFO:tasks.workunit.client.0.smithi002.stdout:3/835: write d1/db6/d15/f108 [359140,58448] 0 2024-03-16T10:45:07.062 INFO:tasks.workunit.client.1.smithi026.stdout:5/56: sync 2024-03-16T10:45:07.063 INFO:tasks.workunit.client.1.smithi026.stdout:6/90: sync 2024-03-16T10:45:07.063 INFO:tasks.workunit.client.1.smithi026.stdout:6/91: fsync d0/f1d 0 2024-03-16T10:45:07.068 INFO:tasks.workunit.client.1.smithi026.stdout:5/57: dwrite d1/fa [0,4194304] 0 2024-03-16T10:45:07.070 INFO:tasks.workunit.client.1.smithi026.stdout:5/58: truncate d1/db/fc 262752 0 2024-03-16T10:45:07.084 INFO:tasks.workunit.client.0.smithi002.stdout:7/707: sync 2024-03-16T10:45:07.085 INFO:tasks.workunit.client.0.smithi002.stdout:0/809: sync 2024-03-16T10:45:07.085 INFO:tasks.workunit.client.0.smithi002.stdout:2/736: sync 2024-03-16T10:45:07.091 INFO:tasks.workunit.client.0.smithi002.stdout:7/708: creat d3/d37/d6a/f101 x:0 0 0 2024-03-16T10:45:07.092 INFO:tasks.workunit.client.0.smithi002.stdout:7/709: dread - d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/f82 zero size 2024-03-16T10:45:07.093 INFO:tasks.workunit.client.0.smithi002.stdout:7/710: read d3/d4/d1d/d87/d96/d5e/fbd [1158914,3183] 0 2024-03-16T10:45:07.097 INFO:tasks.workunit.client.0.smithi002.stdout:7/711: creat d3/d4/d1d/d87/d96/d5e/db7/f102 x:0 0 0 2024-03-16T10:45:07.102 INFO:tasks.workunit.client.1.smithi026.stdout:1/67: mknod d0/d4/c15 0 2024-03-16T10:45:07.103 INFO:tasks.workunit.client.1.smithi026.stdout:2/80: rmdir d2/d14 39 2024-03-16T10:45:07.106 INFO:tasks.workunit.client.0.smithi002.stdout:7/712: rename d3/d4/d1d/d87/d5c/ddb/def/ff3 to d3/d61/d66/f103 0 2024-03-16T10:45:07.111 INFO:tasks.workunit.client.1.smithi026.stdout:8/29: sync 2024-03-16T10:45:07.112 INFO:tasks.workunit.client.1.smithi026.stdout:5/59: creat d1/db/f12 x:0 0 0 2024-03-16T10:45:07.113 INFO:tasks.workunit.client.1.smithi026.stdout:4/78: link d1/l5 d1/db/l1b 0 2024-03-16T10:45:07.115 INFO:tasks.workunit.client.1.smithi026.stdout:1/68: truncate d0/fd 342095 0 2024-03-16T10:45:07.122 INFO:tasks.workunit.client.1.smithi026.stdout:8/30: dwrite d5/f8 [4194304,4194304] 0 2024-03-16T10:45:07.122 INFO:tasks.workunit.client.0.smithi002.stdout:3/836: dwrite d1/dd/d28/d7b/d8a/d93/f51 [0,4194304] 0 2024-03-16T10:45:07.124 INFO:tasks.workunit.client.0.smithi002.stdout:3/837: chown d1/dd/d28/d7b/f77 2 1 2024-03-16T10:45:07.124 INFO:tasks.workunit.client.1.smithi026.stdout:5/60: dwrite d1/f7 [0,4194304] 0 2024-03-16T10:45:07.125 INFO:tasks.workunit.client.1.smithi026.stdout:3/21: link d4/l7 d4/l8 0 2024-03-16T10:45:07.126 INFO:tasks.workunit.client.1.smithi026.stdout:1/69: chown d0/d4/l14 138433 1 2024-03-16T10:45:07.127 INFO:tasks.workunit.client.1.smithi026.stdout:4/79: dwrite d1/f8 [4194304,4194304] 0 2024-03-16T10:45:07.128 INFO:tasks.workunit.client.1.smithi026.stdout:3/22: write d4/f6 [211207,42052] 0 2024-03-16T10:45:07.138 INFO:tasks.workunit.client.0.smithi002.stdout:3/838: dread d1/dd/d28/d7b/d22/f44 [0,4194304] 0 2024-03-16T10:45:07.138 INFO:tasks.workunit.client.1.smithi026.stdout:8/31: creat d5/fb x:0 0 0 2024-03-16T10:45:07.139 INFO:tasks.workunit.client.1.smithi026.stdout:8/32: chown l2 3 1 2024-03-16T10:45:07.139 INFO:tasks.workunit.client.1.smithi026.stdout:8/33: dread - d5/f6 zero size 2024-03-16T10:45:07.140 INFO:tasks.workunit.client.0.smithi002.stdout:6/841: sync 2024-03-16T10:45:07.143 INFO:tasks.workunit.client.1.smithi026.stdout:0/96: rename d1/d18/f1b to d1/f21 0 2024-03-16T10:45:07.145 INFO:tasks.workunit.client.0.smithi002.stdout:3/839: mkdir d1/dd/d28/d7b/d2c/d63/d105/d126 0 2024-03-16T10:45:07.146 INFO:tasks.workunit.client.1.smithi026.stdout:9/47: getdents d0 0 2024-03-16T10:45:07.146 INFO:tasks.workunit.client.1.smithi026.stdout:9/48: chown d0/da 16100 1 2024-03-16T10:45:07.149 INFO:tasks.workunit.client.0.smithi002.stdout:3/840: creat d1/dd/d28/d7b/d2c/d97/f127 x:0 0 0 2024-03-16T10:45:07.150 INFO:tasks.workunit.client.1.smithi026.stdout:3/23: creat d4/f9 x:0 0 0 2024-03-16T10:45:07.150 INFO:tasks.workunit.client.1.smithi026.stdout:6/92: getdents d0/d10/d17 0 2024-03-16T10:45:07.151 INFO:tasks.workunit.client.1.smithi026.stdout:3/24: truncate d4/f6 1124916 0 2024-03-16T10:45:07.152 INFO:tasks.workunit.client.1.smithi026.stdout:8/34: mkdir d5/dc 0 2024-03-16T10:45:07.153 INFO:tasks.workunit.client.1.smithi026.stdout:8/35: write d5/f6 [309454,28058] 0 2024-03-16T10:45:07.153 INFO:tasks.workunit.client.1.smithi026.stdout:8/36: readlink l2 0 2024-03-16T10:45:07.155 INFO:tasks.workunit.client.1.smithi026.stdout:2/81: rename d2/c12 to d2/d14/c19 0 2024-03-16T10:45:07.157 INFO:tasks.workunit.client.1.smithi026.stdout:0/97: mknod d1/d12/c22 0 2024-03-16T10:45:07.159 INFO:tasks.workunit.client.0.smithi002.stdout:3/841: rename d1/da5/l11b to d1/l128 0 2024-03-16T10:45:07.159 INFO:tasks.workunit.client.0.smithi002.stdout:0/810: write d3/d59/dca/f52 [3005991,80883] 0 2024-03-16T10:45:07.161 INFO:tasks.workunit.client.0.smithi002.stdout:2/737: dwrite d7/f2b [0,4194304] 0 2024-03-16T10:45:07.162 INFO:tasks.workunit.client.0.smithi002.stdout:3/842: write d1/db6/d15/d87/dd2/f104 [959024,26998] 0 2024-03-16T10:45:07.163 INFO:tasks.workunit.client.1.smithi026.stdout:6/93: unlink d0/l1c 0 2024-03-16T10:45:07.163 INFO:tasks.workunit.client.1.smithi026.stdout:0/98: dwrite d1/d5/d11/f16 [0,4194304] 0 2024-03-16T10:45:07.165 INFO:tasks.workunit.client.1.smithi026.stdout:3/25: creat d4/fa x:0 0 0 2024-03-16T10:45:07.168 INFO:tasks.workunit.client.1.smithi026.stdout:8/37: symlink d5/ld 0 2024-03-16T10:45:07.168 INFO:tasks.workunit.client.1.smithi026.stdout:8/38: chown d5 43740 1 2024-03-16T10:45:07.169 INFO:tasks.workunit.client.1.smithi026.stdout:8/39: truncate d5/fb 319673 0 2024-03-16T10:45:07.170 INFO:tasks.workunit.client.1.smithi026.stdout:8/40: dread d5/fb [0,4194304] 0 2024-03-16T10:45:07.171 INFO:tasks.workunit.client.0.smithi002.stdout:2/738: creat d7/d2e/d28/d40/fe9 x:0 0 0 2024-03-16T10:45:07.171 INFO:tasks.workunit.client.1.smithi026.stdout:8/41: truncate d5/f6 656886 0 2024-03-16T10:45:07.175 INFO:tasks.workunit.client.1.smithi026.stdout:5/61: creat d1/f13 x:0 0 0 2024-03-16T10:45:07.175 INFO:tasks.workunit.client.0.smithi002.stdout:0/811: getdents d3/d27/d90/dbb 0 2024-03-16T10:45:07.176 INFO:tasks.workunit.client.0.smithi002.stdout:3/843: dread d1/f8f [0,4194304] 0 2024-03-16T10:45:07.178 INFO:tasks.workunit.client.1.smithi026.stdout:8/42: dwrite d5/f6 [0,4194304] 0 2024-03-16T10:45:07.179 INFO:tasks.workunit.client.1.smithi026.stdout:8/43: fdatasync d5/f9 0 2024-03-16T10:45:07.179 INFO:tasks.workunit.client.1.smithi026.stdout:9/49: mkdir d0/da/dc 0 2024-03-16T10:45:07.185 INFO:tasks.workunit.client.0.smithi002.stdout:0/812: fdatasync d3/da0/fb1 0 2024-03-16T10:45:07.185 INFO:tasks.workunit.client.1.smithi026.stdout:5/62: dwrite d1/f7 [0,4194304] 0 2024-03-16T10:45:07.186 INFO:tasks.workunit.client.0.smithi002.stdout:0/813: fdatasync d3/d28/d65/f7a 0 2024-03-16T10:45:07.187 INFO:tasks.workunit.client.1.smithi026.stdout:5/63: write d1/f13 [375437,89760] 0 2024-03-16T10:45:07.188 INFO:tasks.workunit.client.1.smithi026.stdout:6/94: rename d0/c1b to d0/d10/d17/c24 0 2024-03-16T10:45:07.191 INFO:tasks.workunit.client.1.smithi026.stdout:3/26: symlink d4/lb 0 2024-03-16T10:45:07.191 INFO:tasks.workunit.client.1.smithi026.stdout:6/95: write d0/f1d [1414806,45461] 0 2024-03-16T10:45:07.193 INFO:tasks.workunit.client.1.smithi026.stdout:7/64: truncate d6/fc 1585553 0 2024-03-16T10:45:07.194 INFO:tasks.workunit.client.1.smithi026.stdout:9/50: dwrite d0/f9 [0,4194304] 0 2024-03-16T10:45:07.195 INFO:tasks.workunit.client.1.smithi026.stdout:7/65: write d6/de/f11 [63218,2444] 0 2024-03-16T10:45:07.199 INFO:tasks.workunit.client.1.smithi026.stdout:2/82: mkdir d2/d14/d17/d1a 0 2024-03-16T10:45:07.200 INFO:tasks.workunit.client.1.smithi026.stdout:6/96: dread d0/d10/d17/f18 [0,4194304] 0 2024-03-16T10:45:07.200 INFO:tasks.workunit.client.1.smithi026.stdout:2/83: readlink d2/la 0 2024-03-16T10:45:07.204 INFO:tasks.workunit.client.0.smithi002.stdout:2/739: creat d7/d2e/d1d/d38/fea x:0 0 0 2024-03-16T10:45:07.205 INFO:tasks.workunit.client.1.smithi026.stdout:4/80: write d1/f18 [570755,17636] 0 2024-03-16T10:45:07.206 INFO:tasks.workunit.client.0.smithi002.stdout:3/844: creat d1/dd/d28/d7b/d2c/d63/d105/d126/f129 x:0 0 0 2024-03-16T10:45:07.206 INFO:tasks.workunit.client.0.smithi002.stdout:3/845: fsync d1/db6/d15/f108 0 2024-03-16T10:45:07.207 INFO:tasks.workunit.client.1.smithi026.stdout:1/70: getdents d0/d4 0 2024-03-16T10:45:07.209 INFO:tasks.workunit.client.1.smithi026.stdout:5/64: symlink d1/db/l14 0 2024-03-16T10:45:07.209 INFO:tasks.workunit.client.1.smithi026.stdout:4/81: dread d1/db/f11 [0,4194304] 0 2024-03-16T10:45:07.210 INFO:tasks.workunit.client.1.smithi026.stdout:4/82: chown d1/db/f11 24319014 1 2024-03-16T10:45:07.210 INFO:tasks.workunit.client.0.smithi002.stdout:2/740: write d7/d2e/d1d/d2f/d4b/fa0 [4203135,49137] 0 2024-03-16T10:45:07.211 INFO:tasks.workunit.client.1.smithi026.stdout:4/83: read d1/db/f16 [551370,29679] 0 2024-03-16T10:45:07.213 INFO:tasks.workunit.client.1.smithi026.stdout:8/44: rename f1 to d5/dc/fe 0 2024-03-16T10:45:07.215 INFO:tasks.workunit.client.1.smithi026.stdout:3/27: mkdir d4/dc 0 2024-03-16T10:45:07.216 INFO:tasks.workunit.client.0.smithi002.stdout:2/741: dwrite d7/d2e/d1d/d2f/d4b/fa0 [0,4194304] 0 2024-03-16T10:45:07.218 INFO:tasks.workunit.client.0.smithi002.stdout:3/846: creat d1/dd/d28/d7b/d73/f12a x:0 0 0 2024-03-16T10:45:07.218 INFO:tasks.workunit.client.0.smithi002.stdout:2/742: truncate d7/d27/d3b/d95/da5/fbc 285287 0 2024-03-16T10:45:07.220 INFO:tasks.workunit.client.0.smithi002.stdout:3/847: chown d1/dd/d28/d7b/d2c/d63/d115 822 1 2024-03-16T10:45:07.220 INFO:tasks.workunit.client.0.smithi002.stdout:2/743: readlink d7/d2e/d28/d40/dd8/ldb 0 2024-03-16T10:45:07.220 INFO:tasks.workunit.client.1.smithi026.stdout:9/51: symlink d0/ld 0 2024-03-16T10:45:07.222 INFO:tasks.workunit.client.0.smithi002.stdout:0/814: dread d3/d5/f23 [0,4194304] 0 2024-03-16T10:45:07.223 INFO:tasks.workunit.client.0.smithi002.stdout:4/758: sync 2024-03-16T10:45:07.223 INFO:tasks.workunit.client.0.smithi002.stdout:9/865: sync 2024-03-16T10:45:07.223 INFO:tasks.workunit.client.0.smithi002.stdout:5/692: sync 2024-03-16T10:45:07.223 INFO:tasks.workunit.client.0.smithi002.stdout:1/863: sync 2024-03-16T10:45:07.223 INFO:tasks.workunit.client.0.smithi002.stdout:8/893: sync 2024-03-16T10:45:07.223 INFO:tasks.workunit.client.1.smithi026.stdout:3/28: dwrite d4/f9 [0,4194304] 0 2024-03-16T10:45:07.225 INFO:tasks.workunit.client.0.smithi002.stdout:3/848: mknod d1/dd/d28/d7b/dd5/d123/c12b 0 2024-03-16T10:45:07.229 INFO:tasks.workunit.client.0.smithi002.stdout:9/866: chown d5/d9/dc/fa2 33 1 2024-03-16T10:45:07.230 INFO:tasks.workunit.client.0.smithi002.stdout:8/894: truncate d7/de3/f112 259235 0 2024-03-16T10:45:07.232 INFO:tasks.workunit.client.0.smithi002.stdout:2/744: dwrite d7/d2e/d28/d44/d6d/fd0 [0,4194304] 0 2024-03-16T10:45:07.232 INFO:tasks.workunit.client.0.smithi002.stdout:9/867: fsync d5/d9/dc/f92 0 2024-03-16T10:45:07.234 INFO:tasks.workunit.client.0.smithi002.stdout:2/745: chown d7/d2e/d1d/d38/d49/d8c/cca 34871 1 2024-03-16T10:45:07.235 INFO:tasks.workunit.client.1.smithi026.stdout:1/71: dwrite d0/d4/fc [0,4194304] 0 2024-03-16T10:45:07.238 INFO:tasks.workunit.client.1.smithi026.stdout:4/84: rmdir d1/db 39 2024-03-16T10:45:07.239 INFO:tasks.workunit.client.1.smithi026.stdout:8/45: rmdir d5/d7 39 2024-03-16T10:45:07.241 INFO:tasks.workunit.client.0.smithi002.stdout:4/759: write d2/d6/d32/d66/d7f/de7/f7d [129185,39421] 0 2024-03-16T10:45:07.241 INFO:tasks.workunit.client.0.smithi002.stdout:5/693: truncate d0/d16/dab/d2b/f78 118148 0 2024-03-16T10:45:07.242 INFO:tasks.workunit.client.0.smithi002.stdout:6/842: sync 2024-03-16T10:45:07.242 INFO:tasks.workunit.client.0.smithi002.stdout:7/713: sync 2024-03-16T10:45:07.244 INFO:tasks.workunit.client.0.smithi002.stdout:9/868: read d5/f53 [800170,3987] 0 2024-03-16T10:45:07.248 INFO:tasks.workunit.client.0.smithi002.stdout:7/714: chown d3/d61/d66/f4d 529 1 2024-03-16T10:45:07.248 INFO:tasks.workunit.client.0.smithi002.stdout:2/746: dwrite d7/d27/d3f/f69 [4194304,4194304] 0 2024-03-16T10:45:07.249 INFO:tasks.workunit.client.1.smithi026.stdout:2/84: creat d2/d14/d17/d1a/f1b x:0 0 0 2024-03-16T10:45:07.254 INFO:tasks.workunit.client.0.smithi002.stdout:0/815: write d3/d31/daa/fad [4400638,23085] 0 2024-03-16T10:45:07.255 INFO:tasks.workunit.client.0.smithi002.stdout:1/864: mkdir d2/df/d12/d84/d87/d96/db5/d1c/d116 0 2024-03-16T10:45:07.255 INFO:tasks.workunit.client.1.smithi026.stdout:6/97: dwrite d0/f12 [0,4194304] 0 2024-03-16T10:45:07.256 INFO:tasks.workunit.client.0.smithi002.stdout:1/865: chown d2/df/d12/d84/d87/d96/db5/f7e 7367955 1 2024-03-16T10:45:07.257 INFO:tasks.workunit.client.0.smithi002.stdout:1/866: dread - d2/df/d12/d84/d87/d96/db5/d1c/d5d/f82 zero size 2024-03-16T10:45:07.261 INFO:tasks.workunit.client.0.smithi002.stdout:3/849: write d1/db6/d15/f67 [235244,24189] 0 2024-03-16T10:45:07.262 INFO:tasks.workunit.client.0.smithi002.stdout:5/694: dread d0/d16/d4a/d39/f53 [0,4194304] 0 2024-03-16T10:45:07.264 INFO:tasks.workunit.client.0.smithi002.stdout:9/869: write d5/fd [233678,48504] 0 2024-03-16T10:45:07.267 INFO:tasks.workunit.client.0.smithi002.stdout:9/870: fdatasync d5/d9/dc/d1e/d94/df9/f108 0 2024-03-16T10:45:07.270 INFO:tasks.workunit.client.0.smithi002.stdout:6/843: write d15/d2d/d2c/d5d/f72 [1997273,76131] 0 2024-03-16T10:45:07.272 INFO:tasks.workunit.client.0.smithi002.stdout:3/850: dwrite d1/dd/d28/d7b/d8a/d93/f47 [4194304,4194304] 0 2024-03-16T10:45:07.273 INFO:tasks.workunit.client.1.smithi026.stdout:7/66: write d6/fc [1482925,38887] 0 2024-03-16T10:45:07.273 INFO:tasks.workunit.client.0.smithi002.stdout:0/816: dread - d3/d28/d65/fa6 zero size 2024-03-16T10:45:07.274 INFO:tasks.workunit.client.0.smithi002.stdout:1/867: mkdir d2/df/d12/d29/d5c/d117 0 2024-03-16T10:45:07.277 INFO:tasks.workunit.client.0.smithi002.stdout:8/895: write d7/d9/d17/f10b [8301545,55376] 0 2024-03-16T10:45:07.284 INFO:tasks.workunit.client.0.smithi002.stdout:9/871: mknod d5/d9/dc/d1e/def/c114 0 2024-03-16T10:45:07.285 INFO:tasks.workunit.client.0.smithi002.stdout:2/747: dwrite d7/d2e/d1d/d2f/f75 [0,4194304] 0 2024-03-16T10:45:07.286 INFO:tasks.workunit.client.1.smithi026.stdout:9/52: truncate d0/f4 859501 0 2024-03-16T10:45:07.286 INFO:tasks.workunit.client.0.smithi002.stdout:9/872: chown d5/d9/dc/d28/fe6 357 1 2024-03-16T10:45:07.286 INFO:tasks.workunit.client.0.smithi002.stdout:2/748: chown d7/db/d70/f98 1799765 1 2024-03-16T10:45:07.287 INFO:tasks.workunit.client.0.smithi002.stdout:2/749: write d7/d27/d3b/d95/da5/fbb [285388,58096] 0 2024-03-16T10:45:07.288 INFO:tasks.workunit.client.1.smithi026.stdout:8/46: dwrite d5/f6 [4194304,4194304] 0 2024-03-16T10:45:07.289 INFO:tasks.workunit.client.0.smithi002.stdout:7/715: mknod d3/d4/d1d/d87/d96/d5e/db7/de6/c104 0 2024-03-16T10:45:07.290 INFO:tasks.workunit.client.1.smithi026.stdout:5/65: creat d1/f15 x:0 0 0 2024-03-16T10:45:07.292 INFO:tasks.workunit.client.1.smithi026.stdout:5/66: write d1/db/f12 [943466,38666] 0 2024-03-16T10:45:07.293 INFO:tasks.workunit.client.1.smithi026.stdout:5/67: chown d1/db 53 1 2024-03-16T10:45:07.294 INFO:tasks.workunit.client.1.smithi026.stdout:3/29: truncate d4/f9 2423931 0 2024-03-16T10:45:07.295 INFO:tasks.workunit.client.1.smithi026.stdout:8/47: dread d5/f8 [4194304,4194304] 0 2024-03-16T10:45:07.296 INFO:tasks.workunit.client.1.smithi026.stdout:3/30: write d4/f5 [979336,130943] 0 2024-03-16T10:45:07.296 INFO:tasks.workunit.client.1.smithi026.stdout:4/85: rmdir d1/db 39 2024-03-16T10:45:07.297 INFO:tasks.workunit.client.1.smithi026.stdout:8/48: fdatasync d5/f8 0 2024-03-16T10:45:07.298 INFO:tasks.workunit.client.0.smithi002.stdout:8/896: dread d7/d9/d17/d23/f33 [0,4194304] 0 2024-03-16T10:45:07.299 INFO:tasks.workunit.client.1.smithi026.stdout:7/67: rename d6/de/l12 to d6/l14 0 2024-03-16T10:45:07.300 INFO:tasks.workunit.client.1.smithi026.stdout:7/68: chown l3 0 1 2024-03-16T10:45:07.300 INFO:tasks.workunit.client.1.smithi026.stdout:8/49: truncate d5/f9 261026 0 2024-03-16T10:45:07.302 INFO:tasks.workunit.client.1.smithi026.stdout:7/69: truncate d6/de/f11 653454 0 2024-03-16T10:45:07.303 INFO:tasks.workunit.client.1.smithi026.stdout:7/70: write d6/fc [1351139,19179] 0 2024-03-16T10:45:07.303 INFO:tasks.workunit.client.1.smithi026.stdout:2/85: symlink d2/l1c 0 2024-03-16T10:45:07.305 INFO:tasks.workunit.client.0.smithi002.stdout:1/868: rename d2/df/d12/d74/lc1 to d2/df/d12/d84/d100/l118 0 2024-03-16T10:45:07.305 INFO:tasks.workunit.client.1.smithi026.stdout:2/86: write d2/d14/fe [1384067,5555] 0 2024-03-16T10:45:07.306 INFO:tasks.workunit.client.0.smithi002.stdout:1/869: write d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/fe9 [105964,10195] 0 2024-03-16T10:45:07.308 INFO:tasks.workunit.client.0.smithi002.stdout:4/760: creat d2/d33/d35/fe8 x:0 0 0 2024-03-16T10:45:07.309 INFO:tasks.workunit.client.1.smithi026.stdout:7/71: fsync d6/fc 0 2024-03-16T10:45:07.309 INFO:tasks.workunit.client.1.smithi026.stdout:2/87: dread d2/dd/f15 [0,4194304] 0 2024-03-16T10:45:07.310 INFO:tasks.workunit.client.1.smithi026.stdout:2/88: stat d2 0 2024-03-16T10:45:07.313 INFO:tasks.workunit.client.1.smithi026.stdout:9/53: creat d0/da/fe x:0 0 0 2024-03-16T10:45:07.314 INFO:tasks.workunit.client.0.smithi002.stdout:5/695: getdents d0/d16/dab/d43/dcc 0 2024-03-16T10:45:07.314 INFO:tasks.workunit.client.0.smithi002.stdout:4/761: dread - d2/d6/fd6 zero size 2024-03-16T10:45:07.315 INFO:tasks.workunit.client.0.smithi002.stdout:0/817: write d3/d59/dca/faf [793268,72748] 0 2024-03-16T10:45:07.315 INFO:tasks.workunit.client.0.smithi002.stdout:9/873: write d5/d9/dc/d1e/f6b [1278091,130870] 0 2024-03-16T10:45:07.316 INFO:tasks.workunit.client.0.smithi002.stdout:2/750: mknod d7/d2e/d28/ceb 0 2024-03-16T10:45:07.317 INFO:tasks.workunit.client.0.smithi002.stdout:0/818: read - d3/d31/d88/f8b zero size 2024-03-16T10:45:07.318 INFO:tasks.workunit.client.0.smithi002.stdout:9/874: chown d5/d9/dc/d1e/l39 112797 1 2024-03-16T10:45:07.319 INFO:tasks.workunit.client.0.smithi002.stdout:1/870: dread d2/df/d12/d84/d87/d96/da8/fd0 [0,4194304] 0 2024-03-16T10:45:07.319 INFO:tasks.workunit.client.1.smithi026.stdout:7/72: dwrite d6/de/f11 [0,4194304] 0 2024-03-16T10:45:07.320 INFO:tasks.workunit.client.1.smithi026.stdout:2/89: dread d2/dd/f15 [0,4194304] 0 2024-03-16T10:45:07.321 INFO:tasks.workunit.client.1.smithi026.stdout:3/31: chown d4/l7 855849093 1 2024-03-16T10:45:07.322 INFO:tasks.workunit.client.0.smithi002.stdout:6/844: creat d15/dfd/d96/dbc/dcc/f125 x:0 0 0 2024-03-16T10:45:07.323 INFO:tasks.workunit.client.1.smithi026.stdout:2/90: fdatasync d2/d14/f10 0 2024-03-16T10:45:07.324 INFO:tasks.workunit.client.1.smithi026.stdout:1/72: dwrite d0/fd [0,4194304] 0 2024-03-16T10:45:07.325 INFO:tasks.workunit.client.0.smithi002.stdout:0/819: dwrite d3/d31/daa/ddb/ff7 [0,4194304] 0 2024-03-16T10:45:07.335 INFO:tasks.workunit.client.1.smithi026.stdout:2/91: dwrite d2/f13 [0,4194304] 0 2024-03-16T10:45:07.336 INFO:tasks.workunit.client.1.smithi026.stdout:2/92: fdatasync d2/f9 0 2024-03-16T10:45:07.337 INFO:tasks.workunit.client.0.smithi002.stdout:3/851: write d1/dd/d28/f9d [11108,129713] 0 2024-03-16T10:45:07.346 INFO:tasks.workunit.client.1.smithi026.stdout:0/99: sync 2024-03-16T10:45:07.349 INFO:tasks.workunit.client.1.smithi026.stdout:3/32: unlink c3 0 2024-03-16T10:45:07.355 INFO:tasks.workunit.client.1.smithi026.stdout:4/86: mknod d1/db/c1c 0 2024-03-16T10:45:07.356 INFO:tasks.workunit.client.1.smithi026.stdout:1/73: symlink d0/l16 0 2024-03-16T10:45:07.359 INFO:tasks.workunit.client.1.smithi026.stdout:8/50: mknod d5/d7/cf 0 2024-03-16T10:45:07.359 INFO:tasks.workunit.client.1.smithi026.stdout:7/73: write d6/de/f11 [4444627,17528] 0 2024-03-16T10:45:07.361 INFO:tasks.workunit.client.1.smithi026.stdout:1/74: dread d0/fd [0,4194304] 0 2024-03-16T10:45:07.362 INFO:tasks.workunit.client.1.smithi026.stdout:7/74: fdatasync d6/de/f11 0 2024-03-16T10:45:07.363 INFO:tasks.workunit.client.1.smithi026.stdout:6/98: getdents d0/d10 0 2024-03-16T10:45:07.365 INFO:tasks.workunit.client.1.smithi026.stdout:2/93: creat d2/dd/f1d x:0 0 0 2024-03-16T10:45:07.369 INFO:tasks.workunit.client.1.smithi026.stdout:1/75: dwrite d0/d4/fc [4194304,4194304] 0 2024-03-16T10:45:07.374 INFO:tasks.workunit.client.1.smithi026.stdout:5/68: creat d1/f16 x:0 0 0 2024-03-16T10:45:07.378 INFO:tasks.workunit.client.1.smithi026.stdout:4/87: write d1/db/f11 [3022755,15237] 0 2024-03-16T10:45:07.382 INFO:tasks.workunit.client.1.smithi026.stdout:8/51: rename d5/f9 to d5/d7/f10 0 2024-03-16T10:45:07.385 INFO:tasks.workunit.client.1.smithi026.stdout:7/75: creat d6/f15 x:0 0 0 2024-03-16T10:45:07.387 INFO:tasks.workunit.client.1.smithi026.stdout:6/99: creat d0/d10/d17/f25 x:0 0 0 2024-03-16T10:45:07.388 INFO:tasks.workunit.client.1.smithi026.stdout:6/100: write d0/d10/f1a [498937,16616] 0 2024-03-16T10:45:07.388 INFO:tasks.workunit.client.1.smithi026.stdout:2/94: mknod d2/d14/d17/c1e 0 2024-03-16T10:45:07.389 INFO:tasks.workunit.client.1.smithi026.stdout:6/101: dread - d0/d10/f23 zero size 2024-03-16T10:45:07.394 INFO:tasks.workunit.client.1.smithi026.stdout:9/54: rmdir d0/db 0 2024-03-16T10:45:07.395 INFO:tasks.workunit.client.1.smithi026.stdout:9/55: write d0/da/fe [1007620,24732] 0 2024-03-16T10:45:07.399 INFO:tasks.workunit.client.1.smithi026.stdout:9/56: read d0/da/fe [793755,127073] 0 2024-03-16T10:45:07.404 INFO:tasks.workunit.client.1.smithi026.stdout:5/69: creat d1/db/f17 x:0 0 0 2024-03-16T10:45:07.405 INFO:tasks.workunit.client.0.smithi002.stdout:4/762: write d2/d33/fa9 [166129,52363] 0 2024-03-16T10:45:07.406 INFO:tasks.workunit.client.1.smithi026.stdout:3/33: mkdir d4/dc/dd 0 2024-03-16T10:45:07.406 INFO:tasks.workunit.client.0.smithi002.stdout:5/696: dread d0/d16/f4d [0,4194304] 0 2024-03-16T10:45:07.407 INFO:tasks.workunit.client.1.smithi026.stdout:3/34: fdatasync d4/fa 0 2024-03-16T10:45:07.407 INFO:tasks.workunit.client.0.smithi002.stdout:5/697: chown d0/d24/dad/fae 1002421 1 2024-03-16T10:45:07.407 INFO:tasks.workunit.client.0.smithi002.stdout:5/698: dread - d0/d16/dab/d33/d8c/fa5 zero size 2024-03-16T10:45:07.410 INFO:tasks.workunit.client.0.smithi002.stdout:9/875: unlink d5/d9/d55/fc8 0 2024-03-16T10:45:07.411 INFO:tasks.workunit.client.1.smithi026.stdout:5/70: dwrite d1/f7 [0,4194304] 0 2024-03-16T10:45:07.413 INFO:tasks.workunit.client.0.smithi002.stdout:1/871: chown d2/df/d12/d84/d87/d96/db5/fc 303609205 1 2024-03-16T10:45:07.414 INFO:tasks.workunit.client.0.smithi002.stdout:5/699: dwrite d0/f86 [0,4194304] 0 2024-03-16T10:45:07.415 INFO:tasks.workunit.client.0.smithi002.stdout:6/845: mknod d15/d2d/d2c/d86/c126 0 2024-03-16T10:45:07.419 INFO:tasks.workunit.client.1.smithi026.stdout:2/95: rename d2/f13 to d2/dd/f1f 0 2024-03-16T10:45:07.422 INFO:tasks.workunit.client.1.smithi026.stdout:6/102: creat d0/d10/d17/f26 x:0 0 0 2024-03-16T10:45:07.429 INFO:tasks.workunit.client.0.smithi002.stdout:3/852: unlink d1/dd/d28/d37/lb9 0 2024-03-16T10:45:07.429 INFO:tasks.workunit.client.1.smithi026.stdout:6/103: dwrite d0/d10/d17/f1f [0,4194304] 0 2024-03-16T10:45:07.430 INFO:tasks.workunit.client.1.smithi026.stdout:6/104: chown d0/d10/d17/f26 486075 1 2024-03-16T10:45:07.436 INFO:tasks.workunit.client.1.smithi026.stdout:4/88: link d1/db/f11 d1/db/de/f1d 0 2024-03-16T10:45:07.440 INFO:tasks.workunit.client.1.smithi026.stdout:4/89: dread d1/f18 [0,4194304] 0 2024-03-16T10:45:07.441 INFO:tasks.workunit.client.1.smithi026.stdout:4/90: chown d1/f7 701 1 2024-03-16T10:45:07.443 INFO:tasks.workunit.client.0.smithi002.stdout:1/872: rmdir d2/df/d12/d84/d87/d96/db5/d1c/d5d 39 2024-03-16T10:45:07.444 INFO:tasks.workunit.client.1.smithi026.stdout:7/76: truncate d6/fc 1274923 0 2024-03-16T10:45:07.446 INFO:tasks.workunit.client.0.smithi002.stdout:5/700: unlink d0/d16/dab/c9b 0 2024-03-16T10:45:07.448 INFO:tasks.workunit.client.1.smithi026.stdout:0/100: dwrite d1/d5/d6/f19 [0,4194304] 0 2024-03-16T10:45:07.449 INFO:tasks.workunit.client.0.smithi002.stdout:7/716: write d3/d4/d1d/d87/d96/d5e/f84 [815575,28740] 0 2024-03-16T10:45:07.449 INFO:tasks.workunit.client.1.smithi026.stdout:2/96: mknod d2/dd/c20 0 2024-03-16T10:45:07.450 INFO:tasks.workunit.client.1.smithi026.stdout:2/97: fdatasync d2/d14/d17/d1a/f1b 0 2024-03-16T10:45:07.450 INFO:tasks.workunit.client.1.smithi026.stdout:0/101: write d1/d5/d11/f16 [1332255,107098] 0 2024-03-16T10:45:07.451 INFO:tasks.workunit.client.0.smithi002.stdout:1/873: dwrite d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/f95 [0,4194304] 0 2024-03-16T10:45:07.461 INFO:tasks.workunit.client.1.smithi026.stdout:2/98: dwrite d2/f9 [0,4194304] 0 2024-03-16T10:45:07.463 INFO:tasks.workunit.client.1.smithi026.stdout:0/102: dwrite d1/d5/d11/f16 [0,4194304] 0 2024-03-16T10:45:07.464 INFO:tasks.workunit.client.0.smithi002.stdout:8/897: dwrite d7/d9/d17/d23/d2e/de5/f11f [0,4194304] 0 2024-03-16T10:45:07.465 INFO:tasks.workunit.client.1.smithi026.stdout:0/103: write d1/d5/d11/f16 [5300168,118131] 0 2024-03-16T10:45:07.470 INFO:tasks.workunit.client.0.smithi002.stdout:9/876: rename d5/d9/d34/c88 to d5/d9/dc/d1e/d24/ded/da4/c115 0 2024-03-16T10:45:07.470 INFO:tasks.workunit.client.1.smithi026.stdout:6/105: creat d0/d10/f27 x:0 0 0 2024-03-16T10:45:07.475 INFO:tasks.workunit.client.1.smithi026.stdout:2/99: dwrite d2/dd/f18 [0,4194304] 0 2024-03-16T10:45:07.475 INFO:tasks.workunit.client.0.smithi002.stdout:6/846: mkdir d15/d2d/d73/d115/d127 0 2024-03-16T10:45:07.476 INFO:tasks.workunit.client.1.smithi026.stdout:8/52: truncate d5/f8 3737266 0 2024-03-16T10:45:07.478 INFO:tasks.workunit.client.0.smithi002.stdout:5/701: unlink d0/d1f/la4 0 2024-03-16T10:45:07.482 INFO:tasks.workunit.client.0.smithi002.stdout:6/847: dwrite d15/d2d/da8/fc9 [0,4194304] 0 2024-03-16T10:45:07.483 INFO:tasks.workunit.client.1.smithi026.stdout:4/91: mknod d1/db/de/c1e 0 2024-03-16T10:45:07.485 INFO:tasks.workunit.client.0.smithi002.stdout:1/874: chown d2/df/d12/dd7/cda 2 1 2024-03-16T10:45:07.485 INFO:tasks.workunit.client.1.smithi026.stdout:7/77: mkdir d6/de/d16 0 2024-03-16T10:45:07.486 INFO:tasks.workunit.client.1.smithi026.stdout:7/78: dread - d6/f15 zero size 2024-03-16T10:45:07.486 INFO:tasks.workunit.client.0.smithi002.stdout:1/875: stat d2/df/d12/d84/d87/d96/db5/d51/d56/fef 0 2024-03-16T10:45:07.487 INFO:tasks.workunit.client.0.smithi002.stdout:0/820: creat d3/d28/d2a/f107 x:0 0 0 2024-03-16T10:45:07.488 INFO:tasks.workunit.client.0.smithi002.stdout:4/763: write d2/d33/fe3 [417281,85781] 0 2024-03-16T10:45:07.490 INFO:tasks.workunit.client.1.smithi026.stdout:1/76: getdents d0 0 2024-03-16T10:45:07.490 INFO:tasks.workunit.client.0.smithi002.stdout:2/751: getdents d7/d2e/d1d/d38/da8 0 2024-03-16T10:45:07.491 INFO:tasks.workunit.client.1.smithi026.stdout:1/77: write d0/d4/fc [4797598,90964] 0 2024-03-16T10:45:07.492 INFO:tasks.workunit.client.1.smithi026.stdout:1/78: chown d0/l13 3 1 2024-03-16T10:45:07.492 INFO:tasks.workunit.client.1.smithi026.stdout:1/79: read d0/fd [240181,48183] 0 2024-03-16T10:45:07.493 INFO:tasks.workunit.client.1.smithi026.stdout:9/57: rmdir d0/da/dc 0 2024-03-16T10:45:07.494 INFO:tasks.workunit.client.1.smithi026.stdout:0/104: mknod d1/d18/c23 0 2024-03-16T10:45:07.495 INFO:tasks.workunit.client.0.smithi002.stdout:7/717: dread d3/d4/fea [0,4194304] 0 2024-03-16T10:45:07.498 INFO:tasks.workunit.client.1.smithi026.stdout:0/105: dread d1/d5/d6/f19 [0,4194304] 0 2024-03-16T10:45:07.499 INFO:tasks.workunit.client.1.smithi026.stdout:0/106: stat d1/d5/l4 0 2024-03-16T10:45:07.500 INFO:tasks.workunit.client.0.smithi002.stdout:2/752: dread d7/d2e/d28/d40/f63 [0,4194304] 0 2024-03-16T10:45:07.501 INFO:tasks.workunit.client.0.smithi002.stdout:6/848: rmdir d15/dfd/d96/dbc/dcc 39 2024-03-16T10:45:07.503 INFO:tasks.workunit.client.1.smithi026.stdout:2/100: unlink d2/f9 0 2024-03-16T10:45:07.505 INFO:tasks.workunit.client.0.smithi002.stdout:3/853: creat d1/db6/d15/f12c x:0 0 0 2024-03-16T10:45:07.507 INFO:tasks.workunit.client.0.smithi002.stdout:0/821: mkdir d3/d28/d65/d108 0 2024-03-16T10:45:07.509 INFO:tasks.workunit.client.0.smithi002.stdout:4/764: mknod d2/d6/d32/d66/d7f/de7/d2e/db0/ce9 0 2024-03-16T10:45:07.510 INFO:tasks.workunit.client.0.smithi002.stdout:4/765: write d2/d33/d35/fdf [884506,123548] 0 2024-03-16T10:45:07.511 INFO:tasks.workunit.client.1.smithi026.stdout:5/71: rename d1/f15 to d1/f18 0 2024-03-16T10:45:07.514 INFO:tasks.workunit.client.1.smithi026.stdout:4/92: unlink d1/c4 0 2024-03-16T10:45:07.517 INFO:tasks.workunit.client.1.smithi026.stdout:7/79: rename l3 to d6/l17 0 2024-03-16T10:45:07.517 INFO:tasks.workunit.client.0.smithi002.stdout:8/898: write d7/f60 [4477192,77767] 0 2024-03-16T10:45:07.518 INFO:tasks.workunit.client.1.smithi026.stdout:7/80: read - d6/f15 zero size 2024-03-16T10:45:07.519 INFO:tasks.workunit.client.1.smithi026.stdout:5/72: dwrite d1/f13 [0,4194304] 0 2024-03-16T10:45:07.520 INFO:tasks.workunit.client.0.smithi002.stdout:7/718: symlink d3/d4/d1d/d87/d96/d5e/db7/de6/l105 0 2024-03-16T10:45:07.521 INFO:tasks.workunit.client.0.smithi002.stdout:7/719: chown d3/ld9 61413333 1 2024-03-16T10:45:07.522 INFO:tasks.workunit.client.1.smithi026.stdout:1/80: mkdir d0/d17 0 2024-03-16T10:45:07.523 INFO:tasks.workunit.client.0.smithi002.stdout:8/899: dwrite d7/d9/f58 [0,4194304] 0 2024-03-16T10:45:07.526 INFO:tasks.workunit.client.0.smithi002.stdout:5/702: creat d0/d1f/da1/fd8 x:0 0 0 2024-03-16T10:45:07.529 INFO:tasks.workunit.client.0.smithi002.stdout:4/766: dread d2/d6/f1e [0,4194304] 0 2024-03-16T10:45:07.530 INFO:tasks.workunit.client.0.smithi002.stdout:6/849: symlink d15/d2d/da8/l128 0 2024-03-16T10:45:07.531 INFO:tasks.workunit.client.1.smithi026.stdout:3/35: getdents d4/dc 0 2024-03-16T10:45:07.531 INFO:tasks.workunit.client.1.smithi026.stdout:3/36: stat d4 0 2024-03-16T10:45:07.532 INFO:tasks.workunit.client.1.smithi026.stdout:3/37: chown d4/dc/dd 63283 1 2024-03-16T10:45:07.533 INFO:tasks.workunit.client.1.smithi026.stdout:3/38: readlink d4/lb 0 2024-03-16T10:45:07.533 INFO:tasks.workunit.client.1.smithi026.stdout:3/39: read d4/f5 [1019460,82016] 0 2024-03-16T10:45:07.534 INFO:tasks.workunit.client.0.smithi002.stdout:3/854: symlink d1/dd/d28/d7b/d2c/d63/d105/l12d 0 2024-03-16T10:45:07.535 INFO:tasks.workunit.client.1.smithi026.stdout:6/106: readlink d0/d10/lc 0 2024-03-16T10:45:07.536 INFO:tasks.workunit.client.0.smithi002.stdout:0/822: rmdir d3/d31/d75 39 2024-03-16T10:45:07.537 INFO:tasks.workunit.client.0.smithi002.stdout:9/877: truncate d5/d9/d25/d23/f3b 3572097 0 2024-03-16T10:45:07.538 INFO:tasks.workunit.client.0.smithi002.stdout:0/823: write d3/d31/daa/dff/f101 [550776,94109] 0 2024-03-16T10:45:07.539 INFO:tasks.workunit.client.0.smithi002.stdout:1/876: mkdir d2/df/d12/d84/d87/d96/db5/d1c/da0/d119 0 2024-03-16T10:45:07.544 INFO:tasks.workunit.client.0.smithi002.stdout:2/753: symlink d7/d27/d3b/dd4/de1/lec 0 2024-03-16T10:45:07.548 INFO:tasks.workunit.client.0.smithi002.stdout:5/703: creat d0/d16/dab/d43/d48/fd9 x:0 0 0 2024-03-16T10:45:07.549 INFO:tasks.workunit.client.0.smithi002.stdout:5/704: chown d0/d16/dab/d2b/d94/cbe 493 1 2024-03-16T10:45:07.550 INFO:tasks.workunit.client.1.smithi026.stdout:4/93: symlink d1/db/l1f 0 2024-03-16T10:45:07.551 INFO:tasks.workunit.client.1.smithi026.stdout:9/58: dwrite d0/da/fe [0,4194304] 0 2024-03-16T10:45:07.551 INFO:tasks.workunit.client.0.smithi002.stdout:0/824: dread d3/d28/d3d/f46 [0,4194304] 0 2024-03-16T10:45:07.556 INFO:tasks.workunit.client.1.smithi026.stdout:0/107: rename d1/d5 to d1/d18/d1a/d24 0 2024-03-16T10:45:07.558 INFO:tasks.workunit.client.1.smithi026.stdout:7/81: stat d6/l17 0 2024-03-16T10:45:07.560 INFO:tasks.workunit.client.0.smithi002.stdout:4/767: dread d2/d6/d32/d66/d7f/de7/d5e/f75 [0,4194304] 0 2024-03-16T10:45:07.562 INFO:tasks.workunit.client.1.smithi026.stdout:1/81: creat d0/d4/f18 x:0 0 0 2024-03-16T10:45:07.565 INFO:tasks.workunit.client.1.smithi026.stdout:7/82: dwrite d6/f15 [0,4194304] 0 2024-03-16T10:45:07.566 INFO:tasks.workunit.client.0.smithi002.stdout:8/900: dwrite d7/d9/d17/d23/d39/fb4 [0,4194304] 0 2024-03-16T10:45:07.566 INFO:tasks.workunit.client.0.smithi002.stdout:1/877: fsync d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/dee/f111 0 2024-03-16T10:45:07.568 INFO:tasks.workunit.client.0.smithi002.stdout:1/878: chown d2/df/d12/d84/d87 1 1 2024-03-16T10:45:07.570 INFO:tasks.workunit.client.1.smithi026.stdout:7/83: dread d6/f15 [0,4194304] 0 2024-03-16T10:45:07.571 INFO:tasks.workunit.client.0.smithi002.stdout:6/850: dwrite d15/d2d/d2b/d3f/d77/d9f/f91 [0,4194304] 0 2024-03-16T10:45:07.573 INFO:tasks.workunit.client.1.smithi026.stdout:3/40: creat d4/fe x:0 0 0 2024-03-16T10:45:07.579 INFO:tasks.workunit.client.0.smithi002.stdout:5/705: creat d0/d24/dad/db6/db9/fda x:0 0 0 2024-03-16T10:45:07.584 INFO:tasks.workunit.client.1.smithi026.stdout:4/94: symlink d1/db/l20 0 2024-03-16T10:45:07.585 INFO:tasks.workunit.client.0.smithi002.stdout:1/879: dwrite d2/df/d12/d84/d87/d96/da8/fa9 [0,4194304] 0 2024-03-16T10:45:07.587 INFO:tasks.workunit.client.0.smithi002.stdout:4/768: rename d2/d6/d32/d66/d7f/de7/f4f to d2/d33/d1c/d45/fea 0 2024-03-16T10:45:07.588 INFO:tasks.workunit.client.0.smithi002.stdout:7/720: dwrite d3/d61/d66/f103 [0,4194304] 0 2024-03-16T10:45:07.590 INFO:tasks.workunit.client.0.smithi002.stdout:4/769: chown d2/d6/d32/d66/l6b 1 1 2024-03-16T10:45:07.591 INFO:tasks.workunit.client.0.smithi002.stdout:4/770: truncate d2/fa3 1356327 0 2024-03-16T10:45:07.591 INFO:tasks.workunit.client.1.smithi026.stdout:1/82: rename d0/ce to d0/c19 0 2024-03-16T10:45:07.592 INFO:tasks.workunit.client.1.smithi026.stdout:1/83: readlink d0/d4/l14 0 2024-03-16T10:45:07.594 INFO:tasks.workunit.client.1.smithi026.stdout:0/108: dwrite d1/f1e [0,4194304] 0 2024-03-16T10:45:07.605 INFO:tasks.workunit.client.1.smithi026.stdout:3/41: symlink d4/dc/lf 0 2024-03-16T10:45:07.606 INFO:tasks.workunit.client.1.smithi026.stdout:6/107: mkdir d0/d28 0 2024-03-16T10:45:07.607 INFO:tasks.workunit.client.0.smithi002.stdout:6/851: dread d15/d2d/d2c/f55 [0,4194304] 0 2024-03-16T10:45:07.608 INFO:tasks.workunit.client.1.smithi026.stdout:2/101: rmdir d2/d11 0 2024-03-16T10:45:07.616 INFO:tasks.workunit.client.1.smithi026.stdout:2/102: dwrite d2/dd/f18 [4194304,4194304] 0 2024-03-16T10:45:07.617 INFO:tasks.workunit.client.0.smithi002.stdout:0/825: rename d3/d28/d2a/da1 to d3/d31/d75/db4/d109 0 2024-03-16T10:45:07.619 INFO:tasks.workunit.client.0.smithi002.stdout:2/754: truncate d7/f2b 900190 0 2024-03-16T10:45:07.621 INFO:tasks.workunit.client.1.smithi026.stdout:0/109: creat d1/d18/d1a/f25 x:0 0 0 2024-03-16T10:45:07.623 INFO:tasks.workunit.client.1.smithi026.stdout:7/84: creat d6/de/d16/f18 x:0 0 0 2024-03-16T10:45:07.625 INFO:tasks.workunit.client.0.smithi002.stdout:4/771: truncate d2/d6/d32/d66/d7f/de7/f84 758730 0 2024-03-16T10:45:07.625 INFO:tasks.workunit.client.0.smithi002.stdout:0/826: dwrite d3/d31/daa/dff/f101 [0,4194304] 0 2024-03-16T10:45:07.627 INFO:tasks.workunit.client.0.smithi002.stdout:4/772: write d2/d33/d1c/fba [4833807,74525] 0 2024-03-16T10:45:07.630 INFO:tasks.workunit.client.0.smithi002.stdout:4/773: chown d2/d6/fe1 835543 1 2024-03-16T10:45:07.632 INFO:tasks.workunit.client.0.smithi002.stdout:6/852: mknod d15/dfd/d96/dbb/d107/c129 0 2024-03-16T10:45:07.633 INFO:tasks.workunit.client.1.smithi026.stdout:8/53: dread d5/f8 [0,4194304] 0 2024-03-16T10:45:07.634 INFO:tasks.workunit.client.1.smithi026.stdout:8/54: readlink d5/ld 0 2024-03-16T10:45:07.635 INFO:tasks.workunit.client.0.smithi002.stdout:5/706: mknod d0/cdb 0 2024-03-16T10:45:07.637 INFO:tasks.workunit.client.0.smithi002.stdout:8/901: write d7/d9/d17/d23/d39/f7a [1738131,28784] 0 2024-03-16T10:45:07.639 INFO:tasks.workunit.client.0.smithi002.stdout:1/880: dwrite d2/df/d12/d84/d87/d96/db5/d1c/fba [0,4194304] 0 2024-03-16T10:45:07.639 INFO:tasks.workunit.client.0.smithi002.stdout:8/902: chown d7/d13/db8/de0/c12e 958501397 1 2024-03-16T10:45:07.641 INFO:tasks.workunit.client.1.smithi026.stdout:0/110: creat d1/d18/d1a/f26 x:0 0 0 2024-03-16T10:45:07.641 INFO:tasks.workunit.client.0.smithi002.stdout:3/855: rename d1/db6/l18 to d1/dd/d74/dd8/l12e 0 2024-03-16T10:45:07.644 INFO:tasks.workunit.client.1.smithi026.stdout:0/111: dread d1/d18/d1a/d24/f3 [0,4194304] 0 2024-03-16T10:45:07.645 INFO:tasks.workunit.client.0.smithi002.stdout:9/878: getdents d5/d9/dc/d1e/d91/d82/db4 0 2024-03-16T10:45:07.646 INFO:tasks.workunit.client.1.smithi026.stdout:2/103: dwrite d2/dd/f1f [4194304,4194304] 0 2024-03-16T10:45:07.647 INFO:tasks.workunit.client.1.smithi026.stdout:7/85: creat d6/de/d16/f19 x:0 0 0 2024-03-16T10:45:07.647 INFO:tasks.workunit.client.0.smithi002.stdout:3/856: dwrite d1/db6/f4a [0,4194304] 0 2024-03-16T10:45:07.647 INFO:tasks.workunit.client.1.smithi026.stdout:0/112: fdatasync d1/d18/d1a/f25 0 2024-03-16T10:45:07.648 INFO:tasks.workunit.client.1.smithi026.stdout:2/104: chown c1 60483 1 2024-03-16T10:45:07.648 INFO:tasks.workunit.client.1.smithi026.stdout:2/105: readlink d2/l1c 0 2024-03-16T10:45:07.649 INFO:tasks.workunit.client.0.smithi002.stdout:2/755: mknod d7/d2e/d1d/d38/d49/db5/ced 0 2024-03-16T10:45:07.650 INFO:tasks.workunit.client.0.smithi002.stdout:3/857: truncate d1/dd/d28/d7b/d8a/d93/f91 5527502 0 2024-03-16T10:45:07.653 INFO:tasks.workunit.client.0.smithi002.stdout:7/721: truncate d3/ff8 3656998 0 2024-03-16T10:45:07.657 INFO:tasks.workunit.client.1.smithi026.stdout:8/55: creat d5/d7/f11 x:0 0 0 2024-03-16T10:45:07.663 INFO:tasks.workunit.client.1.smithi026.stdout:3/42: dwrite d4/f9 [0,4194304] 0 2024-03-16T10:45:07.663 INFO:tasks.workunit.client.1.smithi026.stdout:7/86: creat d6/de/d16/f1a x:0 0 0 2024-03-16T10:45:07.665 INFO:tasks.workunit.client.1.smithi026.stdout:7/87: write d6/de/d16/f19 [1018082,127585] 0 2024-03-16T10:45:07.670 INFO:tasks.workunit.client.0.smithi002.stdout:1/881: readlink d2/df/d12/d84/d87/d96/db5/d1c/l81 0 2024-03-16T10:45:07.671 INFO:tasks.workunit.client.0.smithi002.stdout:1/882: fdatasync d2/df/d12/d84/d87/d96/db5/dae/ff0 0 2024-03-16T10:45:07.673 INFO:tasks.workunit.client.1.smithi026.stdout:8/56: creat d5/dc/f12 x:0 0 0 2024-03-16T10:45:07.675 INFO:tasks.workunit.client.1.smithi026.stdout:0/113: symlink d1/l27 0 2024-03-16T10:45:07.676 INFO:tasks.workunit.client.1.smithi026.stdout:0/114: chown d1/d12/l13 465 1 2024-03-16T10:45:07.677 INFO:tasks.workunit.client.0.smithi002.stdout:4/774: rename d2/d33/d1c/d45/ce4 to d2/d6/ceb 0 2024-03-16T10:45:07.677 INFO:tasks.workunit.client.1.smithi026.stdout:3/43: symlink d4/l10 0 2024-03-16T10:45:07.678 INFO:tasks.workunit.client.0.smithi002.stdout:6/853: dwrite d15/d2d/d2b/d3f/d77/d7e/fb2 [0,4194304] 0 2024-03-16T10:45:07.680 INFO:tasks.workunit.client.0.smithi002.stdout:9/879: symlink d5/d9/dc/d1e/d94/l116 0 2024-03-16T10:45:07.680 INFO:tasks.workunit.client.1.smithi026.stdout:7/88: stat d6/l14 0 2024-03-16T10:45:07.682 INFO:tasks.workunit.client.1.smithi026.stdout:6/108: rename d0/d10/l16 to d0/d10/l29 0 2024-03-16T10:45:07.685 INFO:tasks.workunit.client.0.smithi002.stdout:2/756: mkdir d7/d2e/d28/d40/d50/db4/dee 0 2024-03-16T10:45:07.686 INFO:tasks.workunit.client.1.smithi026.stdout:4/95: link d1/db/l1b d1/l21 0 2024-03-16T10:45:07.687 INFO:tasks.workunit.client.0.smithi002.stdout:3/858: fdatasync d1/dd/d28/d7b/d8a/d93/f4d 0 2024-03-16T10:45:07.688 INFO:tasks.workunit.client.0.smithi002.stdout:3/859: dread - d1/dd/d28/d7b/d8a/d93/ff6 zero size 2024-03-16T10:45:07.690 INFO:tasks.workunit.client.1.smithi026.stdout:0/115: readlink d1/d18/d1a/d24/l7 0 2024-03-16T10:45:07.692 INFO:tasks.workunit.client.1.smithi026.stdout:3/44: creat d4/dc/f11 x:0 0 0 2024-03-16T10:45:07.694 INFO:tasks.workunit.client.1.smithi026.stdout:2/106: creat d2/d14/d17/f21 x:0 0 0 2024-03-16T10:45:07.696 INFO:tasks.workunit.client.0.smithi002.stdout:5/707: mknod d0/d16/cdc 0 2024-03-16T10:45:07.697 INFO:tasks.workunit.client.1.smithi026.stdout:7/89: creat d6/f1b x:0 0 0 2024-03-16T10:45:07.699 INFO:tasks.workunit.client.0.smithi002.stdout:0/827: truncate d3/d31/fc6 1242953 0 2024-03-16T10:45:07.700 INFO:tasks.workunit.client.0.smithi002.stdout:1/883: chown d2/df/c6f 10360353 1 2024-03-16T10:45:07.701 INFO:tasks.workunit.client.1.smithi026.stdout:7/90: dread d6/de/d16/f19 [0,4194304] 0 2024-03-16T10:45:07.701 INFO:tasks.workunit.client.0.smithi002.stdout:8/903: mknod d7/d13/db8/c131 0 2024-03-16T10:45:07.702 INFO:tasks.workunit.client.0.smithi002.stdout:8/904: chown d7/d9/d17/d23/d39/db7/fc4 81302 1 2024-03-16T10:45:07.703 INFO:tasks.workunit.client.1.smithi026.stdout:7/91: truncate d6/de/d16/f18 81115 0 2024-03-16T10:45:07.703 INFO:tasks.workunit.client.0.smithi002.stdout:1/884: truncate d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/d102/f112 949358 0 2024-03-16T10:45:07.703 INFO:tasks.workunit.client.1.smithi026.stdout:7/92: chown d6/f1b 0 1 2024-03-16T10:45:07.704 INFO:tasks.workunit.client.0.smithi002.stdout:8/905: dread - d7/d9/d17/d23/d2e/de5/f122 zero size 2024-03-16T10:45:07.704 INFO:tasks.workunit.client.1.smithi026.stdout:7/93: fdatasync d6/de/d16/f18 0 2024-03-16T10:45:07.708 INFO:tasks.workunit.client.0.smithi002.stdout:7/722: rename d3/d61/d66/d88 to d3/d4/d1d/d87/d5c/d63/d3f/daa/d106 0 2024-03-16T10:45:07.709 INFO:tasks.workunit.client.0.smithi002.stdout:8/906: dread d7/d9/d17/d23/d39/f7a [0,4194304] 0 2024-03-16T10:45:07.711 INFO:tasks.workunit.client.0.smithi002.stdout:8/907: chown d7/d9/d17/d83/ff1 0 1 2024-03-16T10:45:07.711 INFO:tasks.workunit.client.0.smithi002.stdout:4/775: rmdir d2/d6/d32/d66/d7f/de7/d5e/d93 39 2024-03-16T10:45:07.712 INFO:tasks.workunit.client.0.smithi002.stdout:4/776: chown d2/d6/d32/d66/d7f/de7/la1 4062928 1 2024-03-16T10:45:07.712 INFO:tasks.workunit.client.0.smithi002.stdout:8/908: write d7/d9/d17/d23/d2e/de5/f11f [3358529,114870] 0 2024-03-16T10:45:07.716 INFO:tasks.workunit.client.0.smithi002.stdout:9/880: unlink d5/d9/dc/d1e/d24/fa5 0 2024-03-16T10:45:07.718 INFO:tasks.workunit.client.1.smithi026.stdout:4/96: dwrite d1/f18 [0,4194304] 0 2024-03-16T10:45:07.718 INFO:tasks.workunit.client.0.smithi002.stdout:2/757: mknod d7/d2e/d46/cef 0 2024-03-16T10:45:07.719 INFO:tasks.workunit.client.1.smithi026.stdout:2/107: dwrite d2/d14/fe [0,4194304] 0 2024-03-16T10:45:07.720 INFO:tasks.workunit.client.1.smithi026.stdout:8/57: link c3 d5/d7/c13 0 2024-03-16T10:45:07.722 INFO:tasks.workunit.client.0.smithi002.stdout:3/860: symlink d1/dd/d28/d7b/d2c/d63/d105/l12f 0 2024-03-16T10:45:07.724 INFO:tasks.workunit.client.1.smithi026.stdout:7/94: mknod d6/de/d16/c1c 0 2024-03-16T10:45:07.727 INFO:tasks.workunit.client.1.smithi026.stdout:7/95: write d6/de/d16/f1a [163387,79260] 0 2024-03-16T10:45:07.727 INFO:tasks.workunit.client.0.smithi002.stdout:0/828: symlink d3/d31/d88/dc8/l10a 0 2024-03-16T10:45:07.727 INFO:tasks.workunit.client.1.smithi026.stdout:3/45: creat d4/dc/dd/f12 x:0 0 0 2024-03-16T10:45:07.728 INFO:tasks.workunit.client.1.smithi026.stdout:8/58: dwrite d5/dc/f12 [0,4194304] 0 2024-03-16T10:45:07.728 INFO:tasks.workunit.client.1.smithi026.stdout:7/96: chown d6/c10 22 1 2024-03-16T10:45:07.729 INFO:tasks.workunit.client.1.smithi026.stdout:7/97: stat d6/cd 0 2024-03-16T10:45:07.729 INFO:tasks.workunit.client.0.smithi002.stdout:1/885: creat d2/df/d12/d84/d87/d96/db5/dae/f11a x:0 0 0 2024-03-16T10:45:07.732 INFO:tasks.workunit.client.0.smithi002.stdout:2/758: dread d7/d2e/d28/d40/d50/f68 [0,4194304] 0 2024-03-16T10:45:07.733 INFO:tasks.workunit.client.1.smithi026.stdout:6/109: creat d0/f2a x:0 0 0 2024-03-16T10:45:07.739 INFO:tasks.workunit.client.0.smithi002.stdout:8/909: rmdir d7/d9/d17/d23/d2e 39 2024-03-16T10:45:07.742 INFO:tasks.workunit.client.1.smithi026.stdout:2/108: mknod d2/d14/d17/d1a/c22 0 2024-03-16T10:45:07.742 INFO:tasks.workunit.client.1.smithi026.stdout:2/109: readlink l0 0 2024-03-16T10:45:07.745 INFO:tasks.workunit.client.0.smithi002.stdout:7/723: dread d3/d4/d1d/d87/f92 [0,4194304] 0 2024-03-16T10:45:07.746 INFO:tasks.workunit.client.1.smithi026.stdout:8/59: mknod d5/d7/c14 0 2024-03-16T10:45:07.749 INFO:tasks.workunit.client.1.smithi026.stdout:8/60: write d5/f6 [6601153,114182] 0 2024-03-16T10:45:07.751 INFO:tasks.workunit.client.0.smithi002.stdout:8/910: dread d7/d9/d25/d91/d123/f12f [0,4194304] 0 2024-03-16T10:45:07.751 INFO:tasks.workunit.client.1.smithi026.stdout:3/46: creat d4/dc/f13 x:0 0 0 2024-03-16T10:45:07.752 INFO:tasks.workunit.client.1.smithi026.stdout:8/61: chown c3 841368116 1 2024-03-16T10:45:07.753 INFO:tasks.workunit.client.1.smithi026.stdout:2/110: dwrite d2/d14/d17/d1a/f1b [0,4194304] 0 2024-03-16T10:45:07.753 INFO:tasks.workunit.client.1.smithi026.stdout:7/98: dread d6/fc [0,4194304] 0 2024-03-16T10:45:07.758 INFO:tasks.workunit.client.1.smithi026.stdout:0/116: getdents d1/d18/d1a 0 2024-03-16T10:45:07.759 INFO:tasks.workunit.client.1.smithi026.stdout:0/117: stat d1/d18 0 2024-03-16T10:45:07.759 INFO:tasks.workunit.client.0.smithi002.stdout:0/829: unlink d3/d28/d3d/f50 0 2024-03-16T10:45:07.761 INFO:tasks.workunit.client.1.smithi026.stdout:7/99: dread d6/f15 [0,4194304] 0 2024-03-16T10:45:07.763 INFO:tasks.workunit.client.1.smithi026.stdout:7/100: chown d6/c7 618012 1 2024-03-16T10:45:07.763 INFO:tasks.workunit.client.1.smithi026.stdout:2/111: dwrite d2/dd/f1d [0,4194304] 0 2024-03-16T10:45:07.763 INFO:tasks.workunit.client.0.smithi002.stdout:1/886: fsync d2/df/d12/d84/d87/d96/db5/f39 0 2024-03-16T10:45:07.766 INFO:tasks.workunit.client.1.smithi026.stdout:3/47: unlink d4/f6 0 2024-03-16T10:45:07.772 INFO:tasks.workunit.client.1.smithi026.stdout:6/110: write d0/d10/d17/f25 [104094,56282] 0 2024-03-16T10:45:07.774 INFO:tasks.workunit.client.1.smithi026.stdout:7/101: dwrite d6/de/d16/f18 [0,4194304] 0 2024-03-16T10:45:07.774 INFO:tasks.workunit.client.1.smithi026.stdout:4/97: rename d1/f7 to d1/f22 0 2024-03-16T10:45:07.776 INFO:tasks.workunit.client.0.smithi002.stdout:9/881: getdents d5/d9/dc/d1e/d91/d107 0 2024-03-16T10:45:07.780 INFO:tasks.workunit.client.1.smithi026.stdout:7/102: dread d6/de/f11 [0,4194304] 0 2024-03-16T10:45:07.781 INFO:tasks.workunit.client.1.smithi026.stdout:7/103: readlink d6/l8 0 2024-03-16T10:45:07.782 INFO:tasks.workunit.client.1.smithi026.stdout:7/104: chown d6/de/d16/f19 297722 1 2024-03-16T10:45:07.782 INFO:tasks.workunit.client.1.smithi026.stdout:0/118: rmdir d1/d18/d1a 39 2024-03-16T10:45:07.785 INFO:tasks.workunit.client.1.smithi026.stdout:3/48: mknod d4/dc/dd/c14 0 2024-03-16T10:45:07.786 INFO:tasks.workunit.client.0.smithi002.stdout:5/708: link d0/d16/dab/daa/lac d0/d16/d5b/da0/ldd 0 2024-03-16T10:45:07.790 INFO:tasks.workunit.client.1.smithi026.stdout:6/111: truncate d0/d10/d17/f18 3588995 0 2024-03-16T10:45:07.792 INFO:tasks.workunit.client.0.smithi002.stdout:1/887: mkdir d2/df/d12/dd7/d11b 0 2024-03-16T10:45:07.794 INFO:tasks.workunit.client.1.smithi026.stdout:7/105: dread d6/de/d16/f19 [0,4194304] 0 2024-03-16T10:45:07.795 INFO:tasks.workunit.client.0.smithi002.stdout:4/777: creat d2/d6/d32/d66/fec x:0 0 0 2024-03-16T10:45:07.796 INFO:tasks.workunit.client.1.smithi026.stdout:3/49: creat d4/dc/f15 x:0 0 0 2024-03-16T10:45:07.797 INFO:tasks.workunit.client.0.smithi002.stdout:7/724: dwrite d3/d4/d1d/d87/d5c/d63/fab [0,4194304] 0 2024-03-16T10:45:07.799 INFO:tasks.workunit.client.0.smithi002.stdout:6/854: getdents d15/d2d 0 2024-03-16T10:45:07.803 INFO:tasks.workunit.client.1.smithi026.stdout:0/119: creat d1/d18/d1a/d24/d6/f28 x:0 0 0 2024-03-16T10:45:07.803 INFO:tasks.workunit.client.0.smithi002.stdout:9/882: dread d5/d9/dc/f74 [0,4194304] 0 2024-03-16T10:45:07.804 INFO:tasks.workunit.client.1.smithi026.stdout:4/98: dwrite d1/f3 [4194304,4194304] 0 2024-03-16T10:45:07.805 INFO:tasks.workunit.client.1.smithi026.stdout:7/106: symlink d6/l1d 0 2024-03-16T10:45:07.807 INFO:tasks.workunit.client.1.smithi026.stdout:4/99: chown d1/db/l20 778857479 1 2024-03-16T10:45:07.807 INFO:tasks.workunit.client.1.smithi026.stdout:6/112: rmdir d0 39 2024-03-16T10:45:07.810 INFO:tasks.workunit.client.1.smithi026.stdout:4/100: write d1/f8 [9228617,84963] 0 2024-03-16T10:45:07.811 INFO:tasks.workunit.client.0.smithi002.stdout:0/830: creat d3/d31/d75/de4/f10b x:0 0 0 2024-03-16T10:45:07.812 INFO:tasks.workunit.client.0.smithi002.stdout:0/831: readlink d3/d31/d88/dc8/l10a 0 2024-03-16T10:45:07.812 INFO:tasks.workunit.client.1.smithi026.stdout:4/101: chown d1/db/f11 56 1 2024-03-16T10:45:07.812 INFO:tasks.workunit.client.1.smithi026.stdout:3/50: creat d4/dc/f16 x:0 0 0 2024-03-16T10:45:07.814 INFO:tasks.workunit.client.1.smithi026.stdout:3/51: chown d4/dc/lf 2006071224 1 2024-03-16T10:45:07.815 INFO:tasks.workunit.client.1.smithi026.stdout:7/107: dwrite d6/de/d16/f18 [0,4194304] 0 2024-03-16T10:45:07.816 INFO:tasks.workunit.client.1.smithi026.stdout:7/108: write d6/de/d16/f18 [707099,45136] 0 2024-03-16T10:45:07.817 INFO:tasks.workunit.client.1.smithi026.stdout:8/62: getdents d5/dc 0 2024-03-16T10:45:07.818 INFO:tasks.workunit.client.1.smithi026.stdout:7/109: write d6/de/f11 [4146814,969] 0 2024-03-16T10:45:07.824 INFO:tasks.workunit.client.0.smithi002.stdout:4/778: write d2/d33/fcd [1005165,108960] 0 2024-03-16T10:45:07.827 INFO:tasks.workunit.client.0.smithi002.stdout:8/911: creat d7/d9/d17/d23/d39/db7/f132 x:0 0 0 2024-03-16T10:45:07.828 INFO:tasks.workunit.client.0.smithi002.stdout:4/779: fsync d2/d33/fcd 0 2024-03-16T10:45:07.828 INFO:tasks.workunit.client.1.smithi026.stdout:4/102: symlink d1/db/de/l23 0 2024-03-16T10:45:07.828 INFO:tasks.workunit.client.0.smithi002.stdout:8/912: chown d7/d9/d25/c46 3 1 2024-03-16T10:45:07.830 INFO:tasks.workunit.client.0.smithi002.stdout:5/709: fdatasync d0/d16/dab/d2b/f78 0 2024-03-16T10:45:07.830 INFO:tasks.workunit.client.1.smithi026.stdout:3/52: symlink d4/dc/l17 0 2024-03-16T10:45:07.831 INFO:tasks.workunit.client.1.smithi026.stdout:3/53: chown d4/l10 15523941 1 2024-03-16T10:45:07.832 INFO:tasks.workunit.client.0.smithi002.stdout:3/861: getdents d1/dd/d28/d7b/d8a/dd7/df1 0 2024-03-16T10:45:07.835 INFO:tasks.workunit.client.0.smithi002.stdout:1/888: dwrite d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/f79 [0,4194304] 0 2024-03-16T10:45:07.837 INFO:tasks.workunit.client.0.smithi002.stdout:0/832: creat d3/d59/dca/f10c x:0 0 0 2024-03-16T10:45:07.838 INFO:tasks.workunit.client.0.smithi002.stdout:2/759: link d7/db/d12/l14 d7/d27/d3f/d7f/lf0 0 2024-03-16T10:45:07.839 INFO:tasks.workunit.client.0.smithi002.stdout:1/889: write d2/df/d12/d84/d87/d96/db5/f9d [14426,115535] 0 2024-03-16T10:45:07.840 INFO:tasks.workunit.client.1.smithi026.stdout:3/54: dwrite d4/dc/dd/f12 [0,4194304] 0 2024-03-16T10:45:07.840 INFO:tasks.workunit.client.1.smithi026.stdout:0/120: link d1/d18/d1a/f26 d1/d18/d1a/d24/f29 0 2024-03-16T10:45:07.841 INFO:tasks.workunit.client.1.smithi026.stdout:0/121: readlink d1/l27 0 2024-03-16T10:45:07.843 INFO:tasks.workunit.client.1.smithi026.stdout:6/113: truncate d0/d10/fd 217625 0 2024-03-16T10:45:07.844 INFO:tasks.workunit.client.0.smithi002.stdout:6/855: mkdir d15/dab/d11c/d12a 0 2024-03-16T10:45:07.844 INFO:tasks.workunit.client.0.smithi002.stdout:4/780: dread d2/d6/d32/d66/d7f/f9f [0,4194304] 0 2024-03-16T10:45:07.846 INFO:tasks.workunit.client.1.smithi026.stdout:4/103: symlink d1/db/de/l24 0 2024-03-16T10:45:07.848 INFO:tasks.workunit.client.0.smithi002.stdout:2/760: dwrite d7/db/d12/f85 [0,4194304] 0 2024-03-16T10:45:07.852 INFO:tasks.workunit.client.1.smithi026.stdout:3/55: creat d4/f18 x:0 0 0 2024-03-16T10:45:07.853 INFO:tasks.workunit.client.1.smithi026.stdout:0/122: rename d1/d12/l13 to d1/d12/l2a 0 2024-03-16T10:45:07.853 INFO:tasks.workunit.client.0.smithi002.stdout:5/710: creat d0/d16/dab/d33/d4b/fde x:0 0 0 2024-03-16T10:45:07.853 INFO:tasks.workunit.client.0.smithi002.stdout:2/761: chown d7/d2e/d1d/d38/f73 63141 1 2024-03-16T10:45:07.854 INFO:tasks.workunit.client.0.smithi002.stdout:3/862: mkdir d1/dd/d28/d7b/d2c/d63/d105/d126/d130 0 2024-03-16T10:45:07.854 INFO:tasks.workunit.client.1.smithi026.stdout:6/114: write d0/f11 [1368645,6730] 0 2024-03-16T10:45:07.855 INFO:tasks.workunit.client.0.smithi002.stdout:2/762: chown d7/db/d12/f85 3622907 1 2024-03-16T10:45:07.856 INFO:tasks.workunit.client.1.smithi026.stdout:6/115: write d0/d10/f27 [37841,29640] 0 2024-03-16T10:45:07.857 INFO:tasks.workunit.client.1.smithi026.stdout:4/104: fdatasync d1/db/f16 0 2024-03-16T10:45:07.858 INFO:tasks.workunit.client.1.smithi026.stdout:3/56: creat d4/dc/f19 x:0 0 0 2024-03-16T10:45:07.859 INFO:tasks.workunit.client.0.smithi002.stdout:3/863: truncate d1/dd/d28/d7b/d2c/d63/f125 1575952 0 2024-03-16T10:45:07.859 INFO:tasks.workunit.client.1.smithi026.stdout:4/105: chown d1/db/c1c 28238 1 2024-03-16T10:45:07.860 INFO:tasks.workunit.client.0.smithi002.stdout:1/890: mkdir d2/df/d12/d84/d87/d96/db5/d51/d56/d11c 0 2024-03-16T10:45:07.860 INFO:tasks.workunit.client.0.smithi002.stdout:3/864: chown d1/dd/d28/d7b/d8a/dd7/df1 2 1 2024-03-16T10:45:07.861 INFO:tasks.workunit.client.1.smithi026.stdout:4/106: fdatasync d1/f6 0 2024-03-16T10:45:07.861 INFO:tasks.workunit.client.1.smithi026.stdout:3/57: write d4/dc/f11 [417757,20367] 0 2024-03-16T10:45:07.863 INFO:tasks.workunit.client.1.smithi026.stdout:0/123: mkdir d1/d18/d1a/d2b 0 2024-03-16T10:45:07.864 INFO:tasks.workunit.client.1.smithi026.stdout:6/116: dread d0/f6 [0,4194304] 0 2024-03-16T10:45:07.864 INFO:tasks.workunit.client.0.smithi002.stdout:6/856: mknod d15/d2d/d2b/d3f/d77/d7e/c12b 0 2024-03-16T10:45:07.865 INFO:tasks.workunit.client.0.smithi002.stdout:6/857: write d15/dfd/d96/dbb/f11e [1035929,115663] 0 2024-03-16T10:45:07.870 INFO:tasks.workunit.client.1.smithi026.stdout:0/124: write d1/d18/d1a/d24/d11/f16 [4098821,56747] 0 2024-03-16T10:45:07.871 INFO:tasks.workunit.client.0.smithi002.stdout:4/781: dread d2/d6/d32/d66/d7f/de7/d2e/f56 [0,4194304] 0 2024-03-16T10:45:07.872 INFO:tasks.workunit.client.0.smithi002.stdout:8/913: creat d7/d9/d17/d23/d2e/de5/f133 x:0 0 0 2024-03-16T10:45:07.872 INFO:tasks.workunit.client.0.smithi002.stdout:4/782: read - d2/d6/d32/d66/d7f/de7/d88/fe0 zero size 2024-03-16T10:45:07.873 INFO:tasks.workunit.client.1.smithi026.stdout:4/107: symlink d1/db/de/l25 0 2024-03-16T10:45:07.874 INFO:tasks.workunit.client.0.smithi002.stdout:8/914: fdatasync d7/d9/d17/d23/d2e/d34/dbf/ff7 0 2024-03-16T10:45:07.874 INFO:tasks.workunit.client.1.smithi026.stdout:8/63: truncate f4 3031772 0 2024-03-16T10:45:07.876 INFO:tasks.workunit.client.1.smithi026.stdout:7/110: truncate d6/de/d16/f18 2929259 0 2024-03-16T10:45:07.878 INFO:tasks.workunit.client.0.smithi002.stdout:9/883: mknod d5/d9/dc/d28/d8e/c117 0 2024-03-16T10:45:07.879 INFO:tasks.workunit.client.0.smithi002.stdout:0/833: getdents d3/d28/d65/d108 0 2024-03-16T10:45:07.883 INFO:tasks.workunit.client.0.smithi002.stdout:2/763: unlink d7/f5e 0 2024-03-16T10:45:07.884 INFO:tasks.workunit.client.0.smithi002.stdout:1/891: rmdir d2/df/d12/d84/d87 39 2024-03-16T10:45:07.885 INFO:tasks.workunit.client.1.smithi026.stdout:4/108: creat d1/db/f26 x:0 0 0 2024-03-16T10:45:07.886 INFO:tasks.workunit.client.0.smithi002.stdout:8/915: dread d7/d13/db8/d8f/ffd [0,4194304] 0 2024-03-16T10:45:07.887 INFO:tasks.workunit.client.1.smithi026.stdout:6/117: symlink d0/l2b 0 2024-03-16T10:45:07.890 INFO:tasks.workunit.client.1.smithi026.stdout:7/111: mkdir d6/d1e 0 2024-03-16T10:45:07.891 INFO:tasks.workunit.client.1.smithi026.stdout:7/112: write d6/de/d16/f1a [732884,50762] 0 2024-03-16T10:45:07.892 INFO:tasks.workunit.client.1.smithi026.stdout:7/113: read d6/f15 [1915365,68176] 0 2024-03-16T10:45:07.893 INFO:tasks.workunit.client.1.smithi026.stdout:0/125: link d1/d18/d1a/d24/d11/f16 d1/d18/d1a/f2c 0 2024-03-16T10:45:07.895 INFO:tasks.workunit.client.0.smithi002.stdout:2/764: dread d7/d2e/f67 [0,4194304] 0 2024-03-16T10:45:07.899 INFO:tasks.workunit.client.0.smithi002.stdout:2/765: read d7/d2e/d1d/d2f/d4b/d6e/fab [6448448,113000] 0 2024-03-16T10:45:07.903 INFO:tasks.workunit.client.0.smithi002.stdout:9/884: rename d5/d9/dc/d1e/d94/df9 to d5/d9/d22/d3c/d118 0 2024-03-16T10:45:07.907 INFO:tasks.workunit.client.0.smithi002.stdout:6/858: write d15/d2d/f1f [2760092,29927] 0 2024-03-16T10:45:07.909 INFO:tasks.workunit.client.0.smithi002.stdout:4/783: dread d2/d33/d1c/d45/f99 [0,4194304] 0 2024-03-16T10:45:07.910 INFO:tasks.workunit.client.0.smithi002.stdout:4/784: chown d2/d6/f17 19733445 1 2024-03-16T10:45:07.910 INFO:tasks.workunit.client.1.smithi026.stdout:0/126: rename d1/d18/d1a/d24/d11/f16 to d1/d18/d1a/d24/d11/f2d 0 2024-03-16T10:45:07.911 INFO:tasks.workunit.client.1.smithi026.stdout:0/127: chown d1/d18/d1a/d24/d11/c14 8238284 1 2024-03-16T10:45:07.915 INFO:tasks.workunit.client.1.smithi026.stdout:7/114: getdents d6/d1e 0 2024-03-16T10:45:07.915 INFO:tasks.workunit.client.1.smithi026.stdout:7/115: stat d6/l14 0 2024-03-16T10:45:07.917 INFO:tasks.workunit.client.0.smithi002.stdout:8/916: creat d7/d9/d17/d23/d2e/d90/f134 x:0 0 0 2024-03-16T10:45:07.918 INFO:tasks.workunit.client.0.smithi002.stdout:8/917: fdatasync d7/d9/d17/d23/fd3 0 2024-03-16T10:45:07.918 INFO:tasks.workunit.client.1.smithi026.stdout:0/128: mknod d1/d18/d1a/d24/d11/c2e 0 2024-03-16T10:45:07.919 INFO:tasks.workunit.client.1.smithi026.stdout:7/116: dread d6/de/f11 [0,4194304] 0 2024-03-16T10:45:07.921 INFO:tasks.workunit.client.1.smithi026.stdout:6/118: link d0/d10/f27 d0/f2c 0 2024-03-16T10:45:07.921 INFO:tasks.workunit.client.1.smithi026.stdout:4/109: truncate d1/fa 712353 0 2024-03-16T10:45:07.922 INFO:tasks.workunit.client.1.smithi026.stdout:4/110: read d1/f8 [8116797,124893] 0 2024-03-16T10:45:07.924 INFO:tasks.workunit.client.0.smithi002.stdout:5/711: creat d0/d16/dab/fdf x:0 0 0 2024-03-16T10:45:07.924 INFO:tasks.workunit.client.0.smithi002.stdout:5/712: chown d0/d16/d4a/d39 385 1 2024-03-16T10:45:07.927 INFO:tasks.workunit.client.0.smithi002.stdout:3/865: write d1/dd/d28/f33 [3072948,20039] 0 2024-03-16T10:45:07.928 INFO:tasks.workunit.client.0.smithi002.stdout:3/866: write d1/db6/d15/fa7 [724481,26271] 0 2024-03-16T10:45:07.928 INFO:tasks.workunit.client.1.smithi026.stdout:0/129: mkdir d1/d12/d2f 0 2024-03-16T10:45:07.930 INFO:tasks.workunit.client.1.smithi026.stdout:6/119: chown d0/d10/f1a 699 1 2024-03-16T10:45:07.935 INFO:tasks.workunit.client.1.smithi026.stdout:4/111: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:07.937 INFO:tasks.workunit.client.0.smithi002.stdout:1/892: rmdir d2/df/d12/d84/d87/d96/db5/d51/d56 39 2024-03-16T10:45:07.942 INFO:tasks.workunit.client.0.smithi002.stdout:8/918: rmdir d7/d9/d17/d23/d2e/d34/dbf 39 2024-03-16T10:45:07.943 INFO:tasks.workunit.client.1.smithi026.stdout:6/120: mkdir d0/d28/d2d 0 2024-03-16T10:45:07.947 INFO:tasks.workunit.client.0.smithi002.stdout:5/713: fsync d0/d16/dab/d2b/f78 0 2024-03-16T10:45:07.948 INFO:tasks.workunit.client.0.smithi002.stdout:2/766: truncate d7/d2e/d1d/d2f/d4b/d6e/fab 989449 0 2024-03-16T10:45:07.949 INFO:tasks.workunit.client.0.smithi002.stdout:9/885: mknod d5/d9/dc/d28/c119 0 2024-03-16T10:45:07.951 INFO:tasks.workunit.client.1.smithi026.stdout:4/112: dwrite d1/f22 [0,4194304] 0 2024-03-16T10:45:07.951 INFO:tasks.workunit.client.1.smithi026.stdout:6/121: dwrite d0/d10/d17/f25 [0,4194304] 0 2024-03-16T10:45:07.953 INFO:tasks.workunit.client.0.smithi002.stdout:6/859: mknod d15/dfd/c12c 0 2024-03-16T10:45:07.954 INFO:tasks.workunit.client.1.smithi026.stdout:5/73: sync 2024-03-16T10:45:07.956 INFO:tasks.workunit.client.0.smithi002.stdout:4/785: creat d2/d6/d32/d66/d7f/de7/d5e/d93/fed x:0 0 0 2024-03-16T10:45:07.957 INFO:tasks.workunit.client.0.smithi002.stdout:4/786: chown d2/d33/d1c/d45/faa 1819 1 2024-03-16T10:45:07.960 INFO:tasks.workunit.client.1.smithi026.stdout:6/122: read d0/d10/d17/f18 [239807,114921] 0 2024-03-16T10:45:07.961 INFO:tasks.workunit.client.1.smithi026.stdout:6/123: truncate d0/d10/f1a 875993 0 2024-03-16T10:45:07.963 INFO:tasks.workunit.client.0.smithi002.stdout:5/714: rmdir d0/d1f 39 2024-03-16T10:45:07.964 INFO:tasks.workunit.client.1.smithi026.stdout:5/74: dread - d1/f9 zero size 2024-03-16T10:45:07.964 INFO:tasks.workunit.client.1.smithi026.stdout:5/75: readlink d1/l11 0 2024-03-16T10:45:07.965 INFO:tasks.workunit.client.0.smithi002.stdout:2/767: fdatasync d7/db/d70/f98 0 2024-03-16T10:45:07.968 INFO:tasks.workunit.client.0.smithi002.stdout:0/834: getdents d3/d31/d75/db4/d109/d9c 0 2024-03-16T10:45:07.969 INFO:tasks.workunit.client.0.smithi002.stdout:0/835: readlink d3/d59/l5e 0 2024-03-16T10:45:07.969 INFO:tasks.workunit.client.1.smithi026.stdout:6/124: fdatasync d0/d10/f23 0 2024-03-16T10:45:07.969 INFO:tasks.workunit.client.0.smithi002.stdout:5/715: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:45:07.970 INFO:tasks.workunit.client.0.smithi002.stdout:0/836: chown d3/d28/d65/c66 3 1 2024-03-16T10:45:07.971 INFO:tasks.workunit.client.1.smithi026.stdout:6/125: write d0/d10/d17/f25 [2413482,74581] 0 2024-03-16T10:45:07.972 INFO:tasks.workunit.client.1.smithi026.stdout:4/113: truncate d1/db/fc 3616321 0 2024-03-16T10:45:07.975 INFO:tasks.workunit.client.0.smithi002.stdout:1/893: link d2/df/f108 d2/d58/de8/f11d 0 2024-03-16T10:45:07.976 INFO:tasks.workunit.client.1.smithi026.stdout:5/76: mknod d1/c19 0 2024-03-16T10:45:07.981 INFO:tasks.workunit.client.0.smithi002.stdout:7/725: sync 2024-03-16T10:45:07.982 INFO:tasks.workunit.client.0.smithi002.stdout:7/726: chown d3/d4/d1d/d87/d5c/d63/d55/fa7 2 1 2024-03-16T10:45:07.984 INFO:tasks.workunit.client.1.smithi026.stdout:5/77: dwrite d1/f13 [0,4194304] 0 2024-03-16T10:45:07.984 INFO:tasks.workunit.client.1.smithi026.stdout:4/114: link d1/f6 d1/f27 0 2024-03-16T10:45:07.986 INFO:tasks.workunit.client.0.smithi002.stdout:9/886: mknod d5/d9/d25/d23/c11a 0 2024-03-16T10:45:07.988 INFO:tasks.workunit.client.0.smithi002.stdout:9/887: write d5/d9/dc/d28/d8e/db5/ff8 [691755,97902] 0 2024-03-16T10:45:07.988 INFO:tasks.workunit.client.1.smithi026.stdout:4/115: dread d1/db/f11 [0,4194304] 0 2024-03-16T10:45:07.989 INFO:tasks.workunit.client.1.smithi026.stdout:5/78: mkdir d1/db/d1a 0 2024-03-16T10:45:07.989 INFO:tasks.workunit.client.0.smithi002.stdout:9/888: readlink d5/db1/dec/lfb 0 2024-03-16T10:45:07.992 INFO:tasks.workunit.client.0.smithi002.stdout:9/889: write d5/d9/dc/d1e/d91/f84 [563464,114095] 0 2024-03-16T10:45:07.993 INFO:tasks.workunit.client.1.smithi026.stdout:5/79: readlink l0 0 2024-03-16T10:45:07.993 INFO:tasks.workunit.client.0.smithi002.stdout:0/837: chown d3/d5/fd 249859 1 2024-03-16T10:45:07.993 INFO:tasks.workunit.client.1.smithi026.stdout:5/80: chown d1/db/l10 0 1 2024-03-16T10:45:07.994 INFO:tasks.workunit.client.0.smithi002.stdout:1/894: creat d2/df/d12/dd7/f11e x:0 0 0 2024-03-16T10:45:07.995 INFO:tasks.workunit.client.1.smithi026.stdout:4/116: rmdir d1/db 39 2024-03-16T10:45:08.000 INFO:tasks.workunit.client.1.smithi026.stdout:5/81: dwrite d1/db/fc [0,4194304] 0 2024-03-16T10:45:08.001 INFO:tasks.workunit.client.1.smithi026.stdout:4/117: chown d1/db/l15 0 1 2024-03-16T10:45:08.001 INFO:tasks.workunit.client.1.smithi026.stdout:4/118: rename d1/db/de to d1/db/de/d28 22 2024-03-16T10:45:08.002 INFO:tasks.workunit.client.1.smithi026.stdout:4/119: readlink d1/db/l17 0 2024-03-16T10:45:08.003 INFO:tasks.workunit.client.0.smithi002.stdout:3/867: rename d1/dd/d28/d7b/d2c/d63/f79 to d1/dd/d28/d7b/f131 0 2024-03-16T10:45:08.010 INFO:tasks.workunit.client.0.smithi002.stdout:9/890: symlink d5/d9/d55/l11b 0 2024-03-16T10:45:08.010 INFO:tasks.workunit.client.1.smithi026.stdout:4/120: dwrite d1/f3 [4194304,4194304] 0 2024-03-16T10:45:08.011 INFO:tasks.workunit.client.1.smithi026.stdout:5/82: creat d1/db/d1a/f1b x:0 0 0 2024-03-16T10:45:08.013 INFO:tasks.workunit.client.0.smithi002.stdout:0/838: creat d3/d31/d75/dde/f10d x:0 0 0 2024-03-16T10:45:08.014 INFO:tasks.workunit.client.0.smithi002.stdout:2/768: dwrite d7/f90 [0,4194304] 0 2024-03-16T10:45:08.018 INFO:tasks.workunit.client.1.smithi026.stdout:5/83: mkdir d1/d1c 0 2024-03-16T10:45:08.018 INFO:tasks.workunit.client.0.smithi002.stdout:7/727: truncate d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/fa1 2197721 0 2024-03-16T10:45:08.024 INFO:tasks.workunit.client.0.smithi002.stdout:7/728: fsync d3/d4/d1d/d87/d96/d5e/f84 0 2024-03-16T10:45:08.024 INFO:tasks.workunit.client.0.smithi002.stdout:3/868: mknod d1/dd/d28/d7b/d2c/d97/c132 0 2024-03-16T10:45:08.025 INFO:tasks.workunit.client.0.smithi002.stdout:1/895: symlink d2/df/d12/df6/ded/l11f 0 2024-03-16T10:45:08.025 INFO:tasks.workunit.client.0.smithi002.stdout:6/860: getdents d15/d2d/d73/d97 0 2024-03-16T10:45:08.026 INFO:tasks.workunit.client.1.smithi026.stdout:5/84: dwrite d1/db/fc [4194304,4194304] 0 2024-03-16T10:45:08.026 INFO:tasks.workunit.client.0.smithi002.stdout:0/839: creat d3/d31/d75/db4/ddc/f10e x:0 0 0 2024-03-16T10:45:08.027 INFO:tasks.workunit.client.0.smithi002.stdout:2/769: dwrite d7/d27/d3b/d95/da5/fbb [0,4194304] 0 2024-03-16T10:45:08.028 INFO:tasks.workunit.client.0.smithi002.stdout:6/861: dread - d15/d2d/d2c/f3b zero size 2024-03-16T10:45:08.031 INFO:tasks.workunit.client.0.smithi002.stdout:8/919: rename d7/d9/d17/d23/d2e/d34/cd6 to d7/d9/d17/d118/c135 0 2024-03-16T10:45:08.031 INFO:tasks.workunit.client.1.smithi026.stdout:5/85: symlink d1/db/l1d 0 2024-03-16T10:45:08.035 INFO:tasks.workunit.client.0.smithi002.stdout:5/716: getdents d0/d16/dab 0 2024-03-16T10:45:08.035 INFO:tasks.workunit.client.0.smithi002.stdout:3/869: unlink d1/fb 0 2024-03-16T10:45:08.035 INFO:tasks.workunit.client.0.smithi002.stdout:9/891: dread d5/d9/d25/d23/d81/fa1 [0,4194304] 0 2024-03-16T10:45:08.038 INFO:tasks.workunit.client.0.smithi002.stdout:0/840: mkdir d3/d59/dca/d10f 0 2024-03-16T10:45:08.045 INFO:tasks.workunit.client.0.smithi002.stdout:6/862: dwrite d15/dab/dc4/fd7 [0,4194304] 0 2024-03-16T10:45:08.046 INFO:tasks.workunit.client.0.smithi002.stdout:6/863: chown d15/d2d/lf0 0 1 2024-03-16T10:45:08.049 INFO:tasks.workunit.client.0.smithi002.stdout:6/864: write d15/f2f [3573164,13845] 0 2024-03-16T10:45:08.051 INFO:tasks.workunit.client.0.smithi002.stdout:6/865: dread d15/d2d/f85 [0,4194304] 0 2024-03-16T10:45:08.053 INFO:tasks.workunit.client.0.smithi002.stdout:6/866: write d15/d2d/f22 [777186,12110] 0 2024-03-16T10:45:08.056 INFO:tasks.workunit.client.0.smithi002.stdout:6/867: truncate d15/d2d/d2b/d3f/ff7 1196123 0 2024-03-16T10:45:08.057 INFO:tasks.workunit.client.0.smithi002.stdout:5/717: dread d0/d16/dab/d43/d48/f7f [0,4194304] 0 2024-03-16T10:45:08.093 INFO:tasks.workunit.client.1.smithi026.stdout:9/59: sync 2024-03-16T10:45:08.097 INFO:tasks.workunit.client.1.smithi026.stdout:9/60: dread d0/f9 [0,4194304] 0 2024-03-16T10:45:08.101 INFO:tasks.workunit.client.0.smithi002.stdout:1/896: creat d2/df/d12/d84/d87/d96/db5/de2/f120 x:0 0 0 2024-03-16T10:45:08.104 INFO:tasks.workunit.client.0.smithi002.stdout:7/729: getdents d3/d4/dde 0 2024-03-16T10:45:08.106 INFO:tasks.workunit.client.0.smithi002.stdout:7/730: chown d3/d4/d1d/d87/d5c/d63/l8e 29 1 2024-03-16T10:45:08.110 INFO:tasks.workunit.client.0.smithi002.stdout:0/841: rmdir d3/d28/d2a/dc7 39 2024-03-16T10:45:08.111 INFO:tasks.workunit.client.0.smithi002.stdout:0/842: dread - d3/df8/f100 zero size 2024-03-16T10:45:08.119 INFO:tasks.workunit.client.0.smithi002.stdout:6/868: creat d15/d2d/d2b/de4/f12d x:0 0 0 2024-03-16T10:45:08.120 INFO:tasks.workunit.client.0.smithi002.stdout:6/869: dread - d15/dfd/d96/db5/f10e zero size 2024-03-16T10:45:08.120 INFO:tasks.workunit.client.0.smithi002.stdout:5/718: readlink d0/d1f/l81 0 2024-03-16T10:45:08.121 INFO:tasks.workunit.client.0.smithi002.stdout:6/870: write d15/d2d/f22 [1547754,40189] 0 2024-03-16T10:45:08.124 INFO:tasks.workunit.client.0.smithi002.stdout:1/897: stat d2/df/d12/d84/d87/d96/db5/d51/d56/c104 0 2024-03-16T10:45:08.130 INFO:tasks.workunit.client.0.smithi002.stdout:9/892: creat d5/d9/dc/d28/d8e/dd0/f11c x:0 0 0 2024-03-16T10:45:08.131 INFO:tasks.workunit.client.0.smithi002.stdout:9/893: write d5/d9/dc/d1e/d94/fa3 [671975,18851] 0 2024-03-16T10:45:08.133 INFO:tasks.workunit.client.0.smithi002.stdout:5/719: symlink d0/d16/dab/d2b/d5d/le0 0 2024-03-16T10:45:08.133 INFO:tasks.workunit.client.0.smithi002.stdout:9/894: stat d5/d9/dc/d1e/d91/d82/d10b 0 2024-03-16T10:45:08.139 INFO:tasks.workunit.client.0.smithi002.stdout:5/720: creat d0/d24/fe1 x:0 0 0 2024-03-16T10:45:08.141 INFO:tasks.workunit.client.0.smithi002.stdout:5/721: dread d0/d16/d4a/d55/fd0 [0,4194304] 0 2024-03-16T10:45:08.142 INFO:tasks.workunit.client.0.smithi002.stdout:9/895: write d5/d9/dc/d28/dd6/fd8 [4768694,62810] 0 2024-03-16T10:45:08.144 INFO:tasks.workunit.client.0.smithi002.stdout:6/871: link d15/d2d/d2b/d3f/c59 d15/d2d/d8a/d7c/d87/df9/c12e 0 2024-03-16T10:45:08.148 INFO:tasks.workunit.client.0.smithi002.stdout:5/722: dwrite d0/d16/d4a/d7d/fc1 [0,4194304] 0 2024-03-16T10:45:08.148 INFO:tasks.workunit.client.0.smithi002.stdout:5/723: chown d0/d16 261 1 2024-03-16T10:45:08.150 INFO:tasks.workunit.client.0.smithi002.stdout:5/724: fdatasync d0/d24/dad/db6/db9/fd4 0 2024-03-16T10:45:08.157 INFO:tasks.workunit.client.0.smithi002.stdout:6/872: getdents d15/dab 0 2024-03-16T10:45:08.174 INFO:tasks.workunit.client.0.smithi002.stdout:6/873: dread d15/d2d/d8a/d7c/d87/fa4 [0,4194304] 0 2024-03-16T10:45:08.178 INFO:tasks.workunit.client.0.smithi002.stdout:6/874: creat d15/dfd/d96/dbc/f12f x:0 0 0 2024-03-16T10:45:08.181 INFO:tasks.workunit.client.0.smithi002.stdout:6/875: creat d15/dfd/d96/db5/dd2/f130 x:0 0 0 2024-03-16T10:45:08.187 INFO:tasks.workunit.client.0.smithi002.stdout:6/876: dwrite d15/d2d/f34 [0,4194304] 0 2024-03-16T10:45:08.188 INFO:tasks.workunit.client.0.smithi002.stdout:6/877: stat d15/d80 0 2024-03-16T10:45:08.198 INFO:tasks.workunit.client.0.smithi002.stdout:3/870: dwrite d1/dd/d28/d7b/dd5/dee/fff [0,4194304] 0 2024-03-16T10:45:08.201 INFO:tasks.workunit.client.0.smithi002.stdout:8/920: dwrite d7/d9/d17/d83/da2/fab [0,4194304] 0 2024-03-16T10:45:08.206 INFO:tasks.workunit.client.0.smithi002.stdout:2/770: mkdir d7/d27/d3f/df1 0 2024-03-16T10:45:08.211 INFO:tasks.workunit.client.0.smithi002.stdout:2/771: mkdir d7/d2e/d1d/d2f/df2 0 2024-03-16T10:45:08.213 INFO:tasks.workunit.client.0.smithi002.stdout:7/731: dwrite d3/d37/f48 [0,4194304] 0 2024-03-16T10:45:08.216 INFO:tasks.workunit.client.0.smithi002.stdout:0/843: creat d3/d31/d75/db4/d109/f110 x:0 0 0 2024-03-16T10:45:08.217 INFO:tasks.workunit.client.0.smithi002.stdout:9/896: write d5/d9/dc/d1e/d24/d31/f4a [943298,16915] 0 2024-03-16T10:45:08.218 INFO:tasks.workunit.client.0.smithi002.stdout:8/921: mkdir d7/d9/d136 0 2024-03-16T10:45:08.220 INFO:tasks.workunit.client.0.smithi002.stdout:8/922: readlink d7/d13/db8/de0/l127 0 2024-03-16T10:45:08.222 INFO:tasks.workunit.client.0.smithi002.stdout:9/897: getdents d5 0 2024-03-16T10:45:08.228 INFO:tasks.workunit.client.0.smithi002.stdout:9/898: link d5/d9/d34/f99 d5/d9/dc/d28/dd6/f11d 0 2024-03-16T10:45:08.230 INFO:tasks.workunit.client.0.smithi002.stdout:2/772: dread d7/d2e/d1d/d2f/f9a [0,4194304] 0 2024-03-16T10:45:08.231 INFO:tasks.workunit.client.0.smithi002.stdout:2/773: write d7/d2e/d28/d40/fa3 [3898402,33535] 0 2024-03-16T10:45:08.232 INFO:tasks.workunit.client.0.smithi002.stdout:8/923: unlink d7/d9/d17/d23/d2e/d34/c54 0 2024-03-16T10:45:08.235 INFO:tasks.workunit.client.0.smithi002.stdout:2/774: unlink d7/d2e/d1d/d38/d49/db5/lc0 0 2024-03-16T10:45:08.238 INFO:tasks.workunit.client.0.smithi002.stdout:4/787: rename d2/d6/d32/d68/c61 to d2/d6/d32/d66/cee 0 2024-03-16T10:45:08.242 INFO:tasks.workunit.client.0.smithi002.stdout:1/898: rename d2/df/d12/d84/d87/d96/db5/d1c/fba to d2/df/d12/dd7/d11b/f121 0 2024-03-16T10:45:08.246 INFO:tasks.workunit.client.0.smithi002.stdout:8/924: dwrite d7/d9/d17/d23/d2e/d34/dbf/dd9/f113 [0,4194304] 0 2024-03-16T10:45:08.248 INFO:tasks.workunit.client.0.smithi002.stdout:8/925: dread - d7/d9/d17/dc3/f10e zero size 2024-03-16T10:45:08.251 INFO:tasks.workunit.client.0.smithi002.stdout:1/899: symlink d2/df/d12/l122 0 2024-03-16T10:45:08.252 INFO:tasks.workunit.client.0.smithi002.stdout:4/788: dwrite d2/d6/d32/d66/d7f/fb5 [0,4194304] 0 2024-03-16T10:45:08.254 INFO:tasks.workunit.client.0.smithi002.stdout:5/725: rename d0/d16/cdc to d0/d16/dab/d33/ce2 0 2024-03-16T10:45:08.258 INFO:tasks.workunit.client.0.smithi002.stdout:6/878: rename d15/d2d/d2b/d3f/d77/d7e/f83 to d15/d2d/d8a/d7c/d87/df9/f131 0 2024-03-16T10:45:08.260 INFO:tasks.workunit.client.0.smithi002.stdout:5/726: unlink d0/d1f/da1/fd8 0 2024-03-16T10:45:08.262 INFO:tasks.workunit.client.0.smithi002.stdout:4/789: rmdir d2/d6/d32/d66/d7f/de7/d88/db2 39 2024-03-16T10:45:08.264 INFO:tasks.workunit.client.0.smithi002.stdout:5/727: write d0/d16/dab/daa/faf [302475,101308] 0 2024-03-16T10:45:08.267 INFO:tasks.workunit.client.0.smithi002.stdout:1/900: creat d2/df/d12/d84/d87/d96/db5/d1c/da0/d119/f123 x:0 0 0 2024-03-16T10:45:08.267 INFO:tasks.workunit.client.0.smithi002.stdout:3/871: dwrite d1/dd/d28/d7b/d2c/f81 [0,4194304] 0 2024-03-16T10:45:08.269 INFO:tasks.workunit.client.0.smithi002.stdout:7/732: rename d3/d4/d1d/lbe to d3/d4/d1c/d5b/l107 0 2024-03-16T10:45:08.272 INFO:tasks.workunit.client.0.smithi002.stdout:5/728: write d0/d16/dab/d33/d4b/fd3 [358730,62912] 0 2024-03-16T10:45:08.273 INFO:tasks.workunit.client.0.smithi002.stdout:7/733: chown d3/d4/d1c/d5b/d64/ddf 2866889 1 2024-03-16T10:45:08.274 INFO:tasks.workunit.client.0.smithi002.stdout:1/901: write d2/df/d12/d84/fd5 [15725,9284] 0 2024-03-16T10:45:08.276 INFO:tasks.workunit.client.0.smithi002.stdout:0/844: rename d3/d31/d75/db4/d109/f110 to d3/da0/f111 0 2024-03-16T10:45:08.276 INFO:tasks.workunit.client.0.smithi002.stdout:8/926: dwrite d7/d9/f51 [0,4194304] 0 2024-03-16T10:45:08.276 INFO:tasks.workunit.client.0.smithi002.stdout:5/729: write d0/fc3 [1175353,128851] 0 2024-03-16T10:45:08.278 INFO:tasks.workunit.client.0.smithi002.stdout:6/879: write d15/d2d/f9b [3315750,100205] 0 2024-03-16T10:45:08.279 INFO:tasks.workunit.client.0.smithi002.stdout:3/872: rename d1 to d1/dd/d28/d37/d133 22 2024-03-16T10:45:08.280 INFO:tasks.workunit.client.0.smithi002.stdout:8/927: mknod d7/de3/c137 0 2024-03-16T10:45:08.281 INFO:tasks.workunit.client.0.smithi002.stdout:3/873: fsync d1/db6/f7f 0 2024-03-16T10:45:08.282 INFO:tasks.workunit.client.0.smithi002.stdout:1/902: creat d2/d58/dbb/f124 x:0 0 0 2024-03-16T10:45:08.283 INFO:tasks.workunit.client.1.smithi026.stdout:1/84: sync 2024-03-16T10:45:08.284 INFO:tasks.workunit.client.1.smithi026.stdout:1/85: chown d0/d4/c15 135607091 1 2024-03-16T10:45:08.285 INFO:tasks.workunit.client.1.smithi026.stdout:1/86: stat d0/d4/l9 0 2024-03-16T10:45:08.287 INFO:tasks.workunit.client.1.smithi026.stdout:1/87: rmdir d0/d4 39 2024-03-16T10:45:08.290 INFO:tasks.workunit.client.0.smithi002.stdout:6/880: mkdir d15/d2d/d2b/d3f/d77/de1/d132 0 2024-03-16T10:45:08.292 INFO:tasks.workunit.client.0.smithi002.stdout:4/790: dwrite d2/d6/d32/d66/d7f/de7/ff [0,4194304] 0 2024-03-16T10:45:08.293 INFO:tasks.workunit.client.1.smithi026.stdout:1/88: rename d0/d4/f18 to d0/d17/f1a 0 2024-03-16T10:45:08.294 INFO:tasks.workunit.client.0.smithi002.stdout:3/874: creat d1/dd/d28/d7b/dd5/dee/f134 x:0 0 0 2024-03-16T10:45:08.294 INFO:tasks.workunit.client.0.smithi002.stdout:7/734: creat d3/d4/d1d/d87/d5c/dc3/df7/f108 x:0 0 0 2024-03-16T10:45:08.294 INFO:tasks.workunit.client.0.smithi002.stdout:6/881: dread - d15/d2d/d73/d97/fd1 zero size 2024-03-16T10:45:08.297 INFO:tasks.workunit.client.0.smithi002.stdout:0/845: link d3/d31/d75/db4/d109/d9c/fb9 d3/d28/d65/f112 0 2024-03-16T10:45:08.297 INFO:tasks.workunit.client.1.smithi026.stdout:1/89: rmdir d0 39 2024-03-16T10:45:08.298 INFO:tasks.workunit.client.0.smithi002.stdout:1/903: creat d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/dee/f125 x:0 0 0 2024-03-16T10:45:08.300 INFO:tasks.workunit.client.0.smithi002.stdout:6/882: read d15/d2d/d2c/d5d/f72 [409489,5810] 0 2024-03-16T10:45:08.300 INFO:tasks.workunit.client.1.smithi026.stdout:1/90: rmdir d0 39 2024-03-16T10:45:08.300 INFO:tasks.workunit.client.0.smithi002.stdout:5/730: dread d0/d24/dad/fae [0,4194304] 0 2024-03-16T10:45:08.301 INFO:tasks.workunit.client.0.smithi002.stdout:4/791: read d2/d6/f2d [648186,25588] 0 2024-03-16T10:45:08.302 INFO:tasks.workunit.client.1.smithi026.stdout:1/91: mknod d0/d4/c1b 0 2024-03-16T10:45:08.303 INFO:tasks.workunit.client.0.smithi002.stdout:0/846: chown d3/d28/d2a/d67/c86 2192225 1 2024-03-16T10:45:08.304 INFO:tasks.workunit.client.1.smithi026.stdout:1/92: mkdir d0/d4/d1c 0 2024-03-16T10:45:08.306 INFO:tasks.workunit.client.0.smithi002.stdout:3/875: truncate d1/dd/d28/d7b/d2c/d63/f88 2053723 0 2024-03-16T10:45:08.310 INFO:tasks.workunit.client.0.smithi002.stdout:8/928: dwrite d7/d9/ffa [0,4194304] 0 2024-03-16T10:45:08.310 INFO:tasks.workunit.client.0.smithi002.stdout:4/792: creat d2/d33/d35/fef x:0 0 0 2024-03-16T10:45:08.311 INFO:tasks.workunit.client.0.smithi002.stdout:7/735: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f89 [0,4194304] 0 2024-03-16T10:45:08.312 INFO:tasks.workunit.client.1.smithi026.stdout:1/93: dwrite d0/f11 [0,4194304] 0 2024-03-16T10:45:08.313 INFO:tasks.workunit.client.1.smithi026.stdout:1/94: dread - d0/d17/f1a zero size 2024-03-16T10:45:08.314 INFO:tasks.workunit.client.0.smithi002.stdout:0/847: creat d3/d28/d3d/f113 x:0 0 0 2024-03-16T10:45:08.314 INFO:tasks.workunit.client.0.smithi002.stdout:3/876: mkdir d1/dd/d28/d7b/d2c/d97/df5/d135 0 2024-03-16T10:45:08.315 INFO:tasks.workunit.client.1.smithi026.stdout:1/95: truncate d0/d17/f1a 378361 0 2024-03-16T10:45:08.321 INFO:tasks.workunit.client.1.smithi026.stdout:1/96: getdents d0/d4/d1c 0 2024-03-16T10:45:08.323 INFO:tasks.workunit.client.0.smithi002.stdout:7/736: dread d3/d4/d1d/d87/d96/d5e/f60 [0,4194304] 0 2024-03-16T10:45:08.325 INFO:tasks.workunit.client.0.smithi002.stdout:7/737: truncate d3/d4/d1d/d87/d96/d8c/f95 514261 0 2024-03-16T10:45:08.326 INFO:tasks.workunit.client.0.smithi002.stdout:7/738: chown d3/d4/d1c/d5b/d64/d99/d9c/ffb 3591 1 2024-03-16T10:45:08.327 INFO:tasks.workunit.client.1.smithi026.stdout:1/97: rename d0/l13 to d0/l1d 0 2024-03-16T10:45:08.331 INFO:tasks.workunit.client.1.smithi026.stdout:1/98: creat d0/f1e x:0 0 0 2024-03-16T10:45:08.338 INFO:tasks.workunit.client.0.smithi002.stdout:0/848: chown d3/d5/c17 85197 1 2024-03-16T10:45:08.340 INFO:tasks.workunit.client.0.smithi002.stdout:7/739: stat d3/d4/d1d/d87/l59 0 2024-03-16T10:45:08.340 INFO:tasks.workunit.client.0.smithi002.stdout:0/849: fsync d3/de/f8f 0 2024-03-16T10:45:08.340 INFO:tasks.workunit.client.0.smithi002.stdout:3/877: getdents d1/dd/d28/d7b/d2c/d63/d115 0 2024-03-16T10:45:08.341 INFO:tasks.workunit.client.0.smithi002.stdout:3/878: readlink d1/dd/d28/d7b/d22/l46 0 2024-03-16T10:45:08.343 INFO:tasks.workunit.client.0.smithi002.stdout:4/793: link d2/d6/d32/d66/d7f/cbc d2/d6/d32/d66/d7f/de7/d2e/cf0 0 2024-03-16T10:45:08.343 INFO:tasks.workunit.client.0.smithi002.stdout:3/879: creat d1/db6/d15/d87/d11e/f136 x:0 0 0 2024-03-16T10:45:08.347 INFO:tasks.workunit.client.0.smithi002.stdout:0/850: dwrite d3/fc [0,4194304] 0 2024-03-16T10:45:08.350 INFO:tasks.workunit.client.0.smithi002.stdout:0/851: chown d3/l4 110799010 1 2024-03-16T10:45:08.354 INFO:tasks.workunit.client.0.smithi002.stdout:0/852: fdatasync d3/d31/d88/f8b 0 2024-03-16T10:45:08.361 INFO:tasks.workunit.client.0.smithi002.stdout:3/880: dread d1/dd/d28/d7b/d8a/d93/f47 [4194304,4194304] 0 2024-03-16T10:45:08.361 INFO:tasks.workunit.client.0.smithi002.stdout:0/853: dwrite d3/d28/f8e [0,4194304] 0 2024-03-16T10:45:08.370 INFO:tasks.workunit.client.0.smithi002.stdout:3/881: dwrite d1/dd/d28/d7b/f7c [0,4194304] 0 2024-03-16T10:45:08.376 INFO:tasks.workunit.client.0.smithi002.stdout:3/882: dread d1/db6/d15/fa7 [0,4194304] 0 2024-03-16T10:45:08.398 INFO:tasks.workunit.client.1.smithi026.stdout:2/112: sync 2024-03-16T10:45:08.405 INFO:tasks.workunit.client.1.smithi026.stdout:1/99: fdatasync d0/f11 0 2024-03-16T10:45:08.410 INFO:tasks.workunit.client.0.smithi002.stdout:1/904: write d2/d58/de8/f11d [372191,91095] 0 2024-03-16T10:45:08.410 INFO:tasks.workunit.client.0.smithi002.stdout:6/883: write fa [1367625,108544] 0 2024-03-16T10:45:08.412 INFO:tasks.workunit.client.0.smithi002.stdout:1/905: chown d2/df/d12/d84/d87/d96/db5/d51/d56/c104 0 1 2024-03-16T10:45:08.415 INFO:tasks.workunit.client.1.smithi026.stdout:1/100: dwrite d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:08.417 INFO:tasks.workunit.client.0.smithi002.stdout:1/906: symlink d2/df/d12/d84/l126 0 2024-03-16T10:45:08.419 INFO:tasks.workunit.client.1.smithi026.stdout:1/101: rename d0/d4 to d0/d17/d1f 0 2024-03-16T10:45:08.420 INFO:tasks.workunit.client.0.smithi002.stdout:9/899: sync 2024-03-16T10:45:08.420 INFO:tasks.workunit.client.0.smithi002.stdout:2/775: sync 2024-03-16T10:45:08.424 INFO:tasks.workunit.client.0.smithi002.stdout:2/776: rmdir d7/d27/d3b 39 2024-03-16T10:45:08.425 INFO:tasks.workunit.client.0.smithi002.stdout:9/900: symlink d5/d9/dc/d1e/d94/l11e 0 2024-03-16T10:45:08.427 INFO:tasks.workunit.client.0.smithi002.stdout:2/777: unlink d7/db/d70/f77 0 2024-03-16T10:45:08.427 INFO:tasks.workunit.client.1.smithi026.stdout:1/102: dwrite d0/d17/d1f/fc [0,4194304] 0 2024-03-16T10:45:08.430 INFO:tasks.workunit.client.0.smithi002.stdout:9/901: rename d5/d9/dc/d1e/def to d5/d9/d55/dc9/d11f 0 2024-03-16T10:45:08.431 INFO:tasks.workunit.client.0.smithi002.stdout:2/778: write d7/d2e/d28/d40/fa3 [4931233,62276] 0 2024-03-16T10:45:08.431 INFO:tasks.workunit.client.0.smithi002.stdout:6/884: dread d15/d2d/f2e [0,4194304] 0 2024-03-16T10:45:08.431 INFO:tasks.workunit.client.1.smithi026.stdout:1/103: mkdir d0/d17/d1f/d20 0 2024-03-16T10:45:08.432 INFO:tasks.workunit.client.0.smithi002.stdout:2/779: chown d7/d2e/d28/fd7 7978 1 2024-03-16T10:45:08.435 INFO:tasks.workunit.client.1.smithi026.stdout:1/104: dread d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:08.437 INFO:tasks.workunit.client.0.smithi002.stdout:1/907: dread d2/df/d12/f37 [4194304,4194304] 0 2024-03-16T10:45:08.438 INFO:tasks.workunit.client.0.smithi002.stdout:1/908: chown d2/df/c6f 563 1 2024-03-16T10:45:08.439 INFO:tasks.workunit.client.0.smithi002.stdout:1/909: chown d2/df/d12/d84/d87/d96/da8 273280 1 2024-03-16T10:45:08.439 INFO:tasks.workunit.client.1.smithi026.stdout:1/105: dread d0/f11 [0,4194304] 0 2024-03-16T10:45:08.439 INFO:tasks.workunit.client.0.smithi002.stdout:6/885: creat d15/d2d/d2b/d3f/d77/d112/f133 x:0 0 0 2024-03-16T10:45:08.441 INFO:tasks.workunit.client.0.smithi002.stdout:9/902: getdents d5/d9/d22/d3c/d118 0 2024-03-16T10:45:08.444 INFO:tasks.workunit.client.0.smithi002.stdout:9/903: creat d5/d9/dc/d1e/d94/f120 x:0 0 0 2024-03-16T10:45:08.445 INFO:tasks.workunit.client.0.smithi002.stdout:9/904: chown d5/d9/d25/l21 2133019 1 2024-03-16T10:45:08.446 INFO:tasks.workunit.client.0.smithi002.stdout:6/886: rename d15/d80/f84 to d15/d2d/d2b/d3f/d77/f134 0 2024-03-16T10:45:08.449 INFO:tasks.workunit.client.0.smithi002.stdout:1/910: rename d2/df/d12/d84/d87/d96/db5/d51/d56/ffe to d2/df/d12/d84/d87/d96/db5/de2/d10a/f127 0 2024-03-16T10:45:08.449 INFO:tasks.workunit.client.0.smithi002.stdout:1/911: readlink d2/df/d12/df6/le5 0 2024-03-16T10:45:08.450 INFO:tasks.workunit.client.0.smithi002.stdout:6/887: chown d15/d2d/d8a/d7c/d87/l104 7947 1 2024-03-16T10:45:08.451 INFO:tasks.workunit.client.0.smithi002.stdout:8/929: write d7/f31 [2367480,98293] 0 2024-03-16T10:45:08.452 INFO:tasks.workunit.client.0.smithi002.stdout:9/905: creat d5/d9/d34/f121 x:0 0 0 2024-03-16T10:45:08.456 INFO:tasks.workunit.client.0.smithi002.stdout:6/888: rename d15/d2d/d2c/d5d/c62 to d15/d80/c135 0 2024-03-16T10:45:08.457 INFO:tasks.workunit.client.0.smithi002.stdout:9/906: symlink d5/db1/l122 0 2024-03-16T10:45:08.460 INFO:tasks.workunit.client.0.smithi002.stdout:6/889: unlink d15/d2d/d2b/d3f/f116 0 2024-03-16T10:45:08.461 INFO:tasks.workunit.client.0.smithi002.stdout:1/912: dwrite d2/df/d12/d84/d87/d96/db5/fd4 [0,4194304] 0 2024-03-16T10:45:08.462 INFO:tasks.workunit.client.0.smithi002.stdout:5/731: sync 2024-03-16T10:45:08.464 INFO:tasks.workunit.client.0.smithi002.stdout:9/907: symlink d5/d9/d55/dc9/d11f/l123 0 2024-03-16T10:45:08.464 INFO:tasks.workunit.client.0.smithi002.stdout:7/740: dwrite d3/d4/d1d/d87/d96/d8c/f9b [0,4194304] 0 2024-03-16T10:45:08.465 INFO:tasks.workunit.client.0.smithi002.stdout:3/883: write d1/db6/d15/f2e [2134947,122507] 0 2024-03-16T10:45:08.472 INFO:tasks.workunit.client.0.smithi002.stdout:9/908: unlink d5/d9/c76 0 2024-03-16T10:45:08.476 INFO:tasks.workunit.client.0.smithi002.stdout:7/741: dread - d3/d4/d1d/d87/d5c/dc3/df7/f108 zero size 2024-03-16T10:45:08.477 INFO:tasks.workunit.client.0.smithi002.stdout:1/913: truncate d2/df/d12/dd7/fe4 49198 0 2024-03-16T10:45:08.477 INFO:tasks.workunit.client.0.smithi002.stdout:4/794: dwrite d2/f77 [0,4194304] 0 2024-03-16T10:45:08.477 INFO:tasks.workunit.client.0.smithi002.stdout:3/884: rmdir d1/db6/d15/d87/d8c 39 2024-03-16T10:45:08.479 INFO:tasks.workunit.client.0.smithi002.stdout:0/854: dwrite d3/df8/f100 [0,4194304] 0 2024-03-16T10:45:08.479 INFO:tasks.workunit.client.0.smithi002.stdout:6/890: dread d15/d2d/d2b/d3f/ff7 [0,4194304] 0 2024-03-16T10:45:08.484 INFO:tasks.workunit.client.0.smithi002.stdout:2/780: write d7/d2e/d28/d44/f4d [1209516,70326] 0 2024-03-16T10:45:08.485 INFO:tasks.workunit.client.0.smithi002.stdout:3/885: chown d1/dd/d28/d7b/d8a/d93 575469459 1 2024-03-16T10:45:08.489 INFO:tasks.workunit.client.0.smithi002.stdout:8/930: truncate d7/d13/db8/f7c 880386 0 2024-03-16T10:45:08.489 INFO:tasks.workunit.client.0.smithi002.stdout:5/732: dwrite d0/d16/dab/d43/d48/fd9 [0,4194304] 0 2024-03-16T10:45:08.492 INFO:tasks.workunit.client.0.smithi002.stdout:7/742: mkdir d3/d4/d1d/d87/d5c/dd0/d109 0 2024-03-16T10:45:08.493 INFO:tasks.workunit.client.0.smithi002.stdout:6/891: rmdir d15/d2d/d2b 39 2024-03-16T10:45:08.494 INFO:tasks.workunit.client.0.smithi002.stdout:5/733: dread d0/d16/dab/f85 [0,4194304] 0 2024-03-16T10:45:08.496 INFO:tasks.workunit.client.0.smithi002.stdout:5/734: dread - d0/d16/dab/d33/d8c/f95 zero size 2024-03-16T10:45:08.500 INFO:tasks.workunit.client.0.smithi002.stdout:9/909: mknod d5/d9/dc/d1e/d91/d82/d10b/df0/c124 0 2024-03-16T10:45:08.502 INFO:tasks.workunit.client.0.smithi002.stdout:3/886: symlink d1/dd/d28/d7b/d22/l137 0 2024-03-16T10:45:08.502 INFO:tasks.workunit.client.0.smithi002.stdout:0/855: write d3/d28/d2a/f4f [506808,1052] 0 2024-03-16T10:45:08.512 INFO:tasks.workunit.client.0.smithi002.stdout:2/781: write d7/d27/d3b/f94 [8608489,77553] 0 2024-03-16T10:45:08.513 INFO:tasks.workunit.client.0.smithi002.stdout:6/892: truncate d15/d2d/f9b 4421669 0 2024-03-16T10:45:08.517 INFO:tasks.workunit.client.0.smithi002.stdout:1/914: dwrite d2/df/d12/d84/d87/d96/db5/d51/d56/fc5 [0,4194304] 0 2024-03-16T10:45:08.517 INFO:tasks.workunit.client.0.smithi002.stdout:8/931: rmdir d7/d9/d17/d23/d2e/d34/dbf/dd9/dea/d125 0 2024-03-16T10:45:08.521 INFO:tasks.workunit.client.0.smithi002.stdout:4/795: dwrite d2/d33/d1c/d45/fea [0,4194304] 0 2024-03-16T10:45:08.522 INFO:tasks.workunit.client.0.smithi002.stdout:5/735: dwrite d0/d16/dab/d2b/d94/fbf [0,4194304] 0 2024-03-16T10:45:08.522 INFO:tasks.workunit.client.0.smithi002.stdout:3/887: write d1/dd/f10 [2081835,22641] 0 2024-03-16T10:45:08.529 INFO:tasks.workunit.client.0.smithi002.stdout:1/915: creat d2/df/d12/d84/d87/d96/db5/dae/f128 x:0 0 0 2024-03-16T10:45:08.529 INFO:tasks.workunit.client.0.smithi002.stdout:9/910: getdents d5/d9/dc 0 2024-03-16T10:45:08.530 INFO:tasks.workunit.client.0.smithi002.stdout:5/736: read - d0/d16/d4a/d7d/fc5 zero size 2024-03-16T10:45:08.530 INFO:tasks.workunit.client.0.smithi002.stdout:6/893: write d15/dfd/d96/dbb/f11e [965013,19318] 0 2024-03-16T10:45:08.532 INFO:tasks.workunit.client.0.smithi002.stdout:0/856: dread d3/d28/d2a/d67/fdd [0,4194304] 0 2024-03-16T10:45:08.532 INFO:tasks.workunit.client.0.smithi002.stdout:2/782: truncate d7/d2e/d1d/d38/fa4 54072 0 2024-03-16T10:45:08.533 INFO:tasks.workunit.client.0.smithi002.stdout:4/796: write d2/d33/d35/fdf [1870962,92458] 0 2024-03-16T10:45:08.534 INFO:tasks.workunit.client.0.smithi002.stdout:1/916: fdatasync d2/df/d12/d29/f34 0 2024-03-16T10:45:08.536 INFO:tasks.workunit.client.0.smithi002.stdout:8/932: mknod d7/d9/d17/d23/d39/db7/c138 0 2024-03-16T10:45:08.537 INFO:tasks.workunit.client.0.smithi002.stdout:9/911: fdatasync d5/d9/dc/d1e/d94/f10f 0 2024-03-16T10:45:08.546 INFO:tasks.workunit.client.0.smithi002.stdout:0/857: mkdir d3/d5/d114 0 2024-03-16T10:45:08.546 INFO:tasks.workunit.client.0.smithi002.stdout:3/888: write d1/dd/d28/d7b/d2c/d97/df5/f11a [389134,76502] 0 2024-03-16T10:45:08.547 INFO:tasks.workunit.client.0.smithi002.stdout:9/912: creat d5/d9/d22/f125 x:0 0 0 2024-03-16T10:45:08.547 INFO:tasks.workunit.client.1.smithi026.stdout:3/58: sync 2024-03-16T10:45:08.547 INFO:tasks.workunit.client.1.smithi026.stdout:8/64: sync 2024-03-16T10:45:08.547 INFO:tasks.workunit.client.0.smithi002.stdout:5/737: creat d0/d16/d5b/da0/dc4/fe3 x:0 0 0 2024-03-16T10:45:08.548 INFO:tasks.workunit.client.0.smithi002.stdout:6/894: symlink d15/d2d/d2b/d3f/d77/d7e/l136 0 2024-03-16T10:45:08.551 INFO:tasks.workunit.client.0.smithi002.stdout:1/917: symlink d2/df/d12/d84/d87/d96/d105/l129 0 2024-03-16T10:45:08.553 INFO:tasks.workunit.client.0.smithi002.stdout:8/933: truncate d7/d9/d17/d83/df5/f10c 250505 0 2024-03-16T10:45:08.554 INFO:tasks.workunit.client.0.smithi002.stdout:2/783: dread d7/d27/d3b/f94 [0,4194304] 0 2024-03-16T10:45:08.555 INFO:tasks.workunit.client.1.smithi026.stdout:3/59: dread d4/f5 [0,4194304] 0 2024-03-16T10:45:08.555 INFO:tasks.workunit.client.0.smithi002.stdout:3/889: unlink d1/dd/d28/d7b/dd5/ldc 0 2024-03-16T10:45:08.557 INFO:tasks.workunit.client.0.smithi002.stdout:9/913: mknod d5/d9/dc/d28/dd6/c126 0 2024-03-16T10:45:08.559 INFO:tasks.workunit.client.0.smithi002.stdout:5/738: mknod d0/d16/dab/daa/ce4 0 2024-03-16T10:45:08.561 INFO:tasks.workunit.client.0.smithi002.stdout:6/895: creat d15/d2d/d2b/d3f/d77/de1/f137 x:0 0 0 2024-03-16T10:45:08.563 INFO:tasks.workunit.client.1.smithi026.stdout:3/60: dwrite d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:08.563 INFO:tasks.workunit.client.0.smithi002.stdout:1/918: rename d2/df/d12/d84/d87/d96/db5/d1c/c40 to d2/df/d12/d74/c12a 0 2024-03-16T10:45:08.564 INFO:tasks.workunit.client.1.smithi026.stdout:8/65: dwrite d5/fb [0,4194304] 0 2024-03-16T10:45:08.564 INFO:tasks.workunit.client.0.smithi002.stdout:0/858: symlink d3/l115 0 2024-03-16T10:45:08.565 INFO:tasks.workunit.client.1.smithi026.stdout:3/61: write d4/dc/dd/f12 [4500143,63613] 0 2024-03-16T10:45:08.566 INFO:tasks.workunit.client.1.smithi026.stdout:8/66: dread d5/d7/f10 [0,4194304] 0 2024-03-16T10:45:08.567 INFO:tasks.workunit.client.0.smithi002.stdout:8/934: chown d7/d9/d17/d23/d2e/d34/dbf/l114 432303 1 2024-03-16T10:45:08.567 INFO:tasks.workunit.client.1.smithi026.stdout:3/62: truncate d4/dc/f19 323082 0 2024-03-16T10:45:08.568 INFO:tasks.workunit.client.0.smithi002.stdout:8/935: chown d7/d9/d25/l7e 0 1 2024-03-16T10:45:08.570 INFO:tasks.workunit.client.0.smithi002.stdout:8/936: truncate d7/d13/db8/ff2 963454 0 2024-03-16T10:45:08.572 INFO:tasks.workunit.client.0.smithi002.stdout:9/914: mkdir d5/d9/dc/d1e/d91/d82/db4/d127 0 2024-03-16T10:45:08.572 INFO:tasks.workunit.client.0.smithi002.stdout:8/937: chown d7/d9/d17/d23/d39/db7/fd8 111305 1 2024-03-16T10:45:08.573 INFO:tasks.workunit.client.0.smithi002.stdout:5/739: readlink d0/l84 0 2024-03-16T10:45:08.577 INFO:tasks.workunit.client.0.smithi002.stdout:8/938: chown d7/d9/d17/d23/d2e/d3a/ce6 261 1 2024-03-16T10:45:08.577 INFO:tasks.workunit.client.0.smithi002.stdout:8/939: write d7/d13/db8/f116 [870010,97802] 0 2024-03-16T10:45:08.578 INFO:tasks.workunit.client.0.smithi002.stdout:2/784: mkdir d7/d2e/d1d/d38/df3 0 2024-03-16T10:45:08.579 INFO:tasks.workunit.client.0.smithi002.stdout:8/940: dread - d7/d9/d17/dc3/f10e zero size 2024-03-16T10:45:08.580 INFO:tasks.workunit.client.0.smithi002.stdout:5/740: fdatasync d0/d16/dab/d43/d48/f8e 0 2024-03-16T10:45:08.581 INFO:tasks.workunit.client.0.smithi002.stdout:9/915: mknod d5/d9/d22/c128 0 2024-03-16T10:45:08.581 INFO:tasks.workunit.client.0.smithi002.stdout:2/785: truncate d7/d2e/d28/fd7 647875 0 2024-03-16T10:45:08.582 INFO:tasks.workunit.client.0.smithi002.stdout:4/797: getdents d2/d6/d32/d66/d7f/de7/d88/db2 0 2024-03-16T10:45:08.583 INFO:tasks.workunit.client.0.smithi002.stdout:6/896: getdents d15/dab/d11c/d12a 0 2024-03-16T10:45:08.585 INFO:tasks.workunit.client.0.smithi002.stdout:1/919: getdents d2/df/d12/d84/d87/d96/db5/d51/d56/d11c 0 2024-03-16T10:45:08.585 INFO:tasks.workunit.client.1.smithi026.stdout:8/67: fdatasync d5/fb 0 2024-03-16T10:45:08.586 INFO:tasks.workunit.client.1.smithi026.stdout:8/68: read d5/d7/f10 [155644,14186] 0 2024-03-16T10:45:08.589 INFO:tasks.workunit.client.1.smithi026.stdout:8/69: creat d5/dc/f15 x:0 0 0 2024-03-16T10:45:08.590 INFO:tasks.workunit.client.0.smithi002.stdout:8/941: dwrite d7/d9/d25/d67/fc0 [0,4194304] 0 2024-03-16T10:45:08.593 INFO:tasks.workunit.client.0.smithi002.stdout:3/890: link d1/dd/d28/d37/l10d d1/dd/d28/d7b/d8a/d93/l138 0 2024-03-16T10:45:08.593 INFO:tasks.workunit.client.0.smithi002.stdout:4/798: dwrite d2/d33/d35/f85 [0,4194304] 0 2024-03-16T10:45:08.593 INFO:tasks.workunit.client.1.smithi026.stdout:3/63: dwrite d4/dc/f11 [4194304,4194304] 0 2024-03-16T10:45:08.595 INFO:tasks.workunit.client.0.smithi002.stdout:5/741: creat d0/d16/d5b/da0/dd5/fe5 x:0 0 0 2024-03-16T10:45:08.598 INFO:tasks.workunit.client.1.smithi026.stdout:3/64: dread - d4/dc/f15 zero size 2024-03-16T10:45:08.598 INFO:tasks.workunit.client.1.smithi026.stdout:8/70: dwrite d5/d7/f11 [0,4194304] 0 2024-03-16T10:45:08.603 INFO:tasks.workunit.client.0.smithi002.stdout:2/786: creat d7/d2e/d28/d40/d50/db4/dcd/ff4 x:0 0 0 2024-03-16T10:45:08.603 INFO:tasks.workunit.client.1.smithi026.stdout:8/71: write d5/d7/f11 [3305669,39991] 0 2024-03-16T10:45:08.605 INFO:tasks.workunit.client.0.smithi002.stdout:4/799: dwrite d2/d33/fb3 [0,4194304] 0 2024-03-16T10:45:08.606 INFO:tasks.workunit.client.0.smithi002.stdout:8/942: mknod d7/d9/d25/d91/c139 0 2024-03-16T10:45:08.611 INFO:tasks.workunit.client.0.smithi002.stdout:4/800: truncate d2/fa3 1692726 0 2024-03-16T10:45:08.612 INFO:tasks.workunit.client.0.smithi002.stdout:1/920: dwrite d2/df/d12/d84/d87/d96/db5/d1c/d5d/fb9 [0,4194304] 0 2024-03-16T10:45:08.612 INFO:tasks.workunit.client.1.smithi026.stdout:3/65: rename d4/dc/l17 to d4/dc/dd/l1a 0 2024-03-16T10:45:08.614 INFO:tasks.workunit.client.0.smithi002.stdout:8/943: dread - d7/d9/d17/d83/df5/ffb zero size 2024-03-16T10:45:08.614 INFO:tasks.workunit.client.0.smithi002.stdout:5/742: dread d0/d16/dab/f3b [0,4194304] 0 2024-03-16T10:45:08.617 INFO:tasks.workunit.client.0.smithi002.stdout:5/743: fsync d0/d16/dab/fdf 0 2024-03-16T10:45:08.620 INFO:tasks.workunit.client.0.smithi002.stdout:3/891: mkdir d1/dd/d28/d7b/d139 0 2024-03-16T10:45:08.621 INFO:tasks.workunit.client.0.smithi002.stdout:3/892: fsync d1/db6/d15/d87/d11e/f136 0 2024-03-16T10:45:08.623 INFO:tasks.workunit.client.0.smithi002.stdout:2/787: creat d7/d27/d3f/d7f/ff5 x:0 0 0 2024-03-16T10:45:08.624 INFO:tasks.workunit.client.0.smithi002.stdout:4/801: dread d2/d6/d32/f4e [0,4194304] 0 2024-03-16T10:45:08.624 INFO:tasks.workunit.client.0.smithi002.stdout:2/788: dread - d7/d2e/d28/d40/d50/db4/dcd/ff4 zero size 2024-03-16T10:45:08.627 INFO:tasks.workunit.client.1.smithi026.stdout:8/72: link d5/d7/cf d5/c16 0 2024-03-16T10:45:08.628 INFO:tasks.workunit.client.1.smithi026.stdout:8/73: dread d5/d7/f10 [0,4194304] 0 2024-03-16T10:45:08.631 INFO:tasks.workunit.client.1.smithi026.stdout:8/74: symlink d5/d7/l17 0 2024-03-16T10:45:08.632 INFO:tasks.workunit.client.1.smithi026.stdout:3/66: getdents d4/dc/dd 0 2024-03-16T10:45:08.633 INFO:tasks.workunit.client.0.smithi002.stdout:9/916: dwrite d5/d9/dc/d28/dd6/f11d [0,4194304] 0 2024-03-16T10:45:08.636 INFO:tasks.workunit.client.0.smithi002.stdout:9/917: read d5/d9/dc/f43 [3112445,113772] 0 2024-03-16T10:45:08.639 INFO:tasks.workunit.client.0.smithi002.stdout:6/897: link d15/d2d/d2b/d3f/d77/d9f/c11a d15/d2d/d2c/c138 0 2024-03-16T10:45:08.639 INFO:tasks.workunit.client.0.smithi002.stdout:1/921: dread d2/f68 [0,4194304] 0 2024-03-16T10:45:08.640 INFO:tasks.workunit.client.1.smithi026.stdout:7/117: sync 2024-03-16T10:45:08.640 INFO:tasks.workunit.client.1.smithi026.stdout:0/130: sync 2024-03-16T10:45:08.641 INFO:tasks.workunit.client.0.smithi002.stdout:8/944: link d7/f60 d7/d9/d17/d23/d2e/d34/dbf/dd9/dea/f13a 0 2024-03-16T10:45:08.641 INFO:tasks.workunit.client.1.smithi026.stdout:0/131: fsync d1/f15 0 2024-03-16T10:45:08.641 INFO:tasks.workunit.client.0.smithi002.stdout:8/945: chown d7/d9/d25/d67 239454 1 2024-03-16T10:45:08.642 INFO:tasks.workunit.client.0.smithi002.stdout:1/922: dread - d2/df/d12/d84/d87/d96/db5/d1c/d5d/f82 zero size 2024-03-16T10:45:08.642 INFO:tasks.workunit.client.0.smithi002.stdout:5/744: getdents d0/d16/dab/d43/dcc 0 2024-03-16T10:45:08.643 INFO:tasks.workunit.client.1.smithi026.stdout:3/67: write d4/f5 [1032661,77863] 0 2024-03-16T10:45:08.644 INFO:tasks.workunit.client.0.smithi002.stdout:3/893: creat d1/db6/d15/dcc/f13a x:0 0 0 2024-03-16T10:45:08.646 INFO:tasks.workunit.client.0.smithi002.stdout:2/789: mkdir d7/d2e/d1d/d38/df6 0 2024-03-16T10:45:08.646 INFO:tasks.workunit.client.0.smithi002.stdout:7/743: sync 2024-03-16T10:45:08.647 INFO:tasks.workunit.client.1.smithi026.stdout:7/118: creat d6/de/d16/f1f x:0 0 0 2024-03-16T10:45:08.648 INFO:tasks.workunit.client.0.smithi002.stdout:4/802: symlink d2/d6/d32/d66/d7f/de7/d2e/db4/lf1 0 2024-03-16T10:45:08.648 INFO:tasks.workunit.client.1.smithi026.stdout:3/68: dwrite d4/f9 [0,4194304] 0 2024-03-16T10:45:08.653 INFO:tasks.workunit.client.0.smithi002.stdout:6/898: truncate d15/d2d/d2b/f61 795519 0 2024-03-16T10:45:08.655 INFO:tasks.workunit.client.1.smithi026.stdout:7/119: rename d6/f1b to d6/d1e/f20 0 2024-03-16T10:45:08.658 INFO:tasks.workunit.client.0.smithi002.stdout:5/745: rmdir d0/d16/dab 39 2024-03-16T10:45:08.658 INFO:tasks.workunit.client.0.smithi002.stdout:7/744: dread d3/f52 [8388608,4194304] 0 2024-03-16T10:45:08.658 INFO:tasks.workunit.client.1.smithi026.stdout:0/132: write d1/d18/d1a/f26 [788474,26783] 0 2024-03-16T10:45:08.659 INFO:tasks.workunit.client.0.smithi002.stdout:5/746: dread - d0/d24/dad/db6/db9/fd4 zero size 2024-03-16T10:45:08.660 INFO:tasks.workunit.client.1.smithi026.stdout:0/133: truncate d1/d18/d1a/f2c 5525316 0 2024-03-16T10:45:08.661 INFO:tasks.workunit.client.1.smithi026.stdout:3/69: rename d4/dc/f15 to d4/f1b 0 2024-03-16T10:45:08.661 INFO:tasks.workunit.client.0.smithi002.stdout:2/790: fsync d7/d2e/d46/fdc 0 2024-03-16T10:45:08.662 INFO:tasks.workunit.client.1.smithi026.stdout:7/120: truncate d6/fc 808611 0 2024-03-16T10:45:08.662 INFO:tasks.workunit.client.0.smithi002.stdout:5/747: truncate d0/d16/d5b/da0/dc4/fe3 594196 0 2024-03-16T10:45:08.663 INFO:tasks.workunit.client.1.smithi026.stdout:7/121: read d6/de/d16/f19 [665977,89648] 0 2024-03-16T10:45:08.665 INFO:tasks.workunit.client.0.smithi002.stdout:9/918: mkdir d5/d9/dc/d1e/d91/df1/d129 0 2024-03-16T10:45:08.672 INFO:tasks.workunit.client.1.smithi026.stdout:3/70: rmdir d4/dc 39 2024-03-16T10:45:08.677 INFO:tasks.workunit.client.0.smithi002.stdout:2/791: mknod d7/d27/d3f/d7f/cf7 0 2024-03-16T10:45:08.678 INFO:tasks.workunit.client.1.smithi026.stdout:3/71: stat d4/dc 0 2024-03-16T10:45:08.678 INFO:tasks.workunit.client.1.smithi026.stdout:3/72: dread - d4/f18 zero size 2024-03-16T10:45:08.679 INFO:tasks.workunit.client.0.smithi002.stdout:4/803: dwrite d2/d6/ddb/fe2 [0,4194304] 0 2024-03-16T10:45:08.680 INFO:tasks.workunit.client.0.smithi002.stdout:5/748: stat d0/d16/dab/d43/d48 0 2024-03-16T10:45:08.680 INFO:tasks.workunit.client.1.smithi026.stdout:0/134: getdents d1/d18/d1a/d24 0 2024-03-16T10:45:08.681 INFO:tasks.workunit.client.1.smithi026.stdout:7/122: write d6/de/f11 [2645036,34999] 0 2024-03-16T10:45:08.682 INFO:tasks.workunit.client.1.smithi026.stdout:0/135: write d1/d18/d1a/d24/d6/fe [1175666,54329] 0 2024-03-16T10:45:08.683 INFO:tasks.workunit.client.1.smithi026.stdout:3/73: symlink d4/dc/l1c 0 2024-03-16T10:45:08.685 INFO:tasks.workunit.client.0.smithi002.stdout:1/923: truncate d2/df/d12/d84/d87/d96/db5/d51/d56/fc5 2262595 0 2024-03-16T10:45:08.687 INFO:tasks.workunit.client.0.smithi002.stdout:2/792: dwrite d7/db/d12/f85 [0,4194304] 0 2024-03-16T10:45:08.687 INFO:tasks.workunit.client.0.smithi002.stdout:6/899: dwrite d15/d2d/d2c/f56 [0,4194304] 0 2024-03-16T10:45:08.689 INFO:tasks.workunit.client.0.smithi002.stdout:0/859: sync 2024-03-16T10:45:08.690 INFO:tasks.workunit.client.0.smithi002.stdout:9/919: truncate d5/d9/dc/d1e/d24/d31/f32 4116224 0 2024-03-16T10:45:08.690 INFO:tasks.workunit.client.0.smithi002.stdout:6/900: chown d15/d2d/d2c/c4b 127 1 2024-03-16T10:45:08.690 INFO:tasks.workunit.client.1.smithi026.stdout:7/123: dwrite d6/de/d16/f1a [0,4194304] 0 2024-03-16T10:45:08.691 INFO:tasks.workunit.client.0.smithi002.stdout:1/924: truncate d2/df/d12/d84/d87/d96/db5/f8 1720486 0 2024-03-16T10:45:08.691 INFO:tasks.workunit.client.0.smithi002.stdout:7/745: creat d3/d4/d1d/d87/d5c/dd0/d109/f10a x:0 0 0 2024-03-16T10:45:08.693 INFO:tasks.workunit.client.0.smithi002.stdout:9/920: write d5/d9/dc/f92 [1412431,129451] 0 2024-03-16T10:45:08.693 INFO:tasks.workunit.client.1.smithi026.stdout:0/136: symlink d1/d18/d1a/d2b/l30 0 2024-03-16T10:45:08.694 INFO:tasks.workunit.client.0.smithi002.stdout:9/921: stat d5/d9/dc/d1e/d24/fc6 0 2024-03-16T10:45:08.696 INFO:tasks.workunit.client.1.smithi026.stdout:3/74: rename d4/f5 to d4/f1d 0 2024-03-16T10:45:08.701 INFO:tasks.workunit.client.1.smithi026.stdout:3/75: mknod d4/dc/dd/c1e 0 2024-03-16T10:45:08.702 INFO:tasks.workunit.client.1.smithi026.stdout:3/76: fsync d4/dc/f16 0 2024-03-16T10:45:08.703 INFO:tasks.workunit.client.1.smithi026.stdout:3/77: truncate d4/fa 753284 0 2024-03-16T10:45:08.704 INFO:tasks.workunit.client.0.smithi002.stdout:8/946: getdents d7 0 2024-03-16T10:45:08.704 INFO:tasks.workunit.client.0.smithi002.stdout:3/894: getdents d1/dd/d28/d7b/dd5/d123 0 2024-03-16T10:45:08.709 INFO:tasks.workunit.client.1.smithi026.stdout:3/78: symlink d4/l1f 0 2024-03-16T10:45:08.711 INFO:tasks.workunit.client.0.smithi002.stdout:2/793: creat d7/d2e/d1d/d2f/d4b/ff8 x:0 0 0 2024-03-16T10:45:08.711 INFO:tasks.workunit.client.0.smithi002.stdout:7/746: mkdir d3/d4/d1c/d5b/d64/d10b 0 2024-03-16T10:45:08.712 INFO:tasks.workunit.client.0.smithi002.stdout:1/925: dread d2/df/d12/d29/d5c/f6e [0,4194304] 0 2024-03-16T10:45:08.713 INFO:tasks.workunit.client.1.smithi026.stdout:3/79: creat d4/dc/f20 x:0 0 0 2024-03-16T10:45:08.713 INFO:tasks.workunit.client.0.smithi002.stdout:2/794: truncate d7/d27/d3b/d95/da5/fbc 905888 0 2024-03-16T10:45:08.714 INFO:tasks.workunit.client.0.smithi002.stdout:1/926: chown d2/c16 260232 1 2024-03-16T10:45:08.715 INFO:tasks.workunit.client.0.smithi002.stdout:7/747: truncate d3/d4/d1d/d87/d5c/dc3/df7/f108 276121 0 2024-03-16T10:45:08.716 INFO:tasks.workunit.client.1.smithi026.stdout:3/80: creat d4/dc/dd/f21 x:0 0 0 2024-03-16T10:45:08.717 INFO:tasks.workunit.client.1.smithi026.stdout:3/81: fsync d4/f9 0 2024-03-16T10:45:08.717 INFO:tasks.workunit.client.1.smithi026.stdout:3/82: stat d4/dc/f20 0 2024-03-16T10:45:08.718 INFO:tasks.workunit.client.1.smithi026.stdout:3/83: fsync d4/fa 0 2024-03-16T10:45:08.718 INFO:tasks.workunit.client.0.smithi002.stdout:9/922: mknod d5/d9/dc/d1e/d91/c12a 0 2024-03-16T10:45:08.719 INFO:tasks.workunit.client.0.smithi002.stdout:4/804: rename d2/d33/d35/d59/l5c to d2/d6/d32/d66/d7f/de7/d2e/lf2 0 2024-03-16T10:45:08.722 INFO:tasks.workunit.client.0.smithi002.stdout:2/795: creat d7/d2e/d28/d40/ff9 x:0 0 0 2024-03-16T10:45:08.723 INFO:tasks.workunit.client.0.smithi002.stdout:9/923: mknod d5/d9/d55/dc9/d11f/c12b 0 2024-03-16T10:45:08.724 INFO:tasks.workunit.client.0.smithi002.stdout:5/749: write d0/d1f/f36 [4377193,120765] 0 2024-03-16T10:45:08.724 INFO:tasks.workunit.client.0.smithi002.stdout:1/927: truncate d2/df/d12/d84/d87/d96/db5/d1c/d5d/fce 484062 0 2024-03-16T10:45:08.727 INFO:tasks.workunit.client.0.smithi002.stdout:8/947: creat d7/d9/d17/d23/f13b x:0 0 0 2024-03-16T10:45:08.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:08 smithi026 ceph-mon[37041]: pgmap v37: 65 pgs: 65 active+clean; 1.7 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 75 MiB/s rd, 245 MiB/s wr, 560 op/s 2024-03-16T10:45:08.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:08 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:08.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:08 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:08.753 INFO:tasks.workunit.client.0.smithi002.stdout:9/924: creat d5/d9/dc/d28/d90/f12c x:0 0 0 2024-03-16T10:45:08.754 INFO:tasks.workunit.client.0.smithi002.stdout:7/748: rename d3/l20 to d3/l10c 0 2024-03-16T10:45:08.754 INFO:tasks.workunit.client.0.smithi002.stdout:8/948: dread - d7/d9/d17/d23/d2e/d3a/fa5 zero size 2024-03-16T10:45:08.755 INFO:tasks.workunit.client.0.smithi002.stdout:2/796: write d7/d2e/d28/d40/d50/f68 [4035975,16185] 0 2024-03-16T10:45:08.755 INFO:tasks.workunit.client.0.smithi002.stdout:3/895: rename d1/dd/d28/d7b/d8a/d93/f60 to d1/db6/d15/f13b 0 2024-03-16T10:45:08.755 INFO:tasks.workunit.client.0.smithi002.stdout:5/750: write d0/d16/dab/f23 [900313,58577] 0 2024-03-16T10:45:08.758 INFO:tasks.workunit.client.0.smithi002.stdout:1/928: creat d2/df/d12/d84/d87/d96/db5/dde/f12b x:0 0 0 2024-03-16T10:45:08.759 INFO:tasks.workunit.client.0.smithi002.stdout:6/901: rename l0 to d15/d2d/d8a/d7c/d87/l139 0 2024-03-16T10:45:08.760 INFO:tasks.workunit.client.0.smithi002.stdout:7/749: unlink d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/la3 0 2024-03-16T10:45:08.763 INFO:tasks.workunit.client.0.smithi002.stdout:7/750: write d3/d4/d1d/d87/d5c/dd0/d109/f10a [546247,55880] 0 2024-03-16T10:45:08.763 INFO:tasks.workunit.client.0.smithi002.stdout:1/929: creat d2/df/d12/d84/d100/f12c x:0 0 0 2024-03-16T10:45:08.764 INFO:tasks.workunit.client.0.smithi002.stdout:1/930: chown d2/fe1 4480554 1 2024-03-16T10:45:08.765 INFO:tasks.workunit.client.0.smithi002.stdout:0/860: rename d3/d59/cba to d3/d27/c116 0 2024-03-16T10:45:08.765 INFO:tasks.workunit.client.0.smithi002.stdout:7/751: dread - d3/d4/d1d/d87/d5c/dc3/fdc zero size 2024-03-16T10:45:08.767 INFO:tasks.workunit.client.0.smithi002.stdout:1/931: read - d2/df/d12/dd7/dd9/deb/ff7 zero size 2024-03-16T10:45:08.768 INFO:tasks.workunit.client.0.smithi002.stdout:7/752: chown d3/d4/d1d/d87/d5c/dd0/d109/f10a 849932246 1 2024-03-16T10:45:08.771 INFO:tasks.workunit.client.0.smithi002.stdout:9/925: rename d5/d9/d34/f99 to d5/d9/d22/d3c/d118/f12d 0 2024-03-16T10:45:08.773 INFO:tasks.workunit.client.0.smithi002.stdout:3/896: link d1/dd/f10 d1/dd/f13c 0 2024-03-16T10:45:08.774 INFO:tasks.workunit.client.0.smithi002.stdout:2/797: dread d7/d2e/d1d/f21 [0,4194304] 0 2024-03-16T10:45:08.774 INFO:tasks.workunit.client.0.smithi002.stdout:5/751: dread d0/f57 [4194304,4194304] 0 2024-03-16T10:45:08.776 INFO:tasks.workunit.client.0.smithi002.stdout:8/949: rename d7/d9/d17/d23/d39/db7/fd8 to d7/d9/d17/d23/d2e/de5/f13c 0 2024-03-16T10:45:08.778 INFO:tasks.workunit.client.0.smithi002.stdout:8/950: readlink d7/d9/d17/d23/d2e/d34/dbf/l114 0 2024-03-16T10:45:08.779 INFO:tasks.workunit.client.0.smithi002.stdout:5/752: stat d0/c7e 0 2024-03-16T10:45:08.780 INFO:tasks.workunit.client.0.smithi002.stdout:7/753: symlink d3/d4/l10d 0 2024-03-16T10:45:08.783 INFO:tasks.workunit.client.0.smithi002.stdout:2/798: creat d7/d27/d3b/d95/da5/ffa x:0 0 0 2024-03-16T10:45:08.784 INFO:tasks.workunit.client.0.smithi002.stdout:6/902: rename d15/dfd/c12c to d15/d2d/d2b/de4/dfc/c13a 0 2024-03-16T10:45:08.790 INFO:tasks.workunit.client.0.smithi002.stdout:8/951: mkdir d7/d9/d17/d23/d2e/d34/dde/d13d 0 2024-03-16T10:45:08.791 INFO:tasks.workunit.client.0.smithi002.stdout:7/754: fsync d3/d61/d66/f40 0 2024-03-16T10:45:08.791 INFO:tasks.workunit.client.0.smithi002.stdout:5/753: dread d0/d16/f5f [4194304,4194304] 0 2024-03-16T10:45:08.792 INFO:tasks.workunit.client.0.smithi002.stdout:2/799: creat d7/d2e/d28/ffb x:0 0 0 2024-03-16T10:45:08.792 INFO:tasks.workunit.client.0.smithi002.stdout:8/952: dread - d7/d9/d25/d67/f12b zero size 2024-03-16T10:45:08.793 INFO:tasks.workunit.client.0.smithi002.stdout:1/932: rename d2/df/d12/c18 to d2/df/d12/d84/d87/d96/da8/c12d 0 2024-03-16T10:45:08.794 INFO:tasks.workunit.client.0.smithi002.stdout:8/953: chown d7/d9/d17/dc3/le4 110430 1 2024-03-16T10:45:08.794 INFO:tasks.workunit.client.0.smithi002.stdout:6/903: creat d15/d2d/d2b/d3f/d77/de1/f13b x:0 0 0 2024-03-16T10:45:08.795 INFO:tasks.workunit.client.0.smithi002.stdout:9/926: creat d5/d9/d34/f12e x:0 0 0 2024-03-16T10:45:08.796 INFO:tasks.workunit.client.0.smithi002.stdout:9/927: write d5/d9/dc/d28/fe6 [4624620,5059] 0 2024-03-16T10:45:08.797 INFO:tasks.workunit.client.1.smithi026.stdout:3/84: chown d4/dc/f16 180096284 1 2024-03-16T10:45:08.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:08 smithi002 ceph-mon[29248]: pgmap v37: 65 pgs: 65 active+clean; 1.7 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 75 MiB/s rd, 245 MiB/s wr, 560 op/s 2024-03-16T10:45:08.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:08 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:08.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:08 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:08.799 INFO:tasks.workunit.client.1.smithi026.stdout:3/85: mknod d4/dc/dd/c22 0 2024-03-16T10:45:08.801 INFO:tasks.workunit.client.0.smithi002.stdout:9/928: dread d5/d9/dc/d1e/d24/f3d [0,4194304] 0 2024-03-16T10:45:08.801 INFO:tasks.workunit.client.0.smithi002.stdout:7/755: symlink d3/d4/d1d/d87/d5c/d63/d3f/daa/d106/l10e 0 2024-03-16T10:45:08.802 INFO:tasks.workunit.client.1.smithi026.stdout:3/86: symlink d4/dc/l23 0 2024-03-16T10:45:08.804 INFO:tasks.workunit.client.0.smithi002.stdout:6/904: unlink d15/d2d/d2c/f55 0 2024-03-16T10:45:08.805 INFO:tasks.workunit.client.1.smithi026.stdout:3/87: creat d4/dc/dd/f24 x:0 0 0 2024-03-16T10:45:08.807 INFO:tasks.workunit.client.0.smithi002.stdout:8/954: unlink d7/d9/d17/d23/d39/f8c 0 2024-03-16T10:45:08.808 INFO:tasks.workunit.client.1.smithi026.stdout:3/88: creat d4/dc/f25 x:0 0 0 2024-03-16T10:45:08.809 INFO:tasks.workunit.client.0.smithi002.stdout:9/929: dwrite d5/d9/d34/f12e [0,4194304] 0 2024-03-16T10:45:08.810 INFO:tasks.workunit.client.1.smithi026.stdout:3/89: symlink d4/dc/dd/l26 0 2024-03-16T10:45:08.811 INFO:tasks.workunit.client.0.smithi002.stdout:3/897: dwrite d1/dd/d28/d7b/d2c/d63/f88 [0,4194304] 0 2024-03-16T10:45:08.811 INFO:tasks.workunit.client.0.smithi002.stdout:8/955: dread - d7/d9/d17/d23/d39/db7/f10d zero size 2024-03-16T10:45:08.811 INFO:tasks.workunit.client.1.smithi026.stdout:3/90: write d4/fe [423173,9005] 0 2024-03-16T10:45:08.812 INFO:tasks.workunit.client.0.smithi002.stdout:3/898: chown d1/da5/lf4 24914631 1 2024-03-16T10:45:08.814 INFO:tasks.workunit.client.0.smithi002.stdout:5/754: link d0/d16/dab/d2b/fc6 d0/d16/d4a/d39/fe6 0 2024-03-16T10:45:08.814 INFO:tasks.workunit.client.0.smithi002.stdout:3/899: truncate d1/db6/d15/f67 650439 0 2024-03-16T10:45:08.816 INFO:tasks.workunit.client.0.smithi002.stdout:1/933: mknod d2/df/d12/d84/d87/d96/db5/d1c/c12e 0 2024-03-16T10:45:08.817 INFO:tasks.workunit.client.0.smithi002.stdout:1/934: chown d2/l93 0 1 2024-03-16T10:45:08.819 INFO:tasks.workunit.client.1.smithi026.stdout:3/91: symlink d4/l27 0 2024-03-16T10:45:08.822 INFO:tasks.workunit.client.0.smithi002.stdout:6/905: creat d15/dfd/d96/dbb/f13c x:0 0 0 2024-03-16T10:45:08.824 INFO:tasks.workunit.client.1.smithi026.stdout:3/92: symlink d4/l28 0 2024-03-16T10:45:08.825 INFO:tasks.workunit.client.0.smithi002.stdout:2/800: rmdir d7/d2e/d1d/d38/df3 0 2024-03-16T10:45:08.827 INFO:tasks.workunit.client.0.smithi002.stdout:9/930: unlink d5/d9/f78 0 2024-03-16T10:45:08.828 INFO:tasks.workunit.client.1.smithi026.stdout:3/93: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:08.828 INFO:tasks.workunit.client.0.smithi002.stdout:3/900: chown d1/dd/d28/f31 0 1 2024-03-16T10:45:08.829 INFO:tasks.workunit.client.1.smithi026.stdout:3/94: chown d4/dc/f25 114 1 2024-03-16T10:45:08.829 INFO:tasks.workunit.client.1.smithi026.stdout:3/95: write d4/f9 [1776528,24379] 0 2024-03-16T10:45:08.831 INFO:tasks.workunit.client.0.smithi002.stdout:1/935: rename d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/ldc to d2/d58/l12f 0 2024-03-16T10:45:08.835 INFO:tasks.workunit.client.0.smithi002.stdout:6/906: fdatasync d15/f19 0 2024-03-16T10:45:08.836 INFO:tasks.workunit.client.0.smithi002.stdout:2/801: readlink d7/d2e/d1d/d38/d49/d8c/lc5 0 2024-03-16T10:45:08.837 INFO:tasks.workunit.client.1.smithi026.stdout:3/96: mkdir d4/d29 0 2024-03-16T10:45:08.837 INFO:tasks.workunit.client.1.smithi026.stdout:3/97: chown d4/dc/f25 5000180 1 2024-03-16T10:45:08.838 INFO:tasks.workunit.client.1.smithi026.stdout:3/98: write d4/dc/f11 [6015069,123810] 0 2024-03-16T10:45:08.840 INFO:tasks.workunit.client.0.smithi002.stdout:9/931: creat d5/d9/d55/f12f x:0 0 0 2024-03-16T10:45:08.844 INFO:tasks.workunit.client.0.smithi002.stdout:1/936: mkdir d2/df/d12/d84/d87/d96/db5/de2/d10a/d130 0 2024-03-16T10:45:08.845 INFO:tasks.workunit.client.1.smithi026.stdout:3/99: symlink d4/d29/l2a 0 2024-03-16T10:45:08.846 INFO:tasks.workunit.client.1.smithi026.stdout:3/100: dread d4/f1d [0,4194304] 0 2024-03-16T10:45:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:5/755: dwrite d0/d16/d4a/d39/fe6 [0,4194304] 0 2024-03-16T10:45:08.847 INFO:tasks.workunit.client.1.smithi026.stdout:3/101: chown d4/dc/dd/l26 95503 1 2024-03-16T10:45:08.847 INFO:tasks.workunit.client.0.smithi002.stdout:2/802: dread d7/d2e/d28/d44/d6d/fd0 [0,4194304] 0 2024-03-16T10:45:08.848 INFO:tasks.workunit.client.1.smithi026.stdout:3/102: write d4/dc/f13 [28704,78296] 0 2024-03-16T10:45:08.848 INFO:tasks.workunit.client.0.smithi002.stdout:5/756: chown d0/d16/d4a/d7d/ccb 3874466 1 2024-03-16T10:45:08.849 INFO:tasks.workunit.client.1.smithi026.stdout:3/103: chown d4/dc/l23 37137 1 2024-03-16T10:45:08.851 INFO:tasks.workunit.client.1.smithi026.stdout:2/113: sync 2024-03-16T10:45:08.851 INFO:tasks.workunit.client.1.smithi026.stdout:5/86: sync 2024-03-16T10:45:08.851 INFO:tasks.workunit.client.1.smithi026.stdout:4/121: sync 2024-03-16T10:45:08.851 INFO:tasks.workunit.client.1.smithi026.stdout:1/106: sync 2024-03-16T10:45:08.851 INFO:tasks.workunit.client.1.smithi026.stdout:6/126: sync 2024-03-16T10:45:08.851 INFO:tasks.workunit.client.1.smithi026.stdout:9/61: sync 2024-03-16T10:45:08.852 INFO:tasks.workunit.client.0.smithi002.stdout:6/907: rename d15/dfd/d96/db5/f118 to d15/d80/f13d 0 2024-03-16T10:45:08.853 INFO:tasks.workunit.client.1.smithi026.stdout:3/104: symlink d4/dc/dd/l2b 0 2024-03-16T10:45:08.854 INFO:tasks.workunit.client.1.smithi026.stdout:3/105: fdatasync d4/dc/f11 0 2024-03-16T10:45:08.854 INFO:tasks.workunit.client.0.smithi002.stdout:4/805: sync 2024-03-16T10:45:08.855 INFO:tasks.workunit.client.0.smithi002.stdout:7/756: getdents d3/d4/d1d/d87 0 2024-03-16T10:45:08.856 INFO:tasks.workunit.client.1.smithi026.stdout:6/127: dread d0/f6 [0,4194304] 0 2024-03-16T10:45:08.856 INFO:tasks.workunit.client.1.smithi026.stdout:9/62: symlink d0/lf 0 2024-03-16T10:45:08.857 INFO:tasks.workunit.client.1.smithi026.stdout:6/128: chown d0/f6 2 1 2024-03-16T10:45:08.858 INFO:tasks.workunit.client.1.smithi026.stdout:6/129: write d0/f12 [1591065,129193] 0 2024-03-16T10:45:08.859 INFO:tasks.workunit.client.0.smithi002.stdout:7/757: chown d3/d4/d1d/d87/d96/d5e/db7/de6/l105 3 1 2024-03-16T10:45:08.859 INFO:tasks.workunit.client.1.smithi026.stdout:4/122: mkdir d1/db/d29 0 2024-03-16T10:45:08.860 INFO:tasks.workunit.client.0.smithi002.stdout:7/758: dread - d3/f57 zero size 2024-03-16T10:45:08.861 INFO:tasks.workunit.client.0.smithi002.stdout:3/901: dwrite d1/dd/d28/d7b/f24 [0,4194304] 0 2024-03-16T10:45:08.862 INFO:tasks.workunit.client.1.smithi026.stdout:2/114: rmdir d2/d14/d17 39 2024-03-16T10:45:08.862 INFO:tasks.workunit.client.0.smithi002.stdout:3/902: chown d1/f16 2102 1 2024-03-16T10:45:08.864 INFO:tasks.workunit.client.1.smithi026.stdout:5/87: write d1/f18 [212802,106744] 0 2024-03-16T10:45:08.866 INFO:tasks.workunit.client.0.smithi002.stdout:1/937: creat d2/df/d12/d84/d87/d96/db5/d1c/da0/d119/f131 x:0 0 0 2024-03-16T10:45:08.867 INFO:tasks.workunit.client.1.smithi026.stdout:1/107: mknod d0/d17/d1f/c21 0 2024-03-16T10:45:08.868 INFO:tasks.workunit.client.1.smithi026.stdout:1/108: stat d0/d17/d1f/fc 0 2024-03-16T10:45:08.869 INFO:tasks.workunit.client.1.smithi026.stdout:1/109: write d0/f11 [4187752,120250] 0 2024-03-16T10:45:08.872 INFO:tasks.workunit.client.0.smithi002.stdout:0/861: sync 2024-03-16T10:45:08.876 INFO:tasks.workunit.client.1.smithi026.stdout:9/63: creat d0/f10 x:0 0 0 2024-03-16T10:45:08.877 INFO:tasks.workunit.client.0.smithi002.stdout:9/932: dwrite d5/d9/dc/d1e/d24/ded/f54 [0,4194304] 0 2024-03-16T10:45:08.879 INFO:tasks.workunit.client.0.smithi002.stdout:9/933: stat d5/d9/dc/d1e/d24/d31/f4a 0 2024-03-16T10:45:08.880 INFO:tasks.workunit.client.1.smithi026.stdout:6/130: rename d0/f2a to d0/d28/f2e 0 2024-03-16T10:45:08.880 INFO:tasks.workunit.client.0.smithi002.stdout:5/757: write d0/d16/dab/d2b/d5d/fa7 [997988,26871] 0 2024-03-16T10:45:08.881 INFO:tasks.workunit.client.0.smithi002.stdout:2/803: dwrite d7/d2e/d28/d40/d50/fa1 [0,4194304] 0 2024-03-16T10:45:08.884 INFO:tasks.workunit.client.0.smithi002.stdout:5/758: readlink d0/d16/l35 0 2024-03-16T10:45:08.885 INFO:tasks.workunit.client.0.smithi002.stdout:8/956: getdents d7/d9/d25 0 2024-03-16T10:45:08.885 INFO:tasks.workunit.client.1.smithi026.stdout:2/115: dread - d2/d14/d17/f21 zero size 2024-03-16T10:45:08.887 INFO:tasks.workunit.client.1.smithi026.stdout:2/116: read d2/dd/f1d [2540361,2700] 0 2024-03-16T10:45:08.888 INFO:tasks.workunit.client.1.smithi026.stdout:2/117: chown d2/d14/d17/d1a/f1b 535409 1 2024-03-16T10:45:08.888 INFO:tasks.workunit.client.0.smithi002.stdout:4/806: creat d2/d6/d32/d66/d7f/de7/d2e/db4/ff3 x:0 0 0 2024-03-16T10:45:08.889 INFO:tasks.workunit.client.1.smithi026.stdout:4/123: write d1/db/f16 [1976545,90017] 0 2024-03-16T10:45:08.891 INFO:tasks.workunit.client.0.smithi002.stdout:7/759: write d3/d61/f83 [4309879,130238] 0 2024-03-16T10:45:08.892 INFO:tasks.workunit.client.0.smithi002.stdout:7/760: stat d3/d4/d1d/d87/f92 0 2024-03-16T10:45:08.894 INFO:tasks.workunit.client.1.smithi026.stdout:9/64: rmdir d0 39 2024-03-16T10:45:08.895 INFO:tasks.workunit.client.0.smithi002.stdout:1/938: rmdir d2/df/d12/dd7/dd9/deb 39 2024-03-16T10:45:08.899 INFO:tasks.workunit.client.1.smithi026.stdout:5/88: rename d1/f16 to d1/d1c/f1e 0 2024-03-16T10:45:08.904 INFO:tasks.workunit.client.0.smithi002.stdout:9/934: mkdir d5/d7e/d130 0 2024-03-16T10:45:08.905 INFO:tasks.workunit.client.1.smithi026.stdout:6/131: dwrite d0/d28/f2e [0,4194304] 0 2024-03-16T10:45:08.905 INFO:tasks.workunit.client.1.smithi026.stdout:2/118: unlink d2/c4 0 2024-03-16T10:45:08.906 INFO:tasks.workunit.client.1.smithi026.stdout:6/132: chown d0/d10/d17/f18 6347664 1 2024-03-16T10:45:08.910 INFO:tasks.workunit.client.1.smithi026.stdout:1/110: mknod d0/c22 0 2024-03-16T10:45:08.910 INFO:tasks.workunit.client.1.smithi026.stdout:1/111: chown d0/d17/f1a 14250 1 2024-03-16T10:45:08.911 INFO:tasks.workunit.client.1.smithi026.stdout:1/112: rename d0/d17/d1f to d0/d17/d1f/d1c/d23 22 2024-03-16T10:45:08.913 INFO:tasks.workunit.client.0.smithi002.stdout:5/759: symlink d0/d16/dab/le7 0 2024-03-16T10:45:08.913 INFO:tasks.workunit.client.1.smithi026.stdout:3/106: link d4/l10 d4/d29/l2c 0 2024-03-16T10:45:08.915 INFO:tasks.workunit.client.0.smithi002.stdout:5/760: fsync d0/d1f/f2c 0 2024-03-16T10:45:08.915 INFO:tasks.workunit.client.1.smithi026.stdout:1/113: dread d0/d17/d1f/fc [4194304,4194304] 0 2024-03-16T10:45:08.917 INFO:tasks.workunit.client.0.smithi002.stdout:3/903: mkdir d1/dd/d28/d7b/d8a/d93/d13d 0 2024-03-16T10:45:08.918 INFO:tasks.workunit.client.0.smithi002.stdout:2/804: dwrite d7/d2e/d1d/f4f [0,4194304] 0 2024-03-16T10:45:08.919 INFO:tasks.workunit.client.0.smithi002.stdout:1/939: rmdir d2 39 2024-03-16T10:45:08.920 INFO:tasks.workunit.client.0.smithi002.stdout:6/908: dwrite d15/d2d/d2b/f61 [0,4194304] 0 2024-03-16T10:45:08.925 INFO:tasks.workunit.client.1.smithi026.stdout:3/107: symlink d4/dc/l2d 0 2024-03-16T10:45:08.927 INFO:tasks.workunit.client.0.smithi002.stdout:3/904: write d1/db6/d15/f2e [2009397,109138] 0 2024-03-16T10:45:08.927 INFO:tasks.workunit.client.0.smithi002.stdout:6/909: chown d15/dfd/d96/db5/f10e 265 1 2024-03-16T10:45:08.927 INFO:tasks.workunit.client.0.smithi002.stdout:0/862: rmdir d3/d28 39 2024-03-16T10:45:08.928 INFO:tasks.workunit.client.0.smithi002.stdout:4/807: link d2/d6/fe1 d2/d6/d32/d66/d7f/de7/d5e/d93/ff4 0 2024-03-16T10:45:08.930 INFO:tasks.workunit.client.0.smithi002.stdout:7/761: symlink d3/d4/d1d/l10f 0 2024-03-16T10:45:08.930 INFO:tasks.workunit.client.1.smithi026.stdout:1/114: unlink d0/d17/d1f/c21 0 2024-03-16T10:45:08.931 INFO:tasks.workunit.client.0.smithi002.stdout:6/910: chown d15/d2d/d2c/l37 886764 1 2024-03-16T10:45:08.931 INFO:tasks.workunit.client.1.smithi026.stdout:1/115: chown d0/d17/d1f/c15 0 1 2024-03-16T10:45:08.935 INFO:tasks.workunit.client.1.smithi026.stdout:6/133: symlink d0/l2f 0 2024-03-16T10:45:08.936 INFO:tasks.workunit.client.0.smithi002.stdout:9/935: dwrite d5/d9/dc/d1e/d94/fad [4194304,4194304] 0 2024-03-16T10:45:08.937 INFO:tasks.workunit.client.1.smithi026.stdout:9/65: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:08.938 INFO:tasks.workunit.client.0.smithi002.stdout:9/936: readlink d5/d9/dc/d1e/d24/ded/l50 0 2024-03-16T10:45:08.938 INFO:tasks.workunit.client.1.smithi026.stdout:4/124: truncate d1/f3 974330 0 2024-03-16T10:45:08.941 INFO:tasks.workunit.client.0.smithi002.stdout:9/937: write d5/d9/dc/d1e/d91/d83/f9e [3417537,107335] 0 2024-03-16T10:45:08.941 INFO:tasks.workunit.client.1.smithi026.stdout:3/108: creat d4/dc/dd/f2e x:0 0 0 2024-03-16T10:45:08.942 INFO:tasks.workunit.client.1.smithi026.stdout:3/109: chown d4/dc/dd 62 1 2024-03-16T10:45:08.942 INFO:tasks.workunit.client.1.smithi026.stdout:6/134: dwrite d0/f6 [0,4194304] 0 2024-03-16T10:45:08.943 INFO:tasks.workunit.client.1.smithi026.stdout:3/110: readlink d4/d29/l2c 0 2024-03-16T10:45:08.943 INFO:tasks.workunit.client.0.smithi002.stdout:9/938: write d5/d9/dc/d28/f65 [2801233,49570] 0 2024-03-16T10:45:08.945 INFO:tasks.workunit.client.0.smithi002.stdout:8/957: dwrite d7/d13/db8/d8f/f12d [0,4194304] 0 2024-03-16T10:45:08.946 INFO:tasks.workunit.client.0.smithi002.stdout:4/808: rmdir d2/d6/d32/d66/d7f/de7/d2e 39 2024-03-16T10:45:08.949 INFO:tasks.workunit.client.0.smithi002.stdout:6/911: write d15/d2d/d73/d97/fdd [546218,5916] 0 2024-03-16T10:45:08.950 INFO:tasks.workunit.client.1.smithi026.stdout:9/66: rename d0/f10 to d0/da/f11 0 2024-03-16T10:45:08.952 INFO:tasks.workunit.client.0.smithi002.stdout:7/762: creat d3/d4/d1d/d87/d96/d5e/db7/f110 x:0 0 0 2024-03-16T10:45:08.953 INFO:tasks.workunit.client.1.smithi026.stdout:2/119: write d2/d14/f10 [30396,79214] 0 2024-03-16T10:45:08.955 INFO:tasks.workunit.client.1.smithi026.stdout:6/135: readlink d0/d10/lc 0 2024-03-16T10:45:08.959 INFO:tasks.workunit.client.1.smithi026.stdout:5/89: getdents d1/db/d1a 0 2024-03-16T10:45:08.962 INFO:tasks.workunit.client.0.smithi002.stdout:3/905: symlink d1/dd/d28/d7b/d8a/l13e 0 2024-03-16T10:45:08.966 INFO:tasks.workunit.client.0.smithi002.stdout:0/863: fsync d3/d28/f9b 0 2024-03-16T10:45:08.967 INFO:tasks.workunit.client.0.smithi002.stdout:0/864: write d3/d59/dca/fdf [347737,108341] 0 2024-03-16T10:45:08.969 INFO:tasks.workunit.client.1.smithi026.stdout:6/136: rename d0/f19 to d0/d10/d17/f30 0 2024-03-16T10:45:08.971 INFO:tasks.workunit.client.0.smithi002.stdout:9/939: mknod d5/d9/d25/dac/c131 0 2024-03-16T10:45:08.974 INFO:tasks.workunit.client.1.smithi026.stdout:2/120: write d2/dd/f15 [678534,1966] 0 2024-03-16T10:45:08.977 INFO:tasks.workunit.client.1.smithi026.stdout:9/67: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:08.979 INFO:tasks.workunit.client.0.smithi002.stdout:7/763: creat d3/d4/d1d/d87/d5c/d63/d3f/d75/f111 x:0 0 0 2024-03-16T10:45:08.979 INFO:tasks.workunit.client.1.smithi026.stdout:3/111: truncate d4/f1d 1081183 0 2024-03-16T10:45:08.980 INFO:tasks.workunit.client.0.smithi002.stdout:7/764: fsync d3/d4/d1c/d5b/d64/d99/d9c/ffb 0 2024-03-16T10:45:08.981 INFO:tasks.workunit.client.1.smithi026.stdout:5/90: write d1/f5 [172656,40195] 0 2024-03-16T10:45:08.981 INFO:tasks.workunit.client.1.smithi026.stdout:5/91: dread - d1/d1c/f1e zero size 2024-03-16T10:45:08.982 INFO:tasks.workunit.client.0.smithi002.stdout:5/761: creat d0/d16/dab/d33/d4b/fe8 x:0 0 0 2024-03-16T10:45:08.984 INFO:tasks.workunit.client.0.smithi002.stdout:2/805: creat d7/d27/ffc x:0 0 0 2024-03-16T10:45:08.985 INFO:tasks.workunit.client.0.smithi002.stdout:2/806: chown d7/d27/d3b/c6f 63818 1 2024-03-16T10:45:08.985 INFO:tasks.workunit.client.0.smithi002.stdout:3/906: mkdir d1/dd/d28/d7b/d2c/d97/d13f 0 2024-03-16T10:45:08.989 INFO:tasks.workunit.client.1.smithi026.stdout:2/121: creat d2/d14/d17/f23 x:0 0 0 2024-03-16T10:45:08.989 INFO:tasks.workunit.client.1.smithi026.stdout:2/122: chown c1 275114330 1 2024-03-16T10:45:08.990 INFO:tasks.workunit.client.1.smithi026.stdout:4/125: getdents d1 0 2024-03-16T10:45:08.994 INFO:tasks.workunit.client.1.smithi026.stdout:9/68: creat d0/da/f12 x:0 0 0 2024-03-16T10:45:08.995 INFO:tasks.workunit.client.0.smithi002.stdout:4/809: fsync d2/d33/f94 0 2024-03-16T10:45:08.997 INFO:tasks.workunit.client.1.smithi026.stdout:3/112: rename d4/l10 to d4/d29/l2f 0 2024-03-16T10:45:08.998 INFO:tasks.workunit.client.0.smithi002.stdout:6/912: creat d15/dfd/d96/dbc/dcc/f13e x:0 0 0 2024-03-16T10:45:08.999 INFO:tasks.workunit.client.1.smithi026.stdout:5/92: mkdir d1/db/d1f 0 2024-03-16T10:45:08.999 INFO:tasks.workunit.client.0.smithi002.stdout:6/913: write d15/dfd/d96/dbb/d107/f108 [306540,92806] 0 2024-03-16T10:45:09.000 INFO:tasks.workunit.client.0.smithi002.stdout:6/914: chown d15/d2d/d2b/f61 26032982 1 2024-03-16T10:45:09.001 INFO:tasks.workunit.client.0.smithi002.stdout:2/807: dread d7/f90 [0,4194304] 0 2024-03-16T10:45:09.004 INFO:tasks.workunit.client.1.smithi026.stdout:4/126: creat d1/db/de/f2a x:0 0 0 2024-03-16T10:45:09.007 INFO:tasks.workunit.client.0.smithi002.stdout:0/865: readlink d3/de/da2/lb2 0 2024-03-16T10:45:09.009 INFO:tasks.workunit.client.0.smithi002.stdout:9/940: mkdir d5/d9/dc/d1e/d24/d31/d132 0 2024-03-16T10:45:09.011 INFO:tasks.workunit.client.1.smithi026.stdout:6/137: creat d0/d10/f31 x:0 0 0 2024-03-16T10:45:09.012 INFO:tasks.workunit.client.1.smithi026.stdout:6/138: fdatasync d0/d10/d17/f25 0 2024-03-16T10:45:09.013 INFO:tasks.workunit.client.1.smithi026.stdout:2/123: link d2/d14/d17/f21 d2/d14/d17/d1a/f24 0 2024-03-16T10:45:09.013 INFO:tasks.workunit.client.0.smithi002.stdout:1/940: getdents d2/df/d12/d84/d87/d96/d105 0 2024-03-16T10:45:09.016 INFO:tasks.workunit.client.1.smithi026.stdout:4/127: rmdir d1/db 39 2024-03-16T10:45:09.019 INFO:tasks.workunit.client.0.smithi002.stdout:2/808: fdatasync d7/d2e/d28/f6a 0 2024-03-16T10:45:09.020 INFO:tasks.workunit.client.0.smithi002.stdout:2/809: read d7/f90 [1518848,98756] 0 2024-03-16T10:45:09.023 INFO:tasks.workunit.client.1.smithi026.stdout:3/113: link d4/fe d4/d29/f30 0 2024-03-16T10:45:09.024 INFO:tasks.workunit.client.1.smithi026.stdout:5/93: symlink d1/db/d1f/l20 0 2024-03-16T10:45:09.028 INFO:tasks.workunit.client.1.smithi026.stdout:9/69: truncate d0/f4 3602568 0 2024-03-16T10:45:09.028 INFO:tasks.workunit.client.1.smithi026.stdout:2/124: symlink d2/dd/l25 0 2024-03-16T10:45:09.029 INFO:tasks.workunit.client.0.smithi002.stdout:5/762: write d0/d1f/fbd [621411,112457] 0 2024-03-16T10:45:09.029 INFO:tasks.workunit.client.0.smithi002.stdout:6/915: dwrite d15/d2d/d8a/d7c/d87/fa4 [0,4194304] 0 2024-03-16T10:45:09.030 INFO:tasks.workunit.client.0.smithi002.stdout:6/916: readlink d15/d2d/d8a/d7c/lcd 0 2024-03-16T10:45:09.031 INFO:tasks.workunit.client.0.smithi002.stdout:6/917: chown d15/d80/dbe 4 1 2024-03-16T10:45:09.033 INFO:tasks.workunit.client.0.smithi002.stdout:6/918: truncate d15/d2d/d2b/d3f/d77/d7e/f10d 1498823 0 2024-03-16T10:45:09.033 INFO:tasks.workunit.client.1.smithi026.stdout:3/114: mkdir d4/dc/d31 0 2024-03-16T10:45:09.034 INFO:tasks.workunit.client.1.smithi026.stdout:3/115: stat d4/dc/f13 0 2024-03-16T10:45:09.039 INFO:tasks.workunit.client.0.smithi002.stdout:2/810: truncate d7/d2e/f67 3422094 0 2024-03-16T10:45:09.039 INFO:tasks.workunit.client.1.smithi026.stdout:3/116: dread d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:09.042 INFO:tasks.workunit.client.1.smithi026.stdout:2/125: mknod d2/d14/d17/d1a/c26 0 2024-03-16T10:45:09.043 INFO:tasks.workunit.client.1.smithi026.stdout:2/126: read d2/d14/fe [1214088,78884] 0 2024-03-16T10:45:09.044 INFO:tasks.workunit.client.0.smithi002.stdout:0/866: creat d3/df8/f117 x:0 0 0 2024-03-16T10:45:09.047 INFO:tasks.workunit.client.0.smithi002.stdout:8/958: rename d7/d9/d17/d23/d2e/fc1 to d7/d9/f13e 0 2024-03-16T10:45:09.048 INFO:tasks.workunit.client.1.smithi026.stdout:4/128: dread d1/f27 [0,4194304] 0 2024-03-16T10:45:09.049 INFO:tasks.workunit.client.1.smithi026.stdout:8/75: sync 2024-03-16T10:45:09.049 INFO:tasks.workunit.client.0.smithi002.stdout:1/941: mknod d2/df/d12/dd7/dd9/deb/c132 0 2024-03-16T10:45:09.049 INFO:tasks.workunit.client.1.smithi026.stdout:8/76: write d5/d7/f11 [3282239,29959] 0 2024-03-16T10:45:09.050 INFO:tasks.workunit.client.1.smithi026.stdout:6/139: creat d0/f32 x:0 0 0 2024-03-16T10:45:09.051 INFO:tasks.workunit.client.0.smithi002.stdout:5/763: readlink d0/d16/dab/daa/lac 0 2024-03-16T10:45:09.052 INFO:tasks.workunit.client.0.smithi002.stdout:5/764: chown d0/d16/dab/daa 2336 1 2024-03-16T10:45:09.052 INFO:tasks.workunit.client.1.smithi026.stdout:3/117: symlink d4/dc/l32 0 2024-03-16T10:45:09.053 INFO:tasks.workunit.client.0.smithi002.stdout:7/765: getdents d3/d4/d1d/d87/d5c/dd0/d109 0 2024-03-16T10:45:09.058 INFO:tasks.workunit.client.1.smithi026.stdout:5/94: creat d1/f21 x:0 0 0 2024-03-16T10:45:09.063 INFO:tasks.workunit.client.0.smithi002.stdout:1/942: symlink d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/l133 0 2024-03-16T10:45:09.065 INFO:tasks.workunit.client.0.smithi002.stdout:2/811: dread d7/d27/f80 [0,4194304] 0 2024-03-16T10:45:09.065 INFO:tasks.workunit.client.0.smithi002.stdout:5/765: creat d0/d16/d4a/d39/fe9 x:0 0 0 2024-03-16T10:45:09.065 INFO:tasks.workunit.client.1.smithi026.stdout:5/95: dwrite d1/fd [0,4194304] 0 2024-03-16T10:45:09.066 INFO:tasks.workunit.client.1.smithi026.stdout:5/96: write d1/f5 [4200846,49789] 0 2024-03-16T10:45:09.068 INFO:tasks.workunit.client.1.smithi026.stdout:5/97: write d1/db/f12 [490812,97189] 0 2024-03-16T10:45:09.068 INFO:tasks.workunit.client.0.smithi002.stdout:7/766: symlink d3/d61/d66/l112 0 2024-03-16T10:45:09.073 INFO:tasks.workunit.client.1.smithi026.stdout:5/98: dread d1/fa [0,4194304] 0 2024-03-16T10:45:09.074 INFO:tasks.workunit.client.1.smithi026.stdout:5/99: chown d1/db/d1f/l20 4836396 1 2024-03-16T10:45:09.078 INFO:tasks.workunit.client.0.smithi002.stdout:6/919: write d15/d2d/d2b/fb7 [1460008,127317] 0 2024-03-16T10:45:09.080 INFO:tasks.workunit.client.0.smithi002.stdout:5/766: write d0/f57 [3888073,68849] 0 2024-03-16T10:45:09.080 INFO:tasks.workunit.client.1.smithi026.stdout:4/129: creat d1/db/de/f2b x:0 0 0 2024-03-16T10:45:09.081 INFO:tasks.workunit.client.1.smithi026.stdout:4/130: write d1/db/f26 [222623,98335] 0 2024-03-16T10:45:09.082 INFO:tasks.workunit.client.0.smithi002.stdout:2/812: mknod d7/d2e/d1d/d38/d49/db5/cfd 0 2024-03-16T10:45:09.085 INFO:tasks.workunit.client.0.smithi002.stdout:4/810: rename d2/d6/ddb/ce6 to d2/d6/d32/d66/d7f/de7/cf5 0 2024-03-16T10:45:09.087 INFO:tasks.workunit.client.0.smithi002.stdout:0/867: dread d3/d5/d7c/f8a [0,4194304] 0 2024-03-16T10:45:09.089 INFO:tasks.workunit.client.0.smithi002.stdout:6/920: creat d15/d2d/d2b/d75/f13f x:0 0 0 2024-03-16T10:45:09.089 INFO:tasks.workunit.client.1.smithi026.stdout:3/118: mknod d4/c33 0 2024-03-16T10:45:09.089 INFO:tasks.workunit.client.1.smithi026.stdout:3/119: readlink d4/l1f 0 2024-03-16T10:45:09.090 INFO:tasks.workunit.client.0.smithi002.stdout:3/907: sync 2024-03-16T10:45:09.090 INFO:tasks.workunit.client.0.smithi002.stdout:5/767: creat d0/d16/dab/d33/d8c/fea x:0 0 0 2024-03-16T10:45:09.091 INFO:tasks.workunit.client.1.smithi026.stdout:5/100: rmdir d1 39 2024-03-16T10:45:09.093 INFO:tasks.workunit.client.0.smithi002.stdout:4/811: dwrite d2/d33/fb3 [0,4194304] 0 2024-03-16T10:45:09.093 INFO:tasks.workunit.client.0.smithi002.stdout:2/813: symlink d7/d2e/d1d/lfe 0 2024-03-16T10:45:09.093 INFO:tasks.workunit.client.1.smithi026.stdout:8/77: truncate d5/d7/f11 4139538 0 2024-03-16T10:45:09.094 INFO:tasks.workunit.client.0.smithi002.stdout:2/814: write d7/db/fbd [991798,105697] 0 2024-03-16T10:45:09.094 INFO:tasks.workunit.client.1.smithi026.stdout:2/127: truncate d2/d14/fe 363460 0 2024-03-16T10:45:09.094 INFO:tasks.workunit.client.1.smithi026.stdout:8/78: write d5/fb [3660324,30173] 0 2024-03-16T10:45:09.096 INFO:tasks.workunit.client.1.smithi026.stdout:8/79: write d5/dc/f12 [1209945,48919] 0 2024-03-16T10:45:09.098 INFO:tasks.workunit.client.0.smithi002.stdout:7/767: link d3/d4/d1d/d87/d5c/d63/d55/f5d d3/d4/d1d/d87/d5c/d63/d55/f113 0 2024-03-16T10:45:09.099 INFO:tasks.workunit.client.0.smithi002.stdout:8/959: truncate d7/d9/d25/d91/d123/f12f 4122732 0 2024-03-16T10:45:09.099 INFO:tasks.workunit.client.1.smithi026.stdout:9/70: truncate d0/f4 3861559 0 2024-03-16T10:45:09.100 INFO:tasks.workunit.client.0.smithi002.stdout:7/768: fsync d3/d4/d1d/d87/d5c/d63/fab 0 2024-03-16T10:45:09.100 INFO:tasks.workunit.client.0.smithi002.stdout:8/960: dread - d7/d9/d17/d23/d39/db7/fd1 zero size 2024-03-16T10:45:09.101 INFO:tasks.workunit.client.0.smithi002.stdout:1/943: write d2/df/d12/d84/d87/d96/db5/f7c [3354981,40067] 0 2024-03-16T10:45:09.102 INFO:tasks.workunit.client.1.smithi026.stdout:6/140: creat d0/f33 x:0 0 0 2024-03-16T10:45:09.102 INFO:tasks.workunit.client.0.smithi002.stdout:8/961: chown d7/de3/f10a 30 1 2024-03-16T10:45:09.103 INFO:tasks.workunit.client.1.smithi026.stdout:6/141: read d0/f6 [2953807,18534] 0 2024-03-16T10:45:09.104 INFO:tasks.workunit.client.0.smithi002.stdout:7/769: truncate d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/fbc 121926 0 2024-03-16T10:45:09.105 INFO:tasks.workunit.client.1.smithi026.stdout:6/142: write d0/d28/f2e [2386649,102992] 0 2024-03-16T10:45:09.105 INFO:tasks.workunit.client.0.smithi002.stdout:9/941: rename d5/db1/dec/ffc to d5/d9/dc/d1e/d24/ded/d7f/f133 0 2024-03-16T10:45:09.110 INFO:tasks.workunit.client.0.smithi002.stdout:8/962: dwrite d7/d9/d17/d23/d2e/d34/f79 [4194304,4194304] 0 2024-03-16T10:45:09.112 INFO:tasks.workunit.client.1.smithi026.stdout:5/101: write d1/db/f12 [1861092,46807] 0 2024-03-16T10:45:09.113 INFO:tasks.workunit.client.0.smithi002.stdout:0/868: creat d3/d31/d75/db4/ddc/f118 x:0 0 0 2024-03-16T10:45:09.113 INFO:tasks.workunit.client.1.smithi026.stdout:5/102: truncate d1/f18 590513 0 2024-03-16T10:45:09.114 INFO:tasks.workunit.client.1.smithi026.stdout:5/103: fdatasync d1/db/fc 0 2024-03-16T10:45:09.115 INFO:tasks.workunit.client.1.smithi026.stdout:7/124: sync 2024-03-16T10:45:09.115 INFO:tasks.workunit.client.1.smithi026.stdout:0/137: sync 2024-03-16T10:45:09.118 INFO:tasks.workunit.client.1.smithi026.stdout:2/128: unlink d2/c8 0 2024-03-16T10:45:09.123 INFO:tasks.workunit.client.1.smithi026.stdout:5/104: dwrite d1/fd [0,4194304] 0 2024-03-16T10:45:09.128 INFO:tasks.workunit.client.0.smithi002.stdout:2/815: readlink d7/d2e/d1d/d38/ldf 0 2024-03-16T10:45:09.132 INFO:tasks.workunit.client.0.smithi002.stdout:6/921: write d15/dab/fb6 [1027681,49276] 0 2024-03-16T10:45:09.133 INFO:tasks.workunit.client.1.smithi026.stdout:0/138: dread d1/d18/d1a/f26 [0,4194304] 0 2024-03-16T10:45:09.134 INFO:tasks.workunit.client.1.smithi026.stdout:4/131: creat d1/db/d29/f2c x:0 0 0 2024-03-16T10:45:09.136 INFO:tasks.workunit.client.0.smithi002.stdout:3/908: write d1/dd/d28/d37/f78 [1961778,27817] 0 2024-03-16T10:45:09.138 INFO:tasks.workunit.client.1.smithi026.stdout:0/139: dread d1/d18/d1a/f2c [0,4194304] 0 2024-03-16T10:45:09.140 INFO:tasks.workunit.client.1.smithi026.stdout:3/120: creat d4/dc/d31/f34 x:0 0 0 2024-03-16T10:45:09.141 INFO:tasks.workunit.client.1.smithi026.stdout:3/121: stat d4/fa 0 2024-03-16T10:45:09.141 INFO:tasks.workunit.client.1.smithi026.stdout:3/122: chown d4/dc/dd/l26 520031 1 2024-03-16T10:45:09.142 INFO:tasks.workunit.client.1.smithi026.stdout:8/80: write d5/f8 [697557,110311] 0 2024-03-16T10:45:09.143 INFO:tasks.workunit.client.1.smithi026.stdout:8/81: chown d5/fb 541100206 1 2024-03-16T10:45:09.143 INFO:tasks.workunit.client.1.smithi026.stdout:3/123: chown d4/dc/d31/f34 670 1 2024-03-16T10:45:09.144 INFO:tasks.workunit.client.1.smithi026.stdout:3/124: read - d4/dc/dd/f21 zero size 2024-03-16T10:45:09.148 INFO:tasks.workunit.client.0.smithi002.stdout:8/963: mkdir d7/d9/d17/d20/d13f 0 2024-03-16T10:45:09.148 INFO:tasks.workunit.client.1.smithi026.stdout:3/125: dread d4/dc/dd/f12 [0,4194304] 0 2024-03-16T10:45:09.152 INFO:tasks.workunit.client.1.smithi026.stdout:7/125: mkdir d6/de/d16/d21 0 2024-03-16T10:45:09.153 INFO:tasks.workunit.client.1.smithi026.stdout:7/126: dread - d6/de/d16/f1f zero size 2024-03-16T10:45:09.153 INFO:tasks.workunit.client.1.smithi026.stdout:7/127: chown d6/d1e 447768 1 2024-03-16T10:45:09.155 INFO:tasks.workunit.client.1.smithi026.stdout:2/129: creat d2/d14/d17/f27 x:0 0 0 2024-03-16T10:45:09.156 INFO:tasks.workunit.client.1.smithi026.stdout:2/130: dread - d2/d14/d17/d1a/f24 zero size 2024-03-16T10:45:09.156 INFO:tasks.workunit.client.1.smithi026.stdout:5/105: creat d1/db/f22 x:0 0 0 2024-03-16T10:45:09.157 INFO:tasks.workunit.client.0.smithi002.stdout:9/942: dwrite d5/d9/d25/f4f [0,4194304] 0 2024-03-16T10:45:09.157 INFO:tasks.workunit.client.1.smithi026.stdout:5/106: chown d1/d1c/f1e 41 1 2024-03-16T10:45:09.159 INFO:tasks.workunit.client.0.smithi002.stdout:4/812: mkdir d2/d6/d32/df6 0 2024-03-16T10:45:09.159 INFO:tasks.workunit.client.0.smithi002.stdout:2/816: creat d7/d27/d3b/dd4/de1/fff x:0 0 0 2024-03-16T10:45:09.163 INFO:tasks.workunit.client.1.smithi026.stdout:9/71: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:09.164 INFO:tasks.workunit.client.1.smithi026.stdout:9/72: chown d0 610443229 1 2024-03-16T10:45:09.165 INFO:tasks.workunit.client.0.smithi002.stdout:1/944: creat d2/d58/d115/f134 x:0 0 0 2024-03-16T10:45:09.167 INFO:tasks.workunit.client.0.smithi002.stdout:3/909: readlink d1/dd/l7e 0 2024-03-16T10:45:09.170 INFO:tasks.workunit.client.0.smithi002.stdout:3/910: dread - d1/dd/d28/d7b/d2c/d63/d105/d126/f129 zero size 2024-03-16T10:45:09.172 INFO:tasks.workunit.client.0.smithi002.stdout:1/945: dwrite d2/df/d12/d84/d87/d96/db5/d1c/da0/db1/f95 [0,4194304] 0 2024-03-16T10:45:09.173 INFO:tasks.workunit.client.0.smithi002.stdout:6/922: write d15/f67 [849686,2224] 0 2024-03-16T10:45:09.174 INFO:tasks.workunit.client.0.smithi002.stdout:6/923: stat d15/dab/dc4/lfa 0 2024-03-16T10:45:09.174 INFO:tasks.workunit.client.0.smithi002.stdout:8/964: unlink d7/d13/db8/fe7 0 2024-03-16T10:45:09.178 INFO:tasks.workunit.client.0.smithi002.stdout:5/768: rmdir d0/d16/dab/d33/d4b/d9d 0 2024-03-16T10:45:09.179 INFO:tasks.workunit.client.0.smithi002.stdout:5/769: chown d0/d16/dab/d33/d4b 4 1 2024-03-16T10:45:09.180 INFO:tasks.workunit.client.0.smithi002.stdout:5/770: chown d0/d1f/f36 41 1 2024-03-16T10:45:09.181 INFO:tasks.workunit.client.0.smithi002.stdout:6/924: dread d15/d2d/d73/d97/fdd [0,4194304] 0 2024-03-16T10:45:09.182 INFO:tasks.workunit.client.1.smithi026.stdout:6/143: truncate d0/f11 4117404 0 2024-03-16T10:45:09.184 INFO:tasks.workunit.client.0.smithi002.stdout:9/943: mkdir d5/d9/d22/d3c/d134 0 2024-03-16T10:45:09.187 INFO:tasks.workunit.client.0.smithi002.stdout:0/869: write d3/f3a [87887,921] 0 2024-03-16T10:45:09.188 INFO:tasks.workunit.client.0.smithi002.stdout:2/817: creat d7/d27/d3b/f100 x:0 0 0 2024-03-16T10:45:09.193 INFO:tasks.workunit.client.1.smithi026.stdout:2/131: mknod d2/dd/c28 0 2024-03-16T10:45:09.198 INFO:tasks.workunit.client.1.smithi026.stdout:9/73: mkdir d0/da/d13 0 2024-03-16T10:45:09.198 INFO:tasks.workunit.client.1.smithi026.stdout:9/74: readlink d0/lf 0 2024-03-16T10:45:09.198 INFO:tasks.workunit.client.0.smithi002.stdout:1/946: mknod d2/df/d12/d84/d87/d96/db5/d1c/d5d/d71/c135 0 2024-03-16T10:45:09.199 INFO:tasks.workunit.client.1.smithi026.stdout:4/132: truncate d1/f18 1450582 0 2024-03-16T10:45:09.201 INFO:tasks.workunit.client.1.smithi026.stdout:9/75: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:09.202 INFO:tasks.workunit.client.0.smithi002.stdout:8/965: rmdir d7/d9/d17/d23/d2e/d34/dde 39 2024-03-16T10:45:09.205 INFO:tasks.workunit.client.0.smithi002.stdout:6/925: fdatasync d15/d2d/d2b/d3f/fc0 0 2024-03-16T10:45:09.206 INFO:tasks.workunit.client.1.smithi026.stdout:3/126: mkdir d4/d35 0 2024-03-16T10:45:09.211 INFO:tasks.workunit.client.1.smithi026.stdout:0/140: truncate d1/d18/d1a/d24/d6/fe 1209369 0 2024-03-16T10:45:09.214 INFO:tasks.workunit.client.0.smithi002.stdout:4/813: dwrite d2/d6/d32/d66/d7f/de7/d2e/f8e [0,4194304] 0 2024-03-16T10:45:09.216 INFO:tasks.workunit.client.1.smithi026.stdout:9/76: mkdir d0/da/d14 0 2024-03-16T10:45:09.218 INFO:tasks.workunit.client.1.smithi026.stdout:9/77: write d0/da/f12 [193839,95576] 0 2024-03-16T10:45:09.219 INFO:tasks.workunit.client.0.smithi002.stdout:3/911: write d1/db6/d15/f17 [798640,87240] 0 2024-03-16T10:45:09.221 INFO:tasks.workunit.client.0.smithi002.stdout:8/966: creat d7/d9/d17/d20/f140 x:0 0 0 2024-03-16T10:45:09.222 INFO:tasks.workunit.client.0.smithi002.stdout:3/912: fdatasync d1/dd/d28/d7b/d8a/fc8 0 2024-03-16T10:45:09.223 INFO:tasks.workunit.client.0.smithi002.stdout:1/947: dwrite d2/df/d12/d84/d87/d96/db5/d1c/da0/ff2 [0,4194304] 0 2024-03-16T10:45:09.225 INFO:tasks.workunit.client.0.smithi002.stdout:8/967: chown d7/d9/d17/dc3/fc7 5 1 2024-03-16T10:45:09.225 INFO:tasks.workunit.client.0.smithi002.stdout:5/771: truncate d0/d16/dab/f3b 923802 0 2024-03-16T10:45:09.226 INFO:tasks.workunit.client.0.smithi002.stdout:1/948: read - d2/df/d12/d84/d87/d96/db5/d1c/d5d/f82 zero size 2024-03-16T10:45:09.226 INFO:tasks.workunit.client.0.smithi002.stdout:5/772: read d0/d16/dab/daa/faf [319218,77483] 0 2024-03-16T10:45:09.227 INFO:tasks.workunit.client.1.smithi026.stdout:7/128: rmdir d6/de/d16/d21 0 2024-03-16T10:45:09.227 INFO:tasks.workunit.client.0.smithi002.stdout:8/968: chown d7/d9/d17/d83/ff1 213 1 2024-03-16T10:45:09.229 INFO:tasks.workunit.client.1.smithi026.stdout:0/141: creat d1/d18/d1a/d24/d6/f31 x:0 0 0 2024-03-16T10:45:09.229 INFO:tasks.workunit.client.0.smithi002.stdout:8/969: chown d7/d13/db8/d8f/ffd 2 1 2024-03-16T10:45:09.229 INFO:tasks.workunit.client.0.smithi002.stdout:5/773: write d0/d16/dab/d33/d8c/fa5 [747310,80338] 0 2024-03-16T10:45:09.230 INFO:tasks.workunit.client.1.smithi026.stdout:3/127: write d4/dc/dd/f12 [3639675,25575] 0 2024-03-16T10:45:09.230 INFO:tasks.workunit.client.0.smithi002.stdout:6/926: dread - d15/dab/dc4/fd3 zero size 2024-03-16T10:45:09.233 INFO:tasks.workunit.client.1.smithi026.stdout:7/129: dread d6/de/d16/f1a [0,4194304] 0 2024-03-16T10:45:09.234 INFO:tasks.workunit.client.0.smithi002.stdout:0/870: mkdir d3/d28/d3d/d119 0 2024-03-16T10:45:09.234 INFO:tasks.workunit.client.0.smithi002.stdout:6/927: write d15/d2d/d8a/d7c/d87/fa4 [4897898,94932] 0 2024-03-16T10:45:09.234 INFO:tasks.workunit.client.0.smithi002.stdout:5/774: truncate d0/d24/dad/db6/db9/fc2 86485 0 2024-03-16T10:45:09.235 INFO:tasks.workunit.client.1.smithi026.stdout:7/130: write d6/de/f11 [3665616,83552] 0 2024-03-16T10:45:09.236 INFO:tasks.workunit.client.1.smithi026.stdout:6/144: fdatasync d0/f11 0 2024-03-16T10:45:09.236 INFO:tasks.workunit.client.0.smithi002.stdout:0/871: stat d3/d28/d3d 0 2024-03-16T10:45:09.236 INFO:tasks.workunit.client.0.smithi002.stdout:7/770: getdents d3/d61/d85 0 2024-03-16T10:45:09.239 INFO:tasks.workunit.client.0.smithi002.stdout:4/814: write d2/d33/f64 [3485062,96987] 0 2024-03-16T10:45:09.243 INFO:tasks.workunit.client.1.smithi026.stdout:9/78: dwrite d0/da/f11 [0,4194304] 0 2024-03-16T10:45:09.244 INFO:tasks.workunit.client.1.smithi026.stdout:2/132: rename d2/d14 to d2/d29 0 2024-03-16T10:45:09.247 INFO:tasks.workunit.client.0.smithi002.stdout:1/949: dread d2/df/d12/d29/f34 [0,4194304] 0 2024-03-16T10:45:09.247 INFO:tasks.workunit.client.0.smithi002.stdout:3/913: rename d1/dd/d28/d7b/d73/l92 to d1/dd/d28/d7b/d22/l140 0 2024-03-16T10:45:09.248 INFO:tasks.workunit.client.1.smithi026.stdout:5/107: getdents d1/db/d1f 0 2024-03-16T10:45:09.250 INFO:tasks.workunit.client.0.smithi002.stdout:9/944: rmdir d5/d9/dc/d1e/d24/d31/d132 0 2024-03-16T10:45:09.251 INFO:tasks.workunit.client.1.smithi026.stdout:3/128: creat d4/dc/f36 x:0 0 0 2024-03-16T10:45:09.251 INFO:tasks.workunit.client.0.smithi002.stdout:7/771: dread d3/d4/d1c/d5b/fca [0,4194304] 0 2024-03-16T10:45:09.251 INFO:tasks.workunit.client.1.smithi026.stdout:3/129: read - d4/f1b zero size 2024-03-16T10:45:09.252 INFO:tasks.workunit.client.1.smithi026.stdout:3/130: dread - d4/dc/f25 zero size 2024-03-16T10:45:09.255 INFO:tasks.workunit.client.1.smithi026.stdout:0/142: rename d1/f21 to d1/d18/d1a/d2b/f32 0 2024-03-16T10:45:09.256 INFO:tasks.workunit.client.0.smithi002.stdout:6/928: unlink d15/f19 0 2024-03-16T10:45:09.258 INFO:tasks.workunit.client.1.smithi026.stdout:4/133: fdatasync d1/f27 0 2024-03-16T10:45:09.262 INFO:tasks.workunit.client.1.smithi026.stdout:7/131: creat d6/f22 x:0 0 0 2024-03-16T10:45:09.266 INFO:tasks.workunit.client.0.smithi002.stdout:9/945: dread d5/f97 [0,4194304] 0 2024-03-16T10:45:09.274 INFO:tasks.workunit.client.0.smithi002.stdout:1/950: mkdir d2/df/d12/d84/d87/d96/db5/dde/d136 0 2024-03-16T10:45:09.275 INFO:tasks.workunit.client.1.smithi026.stdout:1/116: sync 2024-03-16T10:45:09.276 INFO:tasks.workunit.client.1.smithi026.stdout:1/117: chown d0/d17/d1f/l9 366755 1 2024-03-16T10:45:09.278 INFO:tasks.workunit.client.1.smithi026.stdout:3/131: rename d4/dc/lf to d4/dc/d31/l37 0 2024-03-16T10:45:09.279 INFO:tasks.workunit.client.1.smithi026.stdout:9/79: write d0/f4 [2053038,52446] 0 2024-03-16T10:45:09.280 INFO:tasks.workunit.client.1.smithi026.stdout:7/132: mknod d6/d1e/c23 0 2024-03-16T10:45:09.281 INFO:tasks.workunit.client.0.smithi002.stdout:1/951: dwrite d2/df/d12/d74/ffa [0,4194304] 0 2024-03-16T10:45:09.281 INFO:tasks.workunit.client.0.smithi002.stdout:6/929: mknod d15/dab/c140 0 2024-03-16T10:45:09.284 INFO:tasks.workunit.client.0.smithi002.stdout:0/872: mknod d3/d5/c11a 0 2024-03-16T10:45:09.287 INFO:tasks.workunit.client.0.smithi002.stdout:3/914: dwrite d1/dd/d28/d7b/d2c/fa3 [0,4194304] 0 2024-03-16T10:45:09.289 INFO:tasks.workunit.client.0.smithi002.stdout:9/946: mkdir d5/d9/d55/d135 0 2024-03-16T10:45:09.291 INFO:tasks.workunit.client.0.smithi002.stdout:4/815: mkdir d2/d6/d32/d66/df7 0 2024-03-16T10:45:09.292 INFO:tasks.workunit.client.1.smithi026.stdout:6/145: link d0/c13 d0/d28/d2d/c34 0 2024-03-16T10:45:09.295 INFO:tasks.workunit.client.1.smithi026.stdout:5/108: link d1/f9 d1/db/d1a/f23 0 2024-03-16T10:45:09.301 INFO:tasks.workunit.client.0.smithi002.stdout:5/775: creat d0/d16/dab/feb x:0 0 0 2024-03-16T10:45:09.303 INFO:tasks.workunit.client.1.smithi026.stdout:1/118: write d0/fd [94631,17466] 0 2024-03-16T10:45:09.304 INFO:tasks.workunit.client.1.smithi026.stdout:2/133: getdents d2/d29 0 2024-03-16T10:45:09.307 INFO:tasks.workunit.client.1.smithi026.stdout:5/109: symlink d1/d1c/l24 0 2024-03-16T10:45:09.307 INFO:tasks.workunit.client.1.smithi026.stdout:7/133: write d6/de/d16/f19 [1376345,7946] 0 2024-03-16T10:45:09.308 INFO:tasks.workunit.client.0.smithi002.stdout:0/873: truncate d3/d28/d65/f6a 1387848 0 2024-03-16T10:45:09.309 INFO:tasks.workunit.client.1.smithi026.stdout:4/134: rename d1/db/f16 to d1/f2d 0 2024-03-16T10:45:09.310 INFO:tasks.workunit.client.1.smithi026.stdout:9/80: rename d0/da to d0/da/d14/d15 22 2024-03-16T10:45:09.312 INFO:tasks.workunit.client.0.smithi002.stdout:7/772: truncate d3/d61/f83 7228833 0 2024-03-16T10:45:09.324 INFO:tasks.workunit.client.0.smithi002.stdout:5/776: creat d0/d16/d4a/d55/fec x:0 0 0 2024-03-16T10:45:09.324 INFO:tasks.workunit.client.0.smithi002.stdout:1/952: mkdir d2/d113/d137 0 2024-03-16T10:45:09.326 INFO:tasks.workunit.client.0.smithi002.stdout:0/874: write d3/d28/d2a/d67/fdd [2769537,20527] 0 2024-03-16T10:45:09.326 INFO:tasks.workunit.client.1.smithi026.stdout:9/81: mkdir d0/da/d16 0 2024-03-16T10:45:09.327 INFO:tasks.workunit.client.1.smithi026.stdout:1/119: dwrite d0/d17/d1f/fc [8388608,4194304] 0 2024-03-16T10:45:09.328 INFO:tasks.workunit.client.0.smithi002.stdout:3/915: mkdir d1/db6/d15/d87/d8c/dfd/d141 0 2024-03-16T10:45:09.329 INFO:tasks.workunit.client.1.smithi026.stdout:1/120: write d0/fd [2143188,6188] 0 2024-03-16T10:45:09.331 INFO:tasks.workunit.client.1.smithi026.stdout:3/132: getdents d4 0 2024-03-16T10:45:09.331 INFO:tasks.workunit.client.1.smithi026.stdout:4/135: dread d1/f3 [0,4194304] 0 2024-03-16T10:45:09.332 INFO:tasks.workunit.client.0.smithi002.stdout:4/816: mknod d2/cf8 0 2024-03-16T10:45:09.332 INFO:tasks.workunit.client.1.smithi026.stdout:6/146: creat d0/f35 x:0 0 0 2024-03-16T10:45:09.333 INFO:tasks.workunit.client.1.smithi026.stdout:9/82: dread d0/da/fe [0,4194304] 0 2024-03-16T10:45:09.334 INFO:tasks.workunit.client.0.smithi002.stdout:1/953: dwrite d2/df/d12/d84/d87/d96/db5/f8 [0,4194304] 0 2024-03-16T10:45:09.335 INFO:tasks.workunit.client.0.smithi002.stdout:8/970: rename d7/d13/db8/f7c to d7/d9/d17/d23/d39/f141 0 2024-03-16T10:45:09.336 INFO:tasks.workunit.client.0.smithi002.stdout:7/773: creat d3/d4/d1d/d87/d96/d5e/db7/f114 x:0 0 0 2024-03-16T10:45:09.337 INFO:tasks.workunit.client.0.smithi002.stdout:9/947: symlink d5/d9/dc/d1e/d91/df1/d129/l136 0 2024-03-16T10:45:09.338 INFO:tasks.workunit.client.0.smithi002.stdout:9/948: fsync d5/fc4 0 2024-03-16T10:45:09.341 INFO:tasks.workunit.client.0.smithi002.stdout:8/971: dread d7/d9/d17/d20/f5e [0,4194304] 0 2024-03-16T10:45:09.343 INFO:tasks.workunit.client.1.smithi026.stdout:9/83: dwrite d0/f4 [4194304,4194304] 0 2024-03-16T10:45:09.345 INFO:tasks.workunit.client.1.smithi026.stdout:1/121: symlink d0/d17/l24 0 2024-03-16T10:45:09.348 INFO:tasks.workunit.client.1.smithi026.stdout:9/84: dread d0/da/f12 [0,4194304] 0 2024-03-16T10:45:09.349 INFO:tasks.workunit.client.1.smithi026.stdout:4/136: fsync d1/db/de/f1d 0 2024-03-16T10:45:09.357 INFO:tasks.workunit.client.1.smithi026.stdout:6/147: dread d0/d10/ff [0,4194304] 0 2024-03-16T10:45:09.362 INFO:tasks.workunit.client.1.smithi026.stdout:1/122: readlink d0/l1d 0 2024-03-16T10:45:09.363 INFO:tasks.workunit.client.1.smithi026.stdout:1/123: write d0/fd [839072,124872] 0 2024-03-16T10:45:09.365 INFO:tasks.workunit.client.1.smithi026.stdout:3/133: rename d4/dc/dd/l1a to d4/l38 0 2024-03-16T10:45:09.371 INFO:tasks.workunit.client.1.smithi026.stdout:2/134: getdents d2/d29/d17/d1a 0 2024-03-16T10:45:09.372 INFO:tasks.workunit.client.1.smithi026.stdout:2/135: fsync d2/d29/d17/f23 0 2024-03-16T10:45:09.372 INFO:tasks.workunit.client.1.smithi026.stdout:2/136: fdatasync d2/dd/f18 0 2024-03-16T10:45:09.373 INFO:tasks.workunit.client.1.smithi026.stdout:5/110: getdents d1/d1c 0 2024-03-16T10:45:09.373 INFO:tasks.workunit.client.1.smithi026.stdout:5/111: readlink d1/l6 0 2024-03-16T10:45:09.374 INFO:tasks.workunit.client.1.smithi026.stdout:5/112: dread - d1/db/d1a/f1b zero size 2024-03-16T10:45:09.375 INFO:tasks.workunit.client.1.smithi026.stdout:6/148: chown d0/d10/cb 3830588 1 2024-03-16T10:45:09.375 INFO:tasks.workunit.client.1.smithi026.stdout:6/149: chown d0/d10/ff 190546 1 2024-03-16T10:45:09.376 INFO:tasks.workunit.client.1.smithi026.stdout:6/150: readlink d0/l2f 0 2024-03-16T10:45:09.376 INFO:tasks.workunit.client.1.smithi026.stdout:6/151: chown d0/f2c 409586 1 2024-03-16T10:45:09.378 INFO:tasks.workunit.client.1.smithi026.stdout:1/124: mkdir d0/d17/d1f/d25 0 2024-03-16T10:45:09.381 INFO:tasks.workunit.client.1.smithi026.stdout:9/85: mkdir d0/da/d16/d17 0 2024-03-16T10:45:09.382 INFO:tasks.workunit.client.0.smithi002.stdout:2/818: sync 2024-03-16T10:45:09.383 INFO:tasks.workunit.client.0.smithi002.stdout:2/819: write d7/d2e/d1d/d2f/d4b/f74 [5978421,77867] 0 2024-03-16T10:45:09.384 INFO:tasks.workunit.client.0.smithi002.stdout:2/820: write d7/d27/d3b/d95/da5/fbc [123749,104680] 0 2024-03-16T10:45:09.393 INFO:tasks.workunit.client.1.smithi026.stdout:5/113: mknod d1/db/d1a/c25 0 2024-03-16T10:45:09.399 INFO:tasks.workunit.client.1.smithi026.stdout:5/114: dwrite d1/fd [0,4194304] 0 2024-03-16T10:45:09.400 INFO:tasks.workunit.client.1.smithi026.stdout:9/86: symlink d0/da/d16/l18 0 2024-03-16T10:45:09.404 INFO:tasks.workunit.client.1.smithi026.stdout:2/137: symlink d2/l2a 0 2024-03-16T10:45:09.411 INFO:tasks.workunit.client.1.smithi026.stdout:2/138: dwrite d2/d29/d17/f27 [0,4194304] 0 2024-03-16T10:45:09.415 INFO:tasks.workunit.client.1.smithi026.stdout:9/87: fdatasync d0/da/f12 0 2024-03-16T10:45:09.418 INFO:tasks.workunit.client.1.smithi026.stdout:6/152: creat d0/d10/f36 x:0 0 0 2024-03-16T10:45:09.421 INFO:tasks.workunit.client.0.smithi002.stdout:5/777: mkdir d0/d16/d4a/d7d/ded 0 2024-03-16T10:45:09.423 INFO:tasks.workunit.client.1.smithi026.stdout:5/115: fsync d1/db/d1a/f23 0 2024-03-16T10:45:09.426 INFO:tasks.workunit.client.0.smithi002.stdout:0/875: symlink d3/d27/d90/dbb/l11b 0 2024-03-16T10:45:09.427 INFO:tasks.workunit.client.0.smithi002.stdout:0/876: read - d3/d31/d88/f8b zero size 2024-03-16T10:45:09.428 INFO:tasks.workunit.client.1.smithi026.stdout:4/137: truncate d1/db/fc 2231463 0 2024-03-16T10:45:09.429 INFO:tasks.workunit.client.1.smithi026.stdout:3/134: creat d4/d35/f39 x:0 0 0 2024-03-16T10:45:09.430 INFO:tasks.workunit.client.0.smithi002.stdout:4/817: creat d2/d6/ff9 x:0 0 0 2024-03-16T10:45:09.437 INFO:tasks.workunit.client.1.smithi026.stdout:9/88: mkdir d0/da/d16/d17/d19 0 2024-03-16T10:45:09.437 INFO:tasks.workunit.client.1.smithi026.stdout:9/89: chown d0/ld 544429007 1 2024-03-16T10:45:09.439 INFO:tasks.workunit.client.1.smithi026.stdout:4/138: rmdir d1/db/de 39 2024-03-16T10:45:09.442 INFO:tasks.workunit.client.1.smithi026.stdout:3/135: dread d4/dc/f13 [0,4194304] 0 2024-03-16T10:45:09.444 INFO:tasks.workunit.client.1.smithi026.stdout:1/125: truncate d0/d17/d1f/fc 6170712 0 2024-03-16T10:45:09.445 INFO:tasks.workunit.client.0.smithi002.stdout:2/821: symlink d7/d2e/d28/d44/l101 0 2024-03-16T10:45:09.447 INFO:tasks.workunit.client.0.smithi002.stdout:6/930: getdents d15/d2d/d2b/d3f/d77/d112 0 2024-03-16T10:45:09.448 INFO:tasks.workunit.client.1.smithi026.stdout:3/136: dwrite d4/dc/d31/f34 [0,4194304] 0 2024-03-16T10:45:09.448 INFO:tasks.workunit.client.1.smithi026.stdout:8/82: sync 2024-03-16T10:45:09.449 INFO:tasks.workunit.client.0.smithi002.stdout:5/778: fdatasync d0/d16/dab/d33/d8c/f98 0 2024-03-16T10:45:09.450 INFO:tasks.workunit.client.1.smithi026.stdout:8/83: dread d5/d7/f10 [0,4194304] 0 2024-03-16T10:45:09.451 INFO:tasks.workunit.client.0.smithi002.stdout:2/822: dwrite d7/d2e/d28/d40/ff9 [0,4194304] 0 2024-03-16T10:45:09.458 INFO:tasks.workunit.client.1.smithi026.stdout:8/84: dwrite d5/f6 [0,4194304] 0 2024-03-16T10:45:09.463 INFO:tasks.workunit.client.1.smithi026.stdout:9/90: mknod d0/da/d14/c1a 0 2024-03-16T10:45:09.464 INFO:tasks.workunit.client.1.smithi026.stdout:2/139: dwrite d2/d29/d17/d1a/f24 [0,4194304] 0 2024-03-16T10:45:09.464 INFO:tasks.workunit.client.0.smithi002.stdout:2/823: dread d7/d2e/d1d/d2f/d4b/fa0 [0,4194304] 0 2024-03-16T10:45:09.465 INFO:tasks.workunit.client.0.smithi002.stdout:1/954: truncate d2/df/d12/d84/d87/d96/db5/d51/d56/fc5 22584 0 2024-03-16T10:45:09.469 INFO:tasks.workunit.client.0.smithi002.stdout:7/774: creat d3/d4/d1d/d87/d5c/d63/d3f/dba/dcc/f115 x:0 0 0 2024-03-16T10:45:09.470 INFO:tasks.workunit.client.1.smithi026.stdout:4/139: fsync d1/f22 0 2024-03-16T10:45:09.480 INFO:tasks.workunit.client.1.smithi026.stdout:6/153: getdents d0/d28/d2d 0 2024-03-16T10:45:09.482 INFO:tasks.workunit.client.0.smithi002.stdout:5/779: creat d0/d24/fee x:0 0 0 2024-03-16T10:45:09.483 INFO:tasks.workunit.client.1.smithi026.stdout:5/116: getdents d1 0 2024-03-16T10:45:09.488 INFO:tasks.workunit.client.1.smithi026.stdout:8/85: rename d5/f8 to d5/dc/f18 0 2024-03-16T10:45:09.490 INFO:tasks.workunit.client.0.smithi002.stdout:0/877: creat d3/da0/f11c x:0 0 0 2024-03-16T10:45:09.490 INFO:tasks.workunit.client.1.smithi026.stdout:1/126: mkdir d0/d26 0 2024-03-16T10:45:09.490 INFO:tasks.workunit.client.1.smithi026.stdout:1/127: readlink d0/l1d 0 2024-03-16T10:45:09.492 INFO:tasks.workunit.client.1.smithi026.stdout:0/143: sync 2024-03-16T10:45:09.492 INFO:tasks.workunit.client.1.smithi026.stdout:7/134: sync 2024-03-16T10:45:09.493 INFO:tasks.workunit.client.0.smithi002.stdout:2/824: creat d7/d27/d3b/d95/da5/f102 x:0 0 0 2024-03-16T10:45:09.493 INFO:tasks.workunit.client.1.smithi026.stdout:4/140: stat d1/db/de/c1e 0 2024-03-16T10:45:09.496 INFO:tasks.workunit.client.1.smithi026.stdout:3/137: creat d4/d29/f3a x:0 0 0 2024-03-16T10:45:09.498 INFO:tasks.workunit.client.1.smithi026.stdout:6/154: rename d0/d28/f2e to d0/d28/f37 0 2024-03-16T10:45:09.500 INFO:tasks.workunit.client.0.smithi002.stdout:8/972: creat d7/d9/d17/d83/f142 x:0 0 0 2024-03-16T10:45:09.504 INFO:tasks.workunit.client.0.smithi002.stdout:5/780: read - d0/d16/dab/d43/d48/f8f zero size 2024-03-16T10:45:09.505 INFO:tasks.workunit.client.1.smithi026.stdout:2/140: rename d2/dd to d2/d29/d17/d1a/d2b 0 2024-03-16T10:45:09.507 INFO:tasks.workunit.client.1.smithi026.stdout:8/86: symlink d5/dc/l19 0 2024-03-16T10:45:09.508 INFO:tasks.workunit.client.0.smithi002.stdout:3/916: getdents d1/dd/d28/d7b/d2c/d63/d105 0 2024-03-16T10:45:09.509 INFO:tasks.workunit.client.0.smithi002.stdout:3/917: chown d1/db6/d15/c10e 100 1 2024-03-16T10:45:09.514 INFO:tasks.workunit.client.1.smithi026.stdout:0/144: symlink d1/d18/d1a/l33 0 2024-03-16T10:45:09.514 INFO:tasks.workunit.client.0.smithi002.stdout:1/955: dwrite d2/df/d12/d74/ff9 [0,4194304] 0 2024-03-16T10:45:09.518 INFO:tasks.workunit.client.1.smithi026.stdout:5/117: dwrite d1/db/d1a/f23 [0,4194304] 0 2024-03-16T10:45:09.518 INFO:tasks.workunit.client.1.smithi026.stdout:3/138: creat d4/dc/d31/f3b x:0 0 0 2024-03-16T10:45:09.518 INFO:tasks.workunit.client.0.smithi002.stdout:9/949: getdents d5/d9/d34 0 2024-03-16T10:45:09.519 INFO:tasks.workunit.client.1.smithi026.stdout:9/91: truncate d0/f4 5491404 0 2024-03-16T10:45:09.520 INFO:tasks.workunit.client.0.smithi002.stdout:6/931: dwrite d15/d2d/d2b/f53 [0,4194304] 0 2024-03-16T10:45:09.520 INFO:tasks.workunit.client.1.smithi026.stdout:3/139: chown d4/dc/dd/c22 52295 1 2024-03-16T10:45:09.520 INFO:tasks.workunit.client.1.smithi026.stdout:3/140: stat d4/dc/f13 0 2024-03-16T10:45:09.521 INFO:tasks.workunit.client.0.smithi002.stdout:9/950: chown d5/d9/dc/d1e/d94/f95 3 1 2024-03-16T10:45:09.522 INFO:tasks.workunit.client.0.smithi002.stdout:8/973: chown d7/d9/l80 11226451 1 2024-03-16T10:45:09.523 INFO:tasks.workunit.client.0.smithi002.stdout:8/974: chown d7/d9/d25/dc9/f100 2063670 1 2024-03-16T10:45:09.526 INFO:tasks.workunit.client.0.smithi002.stdout:8/975: chown d7/d9/d17/d23/d2e/de5 46 1 2024-03-16T10:45:09.526 INFO:tasks.workunit.client.0.smithi002.stdout:5/781: write d0/d16/dab/d2b/d5d/f5a [2810569,69544] 0 2024-03-16T10:45:09.528 INFO:tasks.workunit.client.1.smithi026.stdout:1/128: rename d0/f11 to d0/d17/d1f/d1c/f27 0 2024-03-16T10:45:09.529 INFO:tasks.workunit.client.0.smithi002.stdout:3/918: dread d1/dd/f13c [0,4194304] 0 2024-03-16T10:45:09.530 INFO:tasks.workunit.client.0.smithi002.stdout:0/878: creat d3/d28/d65/d108/f11d x:0 0 0 2024-03-16T10:45:09.530 INFO:tasks.workunit.client.0.smithi002.stdout:4/818: getdents d2/d6/d32/d66/d7f/de7/d88 0 2024-03-16T10:45:09.531 INFO:tasks.workunit.client.0.smithi002.stdout:0/879: read - d3/d27/fc2 zero size 2024-03-16T10:45:09.533 INFO:tasks.workunit.client.0.smithi002.stdout:2/825: link d7/d2e/d28/d40/ff9 d7/d27/d3b/dd4/f103 0 2024-03-16T10:45:09.535 INFO:tasks.workunit.client.1.smithi026.stdout:5/118: rmdir d1/db/d1f 39 2024-03-16T10:45:09.536 INFO:tasks.workunit.client.1.smithi026.stdout:9/92: write d0/da/fe [2703237,79324] 0 2024-03-16T10:45:09.537 INFO:tasks.workunit.client.1.smithi026.stdout:3/141: chown d4/l28 0 1 2024-03-16T10:45:09.539 INFO:tasks.workunit.client.0.smithi002.stdout:8/976: mknod d7/d9/d17/d20/c143 0 2024-03-16T10:45:09.539 INFO:tasks.workunit.client.0.smithi002.stdout:8/977: stat d7/d13/db8/de0/f128 0 2024-03-16T10:45:09.542 INFO:tasks.workunit.client.0.smithi002.stdout:8/978: write d7/d9/d17/d23/d2e/d5c/df0/f12a [739418,80068] 0 2024-03-16T10:45:09.542 INFO:tasks.workunit.client.1.smithi026.stdout:2/141: mknod d2/c2c 0 2024-03-16T10:45:09.544 INFO:tasks.workunit.client.1.smithi026.stdout:2/142: write d2/d29/d17/d1a/d2b/f15 [4636927,16271] 0 2024-03-16T10:45:09.546 INFO:tasks.workunit.client.0.smithi002.stdout:3/919: creat d1/dd/d28/d7b/d2c/d63/d105/f142 x:0 0 0 2024-03-16T10:45:09.547 INFO:tasks.workunit.client.0.smithi002.stdout:4/819: readlink d2/d6/d32/d66/d7f/de7/d5e/l98 0 2024-03-16T10:45:09.548 INFO:tasks.workunit.client.0.smithi002.stdout:4/820: write d2/d33/d1c/fba [4043534,54629] 0 2024-03-16T10:45:09.548 INFO:tasks.workunit.client.1.smithi026.stdout:1/129: mknod d0/d17/c28 0 2024-03-16T10:45:09.549 INFO:tasks.workunit.client.1.smithi026.stdout:7/135: write d6/fc [1580212,130480] 0 2024-03-16T10:45:09.549 INFO:tasks.workunit.client.1.smithi026.stdout:1/130: read d0/d17/f1a [39405,5862] 0 2024-03-16T10:45:09.554 INFO:tasks.workunit.client.0.smithi002.stdout:2/826: rename d7/d2e/d1d/d2f/c65 to d7/d27/d3f/c104 0 2024-03-16T10:45:09.554 INFO:tasks.workunit.client.0.smithi002.stdout:2/827: readlink d7/db/l1b 0 2024-03-16T10:45:09.557 INFO:tasks.workunit.client.0.smithi002.stdout:6/932: link d15/d2d/d2b/d3f/d77/d7e/fcf d15/d2d/d2c/f141 0 2024-03-16T10:45:09.558 INFO:tasks.workunit.client.0.smithi002.stdout:5/782: readlink d0/d16/dab/daa/ld1 0 2024-03-16T10:45:09.558 INFO:tasks.workunit.client.0.smithi002.stdout:2/828: dread d7/d2e/d1d/f4f [0,4194304] 0 2024-03-16T10:45:09.559 INFO:tasks.workunit.client.1.smithi026.stdout:3/142: mknod d4/dc/d31/c3c 0 2024-03-16T10:45:09.559 INFO:tasks.workunit.client.0.smithi002.stdout:5/783: fsync d0/d24/dad/db6/db9/fd4 0 2024-03-16T10:45:09.560 INFO:tasks.workunit.client.0.smithi002.stdout:8/979: mknod d7/d9/d17/d20/c144 0 2024-03-16T10:45:09.561 INFO:tasks.workunit.client.1.smithi026.stdout:6/155: creat d0/d10/f38 x:0 0 0 2024-03-16T10:45:09.561 INFO:tasks.workunit.client.1.smithi026.stdout:6/156: read - d0/d10/d17/f26 zero size 2024-03-16T10:45:09.562 INFO:tasks.workunit.client.0.smithi002.stdout:3/920: stat d1/da5/fc3 0 2024-03-16T10:45:09.564 INFO:tasks.workunit.client.1.smithi026.stdout:0/145: truncate d1/f1e 286178 0 2024-03-16T10:45:09.565 INFO:tasks.workunit.client.1.smithi026.stdout:2/143: creat d2/d29/d17/f2d x:0 0 0 2024-03-16T10:45:09.566 INFO:tasks.workunit.client.1.smithi026.stdout:2/144: dread - d2/d29/d17/f2d zero size 2024-03-16T10:45:09.566 INFO:tasks.workunit.client.0.smithi002.stdout:4/821: fsync d2/d6/d32/f50 0 2024-03-16T10:45:09.567 INFO:tasks.workunit.client.0.smithi002.stdout:0/880: dwrite d3/d31/d88/fab [0,4194304] 0 2024-03-16T10:45:09.567 INFO:tasks.workunit.client.1.smithi026.stdout:7/136: creat d6/f24 x:0 0 0 2024-03-16T10:45:09.568 INFO:tasks.workunit.client.1.smithi026.stdout:7/137: readlink d6/l8 0 2024-03-16T10:45:09.569 INFO:tasks.workunit.client.1.smithi026.stdout:4/141: getdents d1/db/de 0 2024-03-16T10:45:09.571 INFO:tasks.workunit.client.1.smithi026.stdout:4/142: dread - d1/db/d29/f2c zero size 2024-03-16T10:45:09.574 INFO:tasks.workunit.client.0.smithi002.stdout:8/980: dread d7/d9/d17/d83/ff1 [0,4194304] 0 2024-03-16T10:45:09.574 INFO:tasks.workunit.client.0.smithi002.stdout:7/775: sync 2024-03-16T10:45:09.574 INFO:tasks.workunit.client.1.smithi026.stdout:2/145: dwrite d2/d29/d17/f27 [0,4194304] 0 2024-03-16T10:45:09.575 INFO:tasks.workunit.client.0.smithi002.stdout:6/933: write d15/d2d/f22 [108261,12506] 0 2024-03-16T10:45:09.576 INFO:tasks.workunit.client.0.smithi002.stdout:0/881: read d3/d31/f4b [2323320,93830] 0 2024-03-16T10:45:09.577 INFO:tasks.workunit.client.1.smithi026.stdout:1/131: dwrite d0/d17/d1f/d1c/f27 [0,4194304] 0 2024-03-16T10:45:09.579 INFO:tasks.workunit.client.1.smithi026.stdout:3/143: rename d4/d35 to d4/dc/dd/d3d 0 2024-03-16T10:45:09.580 INFO:tasks.workunit.client.0.smithi002.stdout:2/829: truncate d7/db/d70/f98 42085 0 2024-03-16T10:45:09.585 INFO:tasks.workunit.client.0.smithi002.stdout:5/784: fsync d0/d16/dab/fdf 0 2024-03-16T10:45:09.586 INFO:tasks.workunit.client.0.smithi002.stdout:3/921: creat d1/dd/d28/d37/f143 x:0 0 0 2024-03-16T10:45:09.586 INFO:tasks.workunit.client.1.smithi026.stdout:6/157: dread d0/f3 [0,4194304] 0 2024-03-16T10:45:09.586 INFO:tasks.workunit.client.0.smithi002.stdout:7/776: dwrite d3/d37/d6a/f6e [4194304,4194304] 0 2024-03-16T10:45:09.587 INFO:tasks.workunit.client.0.smithi002.stdout:5/785: readlink d0/d16/d5b/l62 0 2024-03-16T10:45:09.588 INFO:tasks.workunit.client.1.smithi026.stdout:6/158: read d0/d10/fd [169568,24254] 0 2024-03-16T10:45:09.589 INFO:tasks.workunit.client.0.smithi002.stdout:1/956: getdents d2/df/d12/d84/d87/d96/db5/de2/d10a 0 2024-03-16T10:45:09.590 INFO:tasks.workunit.client.1.smithi026.stdout:7/138: rmdir d6 39 2024-03-16T10:45:09.593 INFO:tasks.workunit.client.0.smithi002.stdout:4/822: truncate d2/d6/d32/d66/f82 256870 0 2024-03-16T10:45:09.595 INFO:tasks.workunit.client.1.smithi026.stdout:2/146: mkdir d2/d29/d17/d1a/d2b/d2e 0 2024-03-16T10:45:09.596 INFO:tasks.workunit.client.0.smithi002.stdout:9/951: getdents d5/d9/d22/d3c 0 2024-03-16T10:45:09.598 INFO:tasks.workunit.client.0.smithi002.stdout:6/934: dread d15/d2d/d2c/f31 [0,4194304] 0 2024-03-16T10:45:09.600 INFO:tasks.workunit.client.1.smithi026.stdout:2/147: dread d2/d29/d17/d1a/d2b/f1d [0,4194304] 0 2024-03-16T10:45:09.602 INFO:tasks.workunit.client.1.smithi026.stdout:2/148: write d2/d29/d17/d1a/d2b/f1f [4893091,2007] 0 2024-03-16T10:45:09.602 INFO:tasks.workunit.client.1.smithi026.stdout:2/149: readlink d2/l1c 0 2024-03-16T10:45:09.603 INFO:tasks.workunit.client.1.smithi026.stdout:0/146: symlink d1/d18/l34 0 2024-03-16T10:45:09.603 INFO:tasks.workunit.client.1.smithi026.stdout:0/147: chown d1 11 1 2024-03-16T10:45:09.605 INFO:tasks.workunit.client.0.smithi002.stdout:0/882: creat d3/d31/d75/db4/f11e x:0 0 0 2024-03-16T10:45:09.606 INFO:tasks.workunit.client.0.smithi002.stdout:0/883: read d3/d31/d88/fab [2826877,84695] 0 2024-03-16T10:45:09.608 INFO:tasks.workunit.client.1.smithi026.stdout:6/159: creat d0/d28/d2d/f39 x:0 0 0 2024-03-16T10:45:09.611 INFO:tasks.workunit.client.0.smithi002.stdout:3/922: symlink d1/db6/d15/d87/d8c/l144 0 2024-03-16T10:45:09.611 INFO:tasks.workunit.client.1.smithi026.stdout:9/93: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:09.612 INFO:tasks.workunit.client.1.smithi026.stdout:4/143: truncate d1/f8 218655 0 2024-03-16T10:45:09.615 INFO:tasks.workunit.client.1.smithi026.stdout:1/132: symlink d0/d17/d1f/d25/l29 0 2024-03-16T10:45:09.616 INFO:tasks.workunit.client.0.smithi002.stdout:1/957: dread - d2/df/d12/d84/d87/d96/db5/d51/d56/fdb zero size 2024-03-16T10:45:09.618 INFO:tasks.workunit.client.1.smithi026.stdout:9/94: dwrite d0/da/fe [4194304,4194304] 0 2024-03-16T10:45:09.620 INFO:tasks.workunit.client.1.smithi026.stdout:9/95: dread d0/da/f12 [0,4194304] 0 2024-03-16T10:45:09.621 INFO:tasks.workunit.client.1.smithi026.stdout:0/148: dread d1/d18/d1a/d24/f29 [0,4194304] 0 2024-03-16T10:45:09.623 INFO:tasks.workunit.client.0.smithi002.stdout:2/830: dwrite d7/d2e/d28/d44/f64 [0,4194304] 0 2024-03-16T10:45:09.623 INFO:tasks.workunit.client.0.smithi002.stdout:0/884: mkdir d3/d31/d75/d11f 0 2024-03-16T10:45:09.624 INFO:tasks.workunit.client.1.smithi026.stdout:6/160: chown d0/d10/d17/c24 26 1 2024-03-16T10:45:09.626 INFO:tasks.workunit.client.1.smithi026.stdout:7/139: rename d6/c7 to d6/c25 0 2024-03-16T10:45:09.632 INFO:tasks.workunit.client.0.smithi002.stdout:4/823: write d2/d33/d1c/f83 [547206,69496] 0 2024-03-16T10:45:09.633 INFO:tasks.workunit.client.1.smithi026.stdout:6/161: dwrite d0/f11 [0,4194304] 0 2024-03-16T10:45:09.638 INFO:tasks.workunit.client.1.smithi026.stdout:7/140: dwrite d6/f24 [0,4194304] 0 2024-03-16T10:45:09.638 INFO:tasks.workunit.client.0.smithi002.stdout:1/958: mkdir d2/df/d12/d29/d5c/d138 0 2024-03-16T10:45:09.640 INFO:tasks.workunit.client.0.smithi002.stdout:1/959: truncate d2/df/d12/dd7/fe4 590557 0 2024-03-16T10:45:09.649 INFO:tasks.workunit.client.0.smithi002.stdout:2/831: write d7/f90 [2594949,121294] 0 2024-03-16T10:45:09.649 INFO:tasks.workunit.client.0.smithi002.stdout:0/885: creat d3/d31/daa/ddb/f120 x:0 0 0 2024-03-16T10:45:09.649 INFO:tasks.workunit.client.1.smithi026.stdout:2/150: truncate d2/d29/d17/d1a/d2b/f18 7764791 0 2024-03-16T10:45:09.650 INFO:tasks.workunit.client.1.smithi026.stdout:2/151: read d2/d29/d17/f27 [3664122,56341] 0 2024-03-16T10:45:09.650 INFO:tasks.workunit.client.0.smithi002.stdout:6/935: dread d15/d2d/d8a/d7c/d87/fb0 [0,4194304] 0 2024-03-16T10:45:09.650 INFO:tasks.workunit.client.0.smithi002.stdout:5/786: creat d0/d16/dab/d33/fef x:0 0 0 2024-03-16T10:45:09.651 INFO:tasks.workunit.client.1.smithi026.stdout:2/152: chown d2/d29/d17/f23 203360827 1 2024-03-16T10:45:09.652 INFO:tasks.workunit.client.0.smithi002.stdout:2/832: write d7/db/f7c [4917876,30146] 0 2024-03-16T10:45:09.652 INFO:tasks.workunit.client.0.smithi002.stdout:7/777: rename d3/d4/d1c/f2b to d3/d61/f116 0 2024-03-16T10:45:09.653 INFO:tasks.workunit.client.1.smithi026.stdout:9/96: truncate d0/da/f12 301869 0 2024-03-16T10:45:09.653 INFO:tasks.workunit.client.0.smithi002.stdout:1/960: dwrite d2/df/d12/d84/d87/d96/db5/dae/f11a [0,4194304] 0 2024-03-16T10:45:09.655 INFO:tasks.workunit.client.0.smithi002.stdout:4/824: creat d2/d6/d32/d66/d7f/de7/d5e/d93/ffa x:0 0 0 2024-03-16T10:45:09.657 INFO:tasks.workunit.client.0.smithi002.stdout:9/952: creat d5/d9/d25/d23/f137 x:0 0 0 2024-03-16T10:45:09.658 INFO:tasks.workunit.client.1.smithi026.stdout:4/144: mknod d1/db/c2e 0 2024-03-16T10:45:09.662 INFO:tasks.workunit.client.1.smithi026.stdout:6/162: creat d0/d10/d17/f3a x:0 0 0 2024-03-16T10:45:09.663 INFO:tasks.workunit.client.1.smithi026.stdout:6/163: chown d0/d10/d17/c24 573 1 2024-03-16T10:45:09.664 INFO:tasks.workunit.client.1.smithi026.stdout:7/141: mknod d6/de/d16/c26 0 2024-03-16T10:45:09.666 INFO:tasks.workunit.client.0.smithi002.stdout:9/953: dwrite d5/d9/dc/d1e/d94/fa3 [0,4194304] 0 2024-03-16T10:45:09.666 INFO:tasks.workunit.client.0.smithi002.stdout:6/936: creat d15/dfd/d96/f142 x:0 0 0 2024-03-16T10:45:09.666 INFO:tasks.workunit.client.1.smithi026.stdout:1/133: symlink d0/l2a 0 2024-03-16T10:45:09.668 INFO:tasks.workunit.client.0.smithi002.stdout:0/886: symlink d3/d31/d75/db4/d109/d9c/l121 0 2024-03-16T10:45:09.673 INFO:tasks.workunit.client.0.smithi002.stdout:1/961: mkdir d2/df/d12/d84/d87/d96/d139 0 2024-03-16T10:45:09.675 INFO:tasks.workunit.client.1.smithi026.stdout:2/153: symlink d2/d29/d17/d1a/l2f 0 2024-03-16T10:45:09.677 INFO:tasks.workunit.client.1.smithi026.stdout:2/154: write d2/d29/d17/f21 [2678916,112167] 0 2024-03-16T10:45:09.680 INFO:tasks.workunit.client.0.smithi002.stdout:9/954: creat d5/d9/d25/d23/d81/f138 x:0 0 0 2024-03-16T10:45:09.681 INFO:tasks.workunit.client.1.smithi026.stdout:6/164: mknod d0/d10/d17/c3b 0 2024-03-16T10:45:09.682 INFO:tasks.workunit.client.0.smithi002.stdout:6/937: fdatasync d15/d2d/d2b/d3f/f6d 0 2024-03-16T10:45:09.683 INFO:tasks.workunit.client.0.smithi002.stdout:9/955: truncate d5/d9/dc/d1e/d94/f120 615868 0 2024-03-16T10:45:09.683 INFO:tasks.workunit.client.1.smithi026.stdout:7/142: fdatasync d6/d1e/f20 0 2024-03-16T10:45:09.685 INFO:tasks.workunit.client.1.smithi026.stdout:7/143: chown d6/de/d16/f1f 412 1 2024-03-16T10:45:09.686 INFO:tasks.workunit.client.1.smithi026.stdout:7/144: stat d6/de/d16/f1a 0 2024-03-16T10:45:09.688 INFO:tasks.workunit.client.1.smithi026.stdout:6/165: dwrite d0/d10/f31 [0,4194304] 0 2024-03-16T10:45:09.693 INFO:tasks.workunit.client.0.smithi002.stdout:5/787: write d0/d16/dab/d2b/f4f [1489607,56693] 0 2024-03-16T10:45:09.696 INFO:tasks.workunit.client.0.smithi002.stdout:6/938: creat d15/d80/df5/f143 x:0 0 0 2024-03-16T10:45:09.697 INFO:tasks.workunit.client.0.smithi002.stdout:8/981: sync 2024-03-16T10:45:09.697 INFO:tasks.workunit.client.1.smithi026.stdout:2/155: rename d2/l7 to d2/d29/d17/d1a/l30 0 2024-03-16T10:45:09.698 INFO:tasks.workunit.client.1.smithi026.stdout:0/149: dwrite d1/f1e [0,4194304] 0 2024-03-16T10:45:09.699 INFO:tasks.workunit.client.1.smithi026.stdout:0/150: readlink d1/d18/l34 0 2024-03-16T10:45:09.700 INFO:tasks.workunit.client.1.smithi026.stdout:0/151: dread d1/d18/d1a/d24/f3 [0,4194304] 0 2024-03-16T10:45:09.704 INFO:tasks.workunit.client.0.smithi002.stdout:1/962: mknod d2/df/d12/d84/d87/d96/db5/d1c/d116/c13a 0 2024-03-16T10:45:09.705 INFO:tasks.workunit.client.0.smithi002.stdout:4/825: creat d2/d6/d32/d66/d7f/de7/d5e/ffb x:0 0 0 2024-03-16T10:45:09.706 INFO:tasks.workunit.client.0.smithi002.stdout:1/963: fsync d2/df/d12/d84/d87/d96/db5/f8 0 2024-03-16T10:45:09.707 INFO:tasks.workunit.client.0.smithi002.stdout:6/939: creat d15/d80/dbe/f144 x:0 0 0 2024-03-16T10:45:09.708 INFO:tasks.workunit.client.1.smithi026.stdout:7/145: unlink d6/l14 0 2024-03-16T10:45:09.708 INFO:tasks.workunit.client.1.smithi026.stdout:7/146: dread - d6/d1e/f20 zero size 2024-03-16T10:45:09.710 INFO:tasks.workunit.client.0.smithi002.stdout:5/788: rename d0/d16/d4a/d55/fbb to d0/d16/d4a/d55/ff0 0 2024-03-16T10:45:09.711 INFO:tasks.workunit.client.1.smithi026.stdout:3/144: link d4/d29/f30 d4/dc/dd/d3d/f3e 0 2024-03-16T10:45:09.711 INFO:tasks.workunit.client.1.smithi026.stdout:3/145: read d4/dc/dd/f12 [1250788,26478] 0 2024-03-16T10:45:09.713 INFO:tasks.workunit.client.0.smithi002.stdout:8/982: unlink d7/d9/d17/d23/d39/fbb 0 2024-03-16T10:45:09.713 INFO:tasks.workunit.client.0.smithi002.stdout:1/964: dwrite d2/df/d12/d84/d87/d96/db5/f39 [0,4194304] 0 2024-03-16T10:45:09.715 INFO:tasks.workunit.client.1.smithi026.stdout:3/146: dread d4/dc/f11 [4194304,4194304] 0 2024-03-16T10:45:09.717 INFO:tasks.workunit.client.1.smithi026.stdout:1/134: write d0/d17/d1f/fc [745995,64374] 0 2024-03-16T10:45:09.718 INFO:tasks.workunit.client.0.smithi002.stdout:2/833: getdents d7/d2e/d28 0 2024-03-16T10:45:09.720 INFO:tasks.workunit.client.1.smithi026.stdout:4/145: creat d1/f2f x:0 0 0 2024-03-16T10:45:09.722 INFO:tasks.workunit.client.1.smithi026.stdout:1/135: dread d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:09.723 INFO:tasks.workunit.client.1.smithi026.stdout:0/152: fdatasync d1/d18/d1a/f2c 0 2024-03-16T10:45:09.725 INFO:tasks.workunit.client.1.smithi026.stdout:0/153: truncate d1/d18/d1a/d24/d6/f28 927314 0 2024-03-16T10:45:09.726 INFO:tasks.workunit.client.1.smithi026.stdout:0/154: read d1/d18/d1a/d24/d11/f2d [540628,13587] 0 2024-03-16T10:45:09.727 INFO:tasks.workunit.client.1.smithi026.stdout:6/166: symlink d0/l3c 0 2024-03-16T10:45:09.728 INFO:tasks.workunit.client.0.smithi002.stdout:0/887: getdents d3/d28/d65/d108 0 2024-03-16T10:45:09.729 INFO:tasks.workunit.client.1.smithi026.stdout:7/147: mknod d6/c27 0 2024-03-16T10:45:09.730 INFO:tasks.workunit.client.0.smithi002.stdout:5/789: creat d0/d24/ff1 x:0 0 0 2024-03-16T10:45:09.731 INFO:tasks.workunit.client.1.smithi026.stdout:3/147: creat d4/dc/f3f x:0 0 0 2024-03-16T10:45:09.731 INFO:tasks.workunit.client.0.smithi002.stdout:5/790: chown d0/d16/dab/d33/d8c/f95 25700978 1 2024-03-16T10:45:09.732 INFO:tasks.workunit.client.1.smithi026.stdout:6/167: dread d0/f1d [0,4194304] 0 2024-03-16T10:45:09.733 INFO:tasks.workunit.client.1.smithi026.stdout:6/168: fdatasync d0/d10/d17/f26 0 2024-03-16T10:45:09.733 INFO:tasks.workunit.client.1.smithi026.stdout:6/169: read d0/f1d [1233854,30744] 0 2024-03-16T10:45:09.734 INFO:tasks.workunit.client.1.smithi026.stdout:2/156: rename d2/d29/d17/f21 to d2/d29/d17/d1a/d2b/d2e/f31 0 2024-03-16T10:45:09.736 INFO:tasks.workunit.client.0.smithi002.stdout:2/834: fsync d7/db/f35 0 2024-03-16T10:45:09.736 INFO:tasks.workunit.client.0.smithi002.stdout:8/983: dwrite d7/d9/d17/d23/d2e/ff4 [0,4194304] 0 2024-03-16T10:45:09.738 INFO:tasks.workunit.client.0.smithi002.stdout:6/940: fsync d15/d2d/f6c 0 2024-03-16T10:45:09.740 INFO:tasks.workunit.client.0.smithi002.stdout:7/778: sync 2024-03-16T10:45:09.740 INFO:tasks.workunit.client.0.smithi002.stdout:3/923: sync 2024-03-16T10:45:09.744 INFO:tasks.workunit.client.0.smithi002.stdout:9/956: getdents d5/d9/d25/d23 0 2024-03-16T10:45:09.745 INFO:tasks.workunit.client.1.smithi026.stdout:7/148: mknod d6/c28 0 2024-03-16T10:45:09.746 INFO:tasks.workunit.client.0.smithi002.stdout:2/835: dwrite d7/d2e/d1d/d38/d49/fe5 [0,4194304] 0 2024-03-16T10:45:09.753 INFO:tasks.workunit.client.1.smithi026.stdout:2/157: mknod d2/d29/d17/c32 0 2024-03-16T10:45:09.756 INFO:tasks.workunit.client.1.smithi026.stdout:4/146: unlink d1/f2f 0 2024-03-16T10:45:09.757 INFO:tasks.workunit.client.0.smithi002.stdout:4/826: link d2/d33/d35/d59/c7c d2/d33/d1c/cfc 0 2024-03-16T10:45:09.758 INFO:tasks.workunit.client.0.smithi002.stdout:4/827: stat d2/f77 0 2024-03-16T10:45:09.760 INFO:tasks.workunit.client.0.smithi002.stdout:3/924: mknod d1/db6/d15/d87/d8c/c145 0 2024-03-16T10:45:09.762 INFO:tasks.workunit.client.0.smithi002.stdout:0/888: symlink d3/l122 0 2024-03-16T10:45:09.766 INFO:tasks.workunit.client.1.smithi026.stdout:3/148: dwrite d4/dc/f13 [0,4194304] 0 2024-03-16T10:45:09.768 INFO:tasks.workunit.client.1.smithi026.stdout:3/149: chown d4/dc/l1c 12214313 1 2024-03-16T10:45:09.769 INFO:tasks.workunit.client.1.smithi026.stdout:2/158: creat d2/d29/d17/d1a/d2b/d2e/f33 x:0 0 0 2024-03-16T10:45:09.769 INFO:tasks.workunit.client.0.smithi002.stdout:0/889: dwrite d3/df8/f117 [0,4194304] 0 2024-03-16T10:45:09.771 INFO:tasks.workunit.client.0.smithi002.stdout:2/836: mknod d7/db/d12/c105 0 2024-03-16T10:45:09.772 INFO:tasks.workunit.client.1.smithi026.stdout:4/147: dread d1/fa [0,4194304] 0 2024-03-16T10:45:09.773 INFO:tasks.workunit.client.1.smithi026.stdout:1/136: link d0/d17/d1f/c1b d0/d17/d1f/d25/c2b 0 2024-03-16T10:45:09.777 INFO:tasks.workunit.client.1.smithi026.stdout:0/155: link d1/d18/d1a/d24/l1d d1/d18/d1a/l35 0 2024-03-16T10:45:09.778 INFO:tasks.workunit.client.0.smithi002.stdout:8/984: truncate d7/d9/d17/d20/f63 289269 0 2024-03-16T10:45:09.780 INFO:tasks.workunit.client.1.smithi026.stdout:7/149: truncate d6/de/d16/f18 304418 0 2024-03-16T10:45:09.782 INFO:tasks.workunit.client.1.smithi026.stdout:3/150: chown d4/dc/d31/l37 3 1 2024-03-16T10:45:09.783 INFO:tasks.workunit.client.0.smithi002.stdout:4/828: rmdir d2/d33/d1c/d45 39 2024-03-16T10:45:09.784 INFO:tasks.workunit.client.1.smithi026.stdout:2/159: rename d2/la to d2/d29/d17/d1a/d2b/l34 0 2024-03-16T10:45:09.785 INFO:tasks.workunit.client.0.smithi002.stdout:9/957: dread d5/d9/d25/d23/f8d [0,4194304] 0 2024-03-16T10:45:09.786 INFO:tasks.workunit.client.1.smithi026.stdout:4/148: unlink d1/db/c2e 0 2024-03-16T10:45:09.787 INFO:tasks.workunit.client.0.smithi002.stdout:8/985: dwrite d7/d9/d17/d23/d2e/d34/dbf/dd9/f11c [0,4194304] 0 2024-03-16T10:45:09.791 INFO:tasks.workunit.client.1.smithi026.stdout:0/156: rmdir d1/d18/d1a/d24/d6 39 2024-03-16T10:45:09.793 INFO:tasks.workunit.client.0.smithi002.stdout:0/890: symlink d3/d31/d88/l123 0 2024-03-16T10:45:09.795 INFO:tasks.workunit.client.0.smithi002.stdout:2/837: rmdir d7/d27 39 2024-03-16T10:45:09.798 INFO:tasks.workunit.client.1.smithi026.stdout:2/160: dread d2/d29/d17/d1a/d2b/f1f [0,4194304] 0 2024-03-16T10:45:09.799 INFO:tasks.workunit.client.0.smithi002.stdout:1/965: getdents d2/df/d12/d84/d87/d96/db5/d1c 0 2024-03-16T10:45:09.808 INFO:tasks.workunit.client.1.smithi026.stdout:1/137: symlink d0/d17/d1f/d20/l2c 0 2024-03-16T10:45:09.808 INFO:tasks.workunit.client.1.smithi026.stdout:1/138: dread - d0/f1e zero size 2024-03-16T10:45:09.809 INFO:tasks.workunit.client.1.smithi026.stdout:1/139: stat d0/l10 0 2024-03-16T10:45:09.809 INFO:tasks.workunit.client.0.smithi002.stdout:1/966: dread d2/df/d12/f4e [0,4194304] 0 2024-03-16T10:45:09.811 INFO:tasks.workunit.client.0.smithi002.stdout:7/779: link d3/d61/d66/l112 d3/d4/d1d/d87/d5c/d63/d3f/daa/l117 0 2024-03-16T10:45:09.815 INFO:tasks.workunit.client.1.smithi026.stdout:7/150: link d6/fc d6/d1e/f29 0 2024-03-16T10:45:09.817 INFO:tasks.workunit.client.1.smithi026.stdout:6/170: link d0/d10/c14 d0/c3d 0 2024-03-16T10:45:09.818 INFO:tasks.workunit.client.0.smithi002.stdout:5/791: getdents d0/d16/d4a 0 2024-03-16T10:45:09.821 INFO:tasks.workunit.client.0.smithi002.stdout:0/891: unlink d3/d31/d75/df4/cfa 0 2024-03-16T10:45:09.821 INFO:tasks.workunit.client.0.smithi002.stdout:7/780: dread d3/d61/d66/fb4 [0,4194304] 0 2024-03-16T10:45:09.822 INFO:tasks.workunit.client.1.smithi026.stdout:1/140: dwrite d0/f1e [0,4194304] 0 2024-03-16T10:45:09.823 INFO:tasks.workunit.client.1.smithi026.stdout:6/171: dread - d0/d10/d17/f3a zero size 2024-03-16T10:45:09.823 INFO:tasks.workunit.client.1.smithi026.stdout:8/87: sync 2024-03-16T10:45:09.825 INFO:tasks.workunit.client.1.smithi026.stdout:3/151: dwrite d4/dc/dd/d3d/f3e [0,4194304] 0 2024-03-16T10:45:09.825 INFO:tasks.workunit.client.1.smithi026.stdout:0/157: dwrite d1/d18/d1a/d24/d6/f28 [0,4194304] 0 2024-03-16T10:45:09.826 INFO:tasks.workunit.client.0.smithi002.stdout:6/941: getdents d15/d2d/d2b/d3f 0 2024-03-16T10:45:09.828 INFO:tasks.workunit.client.1.smithi026.stdout:6/172: truncate d0/f33 104482 0 2024-03-16T10:45:09.828 INFO:tasks.workunit.client.0.smithi002.stdout:3/925: link d1/db6/d15/d87/d8c/c94 d1/dd/d28/d7b/d8a/dd7/c146 0 2024-03-16T10:45:09.828 INFO:tasks.workunit.client.0.smithi002.stdout:3/926: chown d1/db6/d15/d87 4852654 1 2024-03-16T10:45:09.829 INFO:tasks.workunit.client.1.smithi026.stdout:4/149: mknod d1/db/c30 0 2024-03-16T10:45:09.831 INFO:tasks.workunit.client.0.smithi002.stdout:4/829: mknod d2/d6/d32/df6/cfd 0 2024-03-16T10:45:09.833 INFO:tasks.workunit.client.0.smithi002.stdout:4/830: fdatasync d2/d6/f89 0 2024-03-16T10:45:09.833 INFO:tasks.workunit.client.1.smithi026.stdout:1/141: dread d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:09.834 INFO:tasks.workunit.client.0.smithi002.stdout:1/967: rmdir d2/df/d12/d84/d87/d96/db5/dde 39 2024-03-16T10:45:09.839 INFO:tasks.workunit.client.0.smithi002.stdout:5/792: creat d0/d24/ff2 x:0 0 0 2024-03-16T10:45:09.839 INFO:tasks.workunit.client.0.smithi002.stdout:0/892: symlink d3/d27/l124 0 2024-03-16T10:45:09.841 INFO:tasks.workunit.client.0.smithi002.stdout:7/781: creat d3/d4/d1d/d87/d5c/d63/d55/f118 x:0 0 0 2024-03-16T10:45:09.841 INFO:tasks.workunit.client.0.smithi002.stdout:0/893: fsync d3/d28/d65/f7a 0 2024-03-16T10:45:09.843 INFO:tasks.workunit.client.1.smithi026.stdout:1/142: dwrite d0/d17/d1f/d1c/f27 [0,4194304] 0 2024-03-16T10:45:09.844 INFO:tasks.workunit.client.0.smithi002.stdout:1/968: dwrite d2/df/d12/d84/d87/d96/db5/d1c/f7f [0,4194304] 0 2024-03-16T10:45:09.844 INFO:tasks.workunit.client.0.smithi002.stdout:2/838: creat d7/d27/d3b/dd4/f106 x:0 0 0 2024-03-16T10:45:09.844 INFO:tasks.workunit.client.1.smithi026.stdout:7/151: dwrite d6/de/d16/f1a [0,4194304] 0 2024-03-16T10:45:09.846 INFO:tasks.workunit.client.1.smithi026.stdout:7/152: write d6/de/d16/f1f [583361,82578] 0 2024-03-16T10:45:09.846 INFO:tasks.workunit.client.1.smithi026.stdout:8/88: creat d5/d7/f1a x:0 0 0 2024-03-16T10:45:09.848 INFO:tasks.workunit.client.1.smithi026.stdout:7/153: write d6/de/d16/f1a [309985,73267] 0 2024-03-16T10:45:09.850 INFO:tasks.workunit.client.1.smithi026.stdout:7/154: read d6/f24 [932962,32370] 0 2024-03-16T10:45:09.852 INFO:tasks.workunit.client.0.smithi002.stdout:3/927: creat d1/dd/d28/d7b/dd5/d123/f147 x:0 0 0 2024-03-16T10:45:09.853 INFO:tasks.workunit.client.1.smithi026.stdout:7/155: dread d6/fc [0,4194304] 0 2024-03-16T10:45:09.853 INFO:tasks.workunit.client.1.smithi026.stdout:0/158: truncate d1/d18/d1a/f26 136634 0 2024-03-16T10:45:09.853 INFO:tasks.workunit.client.1.smithi026.stdout:7/156: fsync d6/f22 0 2024-03-16T10:45:09.855 INFO:tasks.workunit.client.1.smithi026.stdout:4/150: mknod d1/db/de/c31 0 2024-03-16T10:45:09.857 INFO:tasks.workunit.client.0.smithi002.stdout:4/831: truncate d2/d6/d32/d66/d7f/f96 411405 0 2024-03-16T10:45:09.858 INFO:tasks.workunit.client.0.smithi002.stdout:8/986: write d7/d9/d17/d23/d2e/de5/f13c [1726774,73687] 0 2024-03-16T10:45:09.859 INFO:tasks.workunit.client.0.smithi002.stdout:8/987: fdatasync d7/d13/db8/f86 0 2024-03-16T10:45:09.860 INFO:tasks.workunit.client.1.smithi026.stdout:2/161: rename d2/d29/d17/d1a/f24 to d2/f35 0 2024-03-16T10:45:09.861 INFO:tasks.workunit.client.1.smithi026.stdout:2/162: chown d2/d29/d17/d1a/f1b 1 1 2024-03-16T10:45:09.862 INFO:tasks.workunit.client.0.smithi002.stdout:5/793: mkdir d0/d16/d5b/df3 0 2024-03-16T10:45:09.863 INFO:tasks.workunit.client.0.smithi002.stdout:0/894: creat d3/d28/d65/f125 x:0 0 0 2024-03-16T10:45:09.864 INFO:tasks.workunit.client.1.smithi026.stdout:8/89: chown d5/dc/fe 8855260 1 2024-03-16T10:45:09.865 INFO:tasks.workunit.client.0.smithi002.stdout:7/782: mkdir d3/d4/d1d/d87/d5c/d63/d3f/dba/dcc/d119 0 2024-03-16T10:45:09.866 INFO:tasks.workunit.client.1.smithi026.stdout:1/143: fdatasync d0/d17/d1f/d1c/f27 0 2024-03-16T10:45:09.866 INFO:tasks.workunit.client.0.smithi002.stdout:9/958: sync 2024-03-16T10:45:09.867 INFO:tasks.workunit.client.0.smithi002.stdout:2/839: mkdir d7/d2e/d1d/d107 0 2024-03-16T10:45:09.867 INFO:tasks.workunit.client.1.smithi026.stdout:5/119: sync 2024-03-16T10:45:09.867 INFO:tasks.workunit.client.1.smithi026.stdout:6/173: mknod d0/c3e 0 2024-03-16T10:45:09.868 INFO:tasks.workunit.client.0.smithi002.stdout:6/942: dread d15/d2d/fbd [0,4194304] 0 2024-03-16T10:45:09.869 INFO:tasks.workunit.client.0.smithi002.stdout:2/840: write d7/d2e/d28/d44/d6d/f9f [2541678,129353] 0 2024-03-16T10:45:09.871 INFO:tasks.workunit.client.0.smithi002.stdout:3/928: chown d1/dd/d28/f9c 13163 1 2024-03-16T10:45:09.874 INFO:tasks.workunit.client.1.smithi026.stdout:1/144: dwrite d0/fd [0,4194304] 0 2024-03-16T10:45:09.875 INFO:tasks.workunit.client.1.smithi026.stdout:1/145: chown d0/d17/d1f/c15 2089486091 1 2024-03-16T10:45:09.875 INFO:tasks.workunit.client.0.smithi002.stdout:3/929: dread d1/db6/d15/d87/dd2/f104 [0,4194304] 0 2024-03-16T10:45:09.876 INFO:tasks.workunit.client.0.smithi002.stdout:3/930: chown d1/dd/d28/d7b/d8a 33233 1 2024-03-16T10:45:09.882 INFO:tasks.workunit.client.0.smithi002.stdout:5/794: creat d0/d16/dab/d33/d4b/ff4 x:0 0 0 2024-03-16T10:45:09.885 INFO:tasks.workunit.client.0.smithi002.stdout:0/895: creat d3/d31/d75/db4/d109/f126 x:0 0 0 2024-03-16T10:45:09.890 INFO:tasks.workunit.client.0.smithi002.stdout:9/959: mkdir d5/d9/d25/d23/d81/d139 0 2024-03-16T10:45:09.892 INFO:tasks.workunit.client.0.smithi002.stdout:1/969: symlink d2/df/d12/d84/d87/d96/db5/d51/d56/d11c/l13b 0 2024-03-16T10:45:09.896 INFO:tasks.workunit.client.0.smithi002.stdout:2/841: symlink d7/d2e/d1d/d38/d49/l108 0 2024-03-16T10:45:09.896 INFO:tasks.workunit.client.1.smithi026.stdout:8/90: dwrite d5/d7/f10 [0,4194304] 0 2024-03-16T10:45:09.897 INFO:tasks.workunit.client.0.smithi002.stdout:2/842: fsync d7/d2e/d28/d44/d6d/f9f 0 2024-03-16T10:45:09.897 INFO:tasks.workunit.client.1.smithi026.stdout:6/174: truncate d0/f3 3991606 0 2024-03-16T10:45:09.898 INFO:tasks.workunit.client.1.smithi026.stdout:4/151: write d1/f3 [951738,16458] 0 2024-03-16T10:45:09.903 INFO:tasks.workunit.client.0.smithi002.stdout:2/843: dwrite d7/d2e/d1d/fc2 [0,4194304] 0 2024-03-16T10:45:09.903 INFO:tasks.workunit.client.1.smithi026.stdout:6/175: dread d0/d10/f31 [0,4194304] 0 2024-03-16T10:45:09.904 INFO:tasks.workunit.client.1.smithi026.stdout:3/152: link d4/d29/l2f d4/dc/l40 0 2024-03-16T10:45:09.905 INFO:tasks.workunit.client.0.smithi002.stdout:5/795: truncate d0/d16/d4a/d55/fd0 1072242 0 2024-03-16T10:45:09.905 INFO:tasks.workunit.client.0.smithi002.stdout:2/844: chown d7/d2e/d28/d40/d50/db4 658 1 2024-03-16T10:45:09.907 INFO:tasks.workunit.client.0.smithi002.stdout:0/896: chown d3/d31/c6e 26816695 1 2024-03-16T10:45:09.908 INFO:tasks.workunit.client.0.smithi002.stdout:7/783: creat d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f11a x:0 0 0 2024-03-16T10:45:09.911 INFO:tasks.workunit.client.0.smithi002.stdout:9/960: write d5/d7e/fd9 [7542329,7904] 0 2024-03-16T10:45:09.912 INFO:tasks.workunit.client.1.smithi026.stdout:5/120: creat d1/db/d1f/f26 x:0 0 0 2024-03-16T10:45:09.913 INFO:tasks.workunit.client.0.smithi002.stdout:5/796: dwrite d0/d24/ff1 [0,4194304] 0 2024-03-16T10:45:09.914 INFO:tasks.workunit.client.1.smithi026.stdout:8/91: rmdir d5/d7 39 2024-03-16T10:45:09.917 INFO:tasks.workunit.client.0.smithi002.stdout:6/943: mknod d15/d2d/d2b/d3f/d77/c145 0 2024-03-16T10:45:09.919 INFO:tasks.workunit.client.0.smithi002.stdout:8/988: truncate d7/d9/d17/d23/d2e/d90/fba 78174 0 2024-03-16T10:45:09.921 INFO:tasks.workunit.client.1.smithi026.stdout:1/146: symlink d0/l2d 0 2024-03-16T10:45:09.925 INFO:tasks.workunit.client.1.smithi026.stdout:9/97: sync 2024-03-16T10:45:09.927 INFO:tasks.workunit.client.1.smithi026.stdout:6/176: unlink d0/d10/d17/f3a 0 2024-03-16T10:45:09.928 INFO:tasks.workunit.client.0.smithi002.stdout:3/931: creat d1/dd/d74/dd8/d110/f148 x:0 0 0 2024-03-16T10:45:09.929 INFO:tasks.workunit.client.1.smithi026.stdout:6/177: truncate d0/d10/f27 131744 0 2024-03-16T10:45:09.932 INFO:tasks.workunit.client.1.smithi026.stdout:9/98: dwrite d0/f4 [4194304,4194304] 0 2024-03-16T10:45:09.934 INFO:tasks.workunit.client.0.smithi002.stdout:0/897: fdatasync d3/d31/d88/fab 0 2024-03-16T10:45:09.935 INFO:tasks.workunit.client.1.smithi026.stdout:8/92: chown d5/d7/l17 109 1 2024-03-16T10:45:09.936 INFO:tasks.workunit.client.0.smithi002.stdout:3/932: dwrite d1/db6/d15/d87/dd2/f10b [0,4194304] 0 2024-03-16T10:45:09.938 INFO:tasks.workunit.client.0.smithi002.stdout:3/933: chown d1/db6/d15/d87/d8c/cc7 2450320 1 2024-03-16T10:45:09.938 INFO:tasks.workunit.client.0.smithi002.stdout:7/784: chown d3/d4/d1d/d87/d5c/d63/d34 2120307 1 2024-03-16T10:45:09.939 INFO:tasks.workunit.client.1.smithi026.stdout:1/147: mkdir d0/d17/d1f/d25/d2e 0 2024-03-16T10:45:09.942 INFO:tasks.workunit.client.0.smithi002.stdout:1/970: mkdir d2/df/d12/d29/d13c 0 2024-03-16T10:45:09.942 INFO:tasks.workunit.client.1.smithi026.stdout:8/93: dwrite d5/dc/f15 [0,4194304] 0 2024-03-16T10:45:09.944 INFO:tasks.workunit.client.0.smithi002.stdout:2/845: write d7/d2e/d1d/d38/f3a [2117243,68814] 0 2024-03-16T10:45:09.945 INFO:tasks.workunit.client.1.smithi026.stdout:7/157: getdents d6 0 2024-03-16T10:45:09.948 INFO:tasks.workunit.client.1.smithi026.stdout:1/148: dread d0/d17/d1f/fc [0,4194304] 0 2024-03-16T10:45:09.949 INFO:tasks.workunit.client.1.smithi026.stdout:0/159: rename d1/d18/d1a/d24/d6/fe to d1/d18/f36 0 2024-03-16T10:45:09.951 INFO:tasks.workunit.client.0.smithi002.stdout:5/797: dread d0/d16/dab/d2b/f78 [0,4194304] 0 2024-03-16T10:45:09.953 INFO:tasks.workunit.client.0.smithi002.stdout:6/944: unlink d15/f29 0 2024-03-16T10:45:09.955 INFO:tasks.workunit.client.0.smithi002.stdout:6/945: read d15/f9a [2133130,24747] 0 2024-03-16T10:45:09.956 INFO:tasks.workunit.client.1.smithi026.stdout:2/163: getdents d2/d29/d17/d1a/d2b/d2e 0 2024-03-16T10:45:09.957 INFO:tasks.workunit.client.1.smithi026.stdout:2/164: chown d2/f35 105 1 2024-03-16T10:45:09.957 INFO:tasks.workunit.client.0.smithi002.stdout:5/798: dwrite d0/d16/dab/d33/f42 [0,4194304] 0 2024-03-16T10:45:09.957 INFO:tasks.workunit.client.0.smithi002.stdout:4/832: getdents d2/d6/d32/d68 0 2024-03-16T10:45:09.958 INFO:tasks.workunit.client.0.smithi002.stdout:5/799: stat d0/d16/dab/d33/d8c/fbc 0 2024-03-16T10:45:09.959 INFO:tasks.workunit.client.0.smithi002.stdout:5/800: fsync d0/d16/dab/d2b/f4f 0 2024-03-16T10:45:09.960 INFO:tasks.workunit.client.0.smithi002.stdout:5/801: dread - d0/d16/dab/d33/d8c/fea zero size 2024-03-16T10:45:09.962 INFO:tasks.workunit.client.0.smithi002.stdout:0/898: creat d3/d59/f127 x:0 0 0 2024-03-16T10:45:09.963 INFO:tasks.workunit.client.1.smithi026.stdout:7/158: mknod d6/c2a 0 2024-03-16T10:45:09.964 INFO:tasks.workunit.client.1.smithi026.stdout:7/159: write d6/f22 [55933,41241] 0 2024-03-16T10:45:09.967 INFO:tasks.workunit.client.1.smithi026.stdout:8/94: creat d5/d7/f1b x:0 0 0 2024-03-16T10:45:09.968 INFO:tasks.workunit.client.1.smithi026.stdout:8/95: dread - d5/d7/f1a zero size 2024-03-16T10:45:09.973 INFO:tasks.workunit.client.0.smithi002.stdout:2/846: creat d7/d2e/d46/f109 x:0 0 0 2024-03-16T10:45:09.974 INFO:tasks.workunit.client.1.smithi026.stdout:2/165: mkdir d2/d29/d36 0 2024-03-16T10:45:09.975 INFO:tasks.workunit.client.0.smithi002.stdout:2/847: chown d7/d2e/d28/f6a 165749 1 2024-03-16T10:45:09.975 INFO:tasks.workunit.client.0.smithi002.stdout:9/961: getdents d5/d7e/d130 0 2024-03-16T10:45:09.975 INFO:tasks.workunit.client.0.smithi002.stdout:2/848: dread - d7/d27/d3b/fd9 zero size 2024-03-16T10:45:09.976 INFO:tasks.workunit.client.1.smithi026.stdout:5/121: rename d1/db/d1a to d1/db/d27 0 2024-03-16T10:45:09.976 INFO:tasks.workunit.client.0.smithi002.stdout:8/989: mkdir d7/d9/d17/d83/d145 0 2024-03-16T10:45:09.977 INFO:tasks.workunit.client.0.smithi002.stdout:3/934: dread d1/dd/d74/f80 [0,4194304] 0 2024-03-16T10:45:09.977 INFO:tasks.workunit.client.1.smithi026.stdout:9/99: truncate d0/da/f12 559523 0 2024-03-16T10:45:09.977 INFO:tasks.workunit.client.0.smithi002.stdout:8/990: fdatasync d7/d9/d17/d23/d2e/de5/f13c 0 2024-03-16T10:45:09.979 INFO:tasks.workunit.client.0.smithi002.stdout:6/946: mknod d15/dfd/d96/dbb/d107/c146 0 2024-03-16T10:45:09.979 INFO:tasks.workunit.client.0.smithi002.stdout:6/947: readlink d15/d2d/d8a/l81 0 2024-03-16T10:45:09.980 INFO:tasks.workunit.client.0.smithi002.stdout:6/948: chown d15/dfd/d96/dbc/dcc 421872 1 2024-03-16T10:45:09.981 INFO:tasks.workunit.client.0.smithi002.stdout:4/833: mkdir d2/d6/d32/d66/d7f/de7/d2e/db0/dfe 0 2024-03-16T10:45:09.984 INFO:tasks.workunit.client.1.smithi026.stdout:8/96: dread d5/d7/f11 [0,4194304] 0 2024-03-16T10:45:09.986 INFO:tasks.workunit.client.0.smithi002.stdout:5/802: symlink d0/d16/dab/d2b/d5d/lf5 0 2024-03-16T10:45:09.988 INFO:tasks.workunit.client.0.smithi002.stdout:5/803: read d0/d16/f4d [825171,98836] 0 2024-03-16T10:45:09.988 INFO:tasks.workunit.client.1.smithi026.stdout:1/149: link d0/d17/f1a d0/d17/d1f/d1c/f2f 0 2024-03-16T10:45:09.989 INFO:tasks.workunit.client.0.smithi002.stdout:5/804: write d0/d16/dab/fdf [876753,122904] 0 2024-03-16T10:45:09.989 INFO:tasks.workunit.client.0.smithi002.stdout:5/805: chown d0/d24/fee 5 1 2024-03-16T10:45:09.991 INFO:tasks.workunit.client.1.smithi026.stdout:9/100: creat d0/da/f1b x:0 0 0 2024-03-16T10:45:09.994 INFO:tasks.workunit.client.1.smithi026.stdout:4/152: rename d1/db/f11 to d1/f32 0 2024-03-16T10:45:09.996 INFO:tasks.workunit.client.0.smithi002.stdout:1/971: creat d2/df/d12/d84/d87/d96/db5/de2/d10a/d130/f13d x:0 0 0 2024-03-16T10:45:09.997 INFO:tasks.workunit.client.0.smithi002.stdout:1/972: write d2/df/d12/dd7/fe4 [1077817,75169] 0 2024-03-16T10:45:09.999 INFO:tasks.workunit.client.0.smithi002.stdout:2/849: creat d7/db/d12/f10a x:0 0 0 2024-03-16T10:45:10.001 INFO:tasks.workunit.client.1.smithi026.stdout:1/150: creat d0/d17/d1f/d25/f30 x:0 0 0 2024-03-16T10:45:10.001 INFO:tasks.workunit.client.1.smithi026.stdout:7/160: dwrite d6/fc [0,4194304] 0 2024-03-16T10:45:10.001 INFO:tasks.workunit.client.0.smithi002.stdout:2/850: read d7/d2e/d1d/d2f/d4b/f62 [3332828,96117] 0 2024-03-16T10:45:10.001 INFO:tasks.workunit.client.0.smithi002.stdout:2/851: chown d7/d2e/d1d/d38/df6 164362765 1 2024-03-16T10:45:10.003 INFO:tasks.workunit.client.1.smithi026.stdout:9/101: mknod d0/da/d14/c1c 0 2024-03-16T10:45:10.005 INFO:tasks.workunit.client.0.smithi002.stdout:3/935: mkdir d1/db6/d15/d87/d11e/d149 0 2024-03-16T10:45:10.007 INFO:tasks.workunit.client.1.smithi026.stdout:0/160: dwrite d1/d18/d1a/f26 [0,4194304] 0 2024-03-16T10:45:10.008 INFO:tasks.workunit.client.0.smithi002.stdout:8/991: rename d7/d9/d17/d23/d39/db7/dc2 to d7/d9/d17/d23/d2e/d5c/df0/d146 0 2024-03-16T10:45:10.009 INFO:tasks.workunit.client.0.smithi002.stdout:7/785: truncate d3/d4/d1d/d87/d5c/d63/d3f/f41 3877314 0 2024-03-16T10:45:10.010 INFO:tasks.workunit.client.0.smithi002.stdout:0/899: dwrite d3/da0/fa3 [0,4194304] 0 2024-03-16T10:45:10.013 INFO:tasks.workunit.client.1.smithi026.stdout:9/102: dwrite d0/f4 [4194304,4194304] 0 2024-03-16T10:45:10.015 INFO:tasks.workunit.client.1.smithi026.stdout:4/153: write d1/f6 [574805,119997] 0 2024-03-16T10:45:10.015 INFO:tasks.workunit.client.0.smithi002.stdout:7/786: fsync d3/d4/d1d/d87/d96/d5e/db7/f114 0 2024-03-16T10:45:10.016 INFO:tasks.workunit.client.0.smithi002.stdout:6/949: mkdir d15/d2d/d2b/de4/dfc/d147 0 2024-03-16T10:45:10.018 INFO:tasks.workunit.client.1.smithi026.stdout:7/161: symlink d6/d1e/l2b 0 2024-03-16T10:45:10.023 INFO:tasks.workunit.client.0.smithi002.stdout:1/973: mkdir d2/df/d12/d84/d87/d96/db5/d51/d56/d11c/d13e 0 2024-03-16T10:45:10.026 INFO:tasks.workunit.client.1.smithi026.stdout:9/103: creat d0/f1d x:0 0 0 2024-03-16T10:45:10.029 INFO:tasks.workunit.client.1.smithi026.stdout:8/97: link d5/dc/fe d5/d7/f1c 0 2024-03-16T10:45:10.032 INFO:tasks.workunit.client.0.smithi002.stdout:5/806: dwrite d0/d16/dab/d43/d48/f7f [4194304,4194304] 0 2024-03-16T10:45:10.032 INFO:tasks.workunit.client.1.smithi026.stdout:4/154: rmdir d1/db 39 2024-03-16T10:45:10.033 INFO:tasks.workunit.client.1.smithi026.stdout:0/161: dread d1/d18/f36 [0,4194304] 0 2024-03-16T10:45:10.035 INFO:tasks.workunit.client.1.smithi026.stdout:7/162: symlink d6/de/l2c 0 2024-03-16T10:45:10.036 INFO:tasks.workunit.client.0.smithi002.stdout:3/936: dread d1/db6/f4a [0,4194304] 0 2024-03-16T10:45:10.036 INFO:tasks.workunit.client.1.smithi026.stdout:7/163: fdatasync d6/de/d16/f19 0 2024-03-16T10:45:10.039 INFO:tasks.workunit.client.0.smithi002.stdout:0/900: dread d3/d28/d65/f77 [0,4194304] 0 2024-03-16T10:45:10.040 INFO:tasks.workunit.client.0.smithi002.stdout:8/992: dread d7/d9/d17/d83/df5/f10c [0,4194304] 0 2024-03-16T10:45:10.041 INFO:tasks.workunit.client.1.smithi026.stdout:3/153: rename d4/dc/f3f to d4/f41 0 2024-03-16T10:45:10.045 INFO:tasks.workunit.client.0.smithi002.stdout:4/834: dwrite d2/d6/f10 [0,4194304] 0 2024-03-16T10:45:10.045 INFO:tasks.workunit.client.0.smithi002.stdout:9/962: rmdir d5/d7e/d130 0 2024-03-16T10:45:10.047 INFO:tasks.workunit.client.0.smithi002.stdout:9/963: chown d5/d9/dc/d1e/d24/ded/lbe 1 1 2024-03-16T10:45:10.047 INFO:tasks.workunit.client.0.smithi002.stdout:4/835: chown d2/d6/d32/d66/d7f/de7/d88/db2/fd1 119877358 1 2024-03-16T10:45:10.048 INFO:tasks.workunit.client.1.smithi026.stdout:8/98: dread d5/dc/f18 [0,4194304] 0 2024-03-16T10:45:10.051 INFO:tasks.workunit.client.0.smithi002.stdout:9/964: dread d5/d9/d55/dc9/d11f/f103 [0,4194304] 0 2024-03-16T10:45:10.052 INFO:tasks.workunit.client.0.smithi002.stdout:9/965: stat d5/d9/dc/d28/dd6/fd8 0 2024-03-16T10:45:10.053 INFO:tasks.workunit.client.1.smithi026.stdout:7/164: mknod d6/de/d16/c2d 0 2024-03-16T10:45:10.054 INFO:tasks.workunit.client.1.smithi026.stdout:1/151: dwrite d0/d17/d1f/d1c/f2f [0,4194304] 0 2024-03-16T10:45:10.057 INFO:tasks.workunit.client.0.smithi002.stdout:1/974: mknod d2/df/d12/d84/d87/d96/db5/d1c/da0/d119/c13f 0 2024-03-16T10:45:10.059 INFO:tasks.workunit.client.1.smithi026.stdout:3/154: readlink d4/l7 0 2024-03-16T10:45:10.060 INFO:tasks.workunit.client.1.smithi026.stdout:1/152: dread d0/fd [0,4194304] 0 2024-03-16T10:45:10.062 INFO:tasks.workunit.client.1.smithi026.stdout:9/104: creat d0/da/d16/d17/d19/f1e x:0 0 0 2024-03-16T10:45:10.064 INFO:tasks.workunit.client.0.smithi002.stdout:5/807: creat d0/d24/dad/ff6 x:0 0 0 2024-03-16T10:45:10.067 INFO:tasks.workunit.client.1.smithi026.stdout:1/153: dwrite d0/d17/d1f/d1c/f27 [0,4194304] 0 2024-03-16T10:45:10.067 INFO:tasks.workunit.client.0.smithi002.stdout:7/787: write d3/d4/d1c/d5b/d64/fc1 [424170,55260] 0 2024-03-16T10:45:10.068 INFO:tasks.workunit.client.0.smithi002.stdout:7/788: fdatasync d3/d4/d1d/d87/d5c/d63/d55/f118 0 2024-03-16T10:45:10.069 INFO:tasks.workunit.client.0.smithi002.stdout:7/789: readlink d3/d37/ld5 0 2024-03-16T10:45:10.069 INFO:tasks.workunit.client.0.smithi002.stdout:3/937: symlink d1/dd/d74/dd8/l14a 0 2024-03-16T10:45:10.072 INFO:tasks.workunit.client.1.smithi026.stdout:8/99: creat d5/d7/f1d x:0 0 0 2024-03-16T10:45:10.074 INFO:tasks.workunit.client.0.smithi002.stdout:0/901: mknod d3/d31/d75/dea/c128 0 2024-03-16T10:45:10.074 INFO:tasks.workunit.client.1.smithi026.stdout:8/100: write d5/d7/f1a [289044,4665] 0 2024-03-16T10:45:10.076 INFO:tasks.workunit.client.1.smithi026.stdout:1/154: dwrite d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:10.078 INFO:tasks.workunit.client.1.smithi026.stdout:1/155: write d0/d17/d1f/d1c/f27 [2840547,42143] 0 2024-03-16T10:45:10.078 INFO:tasks.workunit.client.1.smithi026.stdout:7/165: creat d6/de/d16/f2e x:0 0 0 2024-03-16T10:45:10.078 INFO:tasks.workunit.client.0.smithi002.stdout:8/993: stat d7/d9/d17/d20/l9d 0 2024-03-16T10:45:10.079 INFO:tasks.workunit.client.1.smithi026.stdout:1/156: chown d0/l16 10137 1 2024-03-16T10:45:10.079 INFO:tasks.workunit.client.1.smithi026.stdout:7/166: read d6/fc [2779976,71272] 0 2024-03-16T10:45:10.082 INFO:tasks.workunit.client.0.smithi002.stdout:6/950: mkdir d15/d2d/d148 0 2024-03-16T10:45:10.083 INFO:tasks.workunit.client.1.smithi026.stdout:3/155: symlink d4/dc/l42 0 2024-03-16T10:45:10.084 INFO:tasks.workunit.client.1.smithi026.stdout:3/156: read - d4/dc/f36 zero size 2024-03-16T10:45:10.084 INFO:tasks.workunit.client.1.smithi026.stdout:9/105: creat d0/f1f x:0 0 0 2024-03-16T10:45:10.084 INFO:tasks.workunit.client.0.smithi002.stdout:4/836: dread - d2/d33/d1c/f8d zero size 2024-03-16T10:45:10.085 INFO:tasks.workunit.client.1.smithi026.stdout:9/106: read d0/f4 [8180863,55740] 0 2024-03-16T10:45:10.087 INFO:tasks.workunit.client.0.smithi002.stdout:2/852: creat d7/d2e/f10b x:0 0 0 2024-03-16T10:45:10.089 INFO:tasks.workunit.client.1.smithi026.stdout:0/162: creat d1/f37 x:0 0 0 2024-03-16T10:45:10.089 INFO:tasks.workunit.client.0.smithi002.stdout:9/966: mknod d5/d9/d25/d23/c13a 0 2024-03-16T10:45:10.090 INFO:tasks.workunit.client.1.smithi026.stdout:0/163: dread - d1/d18/d1a/d2b/f32 zero size 2024-03-16T10:45:10.091 INFO:tasks.workunit.client.1.smithi026.stdout:8/101: rmdir d5/dc 39 2024-03-16T10:45:10.092 INFO:tasks.workunit.client.1.smithi026.stdout:9/107: dwrite d0/f1d [0,4194304] 0 2024-03-16T10:45:10.094 INFO:tasks.workunit.client.0.smithi002.stdout:5/808: creat d0/d16/d4a/d39/ff7 x:0 0 0 2024-03-16T10:45:10.096 INFO:tasks.workunit.client.1.smithi026.stdout:9/108: dread d0/da/f11 [0,4194304] 0 2024-03-16T10:45:10.098 INFO:tasks.workunit.client.1.smithi026.stdout:7/167: mkdir d6/d2f 0 2024-03-16T10:45:10.099 INFO:tasks.workunit.client.1.smithi026.stdout:7/168: write d6/f22 [979509,65334] 0 2024-03-16T10:45:10.100 INFO:tasks.workunit.client.1.smithi026.stdout:7/169: dread - d6/de/d16/f2e zero size 2024-03-16T10:45:10.100 INFO:tasks.workunit.client.0.smithi002.stdout:3/938: rename d1/db6/d15/f12c to d1/dd/d28/d7b/d2c/d97/df5/f14b 0 2024-03-16T10:45:10.106 INFO:tasks.workunit.client.0.smithi002.stdout:6/951: mknod d15/d2d/d2b/d3f/d77/d112/c149 0 2024-03-16T10:45:10.107 INFO:tasks.workunit.client.0.smithi002.stdout:8/994: dread d7/d9/d17/f3d [0,4194304] 0 2024-03-16T10:45:10.108 INFO:tasks.workunit.client.1.smithi026.stdout:3/157: creat d4/dc/f43 x:0 0 0 2024-03-16T10:45:10.109 INFO:tasks.workunit.client.1.smithi026.stdout:3/158: truncate d4/dc/f36 12593 0 2024-03-16T10:45:10.111 INFO:tasks.workunit.client.0.smithi002.stdout:6/952: dwrite d15/f67 [0,4194304] 0 2024-03-16T10:45:10.112 INFO:tasks.workunit.client.0.smithi002.stdout:9/967: mkdir d5/d9/dc/d1e/d91/d83/d13b 0 2024-03-16T10:45:10.114 INFO:tasks.workunit.client.0.smithi002.stdout:9/968: chown d5/d9/dc/d1e/d24/ded/l5e 214 1 2024-03-16T10:45:10.120 INFO:tasks.workunit.client.1.smithi026.stdout:9/109: symlink d0/l20 0 2024-03-16T10:45:10.125 INFO:tasks.workunit.client.1.smithi026.stdout:1/157: mknod d0/d17/d1f/d25/d2e/c31 0 2024-03-16T10:45:10.126 INFO:tasks.workunit.client.0.smithi002.stdout:7/790: write d3/f14 [351396,61836] 0 2024-03-16T10:45:10.127 INFO:tasks.workunit.client.1.smithi026.stdout:1/158: write d0/d17/f1a [1498052,64684] 0 2024-03-16T10:45:10.127 INFO:tasks.workunit.client.1.smithi026.stdout:9/110: dwrite d0/da/fe [4194304,4194304] 0 2024-03-16T10:45:10.128 INFO:tasks.workunit.client.1.smithi026.stdout:9/111: chown d0/c6 167630273 1 2024-03-16T10:45:10.129 INFO:tasks.workunit.client.0.smithi002.stdout:0/902: truncate d3/d28/d65/f112 2844946 0 2024-03-16T10:45:10.130 INFO:tasks.workunit.client.0.smithi002.stdout:0/903: read d3/de/da2/fd2 [61502,113879] 0 2024-03-16T10:45:10.130 INFO:tasks.workunit.client.0.smithi002.stdout:0/904: chown d3/d31/d75/db4/ddc/f118 705 1 2024-03-16T10:45:10.132 INFO:tasks.workunit.client.1.smithi026.stdout:4/155: rename d1/c9 to d1/db/c33 0 2024-03-16T10:45:10.133 INFO:tasks.workunit.client.1.smithi026.stdout:4/156: dread - d1/db/de/f2b zero size 2024-03-16T10:45:10.134 INFO:tasks.workunit.client.1.smithi026.stdout:3/159: mknod d4/d29/c44 0 2024-03-16T10:45:10.135 INFO:tasks.workunit.client.1.smithi026.stdout:3/160: chown d4/dc/f11 1601904 1 2024-03-16T10:45:10.135 INFO:tasks.workunit.client.1.smithi026.stdout:8/102: creat d5/dc/f1e x:0 0 0 2024-03-16T10:45:10.144 INFO:tasks.workunit.client.1.smithi026.stdout:4/157: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:10.146 INFO:tasks.workunit.client.1.smithi026.stdout:3/161: rmdir d4/dc/d31 39 2024-03-16T10:45:10.148 INFO:tasks.workunit.client.1.smithi026.stdout:8/103: mkdir d5/dc/d1f 0 2024-03-16T10:45:10.149 INFO:tasks.workunit.client.0.smithi002.stdout:5/809: truncate d0/d1f/f36 3047161 0 2024-03-16T10:45:10.150 INFO:tasks.workunit.client.0.smithi002.stdout:9/969: dread d5/d9/dc/d28/d90/fa7 [0,4194304] 0 2024-03-16T10:45:10.151 INFO:tasks.workunit.client.0.smithi002.stdout:8/995: dwrite d7/d9/d17/d23/d39/db7/fc4 [0,4194304] 0 2024-03-16T10:45:10.152 INFO:tasks.workunit.client.0.smithi002.stdout:5/810: fsync d0/d16/dab/d33/fef 0 2024-03-16T10:45:10.152 INFO:tasks.workunit.client.1.smithi026.stdout:9/112: getdents d0/da/d13 0 2024-03-16T10:45:10.153 INFO:tasks.workunit.client.1.smithi026.stdout:7/170: truncate d6/fc 3464060 0 2024-03-16T10:45:10.153 INFO:tasks.workunit.client.1.smithi026.stdout:7/171: stat d6/d1e/l2b 0 2024-03-16T10:45:10.155 INFO:tasks.workunit.client.0.smithi002.stdout:6/953: symlink d15/dfd/l14a 0 2024-03-16T10:45:10.157 INFO:tasks.workunit.client.0.smithi002.stdout:7/791: dread d3/fd [0,4194304] 0 2024-03-16T10:45:10.157 INFO:tasks.workunit.client.0.smithi002.stdout:3/939: rmdir d1/db6/d15/d87/d8c/dfd/d141 0 2024-03-16T10:45:10.161 INFO:tasks.workunit.client.1.smithi026.stdout:1/159: link d0/fd d0/d17/d1f/d1c/f32 0 2024-03-16T10:45:10.161 INFO:tasks.workunit.client.1.smithi026.stdout:8/104: dread d5/dc/f12 [0,4194304] 0 2024-03-16T10:45:10.162 INFO:tasks.workunit.client.1.smithi026.stdout:8/105: fdatasync d5/fb 0 2024-03-16T10:45:10.163 INFO:tasks.workunit.client.1.smithi026.stdout:9/113: mknod d0/c21 0 2024-03-16T10:45:10.164 INFO:tasks.workunit.client.0.smithi002.stdout:2/853: getdents d7/d27/d3b/dd4/de1 0 2024-03-16T10:45:10.164 INFO:tasks.workunit.client.1.smithi026.stdout:9/114: chown d0/f1d 32 1 2024-03-16T10:45:10.164 INFO:tasks.workunit.client.1.smithi026.stdout:9/115: stat d0/da/d16/d17 0 2024-03-16T10:45:10.165 INFO:tasks.workunit.client.0.smithi002.stdout:8/996: dread d7/f31 [0,4194304] 0 2024-03-16T10:45:10.165 INFO:tasks.workunit.client.0.smithi002.stdout:1/975: sync 2024-03-16T10:45:10.165 INFO:tasks.workunit.client.0.smithi002.stdout:4/837: sync 2024-03-16T10:45:10.166 INFO:tasks.workunit.client.1.smithi026.stdout:0/164: rename d1/d18/d1a/d24/d6/cf to d1/d18/c38 0 2024-03-16T10:45:10.167 INFO:tasks.workunit.client.0.smithi002.stdout:4/838: chown d2/d6/d32/d68/cc6 47 1 2024-03-16T10:45:10.167 INFO:tasks.workunit.client.0.smithi002.stdout:1/976: read d2/d58/de8/f11d [61722,42540] 0 2024-03-16T10:45:10.167 INFO:tasks.workunit.client.1.smithi026.stdout:4/158: mknod d1/c34 0 2024-03-16T10:45:10.168 INFO:tasks.workunit.client.0.smithi002.stdout:4/839: truncate d2/d33/fe3 5143156 0 2024-03-16T10:45:10.169 INFO:tasks.workunit.client.1.smithi026.stdout:4/159: dread d1/f6 [0,4194304] 0 2024-03-16T10:45:10.170 INFO:tasks.workunit.client.0.smithi002.stdout:4/840: chown d2/d6/d32/d66/d7f/de7/d5e/l98 3 1 2024-03-16T10:45:10.174 INFO:tasks.workunit.client.0.smithi002.stdout:6/954: chown d15/d2d/d73/d97/ca7 0 1 2024-03-16T10:45:10.176 INFO:tasks.workunit.client.1.smithi026.stdout:9/116: symlink d0/da/d16/d17/l22 0 2024-03-16T10:45:10.177 INFO:tasks.workunit.client.1.smithi026.stdout:9/117: fdatasync d0/f1f 0 2024-03-16T10:45:10.177 INFO:tasks.workunit.client.0.smithi002.stdout:3/940: write d1/dd/d28/d7b/d8a/d93/f47 [3055611,79084] 0 2024-03-16T10:45:10.180 INFO:tasks.workunit.client.1.smithi026.stdout:3/162: rename d4/dc/l1c to d4/dc/dd/d3d/l45 0 2024-03-16T10:45:10.186 INFO:tasks.workunit.client.1.smithi026.stdout:4/160: rmdir d1 39 2024-03-16T10:45:10.188 INFO:tasks.workunit.client.1.smithi026.stdout:8/106: link d5/d7/f1a d5/dc/d1f/f20 0 2024-03-16T10:45:10.189 INFO:tasks.workunit.client.1.smithi026.stdout:8/107: write d5/d7/f10 [2743937,61846] 0 2024-03-16T10:45:10.191 INFO:tasks.workunit.client.0.smithi002.stdout:8/997: mkdir d7/d9/d17/d23/d39/dd4/d147 0 2024-03-16T10:45:10.193 INFO:tasks.workunit.client.0.smithi002.stdout:2/854: write d7/d2e/d1d/d2f/d4b/fbf [131824,724] 0 2024-03-16T10:45:10.194 INFO:tasks.workunit.client.1.smithi026.stdout:7/172: rename d6/de/d16/c2d to d6/c30 0 2024-03-16T10:45:10.196 INFO:tasks.workunit.client.0.smithi002.stdout:4/841: read d2/d6/f1e [516887,11775] 0 2024-03-16T10:45:10.197 INFO:tasks.workunit.client.1.smithi026.stdout:0/165: dwrite d1/d18/f36 [0,4194304] 0 2024-03-16T10:45:10.198 INFO:tasks.workunit.client.0.smithi002.stdout:7/792: mknod d3/d4/d1d/d87/d5c/d63/d34/c11b 0 2024-03-16T10:45:10.201 INFO:tasks.workunit.client.0.smithi002.stdout:0/905: getdents d3/d31/d75/df4 0 2024-03-16T10:45:10.201 INFO:tasks.workunit.client.1.smithi026.stdout:7/173: dread d6/de/d16/f1a [0,4194304] 0 2024-03-16T10:45:10.203 INFO:tasks.workunit.client.0.smithi002.stdout:4/842: dwrite d2/d33/fa9 [0,4194304] 0 2024-03-16T10:45:10.204 INFO:tasks.workunit.client.0.smithi002.stdout:9/970: symlink d5/d9/d25/d23/l13c 0 2024-03-16T10:45:10.204 INFO:tasks.workunit.client.1.smithi026.stdout:0/166: dread d1/d18/d1a/d24/d11/f2d [0,4194304] 0 2024-03-16T10:45:10.205 INFO:tasks.workunit.client.0.smithi002.stdout:9/971: dread - d5/d9/d34/f121 zero size 2024-03-16T10:45:10.206 INFO:tasks.workunit.client.1.smithi026.stdout:4/161: dwrite d1/f22 [0,4194304] 0 2024-03-16T10:45:10.207 INFO:tasks.workunit.client.1.smithi026.stdout:0/167: write d1/f1e [4705280,83268] 0 2024-03-16T10:45:10.209 INFO:tasks.workunit.client.1.smithi026.stdout:3/163: mkdir d4/d46 0 2024-03-16T10:45:10.209 INFO:tasks.workunit.client.1.smithi026.stdout:8/108: rmdir d5 39 2024-03-16T10:45:10.216 INFO:tasks.workunit.client.0.smithi002.stdout:8/998: mknod d7/d9/d25/d91/d123/c148 0 2024-03-16T10:45:10.217 INFO:tasks.workunit.client.0.smithi002.stdout:8/999: chown d7/d9 36235345 1 2024-03-16T10:45:10.218 INFO:tasks.workunit.client.0.smithi002.stdout:5/811: link d0/d16/dab/d33/fef d0/d16/dab/d33/ff8 0 2024-03-16T10:45:10.219 INFO:tasks.workunit.client.1.smithi026.stdout:6/178: sync 2024-03-16T10:45:10.219 INFO:tasks.workunit.client.0.smithi002.stdout:2/855: mknod d7/d2e/d1d/d2f/d4b/d6e/c10c 0 2024-03-16T10:45:10.224 INFO:tasks.workunit.client.0.smithi002.stdout:3/941: dwrite d1/dd/d28/d7b/d2c/d63/faf [0,4194304] 0 2024-03-16T10:45:10.226 INFO:tasks.workunit.client.1.smithi026.stdout:9/118: truncate d0/da/fe 7720389 0 2024-03-16T10:45:10.226 INFO:tasks.workunit.client.1.smithi026.stdout:3/164: mkdir d4/dc/dd/d47 0 2024-03-16T10:45:10.230 INFO:tasks.workunit.client.0.smithi002.stdout:0/906: mknod d3/d27/c129 0 2024-03-16T10:45:10.231 INFO:tasks.workunit.client.1.smithi026.stdout:6/179: symlink d0/d28/l3f 0 2024-03-16T10:45:10.234 INFO:tasks.workunit.client.0.smithi002.stdout:1/977: truncate d2/df/d12/d84/d87/d96/db5/f55 1191849 0 2024-03-16T10:45:10.235 INFO:tasks.workunit.client.1.smithi026.stdout:6/180: dread d0/d10/f31 [0,4194304] 0 2024-03-16T10:45:10.236 INFO:tasks.workunit.client.0.smithi002.stdout:5/812: symlink d0/d16/dab/d2b/d5d/lf9 0 2024-03-16T10:45:10.239 INFO:tasks.workunit.client.0.smithi002.stdout:5/813: write d0/d16/dab/d2b/fc6 [2087917,76593] 0 2024-03-16T10:45:10.241 INFO:tasks.workunit.client.0.smithi002.stdout:7/793: dread d3/d37/d6a/f6e [0,4194304] 0 2024-03-16T10:45:10.242 INFO:tasks.workunit.client.1.smithi026.stdout:1/160: rename d0/d17/d1f/d25 to d0/d33 0 2024-03-16T10:45:10.242 INFO:tasks.workunit.client.1.smithi026.stdout:1/161: dread - d0/d33/f30 zero size 2024-03-16T10:45:10.243 INFO:tasks.workunit.client.0.smithi002.stdout:7/794: write d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f11a [770818,57406] 0 2024-03-16T10:45:10.243 INFO:tasks.workunit.client.1.smithi026.stdout:9/119: dwrite d0/da/f11 [0,4194304] 0 2024-03-16T10:45:10.244 INFO:tasks.workunit.client.0.smithi002.stdout:3/942: rename d1/da5/fc3 to d1/dd/d28/d7b/d2c/d63/d115/f14c 0 2024-03-16T10:45:10.244 INFO:tasks.workunit.client.1.smithi026.stdout:9/120: write d0/f1f [140693,43154] 0 2024-03-16T10:45:10.245 INFO:tasks.workunit.client.1.smithi026.stdout:9/121: write d0/f1f [639129,37566] 0 2024-03-16T10:45:10.249 INFO:tasks.workunit.client.0.smithi002.stdout:0/907: creat d3/d31/d75/de4/f12a x:0 0 0 2024-03-16T10:45:10.251 INFO:tasks.workunit.client.1.smithi026.stdout:6/181: creat d0/d28/f40 x:0 0 0 2024-03-16T10:45:10.252 INFO:tasks.workunit.client.1.smithi026.stdout:6/182: dread - d0/d10/f36 zero size 2024-03-16T10:45:10.253 INFO:tasks.workunit.client.0.smithi002.stdout:9/972: mknod d5/d9/dc/d28/c13d 0 2024-03-16T10:45:10.257 INFO:tasks.workunit.client.1.smithi026.stdout:1/162: mknod d0/d17/d1f/c34 0 2024-03-16T10:45:10.258 INFO:tasks.workunit.client.1.smithi026.stdout:1/163: chown d0/c22 96 1 2024-03-16T10:45:10.258 INFO:tasks.workunit.client.0.smithi002.stdout:6/955: getdents d15/d2d/d73/d115 0 2024-03-16T10:45:10.258 INFO:tasks.workunit.client.0.smithi002.stdout:2/856: dwrite d7/d2e/d28/d40/d6c/f7a [0,4194304] 0 2024-03-16T10:45:10.260 INFO:tasks.workunit.client.0.smithi002.stdout:2/857: chown d7/d27/d3b/c6b 2 1 2024-03-16T10:45:10.261 INFO:tasks.workunit.client.1.smithi026.stdout:9/122: mknod d0/da/d16/c23 0 2024-03-16T10:45:10.263 INFO:tasks.workunit.client.1.smithi026.stdout:7/174: link d6/de/d16/f18 d6/f31 0 2024-03-16T10:45:10.264 INFO:tasks.workunit.client.0.smithi002.stdout:6/956: dread d15/d2d/d2c/f50 [0,4194304] 0 2024-03-16T10:45:10.265 INFO:tasks.workunit.client.1.smithi026.stdout:0/168: getdents d1/d18/d1a 0 2024-03-16T10:45:10.266 INFO:tasks.workunit.client.1.smithi026.stdout:0/169: dread - d1/d18/d1a/d2b/f32 zero size 2024-03-16T10:45:10.267 INFO:tasks.workunit.client.0.smithi002.stdout:3/943: creat d1/dd/d28/d7b/d8a/dd7/df1/f14d x:0 0 0 2024-03-16T10:45:10.269 INFO:tasks.workunit.client.0.smithi002.stdout:0/908: creat d3/d28/d65/f12b x:0 0 0 2024-03-16T10:45:10.270 INFO:tasks.workunit.client.1.smithi026.stdout:8/109: creat d5/f21 x:0 0 0 2024-03-16T10:45:10.275 INFO:tasks.workunit.client.1.smithi026.stdout:1/164: creat d0/d17/d1f/f35 x:0 0 0 2024-03-16T10:45:10.275 INFO:tasks.workunit.client.0.smithi002.stdout:0/909: dwrite d3/d27/d90/ffd [0,4194304] 0 2024-03-16T10:45:10.276 INFO:tasks.workunit.client.0.smithi002.stdout:0/910: stat d3/d28/d65/d108/f11d 0 2024-03-16T10:45:10.277 INFO:tasks.workunit.client.1.smithi026.stdout:9/123: creat d0/da/d16/d17/d19/f24 x:0 0 0 2024-03-16T10:45:10.277 INFO:tasks.workunit.client.0.smithi002.stdout:0/911: chown d3/d59/dca 4834688 1 2024-03-16T10:45:10.279 INFO:tasks.workunit.client.0.smithi002.stdout:7/795: getdents d3/d4/d1d/d87/d5c/d63/d3f/dba/dcc/d119 0 2024-03-16T10:45:10.283 INFO:tasks.workunit.client.0.smithi002.stdout:6/957: rename d15/d2d/d8a/c88 to d15/dfd/d96/c14b 0 2024-03-16T10:45:10.284 INFO:tasks.workunit.client.1.smithi026.stdout:6/183: mkdir d0/d41 0 2024-03-16T10:45:10.285 INFO:tasks.workunit.client.0.smithi002.stdout:3/944: fsync d1/dd/d28/d7b/d2c/d97/fc4 0 2024-03-16T10:45:10.286 INFO:tasks.workunit.client.1.smithi026.stdout:8/110: rmdir d5/dc/d1f 39 2024-03-16T10:45:10.288 INFO:tasks.workunit.client.0.smithi002.stdout:4/843: link d2/d6/d32/d66/d7f/de7/f84 d2/d6/d32/d66/df7/fff 0 2024-03-16T10:45:10.291 INFO:tasks.workunit.client.0.smithi002.stdout:0/912: fsync d3/d28/f29 0 2024-03-16T10:45:10.294 INFO:tasks.workunit.client.0.smithi002.stdout:9/973: dread d5/d7e/f8f [0,4194304] 0 2024-03-16T10:45:10.298 INFO:tasks.workunit.client.0.smithi002.stdout:0/913: rmdir d3/d31/d75/dde 39 2024-03-16T10:45:10.299 INFO:tasks.workunit.client.0.smithi002.stdout:2/858: creat d7/d2e/f10d x:0 0 0 2024-03-16T10:45:10.299 INFO:tasks.workunit.client.0.smithi002.stdout:0/914: chown d3/d59/dca/fee 31282766 1 2024-03-16T10:45:10.299 INFO:tasks.workunit.client.0.smithi002.stdout:7/796: symlink d3/l11c 0 2024-03-16T10:45:10.300 INFO:tasks.workunit.client.0.smithi002.stdout:6/958: getdents d15/dab/d11c/d12a 0 2024-03-16T10:45:10.303 INFO:tasks.workunit.client.0.smithi002.stdout:5/814: rename d0/d24/dad/db6/db9/cd2 to d0/d16/dab/d33/cfa 0 2024-03-16T10:45:10.306 INFO:tasks.workunit.client.0.smithi002.stdout:7/797: creat d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f11d x:0 0 0 2024-03-16T10:45:10.306 INFO:tasks.workunit.client.0.smithi002.stdout:1/978: sync 2024-03-16T10:45:10.308 INFO:tasks.workunit.client.0.smithi002.stdout:4/844: rename d2/d33/d1c/cfc to d2/d6/ddb/c100 0 2024-03-16T10:45:10.310 INFO:tasks.workunit.client.0.smithi002.stdout:4/845: chown d2/d6/d32/d66/d7f/de7/d5e/d93/fa2 4 1 2024-03-16T10:45:10.310 INFO:tasks.workunit.client.0.smithi002.stdout:9/974: truncate d5/d9/d22/f47 2046945 0 2024-03-16T10:45:10.311 INFO:tasks.workunit.client.0.smithi002.stdout:2/859: truncate d7/d2e/d28/d40/ff9 217821 0 2024-03-16T10:45:10.320 INFO:tasks.workunit.client.0.smithi002.stdout:3/945: getdents d1 0 2024-03-16T10:45:10.321 INFO:tasks.workunit.client.0.smithi002.stdout:1/979: dwrite d2/d58/d115/f134 [0,4194304] 0 2024-03-16T10:45:10.322 INFO:tasks.workunit.client.0.smithi002.stdout:6/959: truncate d15/d2d/d73/de0/f119 3639122 0 2024-03-16T10:45:10.323 INFO:tasks.workunit.client.0.smithi002.stdout:5/815: creat d0/d16/dab/d33/ffb x:0 0 0 2024-03-16T10:45:10.330 INFO:tasks.workunit.client.0.smithi002.stdout:0/915: link d3/d27/lf1 d3/d28/d2a/dc7/l12c 0 2024-03-16T10:45:10.331 INFO:tasks.workunit.client.0.smithi002.stdout:6/960: chown d15/d2d/d73/fe5 10 1 2024-03-16T10:45:10.331 INFO:tasks.workunit.client.0.smithi002.stdout:3/946: mknod d1/dd/d28/d7b/d25/c14e 0 2024-03-16T10:45:10.333 INFO:tasks.workunit.client.1.smithi026.stdout:3/165: getdents d4/dc/dd 0 2024-03-16T10:45:10.333 INFO:tasks.workunit.client.1.smithi026.stdout:1/165: creat d0/d17/d1f/d1c/f36 x:0 0 0 2024-03-16T10:45:10.334 INFO:tasks.workunit.client.0.smithi002.stdout:2/860: dread d7/d2e/f16 [0,4194304] 0 2024-03-16T10:45:10.334 INFO:tasks.workunit.client.1.smithi026.stdout:9/124: creat d0/da/d14/f25 x:0 0 0 2024-03-16T10:45:10.334 INFO:tasks.workunit.client.0.smithi002.stdout:0/916: write d3/d28/d65/d108/f11d [121281,51951] 0 2024-03-16T10:45:10.335 INFO:tasks.workunit.client.1.smithi026.stdout:0/170: mkdir d1/d12/d2f/d39 0 2024-03-16T10:45:10.335 INFO:tasks.workunit.client.1.smithi026.stdout:6/184: creat d0/d28/f42 x:0 0 0 2024-03-16T10:45:10.337 INFO:tasks.workunit.client.1.smithi026.stdout:8/111: creat d5/d7/f22 x:0 0 0 2024-03-16T10:45:10.338 INFO:tasks.workunit.client.0.smithi002.stdout:4/846: dwrite d2/d33/f3e [0,4194304] 0 2024-03-16T10:45:10.339 INFO:tasks.workunit.client.1.smithi026.stdout:1/166: creat d0/d17/f37 x:0 0 0 2024-03-16T10:45:10.344 INFO:tasks.workunit.client.0.smithi002.stdout:4/847: dread d2/d6/d32/d66/d7f/de7/d5e/f75 [0,4194304] 0 2024-03-16T10:45:10.346 INFO:tasks.workunit.client.0.smithi002.stdout:9/975: dread d5/d9/dc/d28/dd6/fd8 [0,4194304] 0 2024-03-16T10:45:10.346 INFO:tasks.workunit.client.0.smithi002.stdout:4/848: fdatasync d2/d6/d32/d66/d7f/de7/d88/db2/fd0 0 2024-03-16T10:45:10.348 INFO:tasks.workunit.client.1.smithi026.stdout:0/171: dwrite d1/d18/d1a/d24/d6/f31 [0,4194304] 0 2024-03-16T10:45:10.349 INFO:tasks.workunit.client.1.smithi026.stdout:8/112: dwrite d5/f21 [0,4194304] 0 2024-03-16T10:45:10.349 INFO:tasks.workunit.client.1.smithi026.stdout:0/172: chown d1/f15 54 1 2024-03-16T10:45:10.351 INFO:tasks.workunit.client.1.smithi026.stdout:1/167: mkdir d0/d17/d1f/d38 0 2024-03-16T10:45:10.353 INFO:tasks.workunit.client.1.smithi026.stdout:1/168: read - d0/d17/d1f/d1c/f36 zero size 2024-03-16T10:45:10.354 INFO:tasks.workunit.client.1.smithi026.stdout:1/169: chown d0/d17/d1f/cb 1 1 2024-03-16T10:45:10.355 INFO:tasks.workunit.client.1.smithi026.stdout:1/170: fsync d0/d33/f30 0 2024-03-16T10:45:10.360 INFO:tasks.workunit.client.1.smithi026.stdout:9/125: unlink d0/da/fe 0 2024-03-16T10:45:10.370 INFO:tasks.workunit.client.1.smithi026.stdout:0/173: symlink d1/d18/d1a/d24/d11/l3a 0 2024-03-16T10:45:10.375 INFO:tasks.workunit.client.1.smithi026.stdout:3/166: getdents d4/dc/dd 0 2024-03-16T10:45:10.377 INFO:tasks.workunit.client.1.smithi026.stdout:8/113: mkdir d5/d23 0 2024-03-16T10:45:10.379 INFO:tasks.workunit.client.1.smithi026.stdout:0/174: mknod d1/d18/d1a/d24/d11/c3b 0 2024-03-16T10:45:10.381 INFO:tasks.workunit.client.1.smithi026.stdout:6/185: dwrite d0/f1d [0,4194304] 0 2024-03-16T10:45:10.381 INFO:tasks.workunit.client.1.smithi026.stdout:0/175: truncate d1/d18/d1a/f2c 6213862 0 2024-03-16T10:45:10.386 INFO:tasks.workunit.client.1.smithi026.stdout:0/176: dread d1/f1e [0,4194304] 0 2024-03-16T10:45:10.387 INFO:tasks.workunit.client.1.smithi026.stdout:9/126: creat d0/da/d13/f26 x:0 0 0 2024-03-16T10:45:10.388 INFO:tasks.workunit.client.0.smithi002.stdout:7/798: sync 2024-03-16T10:45:10.398 INFO:tasks.workunit.client.1.smithi026.stdout:9/127: dwrite d0/da/d16/d17/d19/f1e [0,4194304] 0 2024-03-16T10:45:10.401 INFO:tasks.workunit.client.1.smithi026.stdout:5/122: sync 2024-03-16T10:45:10.401 INFO:tasks.workunit.client.1.smithi026.stdout:2/166: sync 2024-03-16T10:45:10.401 INFO:tasks.workunit.client.1.smithi026.stdout:6/186: creat d0/d10/d17/f43 x:0 0 0 2024-03-16T10:45:10.401 INFO:tasks.workunit.client.1.smithi026.stdout:3/167: dwrite d4/dc/d31/f34 [0,4194304] 0 2024-03-16T10:45:10.404 INFO:tasks.workunit.client.1.smithi026.stdout:5/123: fsync d1/db/d1f/f26 0 2024-03-16T10:45:10.409 INFO:tasks.workunit.client.1.smithi026.stdout:0/177: dread d1/d18/d1a/d24/d6/f19 [0,4194304] 0 2024-03-16T10:45:10.418 INFO:tasks.workunit.client.1.smithi026.stdout:1/171: write d0/fd [3701493,33771] 0 2024-03-16T10:45:10.420 INFO:tasks.workunit.client.1.smithi026.stdout:3/168: mkdir d4/dc/d31/d48 0 2024-03-16T10:45:10.421 INFO:tasks.workunit.client.1.smithi026.stdout:3/169: chown d4/dc/d31/c3c 0 1 2024-03-16T10:45:10.422 INFO:tasks.workunit.client.1.smithi026.stdout:3/170: dread - d4/dc/dd/d3d/f39 zero size 2024-03-16T10:45:10.424 INFO:tasks.workunit.client.1.smithi026.stdout:0/178: write d1/d18/d1a/d24/d6/f19 [1988782,88548] 0 2024-03-16T10:45:10.426 INFO:tasks.workunit.client.1.smithi026.stdout:1/172: dwrite d0/d17/d1f/d1c/f2f [4194304,4194304] 0 2024-03-16T10:45:10.427 INFO:tasks.workunit.client.1.smithi026.stdout:0/179: write d1/f37 [85770,92141] 0 2024-03-16T10:45:10.435 INFO:tasks.workunit.client.1.smithi026.stdout:2/167: dread d2/d29/d17/d1a/d2b/d2e/f31 [0,4194304] 0 2024-03-16T10:45:10.437 INFO:tasks.workunit.client.1.smithi026.stdout:3/171: creat d4/dc/dd/d3d/f49 x:0 0 0 2024-03-16T10:45:10.444 INFO:tasks.workunit.client.1.smithi026.stdout:3/172: dwrite d4/d29/f30 [0,4194304] 0 2024-03-16T10:45:10.445 INFO:tasks.workunit.client.1.smithi026.stdout:8/114: dwrite d5/d7/f1c [0,4194304] 0 2024-03-16T10:45:10.446 INFO:tasks.workunit.client.1.smithi026.stdout:6/187: creat d0/f44 x:0 0 0 2024-03-16T10:45:10.457 INFO:tasks.workunit.client.1.smithi026.stdout:9/128: dwrite d0/da/f12 [0,4194304] 0 2024-03-16T10:45:10.467 INFO:tasks.workunit.client.1.smithi026.stdout:8/115: mknod d5/dc/d1f/c24 0 2024-03-16T10:45:10.471 INFO:tasks.workunit.client.1.smithi026.stdout:2/168: dwrite d2/d29/d17/d1a/d2b/d2e/f31 [0,4194304] 0 2024-03-16T10:45:10.474 INFO:tasks.workunit.client.1.smithi026.stdout:0/180: creat d1/f3c x:0 0 0 2024-03-16T10:45:10.475 INFO:tasks.workunit.client.0.smithi002.stdout:5/816: creat d0/d16/dab/d33/d8c/ffc x:0 0 0 2024-03-16T10:45:10.476 INFO:tasks.workunit.client.1.smithi026.stdout:0/181: write d1/d18/d1a/d24/d6/f28 [5145946,58427] 0 2024-03-16T10:45:10.476 INFO:tasks.workunit.client.1.smithi026.stdout:1/173: truncate d0/d17/d1f/d1c/f27 3231028 0 2024-03-16T10:45:10.478 INFO:tasks.workunit.client.1.smithi026.stdout:1/174: write d0/d17/f37 [744790,94490] 0 2024-03-16T10:45:10.478 INFO:tasks.workunit.client.1.smithi026.stdout:9/129: creat d0/da/f27 x:0 0 0 2024-03-16T10:45:10.481 INFO:tasks.workunit.client.0.smithi002.stdout:3/947: write d1/db6/d15/d87/f113 [3382009,122734] 0 2024-03-16T10:45:10.482 INFO:tasks.workunit.client.1.smithi026.stdout:3/173: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:10.484 INFO:tasks.workunit.client.1.smithi026.stdout:8/116: mknod d5/dc/d1f/c25 0 2024-03-16T10:45:10.485 INFO:tasks.workunit.client.1.smithi026.stdout:3/174: fdatasync d4/dc/dd/d3d/f3e 0 2024-03-16T10:45:10.486 INFO:tasks.workunit.client.1.smithi026.stdout:8/117: fsync d5/f21 0 2024-03-16T10:45:10.487 INFO:tasks.workunit.client.1.smithi026.stdout:1/175: dwrite d0/d17/d1f/d1c/f32 [0,4194304] 0 2024-03-16T10:45:10.488 INFO:tasks.workunit.client.1.smithi026.stdout:3/175: readlink d4/dc/d31/l37 0 2024-03-16T10:45:10.490 INFO:tasks.workunit.client.0.smithi002.stdout:0/917: rename d3/df8/f117 to d3/d28/d65/d108/f12d 0 2024-03-16T10:45:10.492 INFO:tasks.workunit.client.1.smithi026.stdout:2/169: write d2/d29/f10 [1779711,54155] 0 2024-03-16T10:45:10.494 INFO:tasks.workunit.client.0.smithi002.stdout:4/849: unlink d2/d33/d35/l62 0 2024-03-16T10:45:10.496 INFO:tasks.workunit.client.0.smithi002.stdout:9/976: symlink d5/d9/dc/d28/d90/l13e 0 2024-03-16T10:45:10.497 INFO:tasks.workunit.client.1.smithi026.stdout:9/130: symlink d0/da/d16/d17/l28 0 2024-03-16T10:45:10.498 INFO:tasks.workunit.client.1.smithi026.stdout:9/131: write d0/da/d14/f25 [90420,6921] 0 2024-03-16T10:45:10.502 INFO:tasks.workunit.client.0.smithi002.stdout:1/980: creat d2/df/f140 x:0 0 0 2024-03-16T10:45:10.506 INFO:tasks.workunit.client.1.smithi026.stdout:6/188: creat d0/f45 x:0 0 0 2024-03-16T10:45:10.506 INFO:tasks.workunit.client.1.smithi026.stdout:6/189: chown d0/d41 853 1 2024-03-16T10:45:10.507 INFO:tasks.workunit.client.0.smithi002.stdout:6/961: symlink d15/d2d/d2b/l14c 0 2024-03-16T10:45:10.508 INFO:tasks.workunit.client.1.smithi026.stdout:8/118: fsync d5/d7/f1a 0 2024-03-16T10:45:10.509 INFO:tasks.workunit.client.0.smithi002.stdout:2/861: getdents d7/d2e/d1d/d2f/df2 0 2024-03-16T10:45:10.511 INFO:tasks.workunit.client.1.smithi026.stdout:1/176: rename d0/l10 to d0/d33/d2e/l39 0 2024-03-16T10:45:10.514 INFO:tasks.workunit.client.0.smithi002.stdout:5/817: dread d0/d1f/f36 [0,4194304] 0 2024-03-16T10:45:10.514 INFO:tasks.workunit.client.0.smithi002.stdout:5/818: stat d0/d16 0 2024-03-16T10:45:10.515 INFO:tasks.workunit.client.0.smithi002.stdout:0/918: readlink d3/d5/la8 0 2024-03-16T10:45:10.517 INFO:tasks.workunit.client.1.smithi026.stdout:9/132: symlink d0/da/l29 0 2024-03-16T10:45:10.518 INFO:tasks.workunit.client.0.smithi002.stdout:4/850: creat d2/d6/d32/d66/d7f/de7/d5e/d93/f101 x:0 0 0 2024-03-16T10:45:10.519 INFO:tasks.workunit.client.1.smithi026.stdout:6/190: symlink d0/d28/l46 0 2024-03-16T10:45:10.521 INFO:tasks.workunit.client.0.smithi002.stdout:1/981: write d2/df/d12/d84/d87/d96/db5/d51/d56/f8c [5899655,115543] 0 2024-03-16T10:45:10.523 INFO:tasks.workunit.client.0.smithi002.stdout:3/948: creat d1/dd/d28/d7b/d2c/d63/d105/d126/d130/f14f x:0 0 0 2024-03-16T10:45:10.525 INFO:tasks.workunit.client.0.smithi002.stdout:6/962: creat d15/d2d/d2b/de4/f14d x:0 0 0 2024-03-16T10:45:10.525 INFO:tasks.workunit.client.0.smithi002.stdout:6/963: chown d15/d2d/d73/d115/d127 1 1 2024-03-16T10:45:10.527 INFO:tasks.workunit.client.1.smithi026.stdout:3/176: rename d4/dc/f36 to d4/dc/dd/d3d/f4a 0 2024-03-16T10:45:10.528 INFO:tasks.workunit.client.1.smithi026.stdout:3/177: read - d4/dc/f43 zero size 2024-03-16T10:45:10.528 INFO:tasks.workunit.client.1.smithi026.stdout:4/162: sync 2024-03-16T10:45:10.530 INFO:tasks.workunit.client.0.smithi002.stdout:5/819: creat d0/d24/dad/db6/db9/ffd x:0 0 0 2024-03-16T10:45:10.530 INFO:tasks.workunit.client.0.smithi002.stdout:5/820: dread - d0/d24/fee zero size 2024-03-16T10:45:10.532 INFO:tasks.workunit.client.1.smithi026.stdout:2/170: mknod d2/d29/c37 0 2024-03-16T10:45:10.533 INFO:tasks.workunit.client.0.smithi002.stdout:4/851: stat d2/d33/l11 0 2024-03-16T10:45:10.535 INFO:tasks.workunit.client.1.smithi026.stdout:9/133: unlink d0/ld 0 2024-03-16T10:45:10.536 INFO:tasks.workunit.client.0.smithi002.stdout:1/982: readlink d2/df/d12/d84/d100/l118 0 2024-03-16T10:45:10.537 INFO:tasks.workunit.client.1.smithi026.stdout:6/191: rmdir d0/d10/d17 39 2024-03-16T10:45:10.538 INFO:tasks.workunit.client.1.smithi026.stdout:6/192: read - d0/d10/f36 zero size 2024-03-16T10:45:10.538 INFO:tasks.workunit.client.0.smithi002.stdout:4/852: dwrite d2/d6/f89 [0,4194304] 0 2024-03-16T10:45:10.538 INFO:tasks.workunit.client.0.smithi002.stdout:1/983: dread d2/df/d12/df6/ded/ff1 [0,4194304] 0 2024-03-16T10:45:10.539 INFO:tasks.workunit.client.0.smithi002.stdout:4/853: dread - d2/d6/d32/d66/d7f/de7/d5e/f9a zero size 2024-03-16T10:45:10.539 INFO:tasks.workunit.client.0.smithi002.stdout:3/949: creat d1/dd/d28/d7b/d22/f150 x:0 0 0 2024-03-16T10:45:10.541 INFO:tasks.workunit.client.0.smithi002.stdout:1/984: dread - d2/df/d12/d84/d87/d96/db5/d1c/da0/d119/f131 zero size 2024-03-16T10:45:10.542 INFO:tasks.workunit.client.0.smithi002.stdout:6/964: symlink d15/dfd/d96/l14e 0 2024-03-16T10:45:10.544 INFO:tasks.workunit.client.0.smithi002.stdout:6/965: write d15/d2d/d2b/ff4 [882809,87219] 0 2024-03-16T10:45:10.544 INFO:tasks.workunit.client.1.smithi026.stdout:8/119: mknod d5/dc/c26 0 2024-03-16T10:45:10.547 INFO:tasks.workunit.client.0.smithi002.stdout:7/799: rename d3/f57 to d3/d4/d1d/d87/d5c/d63/f11e 0 2024-03-16T10:45:10.550 INFO:tasks.workunit.client.0.smithi002.stdout:5/821: fsync d0/d16/f19 0 2024-03-16T10:45:10.550 INFO:tasks.workunit.client.1.smithi026.stdout:8/120: dwrite d5/d7/f22 [0,4194304] 0 2024-03-16T10:45:10.552 INFO:tasks.workunit.client.1.smithi026.stdout:3/178: mkdir d4/dc/dd/d4b 0 2024-03-16T10:45:10.552 INFO:tasks.workunit.client.1.smithi026.stdout:8/121: write d5/f21 [4711925,101660] 0 2024-03-16T10:45:10.553 INFO:tasks.workunit.client.0.smithi002.stdout:5/822: write d0/d16/d4a/d55/fec [412927,23445] 0 2024-03-16T10:45:10.555 INFO:tasks.workunit.client.1.smithi026.stdout:3/179: truncate d4/dc/dd/f2e 413575 0 2024-03-16T10:45:10.556 INFO:tasks.workunit.client.0.smithi002.stdout:0/919: dwrite d3/de/f83 [0,4194304] 0 2024-03-16T10:45:10.559 INFO:tasks.workunit.client.0.smithi002.stdout:3/950: creat d1/d5a/f151 x:0 0 0 2024-03-16T10:45:10.559 INFO:tasks.workunit.client.0.smithi002.stdout:4/854: mknod d2/d6/d32/d66/d7f/de7/d88/c102 0 2024-03-16T10:45:10.560 INFO:tasks.workunit.client.0.smithi002.stdout:1/985: dread d2/df/d12/d84/d87/d96/db5/d51/d56/fef [0,4194304] 0 2024-03-16T10:45:10.561 INFO:tasks.workunit.client.1.smithi026.stdout:8/122: dwrite d5/d7/f1c [4194304,4194304] 0 2024-03-16T10:45:10.561 INFO:tasks.workunit.client.1.smithi026.stdout:2/171: mkdir d2/d29/d17/d38 0 2024-03-16T10:45:10.567 INFO:tasks.workunit.client.0.smithi002.stdout:7/800: chown d3/l10c 463318361 1 2024-03-16T10:45:10.569 INFO:tasks.workunit.client.1.smithi026.stdout:1/177: truncate d0/d17/d1f/d1c/f32 205283 0 2024-03-16T10:45:10.573 INFO:tasks.workunit.client.0.smithi002.stdout:5/823: mknod d0/d16/dab/d2b/d94/cfe 0 2024-03-16T10:45:10.573 INFO:tasks.workunit.client.0.smithi002.stdout:5/824: stat d0/d16/dab 0 2024-03-16T10:45:10.573 INFO:tasks.workunit.client.0.smithi002.stdout:5/825: dread - d0/d24/fe1 zero size 2024-03-16T10:45:10.574 INFO:tasks.workunit.client.0.smithi002.stdout:5/826: fdatasync d0/d16/dab/d2b/f4f 0 2024-03-16T10:45:10.575 INFO:tasks.workunit.client.1.smithi026.stdout:3/180: dread - d4/f18 zero size 2024-03-16T10:45:10.577 INFO:tasks.workunit.client.1.smithi026.stdout:2/172: readlink d2/d29/d17/d1a/l30 0 2024-03-16T10:45:10.578 INFO:tasks.workunit.client.1.smithi026.stdout:2/173: chown d2/d29/d17 2951449 1 2024-03-16T10:45:10.579 INFO:tasks.workunit.client.0.smithi002.stdout:1/986: symlink d2/df/d12/d84/d87/d96/db5/d1c/d116/l141 0 2024-03-16T10:45:10.579 INFO:tasks.workunit.client.1.smithi026.stdout:2/174: stat d2/d29/d17/f2d 0 2024-03-16T10:45:10.580 INFO:tasks.workunit.client.0.smithi002.stdout:5/827: dwrite d0/d16/dab/d43/d48/f8f [0,4194304] 0 2024-03-16T10:45:10.582 INFO:tasks.workunit.client.0.smithi002.stdout:9/977: rename d5/d9/d22/d3c/d118/f12d to d5/d9/dc/d28/d90/f13f 0 2024-03-16T10:45:10.583 INFO:tasks.workunit.client.1.smithi026.stdout:6/193: mkdir d0/d10/d47 0 2024-03-16T10:45:10.584 INFO:tasks.workunit.client.1.smithi026.stdout:6/194: truncate d0/f2c 911600 0 2024-03-16T10:45:10.588 INFO:tasks.workunit.client.0.smithi002.stdout:7/801: creat d3/d4/d1d/d87/d5c/dc3/f11f x:0 0 0 2024-03-16T10:45:10.589 INFO:tasks.workunit.client.1.smithi026.stdout:4/163: write d1/f8 [392436,46878] 0 2024-03-16T10:45:10.589 INFO:tasks.workunit.client.1.smithi026.stdout:4/164: stat d1/f22 0 2024-03-16T10:45:10.591 INFO:tasks.workunit.client.1.smithi026.stdout:6/195: read d0/d28/f37 [4133525,100661] 0 2024-03-16T10:45:10.592 INFO:tasks.workunit.client.1.smithi026.stdout:3/181: creat d4/dc/dd/f4c x:0 0 0 2024-03-16T10:45:10.592 INFO:tasks.workunit.client.0.smithi002.stdout:4/855: write d2/f79 [1301200,1414] 0 2024-03-16T10:45:10.594 INFO:tasks.workunit.client.1.smithi026.stdout:8/123: mknod d5/c27 0 2024-03-16T10:45:10.597 INFO:tasks.workunit.client.1.smithi026.stdout:2/175: creat d2/d29/d17/d1a/f39 x:0 0 0 2024-03-16T10:45:10.599 INFO:tasks.workunit.client.1.smithi026.stdout:3/182: dwrite d4/dc/dd/d3d/f39 [0,4194304] 0 2024-03-16T10:45:10.600 INFO:tasks.workunit.client.1.smithi026.stdout:3/183: dread - d4/d29/f3a zero size 2024-03-16T10:45:10.601 INFO:tasks.workunit.client.0.smithi002.stdout:1/987: mknod d2/d58/c142 0 2024-03-16T10:45:10.602 INFO:tasks.workunit.client.0.smithi002.stdout:0/920: write d3/d31/d75/db4/d109/d9c/fb9 [1564441,118370] 0 2024-03-16T10:45:10.603 INFO:tasks.workunit.client.0.smithi002.stdout:0/921: fsync d3/d28/d65/f125 0 2024-03-16T10:45:10.604 INFO:tasks.workunit.client.0.smithi002.stdout:5/828: creat d0/d24/dad/db6/fff x:0 0 0 2024-03-16T10:45:10.608 INFO:tasks.workunit.client.0.smithi002.stdout:2/862: rename d7/db/d12 to d7/d2e/d1d/d107/d10e 0 2024-03-16T10:45:10.609 INFO:tasks.workunit.client.1.smithi026.stdout:6/196: creat d0/d28/f48 x:0 0 0 2024-03-16T10:45:10.614 INFO:tasks.workunit.client.1.smithi026.stdout:2/176: write d2/d29/d17/d1a/d2b/f1d [4973248,96073] 0 2024-03-16T10:45:10.615 INFO:tasks.workunit.client.0.smithi002.stdout:3/951: creat d1/dd/d28/d7b/d8a/f152 x:0 0 0 2024-03-16T10:45:10.616 INFO:tasks.workunit.client.1.smithi026.stdout:9/134: getdents d0/da/d16/d17/d19 0 2024-03-16T10:45:10.617 INFO:tasks.workunit.client.1.smithi026.stdout:9/135: dread - d0/da/d16/d17/d19/f24 zero size 2024-03-16T10:45:10.617 INFO:tasks.workunit.client.0.smithi002.stdout:1/988: chown d2/df/d12/d84/d87/d96/db5/f55 62453681 1 2024-03-16T10:45:10.618 INFO:tasks.workunit.client.1.smithi026.stdout:9/136: dread - d0/da/d13/f26 zero size 2024-03-16T10:45:10.618 INFO:tasks.workunit.client.0.smithi002.stdout:1/989: readlink d2/df/d12/d84/l94 0 2024-03-16T10:45:10.620 INFO:tasks.workunit.client.0.smithi002.stdout:5/829: rmdir d0/d16/d4a/d55 39 2024-03-16T10:45:10.622 INFO:tasks.workunit.client.0.smithi002.stdout:9/978: dwrite d5/d9/dc/f11 [4194304,4194304] 0 2024-03-16T10:45:10.623 INFO:tasks.workunit.client.1.smithi026.stdout:6/197: mkdir d0/d28/d49 0 2024-03-16T10:45:10.623 INFO:tasks.workunit.client.1.smithi026.stdout:6/198: read d0/d10/f31 [1720457,70324] 0 2024-03-16T10:45:10.624 INFO:tasks.workunit.client.0.smithi002.stdout:2/863: creat d7/d2e/d46/f10f x:0 0 0 2024-03-16T10:45:10.625 INFO:tasks.workunit.client.0.smithi002.stdout:0/922: dread d3/d28/d3d/f74 [0,4194304] 0 2024-03-16T10:45:10.625 INFO:tasks.workunit.client.1.smithi026.stdout:8/124: truncate d5/fb 430897 0 2024-03-16T10:45:10.626 INFO:tasks.workunit.client.0.smithi002.stdout:0/923: chown d3/d28/f34 176406 1 2024-03-16T10:45:10.627 INFO:tasks.workunit.client.0.smithi002.stdout:7/802: mknod d3/d4/d1c/d5b/d64/d10b/c120 0 2024-03-16T10:45:10.628 INFO:tasks.workunit.client.0.smithi002.stdout:0/924: chown d3/d31/d75/db4/ddc/f118 108296304 1 2024-03-16T10:45:10.629 INFO:tasks.workunit.client.0.smithi002.stdout:0/925: write d3/f3a [292139,83550] 0 2024-03-16T10:45:10.629 INFO:tasks.workunit.client.1.smithi026.stdout:3/184: mkdir d4/d46/d4d 0 2024-03-16T10:45:10.629 INFO:tasks.workunit.client.0.smithi002.stdout:0/926: chown d3/d28/d3d/f46 716 1 2024-03-16T10:45:10.632 INFO:tasks.workunit.client.1.smithi026.stdout:9/137: creat d0/da/f2a x:0 0 0 2024-03-16T10:45:10.633 INFO:tasks.workunit.client.1.smithi026.stdout:8/125: dwrite d5/d7/f10 [4194304,4194304] 0 2024-03-16T10:45:10.634 INFO:tasks.workunit.client.1.smithi026.stdout:3/185: truncate d4/dc/d31/f3b 885322 0 2024-03-16T10:45:10.635 INFO:tasks.workunit.client.1.smithi026.stdout:1/178: getdents d0 0 2024-03-16T10:45:10.636 INFO:tasks.workunit.client.1.smithi026.stdout:3/186: fdatasync d4/dc/f25 0 2024-03-16T10:45:10.638 INFO:tasks.workunit.client.1.smithi026.stdout:3/187: write d4/d29/f30 [3202450,4421] 0 2024-03-16T10:45:10.639 INFO:tasks.workunit.client.1.smithi026.stdout:4/165: dwrite d1/db/de/f1d [0,4194304] 0 2024-03-16T10:45:10.645 INFO:tasks.workunit.client.1.smithi026.stdout:9/138: read d0/f9 [4159625,98324] 0 2024-03-16T10:45:10.646 INFO:tasks.workunit.client.0.smithi002.stdout:3/952: dwrite d1/dd/d28/d7b/d25/fc9 [0,4194304] 0 2024-03-16T10:45:10.646 INFO:tasks.workunit.client.1.smithi026.stdout:9/139: chown d0/da/l29 0 1 2024-03-16T10:45:10.647 INFO:tasks.workunit.client.0.smithi002.stdout:3/953: chown d1/dd/d28/d7b/d8a/d93/l8e 354476204 1 2024-03-16T10:45:10.647 INFO:tasks.workunit.client.1.smithi026.stdout:1/179: rmdir d0/d17 39 2024-03-16T10:45:10.648 INFO:tasks.workunit.client.0.smithi002.stdout:3/954: chown d1/d5a/f75 1772 1 2024-03-16T10:45:10.649 INFO:tasks.workunit.client.1.smithi026.stdout:8/126: fdatasync d5/d7/f1a 0 2024-03-16T10:45:10.650 INFO:tasks.workunit.client.1.smithi026.stdout:8/127: readlink l2 0 2024-03-16T10:45:10.652 INFO:tasks.workunit.client.0.smithi002.stdout:5/830: dread - d0/d16/d4a/d39/ff7 zero size 2024-03-16T10:45:10.654 INFO:tasks.workunit.client.0.smithi002.stdout:5/831: write d0/d16/dab/d43/d48/fd9 [1376509,31619] 0 2024-03-16T10:45:10.654 INFO:tasks.workunit.client.1.smithi026.stdout:9/140: dwrite d0/f1f [0,4194304] 0 2024-03-16T10:45:10.655 INFO:tasks.workunit.client.1.smithi026.stdout:2/177: creat d2/d29/f3a x:0 0 0 2024-03-16T10:45:10.657 INFO:tasks.workunit.client.0.smithi002.stdout:4/856: link d2/d6/d32/f4e d2/d6/d32/d66/d7f/de7/d5e/f103 0 2024-03-16T10:45:10.658 INFO:tasks.workunit.client.0.smithi002.stdout:4/857: stat d2/d6/d32/d66/d7f/f9f 0 2024-03-16T10:45:10.659 INFO:tasks.workunit.client.0.smithi002.stdout:4/858: chown d2/d6/d32/d66/d7f/de7/d2e/db4/cbf 0 1 2024-03-16T10:45:10.664 INFO:tasks.workunit.client.1.smithi026.stdout:7/175: sync 2024-03-16T10:45:10.665 INFO:tasks.workunit.client.1.smithi026.stdout:5/124: sync 2024-03-16T10:45:10.669 INFO:tasks.workunit.client.0.smithi002.stdout:7/803: symlink d3/d4/dde/l121 0 2024-03-16T10:45:10.672 INFO:tasks.workunit.client.0.smithi002.stdout:2/864: truncate d7/d2e/d28/d40/d50/f68 3526011 0 2024-03-16T10:45:10.672 INFO:tasks.workunit.client.0.smithi002.stdout:2/865: read - d7/d2e/d28/ffb zero size 2024-03-16T10:45:10.673 INFO:tasks.workunit.client.0.smithi002.stdout:3/955: mknod d1/dd/d28/d7b/d2c/d63/d105/c153 0 2024-03-16T10:45:10.673 INFO:tasks.workunit.client.0.smithi002.stdout:2/866: chown d7/d27/d3f/d7f/l9c 2 1 2024-03-16T10:45:10.675 INFO:tasks.workunit.client.0.smithi002.stdout:1/990: creat d2/f143 x:0 0 0 2024-03-16T10:45:10.675 INFO:tasks.workunit.client.1.smithi026.stdout:4/166: symlink d1/l35 0 2024-03-16T10:45:10.676 INFO:tasks.workunit.client.1.smithi026.stdout:4/167: chown d1/db/c14 29921 1 2024-03-16T10:45:10.678 INFO:tasks.workunit.client.1.smithi026.stdout:6/199: link d0/f33 d0/d10/f4a 0 2024-03-16T10:45:10.680 INFO:tasks.workunit.client.0.smithi002.stdout:0/927: write d3/f22 [1217336,16277] 0 2024-03-16T10:45:10.683 INFO:tasks.workunit.client.0.smithi002.stdout:4/859: fdatasync d2/d33/d35/f6f 0 2024-03-16T10:45:10.683 INFO:tasks.workunit.client.0.smithi002.stdout:5/832: dread d0/d16/dab/daa/faf [0,4194304] 0 2024-03-16T10:45:10.684 INFO:tasks.workunit.client.0.smithi002.stdout:4/860: dread - d2/d6/d32/d66/fcf zero size 2024-03-16T10:45:10.685 INFO:tasks.workunit.client.1.smithi026.stdout:7/176: creat d6/d1e/f32 x:0 0 0 2024-03-16T10:45:10.686 INFO:tasks.workunit.client.0.smithi002.stdout:7/804: stat d3/d37/f8f 0 2024-03-16T10:45:10.689 INFO:tasks.workunit.client.1.smithi026.stdout:8/128: mkdir d5/d28 0 2024-03-16T10:45:10.689 INFO:tasks.workunit.client.0.smithi002.stdout:2/867: rmdir d7/d27/d3b/dd4/de1 39 2024-03-16T10:45:10.690 INFO:tasks.workunit.client.0.smithi002.stdout:9/979: dwrite d5/d9/d25/d23/f8d [0,4194304] 0 2024-03-16T10:45:10.690 INFO:tasks.workunit.client.0.smithi002.stdout:2/868: chown d7/db/d70/ldd 87 1 2024-03-16T10:45:10.691 INFO:tasks.workunit.client.1.smithi026.stdout:5/125: dwrite d1/fa [0,4194304] 0 2024-03-16T10:45:10.691 INFO:tasks.workunit.client.1.smithi026.stdout:3/188: creat d4/dc/f4e x:0 0 0 2024-03-16T10:45:10.692 INFO:tasks.workunit.client.0.smithi002.stdout:2/869: fsync d7/d2e/d1d/fc2 0 2024-03-16T10:45:10.693 INFO:tasks.workunit.client.0.smithi002.stdout:3/956: mkdir d1/dd/d28/d7b/d2c/d63/d105/d154 0 2024-03-16T10:45:10.693 INFO:tasks.workunit.client.1.smithi026.stdout:5/126: write d1/f7 [5045072,19691] 0 2024-03-16T10:45:10.694 INFO:tasks.workunit.client.1.smithi026.stdout:3/189: write d4/dc/f13 [835774,117758] 0 2024-03-16T10:45:10.695 INFO:tasks.workunit.client.0.smithi002.stdout:5/833: dread d0/d16/dab/d2b/fc6 [0,4194304] 0 2024-03-16T10:45:10.696 INFO:tasks.workunit.client.1.smithi026.stdout:4/168: rename d1/c34 to d1/db/de/c36 0 2024-03-16T10:45:10.696 INFO:tasks.workunit.client.1.smithi026.stdout:4/169: fdatasync d1/db/de/f1d 0 2024-03-16T10:45:10.697 INFO:tasks.workunit.client.1.smithi026.stdout:6/200: readlink d0/d10/l29 0 2024-03-16T10:45:10.698 INFO:tasks.workunit.client.1.smithi026.stdout:3/190: truncate d4/dc/f19 541875 0 2024-03-16T10:45:10.698 INFO:tasks.workunit.client.1.smithi026.stdout:2/178: mkdir d2/d29/d3b 0 2024-03-16T10:45:10.702 INFO:tasks.workunit.client.0.smithi002.stdout:0/928: rename d3/d28/d3d to d3/d31/d75/db4/d109/d9c/d12e 0 2024-03-16T10:45:10.703 INFO:tasks.workunit.client.0.smithi002.stdout:0/929: fdatasync d3/d31/daa/f102 0 2024-03-16T10:45:10.706 INFO:tasks.workunit.client.1.smithi026.stdout:6/201: dwrite d0/d10/f23 [0,4194304] 0 2024-03-16T10:45:10.713 INFO:tasks.workunit.client.1.smithi026.stdout:7/177: dwrite d6/f24 [4194304,4194304] 0 2024-03-16T10:45:10.718 INFO:tasks.workunit.client.0.smithi002.stdout:9/980: symlink d5/d9/dc/d1e/d91/df1/l140 0 2024-03-16T10:45:10.719 INFO:tasks.workunit.client.1.smithi026.stdout:4/170: write d1/f2d [70826,34377] 0 2024-03-16T10:45:10.720 INFO:tasks.workunit.client.0.smithi002.stdout:9/981: truncate d5/d9/d25/fbb 65402 0 2024-03-16T10:45:10.721 INFO:tasks.workunit.client.1.smithi026.stdout:4/171: write d1/db/f26 [552441,16055] 0 2024-03-16T10:45:10.722 INFO:tasks.workunit.client.1.smithi026.stdout:3/191: mkdir d4/d46/d4f 0 2024-03-16T10:45:10.722 INFO:tasks.workunit.client.0.smithi002.stdout:9/982: write d5/d9/d22/f125 [443391,71393] 0 2024-03-16T10:45:10.724 INFO:tasks.workunit.client.0.smithi002.stdout:2/870: creat d7/d2e/d1d/d38/da8/f110 x:0 0 0 2024-03-16T10:45:10.725 INFO:tasks.workunit.client.0.smithi002.stdout:3/957: mkdir d1/db6/d15/d87/d11e/d155 0 2024-03-16T10:45:10.727 INFO:tasks.workunit.client.0.smithi002.stdout:1/991: mknod d2/df/d12/d29/d5c/d117/c144 0 2024-03-16T10:45:10.728 INFO:tasks.workunit.client.0.smithi002.stdout:0/930: dread d3/da0/fb1 [0,4194304] 0 2024-03-16T10:45:10.728 INFO:tasks.workunit.client.1.smithi026.stdout:6/202: write d0/d10/d17/f26 [863226,77266] 0 2024-03-16T10:45:10.730 INFO:tasks.workunit.client.0.smithi002.stdout:5/834: unlink d0/d16/f66 0 2024-03-16T10:45:10.731 INFO:tasks.workunit.client.0.smithi002.stdout:7/805: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f41 [4194304,4194304] 0 2024-03-16T10:45:10.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:10 smithi026 ceph-mon[37041]: pgmap v38: 65 pgs: 65 active+clean; 1.7 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 63 MiB/s rd, 164 MiB/s wr, 382 op/s 2024-03-16T10:45:10.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:10 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:10.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:10 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:10.736 INFO:tasks.workunit.client.0.smithi002.stdout:4/861: dread d2/d6/f13 [4194304,4194304] 0 2024-03-16T10:45:10.736 INFO:tasks.workunit.client.1.smithi026.stdout:7/178: mknod d6/de/d16/c33 0 2024-03-16T10:45:10.736 INFO:tasks.workunit.client.1.smithi026.stdout:7/179: dread - d6/de/d16/f2e zero size 2024-03-16T10:45:10.737 INFO:tasks.workunit.client.1.smithi026.stdout:7/180: stat d6/d1e/f32 0 2024-03-16T10:45:10.741 INFO:tasks.workunit.client.1.smithi026.stdout:4/172: unlink d1/fa 0 2024-03-16T10:45:10.744 INFO:tasks.workunit.client.1.smithi026.stdout:9/141: getdents d0 0 2024-03-16T10:45:10.747 INFO:tasks.workunit.client.1.smithi026.stdout:6/203: creat d0/d10/d17/f4b x:0 0 0 2024-03-16T10:45:10.748 INFO:tasks.workunit.client.1.smithi026.stdout:4/173: dwrite d1/f8 [0,4194304] 0 2024-03-16T10:45:10.749 INFO:tasks.workunit.client.1.smithi026.stdout:4/174: chown d1/db/de 4 1 2024-03-16T10:45:10.750 INFO:tasks.workunit.client.1.smithi026.stdout:4/175: write d1/db/de/f2b [162528,99055] 0 2024-03-16T10:45:10.752 INFO:tasks.workunit.client.0.smithi002.stdout:3/958: creat d1/dd/d28/d7b/d2c/d63/d105/d126/d130/f156 x:0 0 0 2024-03-16T10:45:10.756 INFO:tasks.workunit.client.1.smithi026.stdout:9/142: creat d0/da/d16/d17/d19/f2b x:0 0 0 2024-03-16T10:45:10.757 INFO:tasks.workunit.client.1.smithi026.stdout:9/143: stat d0/da/d16/l18 0 2024-03-16T10:45:10.758 INFO:tasks.workunit.client.1.smithi026.stdout:9/144: write d0/da/d16/d17/d19/f1e [1171350,120241] 0 2024-03-16T10:45:10.758 INFO:tasks.workunit.client.1.smithi026.stdout:6/204: unlink d0/d28/f48 0 2024-03-16T10:45:10.760 INFO:tasks.workunit.client.1.smithi026.stdout:9/145: chown d0/da/d16/d17/d19/f24 20 1 2024-03-16T10:45:10.760 INFO:tasks.workunit.client.1.smithi026.stdout:3/192: write d4/f1d [785734,44098] 0 2024-03-16T10:45:10.761 INFO:tasks.workunit.client.1.smithi026.stdout:9/146: chown d0/f9 3267340 1 2024-03-16T10:45:10.762 INFO:tasks.workunit.client.1.smithi026.stdout:8/129: rename d5/fb to d5/dc/f29 0 2024-03-16T10:45:10.762 INFO:tasks.workunit.client.1.smithi026.stdout:3/193: chown d4/dc/dd/d3d/f39 51 1 2024-03-16T10:45:10.764 INFO:tasks.workunit.client.1.smithi026.stdout:3/194: fsync d4/f1d 0 2024-03-16T10:45:10.764 INFO:tasks.workunit.client.1.smithi026.stdout:7/181: creat d6/d2f/f34 x:0 0 0 2024-03-16T10:45:10.768 INFO:tasks.workunit.client.1.smithi026.stdout:7/182: write d6/d1e/f20 [939634,31060] 0 2024-03-16T10:45:10.768 INFO:tasks.workunit.client.0.smithi002.stdout:0/931: fdatasync d3/d5/f72 0 2024-03-16T10:45:10.769 INFO:tasks.workunit.client.0.smithi002.stdout:0/932: write d3/d59/dca/faf [411514,51884] 0 2024-03-16T10:45:10.770 INFO:tasks.workunit.client.0.smithi002.stdout:0/933: write d3/d59/dca/f52 [3564753,21536] 0 2024-03-16T10:45:10.770 INFO:tasks.workunit.client.0.smithi002.stdout:3/959: chown d1/dd/d28/d7b/d22/l140 6 1 2024-03-16T10:45:10.771 INFO:tasks.workunit.client.0.smithi002.stdout:6/966: sync 2024-03-16T10:45:10.772 INFO:tasks.workunit.client.1.smithi026.stdout:9/147: dwrite d0/f1f [0,4194304] 0 2024-03-16T10:45:10.773 INFO:tasks.workunit.client.1.smithi026.stdout:8/130: dread d5/d7/f1c [0,4194304] 0 2024-03-16T10:45:10.774 INFO:tasks.workunit.client.0.smithi002.stdout:5/835: creat d0/d16/d4a/d7d/f100 x:0 0 0 2024-03-16T10:45:10.775 INFO:tasks.workunit.client.1.smithi026.stdout:7/183: truncate d6/de/d16/f2e 685982 0 2024-03-16T10:45:10.777 INFO:tasks.workunit.client.1.smithi026.stdout:3/195: dwrite d4/dc/f25 [0,4194304] 0 2024-03-16T10:45:10.780 INFO:tasks.workunit.client.1.smithi026.stdout:8/131: dread d5/d7/f10 [0,4194304] 0 2024-03-16T10:45:10.781 INFO:tasks.workunit.client.1.smithi026.stdout:8/132: chown d5/d7/f22 250 1 2024-03-16T10:45:10.782 INFO:tasks.workunit.client.1.smithi026.stdout:8/133: chown d5/dc/d1f 122 1 2024-03-16T10:45:10.783 INFO:tasks.workunit.client.1.smithi026.stdout:8/134: write f4 [8560782,67385] 0 2024-03-16T10:45:10.783 INFO:tasks.workunit.client.1.smithi026.stdout:8/135: chown d5/dc 27657 1 2024-03-16T10:45:10.785 INFO:tasks.workunit.client.0.smithi002.stdout:9/983: rename d5/db1/dec to d5/d9/dc/d1e/d91/d141 0 2024-03-16T10:45:10.788 INFO:tasks.workunit.client.1.smithi026.stdout:8/136: dread d5/dc/f12 [0,4194304] 0 2024-03-16T10:45:10.788 INFO:tasks.workunit.client.0.smithi002.stdout:2/871: creat d7/db/d70/f111 x:0 0 0 2024-03-16T10:45:10.789 INFO:tasks.workunit.client.0.smithi002.stdout:2/872: chown d7/db/d70/ldd 410141796 1 2024-03-16T10:45:10.790 INFO:tasks.workunit.client.0.smithi002.stdout:2/873: chown d7/d2e/d46/cef 349 1 2024-03-16T10:45:10.790 INFO:tasks.workunit.client.0.smithi002.stdout:2/874: chown d7/d27/d3b/fd9 4465 1 2024-03-16T10:45:10.792 INFO:tasks.workunit.client.1.smithi026.stdout:8/137: dread d5/f21 [0,4194304] 0 2024-03-16T10:45:10.794 INFO:tasks.workunit.client.1.smithi026.stdout:0/182: sync 2024-03-16T10:45:10.794 INFO:tasks.workunit.client.1.smithi026.stdout:0/183: stat d1/d18 0 2024-03-16T10:45:10.795 INFO:tasks.workunit.client.1.smithi026.stdout:0/184: write d1/d18/f36 [1572283,8667] 0 2024-03-16T10:45:10.797 INFO:tasks.workunit.client.1.smithi026.stdout:6/205: link d0/d10/f1a d0/d28/d2d/f4c 0 2024-03-16T10:45:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:10 smithi002 ceph-mon[29248]: pgmap v38: 65 pgs: 65 active+clean; 1.7 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 63 MiB/s rd, 164 MiB/s wr, 382 op/s 2024-03-16T10:45:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:10 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:10 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:10.798 INFO:tasks.workunit.client.0.smithi002.stdout:3/960: dread - d1/db6/d15/dcc/fd4 zero size 2024-03-16T10:45:10.800 INFO:tasks.workunit.client.1.smithi026.stdout:7/184: mkdir d6/d35 0 2024-03-16T10:45:10.804 INFO:tasks.workunit.client.1.smithi026.stdout:6/206: dwrite d0/d10/d17/f26 [0,4194304] 0 2024-03-16T10:45:10.805 INFO:tasks.workunit.client.0.smithi002.stdout:4/862: creat d2/d6/d32/d66/d7f/de7/d2e/f104 x:0 0 0 2024-03-16T10:45:10.807 INFO:tasks.workunit.client.0.smithi002.stdout:0/934: dwrite d3/d27/fc2 [0,4194304] 0 2024-03-16T10:45:10.810 INFO:tasks.workunit.client.0.smithi002.stdout:1/992: rename d2/df/d12/d29/d13c to d2/df/d12/df6/ded/d145 0 2024-03-16T10:45:10.810 INFO:tasks.workunit.client.0.smithi002.stdout:0/935: truncate d3/d31/d75/db4/d109/d9c/d12e/f113 51530 0 2024-03-16T10:45:10.812 INFO:tasks.workunit.client.0.smithi002.stdout:1/993: write d2/df/d12/dd7/dd9/deb/ff7 [998333,110320] 0 2024-03-16T10:45:10.813 INFO:tasks.workunit.client.0.smithi002.stdout:9/984: creat d5/d9/dc/d28/d8e/db5/f142 x:0 0 0 2024-03-16T10:45:10.813 INFO:tasks.workunit.client.0.smithi002.stdout:0/936: read d3/d31/d75/db4/d109/d9c/d12e/f113 [35922,112432] 0 2024-03-16T10:45:10.814 INFO:tasks.workunit.client.0.smithi002.stdout:1/994: dread - d2/df/d12/d84/d100/f12c zero size 2024-03-16T10:45:10.815 INFO:tasks.workunit.client.0.smithi002.stdout:0/937: dread - d3/d31/d75/de4/f12a zero size 2024-03-16T10:45:10.816 INFO:tasks.workunit.client.1.smithi026.stdout:4/176: rename d1/db/l20 to d1/db/l37 0 2024-03-16T10:45:10.816 INFO:tasks.workunit.client.1.smithi026.stdout:8/138: symlink d5/d7/l2a 0 2024-03-16T10:45:10.818 INFO:tasks.workunit.client.0.smithi002.stdout:2/875: creat d7/d2e/d28/f112 x:0 0 0 2024-03-16T10:45:10.821 INFO:tasks.workunit.client.0.smithi002.stdout:7/806: creat d3/d4/d1d/d87/d5c/d63/d34/f122 x:0 0 0 2024-03-16T10:45:10.822 INFO:tasks.workunit.client.0.smithi002.stdout:6/967: dread d15/f24 [0,4194304] 0 2024-03-16T10:45:10.822 INFO:tasks.workunit.client.0.smithi002.stdout:1/995: dwrite d2/df/d12/d74/ff9 [0,4194304] 0 2024-03-16T10:45:10.822 INFO:tasks.workunit.client.0.smithi002.stdout:6/968: stat fa 0 2024-03-16T10:45:10.823 INFO:tasks.workunit.client.1.smithi026.stdout:7/185: creat d6/de/f36 x:0 0 0 2024-03-16T10:45:10.824 INFO:tasks.workunit.client.1.smithi026.stdout:7/186: write d6/de/d16/f19 [1977275,78112] 0 2024-03-16T10:45:10.828 INFO:tasks.workunit.client.0.smithi002.stdout:4/863: symlink d2/d6/d32/d66/d7f/de7/d5e/d93/l105 0 2024-03-16T10:45:10.833 INFO:tasks.workunit.client.1.smithi026.stdout:5/127: rename d1/fa to d1/db/f28 0 2024-03-16T10:45:10.841 INFO:tasks.workunit.client.1.smithi026.stdout:4/177: symlink d1/db/d29/l38 0 2024-03-16T10:45:10.841 INFO:tasks.workunit.client.1.smithi026.stdout:4/178: fsync d1/f8 0 2024-03-16T10:45:10.842 INFO:tasks.workunit.client.0.smithi002.stdout:3/961: symlink d1/db6/d15/d87/d11e/d155/l157 0 2024-03-16T10:45:10.843 INFO:tasks.workunit.client.0.smithi002.stdout:3/962: chown d1/dd/d28/d7b/d2c/d97/d13f 0 1 2024-03-16T10:45:10.843 INFO:tasks.workunit.client.1.smithi026.stdout:8/139: write f4 [4715536,20377] 0 2024-03-16T10:45:10.843 INFO:tasks.workunit.client.0.smithi002.stdout:3/963: dread - d1/dd/d74/dd8/d110/f148 zero size 2024-03-16T10:45:10.844 INFO:tasks.workunit.client.1.smithi026.stdout:8/140: write d5/dc/fe [2447561,66292] 0 2024-03-16T10:45:10.845 INFO:tasks.workunit.client.1.smithi026.stdout:0/185: symlink d1/l3d 0 2024-03-16T10:45:10.846 INFO:tasks.workunit.client.1.smithi026.stdout:0/186: chown d1/f1e 22569 1 2024-03-16T10:45:10.848 INFO:tasks.workunit.client.1.smithi026.stdout:3/196: truncate d4/dc/f25 4143317 0 2024-03-16T10:45:10.849 INFO:tasks.workunit.client.1.smithi026.stdout:0/187: dread d1/d18/d1a/d24/f3 [0,4194304] 0 2024-03-16T10:45:10.850 INFO:tasks.workunit.client.1.smithi026.stdout:0/188: write d1/d18/d1a/d24/d11/f2d [5036960,69149] 0 2024-03-16T10:45:10.851 INFO:tasks.workunit.client.0.smithi002.stdout:1/996: readlink d2/df/l48 0 2024-03-16T10:45:10.851 INFO:tasks.workunit.client.0.smithi002.stdout:9/985: write d5/d9/dc/fa2 [1588767,75651] 0 2024-03-16T10:45:10.852 INFO:tasks.workunit.client.0.smithi002.stdout:1/997: chown d2/df/d12/dd7/dd9/lf5 18917 1 2024-03-16T10:45:10.856 INFO:tasks.workunit.client.1.smithi026.stdout:0/189: dwrite d1/f37 [0,4194304] 0 2024-03-16T10:45:10.856 INFO:tasks.workunit.client.1.smithi026.stdout:2/179: rename d2/d29/d17/d1a/d2b/f1d to d2/d29/d17/d1a/d2b/d2e/f3c 0 2024-03-16T10:45:10.857 INFO:tasks.workunit.client.1.smithi026.stdout:2/180: readlink l0 0 2024-03-16T10:45:10.857 INFO:tasks.workunit.client.0.smithi002.stdout:0/938: dwrite d3/d31/d75/db4/ddc/fc0 [0,4194304] 0 2024-03-16T10:45:10.857 INFO:tasks.workunit.client.1.smithi026.stdout:6/207: truncate d0/f6 2763283 0 2024-03-16T10:45:10.858 INFO:tasks.workunit.client.1.smithi026.stdout:0/190: fsync d1/d18/d1a/d24/d6/f19 0 2024-03-16T10:45:10.858 INFO:tasks.workunit.client.0.smithi002.stdout:0/939: readlink d3/d28/ld4 0 2024-03-16T10:45:10.859 INFO:tasks.workunit.client.0.smithi002.stdout:2/876: creat d7/d27/d3f/df1/f113 x:0 0 0 2024-03-16T10:45:10.862 INFO:tasks.workunit.client.0.smithi002.stdout:3/964: mkdir d1/dd/d28/d7b/d22/d158 0 2024-03-16T10:45:10.864 INFO:tasks.workunit.client.0.smithi002.stdout:3/965: stat d1/dd/d28/d7b/d2c/d63/d105/c153 0 2024-03-16T10:45:10.865 INFO:tasks.workunit.client.1.smithi026.stdout:6/208: dwrite d0/d28/d2d/f39 [0,4194304] 0 2024-03-16T10:45:10.865 INFO:tasks.workunit.client.0.smithi002.stdout:3/966: write d1/db6/d15/f67 [1315319,63563] 0 2024-03-16T10:45:10.870 INFO:tasks.workunit.client.0.smithi002.stdout:2/877: dwrite d7/d2e/d1d/d2f/d4b/f74 [4194304,4194304] 0 2024-03-16T10:45:10.871 INFO:tasks.workunit.client.1.smithi026.stdout:9/148: getdents d0/da/d16 0 2024-03-16T10:45:10.877 INFO:tasks.workunit.client.1.smithi026.stdout:3/197: mknod d4/dc/dd/d3d/c50 0 2024-03-16T10:45:10.879 INFO:tasks.workunit.client.1.smithi026.stdout:8/141: dwrite d5/dc/f18 [0,4194304] 0 2024-03-16T10:45:10.889 INFO:tasks.workunit.client.0.smithi002.stdout:5/836: rename d0/d16/dab/d33/d8c/fa5 to d0/d16/dab/d33/f101 0 2024-03-16T10:45:10.891 INFO:tasks.workunit.client.0.smithi002.stdout:7/807: creat d3/d4/d1c/d5b/f123 x:0 0 0 2024-03-16T10:45:10.892 INFO:tasks.workunit.client.1.smithi026.stdout:0/191: write d1/d18/d1a/d2b/f32 [614200,42556] 0 2024-03-16T10:45:10.894 INFO:tasks.workunit.client.0.smithi002.stdout:9/986: dwrite d5/d9/d55/dc9/ffe [0,4194304] 0 2024-03-16T10:45:10.897 INFO:tasks.workunit.client.1.smithi026.stdout:6/209: creat d0/d28/f4d x:0 0 0 2024-03-16T10:45:10.898 INFO:tasks.workunit.client.0.smithi002.stdout:2/878: write d7/d2e/d1d/d2f/f9a [7556507,121245] 0 2024-03-16T10:45:10.898 INFO:tasks.workunit.client.1.smithi026.stdout:9/149: mknod d0/da/d13/c2c 0 2024-03-16T10:45:10.899 INFO:tasks.workunit.client.1.smithi026.stdout:6/210: truncate d0/f44 167377 0 2024-03-16T10:45:10.902 INFO:tasks.workunit.client.0.smithi002.stdout:1/998: dwrite d2/df/d12/d29/d5c/f6e [4194304,4194304] 0 2024-03-16T10:45:10.907 INFO:tasks.workunit.client.0.smithi002.stdout:5/837: creat d0/d16/d5b/f102 x:0 0 0 2024-03-16T10:45:10.908 INFO:tasks.workunit.client.0.smithi002.stdout:0/940: dwrite d3/d28/d2a/d67/f8d [0,4194304] 0 2024-03-16T10:45:10.908 INFO:tasks.workunit.client.0.smithi002.stdout:9/987: dwrite d5/d9/dc/d28/fe6 [4194304,4194304] 0 2024-03-16T10:45:10.909 INFO:tasks.workunit.client.1.smithi026.stdout:5/128: mknod d1/c29 0 2024-03-16T10:45:10.910 INFO:tasks.workunit.client.0.smithi002.stdout:3/967: getdents d1/dd/d28/d7b/d22/d158 0 2024-03-16T10:45:10.911 INFO:tasks.workunit.client.0.smithi002.stdout:0/941: write d3/d59/dca/f10c [210071,102961] 0 2024-03-16T10:45:10.911 INFO:tasks.workunit.client.0.smithi002.stdout:9/988: chown d5/d9/d25/d23/l13c 261118212 1 2024-03-16T10:45:10.917 INFO:tasks.workunit.client.1.smithi026.stdout:0/192: rmdir d1/d18/d1a/d24/d6 39 2024-03-16T10:45:10.918 INFO:tasks.workunit.client.0.smithi002.stdout:5/838: dwrite d0/d16/d4a/d39/ff7 [0,4194304] 0 2024-03-16T10:45:10.919 INFO:tasks.workunit.client.1.smithi026.stdout:3/198: dwrite d4/dc/dd/f12 [0,4194304] 0 2024-03-16T10:45:10.919 INFO:tasks.workunit.client.1.smithi026.stdout:0/193: write d1/f37 [3259533,113918] 0 2024-03-16T10:45:10.920 INFO:tasks.workunit.client.0.smithi002.stdout:1/999: stat d2/df/d12/d84/d87/d96/db5/f1f 0 2024-03-16T10:45:10.923 INFO:tasks.workunit.client.1.smithi026.stdout:4/179: rename d1/db/l1b to d1/db/l39 0 2024-03-16T10:45:10.924 INFO:tasks.workunit.client.0.smithi002.stdout:7/808: mknod d3/d4/d1d/d87/d5c/dc3/dc7/c124 0 2024-03-16T10:45:10.925 INFO:tasks.workunit.client.0.smithi002.stdout:7/809: chown d3/d61/d66/l112 7916 1 2024-03-16T10:45:10.927 INFO:tasks.workunit.client.0.smithi002.stdout:7/810: read - d3/d4/d1d/d87/d5c/dc3/f11f zero size 2024-03-16T10:45:10.929 INFO:tasks.workunit.client.0.smithi002.stdout:3/968: read d1/dd/d28/d7b/d2c/d63/d115/f14c [487856,127614] 0 2024-03-16T10:45:10.932 INFO:tasks.workunit.client.0.smithi002.stdout:9/989: creat d5/d9/dc/d1e/d94/f143 x:0 0 0 2024-03-16T10:45:10.933 INFO:tasks.workunit.client.1.smithi026.stdout:8/142: creat d5/d23/f2b x:0 0 0 2024-03-16T10:45:10.933 INFO:tasks.workunit.client.1.smithi026.stdout:8/143: stat d5/d7/f11 0 2024-03-16T10:45:10.934 INFO:tasks.workunit.client.1.smithi026.stdout:2/181: truncate d2/d29/d17/d1a/d2b/f15 1941957 0 2024-03-16T10:45:10.934 INFO:tasks.workunit.client.0.smithi002.stdout:0/942: dread d3/d5/f30 [0,4194304] 0 2024-03-16T10:45:10.937 INFO:tasks.workunit.client.0.smithi002.stdout:4/864: rename d2/d33/d1c/f67 to d2/d6/d32/f106 0 2024-03-16T10:45:10.939 INFO:tasks.workunit.client.1.smithi026.stdout:3/199: creat d4/dc/d31/f51 x:0 0 0 2024-03-16T10:45:10.943 INFO:tasks.workunit.client.1.smithi026.stdout:6/211: symlink d0/d28/d49/l4e 0 2024-03-16T10:45:10.944 INFO:tasks.workunit.client.1.smithi026.stdout:6/212: chown d0/l2f 0 1 2024-03-16T10:45:10.944 INFO:tasks.workunit.client.1.smithi026.stdout:6/213: chown d0/d10 341666741 1 2024-03-16T10:45:10.946 INFO:tasks.workunit.client.0.smithi002.stdout:9/990: rename d5/d9/d22/c128 to d5/d9/dc/d1e/d91/d82/d10b/c144 0 2024-03-16T10:45:10.952 INFO:tasks.workunit.client.1.smithi026.stdout:2/182: unlink d2/d29/f10 0 2024-03-16T10:45:10.953 INFO:tasks.workunit.client.0.smithi002.stdout:7/811: symlink d3/d4/d1d/d87/d96/d5e/db9/l125 0 2024-03-16T10:45:10.954 INFO:tasks.workunit.client.1.smithi026.stdout:5/129: read d1/f18 [527553,30443] 0 2024-03-16T10:45:10.957 INFO:tasks.workunit.client.0.smithi002.stdout:5/839: rmdir d0/d16/d4a/d7d/ded 0 2024-03-16T10:45:10.958 INFO:tasks.workunit.client.1.smithi026.stdout:0/194: truncate d1/f15 2146631 0 2024-03-16T10:45:10.958 INFO:tasks.workunit.client.0.smithi002.stdout:7/812: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f46 [0,4194304] 0 2024-03-16T10:45:10.960 INFO:tasks.workunit.client.1.smithi026.stdout:8/144: dwrite d5/dc/d1f/f20 [0,4194304] 0 2024-03-16T10:45:10.961 INFO:tasks.workunit.client.1.smithi026.stdout:8/145: truncate d5/dc/f18 4348926 0 2024-03-16T10:45:10.962 INFO:tasks.workunit.client.1.smithi026.stdout:3/200: rename d4/f1d to d4/d46/f52 0 2024-03-16T10:45:10.964 INFO:tasks.workunit.client.1.smithi026.stdout:1/180: sync 2024-03-16T10:45:10.966 INFO:tasks.workunit.client.0.smithi002.stdout:9/991: mknod d5/d9/dc/d28/d8e/c145 0 2024-03-16T10:45:10.967 INFO:tasks.workunit.client.0.smithi002.stdout:9/992: stat d5/d9/dc/d28/d90/f12c 0 2024-03-16T10:45:10.968 INFO:tasks.workunit.client.0.smithi002.stdout:9/993: stat d5/d9/dc/d28/f65 0 2024-03-16T10:45:10.969 INFO:tasks.workunit.client.1.smithi026.stdout:8/146: dwrite d5/dc/d1f/f20 [0,4194304] 0 2024-03-16T10:45:10.970 INFO:tasks.workunit.client.1.smithi026.stdout:8/147: fdatasync d5/d7/f22 0 2024-03-16T10:45:10.971 INFO:tasks.workunit.client.0.smithi002.stdout:4/865: rename d2/d33/d35/fdf to d2/d33/d35/f107 0 2024-03-16T10:45:10.973 INFO:tasks.workunit.client.0.smithi002.stdout:7/813: mkdir d3/d4/d1d/d87/d5c/d63/d3f/daa/d106/d126 0 2024-03-16T10:45:10.974 INFO:tasks.workunit.client.0.smithi002.stdout:7/814: chown d3/d4/d1d/d87/d5c/d63/d55/fa7 46 1 2024-03-16T10:45:10.979 INFO:tasks.workunit.client.0.smithi002.stdout:5/840: creat d0/d1f/da1/f103 x:0 0 0 2024-03-16T10:45:10.981 INFO:tasks.workunit.client.1.smithi026.stdout:6/214: write d0/f33 [466189,126967] 0 2024-03-16T10:45:10.981 INFO:tasks.workunit.client.1.smithi026.stdout:5/130: stat d1/c3 0 2024-03-16T10:45:10.982 INFO:tasks.workunit.client.0.smithi002.stdout:4/866: mkdir d2/d6/d32/d68/d108 0 2024-03-16T10:45:10.983 INFO:tasks.workunit.client.1.smithi026.stdout:6/215: fdatasync d0/d10/f4a 0 2024-03-16T10:45:10.984 INFO:tasks.workunit.client.0.smithi002.stdout:7/815: mkdir d3/d37/d6a/d127 0 2024-03-16T10:45:10.986 INFO:tasks.workunit.client.0.smithi002.stdout:9/994: unlink d5/d9/dc/d28/dd6/ce1 0 2024-03-16T10:45:10.988 INFO:tasks.workunit.client.1.smithi026.stdout:0/195: unlink d1/d18/d1a/d24/l10 0 2024-03-16T10:45:10.988 INFO:tasks.workunit.client.1.smithi026.stdout:7/187: sync 2024-03-16T10:45:10.989 INFO:tasks.workunit.client.1.smithi026.stdout:0/196: chown d1/d18/d1a/f26 1 1 2024-03-16T10:45:10.990 INFO:tasks.workunit.client.0.smithi002.stdout:5/841: rename d0/d16/dab/d2b/d5d/l5e to d0/d16/dab/daa/l104 0 2024-03-16T10:45:10.994 INFO:tasks.workunit.client.0.smithi002.stdout:9/995: creat d5/d9/dc/d1e/d24/ded/f146 x:0 0 0 2024-03-16T10:45:10.996 INFO:tasks.workunit.client.0.smithi002.stdout:7/816: dread d3/f15 [0,4194304] 0 2024-03-16T10:45:10.997 INFO:tasks.workunit.client.1.smithi026.stdout:3/201: rmdir d4/dc/dd 39 2024-03-16T10:45:11.002 INFO:tasks.workunit.client.0.smithi002.stdout:9/996: write d5/d9/d22/d3c/d118/f108 [922983,45083] 0 2024-03-16T10:45:11.003 INFO:tasks.workunit.client.1.smithi026.stdout:4/180: creat d1/db/f3a x:0 0 0 2024-03-16T10:45:11.004 INFO:tasks.workunit.client.1.smithi026.stdout:3/202: dwrite d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:11.005 INFO:tasks.workunit.client.0.smithi002.stdout:4/867: creat d2/d33/d1c/f109 x:0 0 0 2024-03-16T10:45:11.007 INFO:tasks.workunit.client.1.smithi026.stdout:9/150: getdents d0/da/d16/d17 0 2024-03-16T10:45:11.008 INFO:tasks.workunit.client.1.smithi026.stdout:4/181: dread d1/f32 [0,4194304] 0 2024-03-16T10:45:11.009 INFO:tasks.workunit.client.1.smithi026.stdout:4/182: read d1/f8 [856834,48818] 0 2024-03-16T10:45:11.009 INFO:tasks.workunit.client.0.smithi002.stdout:5/842: rename d0/d16/d4a/d7d/ccb to d0/d16/d4a/d55/c105 0 2024-03-16T10:45:11.011 INFO:tasks.workunit.client.0.smithi002.stdout:9/997: fsync d5/d9/dc/d1e/d24/f4b 0 2024-03-16T10:45:11.012 INFO:tasks.workunit.client.1.smithi026.stdout:8/148: read d5/f21 [3784122,6830] 0 2024-03-16T10:45:11.014 INFO:tasks.workunit.client.0.smithi002.stdout:7/817: symlink d3/d61/l128 0 2024-03-16T10:45:11.015 INFO:tasks.workunit.client.0.smithi002.stdout:5/843: mknod d0/d16/d4a/d7d/c106 0 2024-03-16T10:45:11.016 INFO:tasks.workunit.client.0.smithi002.stdout:5/844: dread - d0/d16/dab/d33/d4b/fe8 zero size 2024-03-16T10:45:11.020 INFO:tasks.workunit.client.0.smithi002.stdout:4/868: truncate d2/d33/d1c/d45/f54 985331 0 2024-03-16T10:45:11.022 INFO:tasks.workunit.client.0.smithi002.stdout:5/845: dwrite d0/d16/dab/d33/ffb [0,4194304] 0 2024-03-16T10:45:11.022 INFO:tasks.workunit.client.0.smithi002.stdout:7/818: creat d3/d4/d1d/d87/d5c/dd0/d109/f129 x:0 0 0 2024-03-16T10:45:11.022 INFO:tasks.workunit.client.1.smithi026.stdout:5/131: dread d1/f18 [0,4194304] 0 2024-03-16T10:45:11.023 INFO:tasks.workunit.client.0.smithi002.stdout:7/819: chown d3/d61/d66/f4d 265774 1 2024-03-16T10:45:11.023 INFO:tasks.workunit.client.1.smithi026.stdout:9/151: dread d0/f9 [0,4194304] 0 2024-03-16T10:45:11.024 INFO:tasks.workunit.client.1.smithi026.stdout:2/183: rename d2/d29/c19 to d2/d29/d17/c3d 0 2024-03-16T10:45:11.025 INFO:tasks.workunit.client.0.smithi002.stdout:9/998: rename d5/d9/d22/d3c/c3e to d5/d9/d55/dc9/c147 0 2024-03-16T10:45:11.028 INFO:tasks.workunit.client.0.smithi002.stdout:5/846: dwrite d0/d16/dab/d33/d8c/f98 [0,4194304] 0 2024-03-16T10:45:11.030 INFO:tasks.workunit.client.0.smithi002.stdout:7/820: rename d3/d4/d1d/d87/d5c/d63/d3f/la0 to d3/d4/d1d/d87/d5c/dc3/l12a 0 2024-03-16T10:45:11.032 INFO:tasks.workunit.client.1.smithi026.stdout:4/183: mkdir d1/db/de/d3b 0 2024-03-16T10:45:11.033 INFO:tasks.workunit.client.1.smithi026.stdout:4/184: dread - d1/db/d29/f2c zero size 2024-03-16T10:45:11.034 INFO:tasks.workunit.client.0.smithi002.stdout:9/999: truncate d5/d9/d22/f6e 2270737 0 2024-03-16T10:45:11.036 INFO:tasks.workunit.client.0.smithi002.stdout:4/869: mkdir d2/d33/d10a 0 2024-03-16T10:45:11.037 INFO:tasks.workunit.client.0.smithi002.stdout:4/870: readlink d2/d33/d35/d59/l7a 0 2024-03-16T10:45:11.039 INFO:tasks.workunit.client.0.smithi002.stdout:4/871: read d2/d6/d32/d66/d7f/de7/d2e/fdc [2475024,66849] 0 2024-03-16T10:45:11.040 INFO:tasks.workunit.client.0.smithi002.stdout:5/847: dread d0/d1f/f2c [0,4194304] 0 2024-03-16T10:45:11.041 INFO:tasks.workunit.client.0.smithi002.stdout:7/821: dread - d3/d4/d1d/d87/d5c/d63/d34/f122 zero size 2024-03-16T10:45:11.044 INFO:tasks.workunit.client.0.smithi002.stdout:4/872: write d2/d33/d1c/fc8 [2788341,82889] 0 2024-03-16T10:45:11.046 INFO:tasks.workunit.client.0.smithi002.stdout:5/848: dread d0/d1f/f58 [0,4194304] 0 2024-03-16T10:45:11.052 INFO:tasks.workunit.client.1.smithi026.stdout:6/216: truncate d0/d10/f23 4089547 0 2024-03-16T10:45:11.053 INFO:tasks.workunit.client.1.smithi026.stdout:9/152: mknod d0/da/d13/c2d 0 2024-03-16T10:45:11.053 INFO:tasks.workunit.client.1.smithi026.stdout:5/132: dread d1/db/f12 [0,4194304] 0 2024-03-16T10:45:11.055 INFO:tasks.workunit.client.1.smithi026.stdout:2/184: fsync d2/d29/d17/d1a/d2b/d2e/f3c 0 2024-03-16T10:45:11.056 INFO:tasks.workunit.client.1.smithi026.stdout:2/185: write d2/f35 [3750999,66645] 0 2024-03-16T10:45:11.057 INFO:tasks.workunit.client.1.smithi026.stdout:1/181: symlink d0/d17/d1f/d1c/l3a 0 2024-03-16T10:45:11.060 INFO:tasks.workunit.client.1.smithi026.stdout:0/197: dwrite d1/f15 [0,4194304] 0 2024-03-16T10:45:11.061 INFO:tasks.workunit.client.1.smithi026.stdout:4/185: creat d1/db/de/f3c x:0 0 0 2024-03-16T10:45:11.066 INFO:tasks.workunit.client.0.smithi002.stdout:7/822: rename d3/d4/d1d/d87/d96/d5e/db7 to d3/d12b 0 2024-03-16T10:45:11.067 INFO:tasks.workunit.client.0.smithi002.stdout:7/823: dread - d3/d4/d1d/d87/d5c/d63/ffa zero size 2024-03-16T10:45:11.069 INFO:tasks.workunit.client.0.smithi002.stdout:4/873: creat d2/d33/d35/f10b x:0 0 0 2024-03-16T10:45:11.070 INFO:tasks.workunit.client.1.smithi026.stdout:6/217: write d0/d28/f37 [1404827,92188] 0 2024-03-16T10:45:11.071 INFO:tasks.workunit.client.0.smithi002.stdout:5/849: creat d0/d16/d4a/f107 x:0 0 0 2024-03-16T10:45:11.073 INFO:tasks.workunit.client.1.smithi026.stdout:9/153: creat d0/da/d16/d17/f2e x:0 0 0 2024-03-16T10:45:11.074 INFO:tasks.workunit.client.1.smithi026.stdout:9/154: chown d0/da/f2a 463 1 2024-03-16T10:45:11.074 INFO:tasks.workunit.client.0.smithi002.stdout:7/824: symlink d3/d4/d1d/d87/d96/d5e/db9/l12c 0 2024-03-16T10:45:11.075 INFO:tasks.workunit.client.1.smithi026.stdout:9/155: truncate d0/da/f1b 193261 0 2024-03-16T10:45:11.076 INFO:tasks.workunit.client.1.smithi026.stdout:9/156: write d0/da/f1b [358076,106614] 0 2024-03-16T10:45:11.077 INFO:tasks.workunit.client.1.smithi026.stdout:8/149: truncate d5/d7/f1a 1302684 0 2024-03-16T10:45:11.079 INFO:tasks.workunit.client.0.smithi002.stdout:5/850: write d0/d24/fe1 [378441,120812] 0 2024-03-16T10:45:11.080 INFO:tasks.workunit.client.1.smithi026.stdout:9/157: dread d0/da/f12 [0,4194304] 0 2024-03-16T10:45:11.082 INFO:tasks.workunit.client.1.smithi026.stdout:2/186: rmdir d2 39 2024-03-16T10:45:11.085 INFO:tasks.workunit.client.0.smithi002.stdout:4/874: stat d2/d6/d32/d66/d7f/de7/d88/fe0 0 2024-03-16T10:45:11.086 INFO:tasks.workunit.client.0.smithi002.stdout:4/875: read d2/d6/d32/d66/d7f/f9f [2244938,46322] 0 2024-03-16T10:45:11.090 INFO:tasks.workunit.client.1.smithi026.stdout:0/198: dread d1/d18/d1a/d24/d6/f31 [0,4194304] 0 2024-03-16T10:45:11.092 INFO:tasks.workunit.client.1.smithi026.stdout:0/199: write d1/d18/d1a/d24/d6/f28 [5664118,54291] 0 2024-03-16T10:45:11.092 INFO:tasks.workunit.client.0.smithi002.stdout:7/825: dread d3/d37/d6a/f72 [0,4194304] 0 2024-03-16T10:45:11.096 INFO:tasks.workunit.client.0.smithi002.stdout:5/851: creat d0/d16/d4a/f108 x:0 0 0 2024-03-16T10:45:11.096 INFO:tasks.workunit.client.0.smithi002.stdout:7/826: rmdir d3/d37 39 2024-03-16T10:45:11.098 INFO:tasks.workunit.client.0.smithi002.stdout:7/827: chown d3/d4/d1d/d87/d5c/dc3/dc7/c124 58094714 1 2024-03-16T10:45:11.101 INFO:tasks.workunit.client.1.smithi026.stdout:2/187: truncate d2/d29/d17/d1a/f39 885554 0 2024-03-16T10:45:11.102 INFO:tasks.workunit.client.1.smithi026.stdout:2/188: stat d2/d29/d17/d1a/d2b/c20 0 2024-03-16T10:45:11.103 INFO:tasks.workunit.client.0.smithi002.stdout:7/828: rmdir d3/d12b/de6 39 2024-03-16T10:45:11.106 INFO:tasks.workunit.client.1.smithi026.stdout:1/182: rename d0/d17/d1f/d1c to d0/d33/d3b 0 2024-03-16T10:45:11.107 INFO:tasks.workunit.client.0.smithi002.stdout:5/852: mknod d0/c109 0 2024-03-16T10:45:11.108 INFO:tasks.workunit.client.0.smithi002.stdout:4/876: getdents d2/d6/d32/d66/d7f/de7/d5e 0 2024-03-16T10:45:11.109 INFO:tasks.workunit.client.0.smithi002.stdout:4/877: dread - d2/d33/d1c/f109 zero size 2024-03-16T10:45:11.113 INFO:tasks.workunit.client.1.smithi026.stdout:0/200: rmdir d1/d18/d1a/d24/d11 39 2024-03-16T10:45:11.114 INFO:tasks.workunit.client.1.smithi026.stdout:0/201: chown d1/f37 0 1 2024-03-16T10:45:11.117 INFO:tasks.workunit.client.1.smithi026.stdout:6/218: fdatasync d0/d10/f23 0 2024-03-16T10:45:11.119 INFO:tasks.workunit.client.0.smithi002.stdout:7/829: rename d3/d4/d1d/d87/d5c/d63/d55/la5 to d3/l12d 0 2024-03-16T10:45:11.121 INFO:tasks.workunit.client.1.smithi026.stdout:0/202: dwrite d1/d18/d1a/d24/f3 [0,4194304] 0 2024-03-16T10:45:11.122 INFO:tasks.workunit.client.1.smithi026.stdout:6/219: dread d0/d10/f4a [0,4194304] 0 2024-03-16T10:45:11.122 INFO:tasks.workunit.client.1.smithi026.stdout:6/220: chown d0/l3c 87 1 2024-03-16T10:45:11.124 INFO:tasks.workunit.client.0.smithi002.stdout:5/853: rename d0/d16/dab/d33/d8c/l96 to d0/d24/dad/db6/db9/l10a 0 2024-03-16T10:45:11.125 INFO:tasks.workunit.client.0.smithi002.stdout:4/878: dwrite d2/d6/d32/d66/d7f/de7/d2e/db4/fd2 [4194304,4194304] 0 2024-03-16T10:45:11.127 INFO:tasks.workunit.client.1.smithi026.stdout:8/150: dread d5/dc/d1f/f20 [0,4194304] 0 2024-03-16T10:45:11.129 INFO:tasks.workunit.client.1.smithi026.stdout:9/158: link d0/f1f d0/f2f 0 2024-03-16T10:45:11.132 INFO:tasks.workunit.client.1.smithi026.stdout:1/183: readlink d0/l6 0 2024-03-16T10:45:11.133 INFO:tasks.workunit.client.0.smithi002.stdout:5/854: mknod d0/d16/dab/c10b 0 2024-03-16T10:45:11.135 INFO:tasks.workunit.client.1.smithi026.stdout:3/203: getdents d4/dc/dd/d3d 0 2024-03-16T10:45:11.136 INFO:tasks.workunit.client.0.smithi002.stdout:7/830: creat d3/d4/f12e x:0 0 0 2024-03-16T10:45:11.137 INFO:tasks.workunit.client.1.smithi026.stdout:3/204: truncate d4/dc/d31/f3b 1441682 0 2024-03-16T10:45:11.137 INFO:tasks.workunit.client.1.smithi026.stdout:9/159: dwrite d0/da/d16/d17/d19/f1e [0,4194304] 0 2024-03-16T10:45:11.139 INFO:tasks.workunit.client.1.smithi026.stdout:9/160: chown d0/da/d16/d17/d19/f24 32487633 1 2024-03-16T10:45:11.140 INFO:tasks.workunit.client.1.smithi026.stdout:4/186: link d1/f6 d1/db/de/f3d 0 2024-03-16T10:45:11.140 INFO:tasks.workunit.client.0.smithi002.stdout:4/879: symlink d2/d33/l10c 0 2024-03-16T10:45:11.144 INFO:tasks.workunit.client.0.smithi002.stdout:7/831: creat d3/d4/d1c/f12f x:0 0 0 2024-03-16T10:45:11.144 INFO:tasks.workunit.client.1.smithi026.stdout:3/205: dread d4/d46/f52 [0,4194304] 0 2024-03-16T10:45:11.145 INFO:tasks.workunit.client.1.smithi026.stdout:3/206: truncate d4/f41 890028 0 2024-03-16T10:45:11.145 INFO:tasks.workunit.client.0.smithi002.stdout:7/832: chown d3/d4/d1c/d5b/d64/d99/d9c/cd1 79 1 2024-03-16T10:45:11.146 INFO:tasks.workunit.client.1.smithi026.stdout:5/133: getdents d1/db 0 2024-03-16T10:45:11.147 INFO:tasks.workunit.client.0.smithi002.stdout:4/880: fdatasync d2/d6/f1e 0 2024-03-16T10:45:11.148 INFO:tasks.workunit.client.0.smithi002.stdout:4/881: chown d2/d6/f2d 159064 1 2024-03-16T10:45:11.148 INFO:tasks.workunit.client.1.smithi026.stdout:0/203: rmdir d1/d18/d1a/d2b 39 2024-03-16T10:45:11.149 INFO:tasks.workunit.client.1.smithi026.stdout:0/204: chown d1/d18/d1a/d24/d6/f28 91 1 2024-03-16T10:45:11.150 INFO:tasks.workunit.client.0.smithi002.stdout:7/833: fdatasync d3/d4/d1d/d87/d5c/d63/f3c 0 2024-03-16T10:45:11.154 INFO:tasks.workunit.client.0.smithi002.stdout:5/855: unlink d0/d24/dad/db6/db9/l10a 0 2024-03-16T10:45:11.155 INFO:tasks.workunit.client.0.smithi002.stdout:4/882: dwrite d2/d6/d32/d66/d7f/de7/d5e/d93/fac [0,4194304] 0 2024-03-16T10:45:11.156 INFO:tasks.workunit.client.0.smithi002.stdout:5/856: fdatasync d0/d16/dab/d33/d4b/fd3 0 2024-03-16T10:45:11.156 INFO:tasks.workunit.client.1.smithi026.stdout:8/151: read d5/dc/f29 [127336,65029] 0 2024-03-16T10:45:11.157 INFO:tasks.workunit.client.0.smithi002.stdout:4/883: fsync d2/d33/d1c/d45/fea 0 2024-03-16T10:45:11.158 INFO:tasks.workunit.client.0.smithi002.stdout:4/884: truncate d2/d6/d32/d66/d7f/de7/d88/fe0 165773 0 2024-03-16T10:45:11.161 INFO:tasks.workunit.client.1.smithi026.stdout:8/152: dread d5/d7/f1c [0,4194304] 0 2024-03-16T10:45:11.164 INFO:tasks.workunit.client.1.smithi026.stdout:9/161: mknod d0/da/d16/d17/d19/c30 0 2024-03-16T10:45:11.164 INFO:tasks.workunit.client.0.smithi002.stdout:4/885: dwrite d2/d33/d1c/fba [4194304,4194304] 0 2024-03-16T10:45:11.165 INFO:tasks.workunit.client.1.smithi026.stdout:2/189: write d2/d29/d17/d1a/d2b/f18 [5087499,26359] 0 2024-03-16T10:45:11.167 INFO:tasks.workunit.client.0.smithi002.stdout:5/857: rmdir d0/d16/dab/d33/d8c 39 2024-03-16T10:45:11.168 INFO:tasks.workunit.client.0.smithi002.stdout:5/858: dread - d0/d24/dad/db6/db9/fd4 zero size 2024-03-16T10:45:11.169 INFO:tasks.workunit.client.0.smithi002.stdout:5/859: readlink d0/d16/dab/d2b/d94/l6c 0 2024-03-16T10:45:11.172 INFO:tasks.workunit.client.1.smithi026.stdout:8/153: dwrite d5/d7/f1d [0,4194304] 0 2024-03-16T10:45:11.174 INFO:tasks.workunit.client.0.smithi002.stdout:7/834: rename d3/d4/d1d/d87/d5c/d63/d55/f5d to d3/d37/f130 0 2024-03-16T10:45:11.174 INFO:tasks.workunit.client.1.smithi026.stdout:3/207: dread d4/fe [0,4194304] 0 2024-03-16T10:45:11.175 INFO:tasks.workunit.client.1.smithi026.stdout:6/221: mkdir d0/d41/d4f 0 2024-03-16T10:45:11.176 INFO:tasks.workunit.client.1.smithi026.stdout:2/190: dread d2/f35 [0,4194304] 0 2024-03-16T10:45:11.177 INFO:tasks.workunit.client.0.smithi002.stdout:6/969: sync 2024-03-16T10:45:11.177 INFO:tasks.workunit.client.0.smithi002.stdout:3/969: sync 2024-03-16T10:45:11.177 INFO:tasks.workunit.client.0.smithi002.stdout:2/879: sync 2024-03-16T10:45:11.177 INFO:tasks.workunit.client.0.smithi002.stdout:0/943: sync 2024-03-16T10:45:11.178 INFO:tasks.workunit.client.0.smithi002.stdout:4/886: dread d2/d33/d1c/fc8 [0,4194304] 0 2024-03-16T10:45:11.178 INFO:tasks.workunit.client.0.smithi002.stdout:0/944: chown d3/f3a 0 1 2024-03-16T10:45:11.179 INFO:tasks.workunit.client.0.smithi002.stdout:5/860: creat d0/d16/dab/daa/f10c x:0 0 0 2024-03-16T10:45:11.179 INFO:tasks.workunit.client.1.smithi026.stdout:1/184: rename d0/d26 to d0/d33/d3c 0 2024-03-16T10:45:11.180 INFO:tasks.workunit.client.0.smithi002.stdout:2/880: fdatasync d7/d2e/d28/d40/d50/fa1 0 2024-03-16T10:45:11.180 INFO:tasks.workunit.client.1.smithi026.stdout:9/162: creat d0/da/d13/f31 x:0 0 0 2024-03-16T10:45:11.183 INFO:tasks.workunit.client.0.smithi002.stdout:7/835: read d3/d4/d1d/d87/d96/fb8 [367898,120095] 0 2024-03-16T10:45:11.184 INFO:tasks.workunit.client.1.smithi026.stdout:9/163: dread d0/f2f [0,4194304] 0 2024-03-16T10:45:11.185 INFO:tasks.workunit.client.1.smithi026.stdout:5/134: mkdir d1/d1c/d2a 0 2024-03-16T10:45:11.186 INFO:tasks.workunit.client.1.smithi026.stdout:0/205: creat d1/d12/d2f/d39/f3e x:0 0 0 2024-03-16T10:45:11.187 INFO:tasks.workunit.client.1.smithi026.stdout:0/206: write d1/d18/d1a/f2c [5850064,88693] 0 2024-03-16T10:45:11.188 INFO:tasks.workunit.client.1.smithi026.stdout:8/154: mkdir d5/d7/d2c 0 2024-03-16T10:45:11.190 INFO:tasks.workunit.client.0.smithi002.stdout:7/836: read - d3/d4/f12e zero size 2024-03-16T10:45:11.191 INFO:tasks.workunit.client.0.smithi002.stdout:6/970: mkdir d15/d2d/da8/d14f 0 2024-03-16T10:45:11.192 INFO:tasks.workunit.client.0.smithi002.stdout:3/970: dwrite d1/dd/d74/dd8/d110/f148 [0,4194304] 0 2024-03-16T10:45:11.192 INFO:tasks.workunit.client.0.smithi002.stdout:2/881: creat d7/d2e/d1d/d2f/d4b/f114 x:0 0 0 2024-03-16T10:45:11.195 INFO:tasks.workunit.client.0.smithi002.stdout:2/882: chown d7/d27/c3d 14 1 2024-03-16T10:45:11.196 INFO:tasks.workunit.client.0.smithi002.stdout:4/887: symlink d2/d6/d32/d66/d7f/de7/d2e/l10d 0 2024-03-16T10:45:11.197 INFO:tasks.workunit.client.0.smithi002.stdout:5/861: dwrite d0/d16/dab/d43/d48/f8f [4194304,4194304] 0 2024-03-16T10:45:11.198 INFO:tasks.workunit.client.0.smithi002.stdout:2/883: stat d7/d27/d3b/c53 0 2024-03-16T10:45:11.198 INFO:tasks.workunit.client.0.smithi002.stdout:7/837: chown d3/d4/d1d/d87/d5c/d63/d55/f118 1 1 2024-03-16T10:45:11.201 INFO:tasks.workunit.client.0.smithi002.stdout:7/838: chown d3/d4/d1d/d87/d5c/dd0 170083570 1 2024-03-16T10:45:11.203 INFO:tasks.workunit.client.0.smithi002.stdout:7/839: chown d3/d4/d1d/d87/d5c/d63/d3f/daa/d106/d126 76019 1 2024-03-16T10:45:11.206 INFO:tasks.workunit.client.0.smithi002.stdout:4/888: fdatasync d2/d33/d1c/f8d 0 2024-03-16T10:45:11.208 INFO:tasks.workunit.client.0.smithi002.stdout:3/971: write d1/dd/d28/d7b/d8a/d93/fbf [471068,93496] 0 2024-03-16T10:45:11.209 INFO:tasks.workunit.client.1.smithi026.stdout:9/164: mknod d0/da/d13/c32 0 2024-03-16T10:45:11.210 INFO:tasks.workunit.client.0.smithi002.stdout:0/945: truncate d3/f2d 184645 0 2024-03-16T10:45:11.211 INFO:tasks.workunit.client.1.smithi026.stdout:5/135: write d1/f21 [140592,31111] 0 2024-03-16T10:45:11.212 INFO:tasks.workunit.client.0.smithi002.stdout:0/946: write d3/d31/daa/dff/f101 [488881,36406] 0 2024-03-16T10:45:11.212 INFO:tasks.workunit.client.1.smithi026.stdout:5/136: dread - d1/db/f17 zero size 2024-03-16T10:45:11.213 INFO:tasks.workunit.client.1.smithi026.stdout:5/137: readlink d1/d1c/l24 0 2024-03-16T10:45:11.213 INFO:tasks.workunit.client.0.smithi002.stdout:5/862: rmdir d0/d16/dab/daa 39 2024-03-16T10:45:11.214 INFO:tasks.workunit.client.0.smithi002.stdout:5/863: chown d0/d16/dab/d2b/c75 36553 1 2024-03-16T10:45:11.215 INFO:tasks.workunit.client.0.smithi002.stdout:2/884: rename d7/d2e/d46/la7 to d7/d2e/d28/d40/dd8/l115 0 2024-03-16T10:45:11.216 INFO:tasks.workunit.client.1.smithi026.stdout:0/207: unlink d1/l27 0 2024-03-16T10:45:11.218 INFO:tasks.workunit.client.0.smithi002.stdout:0/947: dwrite d3/d27/fc2 [4194304,4194304] 0 2024-03-16T10:45:11.219 INFO:tasks.workunit.client.1.smithi026.stdout:8/155: rmdir d5/d23 39 2024-03-16T10:45:11.221 INFO:tasks.workunit.client.1.smithi026.stdout:6/222: mknod d0/d41/d4f/c50 0 2024-03-16T10:45:11.222 INFO:tasks.workunit.client.0.smithi002.stdout:5/864: dwrite d0/d24/dad/db6/db9/ffd [0,4194304] 0 2024-03-16T10:45:11.223 INFO:tasks.workunit.client.0.smithi002.stdout:4/889: creat d2/d6/d32/d66/d7f/de7/d5e/d93/f10e x:0 0 0 2024-03-16T10:45:11.224 INFO:tasks.workunit.client.0.smithi002.stdout:5/865: write d0/d16/d4a/d39/ff7 [3824999,94889] 0 2024-03-16T10:45:11.224 INFO:tasks.workunit.client.1.smithi026.stdout:2/191: creat d2/d29/d36/f3e x:0 0 0 2024-03-16T10:45:11.225 INFO:tasks.workunit.client.0.smithi002.stdout:5/866: chown d0/d16/dab/d2b/d5d/la9 0 1 2024-03-16T10:45:11.230 INFO:tasks.workunit.client.1.smithi026.stdout:8/156: dwrite d5/dc/f1e [0,4194304] 0 2024-03-16T10:45:11.234 INFO:tasks.workunit.client.1.smithi026.stdout:6/223: dwrite d0/d10/f36 [0,4194304] 0 2024-03-16T10:45:11.237 INFO:tasks.workunit.client.1.smithi026.stdout:9/165: mknod d0/da/d13/c33 0 2024-03-16T10:45:11.238 INFO:tasks.workunit.client.0.smithi002.stdout:2/885: symlink d7/d2e/d1d/l116 0 2024-03-16T10:45:11.239 INFO:tasks.workunit.client.1.smithi026.stdout:5/138: creat d1/db/d1f/f2b x:0 0 0 2024-03-16T10:45:11.247 INFO:tasks.workunit.client.1.smithi026.stdout:3/208: truncate d4/dc/dd/d3d/f39 3715862 0 2024-03-16T10:45:11.248 INFO:tasks.workunit.client.1.smithi026.stdout:3/209: fsync d4/dc/d31/f51 0 2024-03-16T10:45:11.249 INFO:tasks.workunit.client.1.smithi026.stdout:3/210: dread - d4/dc/dd/d3d/f49 zero size 2024-03-16T10:45:11.251 INFO:tasks.workunit.client.0.smithi002.stdout:4/890: rename d2/d6/fca to d2/d33/d10a/f10f 0 2024-03-16T10:45:11.254 INFO:tasks.workunit.client.0.smithi002.stdout:5/867: creat d0/d16/dab/d2b/d94/f10d x:0 0 0 2024-03-16T10:45:11.254 INFO:tasks.workunit.client.1.smithi026.stdout:7/188: sync 2024-03-16T10:45:11.254 INFO:tasks.workunit.client.1.smithi026.stdout:5/139: fdatasync d1/db/f12 0 2024-03-16T10:45:11.255 INFO:tasks.workunit.client.0.smithi002.stdout:6/971: link d15/d2d/d2b/d3f/c5b d15/dfd/d96/dbc/c150 0 2024-03-16T10:45:11.255 INFO:tasks.workunit.client.1.smithi026.stdout:7/189: write d6/f22 [772197,124499] 0 2024-03-16T10:45:11.256 INFO:tasks.workunit.client.0.smithi002.stdout:6/972: chown d15/d2d/d8a/d7c/d87/l104 0 1 2024-03-16T10:45:11.257 INFO:tasks.workunit.client.0.smithi002.stdout:6/973: dread - d15/d2d/d73/fee zero size 2024-03-16T10:45:11.257 INFO:tasks.workunit.client.1.smithi026.stdout:1/185: getdents d0/d17/d1f/d20 0 2024-03-16T10:45:11.258 INFO:tasks.workunit.client.0.smithi002.stdout:3/972: getdents d1/dd/d28/d7b/d8a/d93/d13d 0 2024-03-16T10:45:11.259 INFO:tasks.workunit.client.0.smithi002.stdout:3/973: chown d1/d5a/l7a 5202 1 2024-03-16T10:45:11.260 INFO:tasks.workunit.client.0.smithi002.stdout:7/840: creat d3/d61/f131 x:0 0 0 2024-03-16T10:45:11.263 INFO:tasks.workunit.client.1.smithi026.stdout:7/190: dwrite d6/f22 [0,4194304] 0 2024-03-16T10:45:11.264 INFO:tasks.workunit.client.1.smithi026.stdout:2/192: dread d2/d29/d17/d1a/f1b [0,4194304] 0 2024-03-16T10:45:11.265 INFO:tasks.workunit.client.1.smithi026.stdout:2/193: fdatasync d2/d29/d17/d1a/d2b/d2e/f3c 0 2024-03-16T10:45:11.265 INFO:tasks.workunit.client.1.smithi026.stdout:9/166: link d0/da/d16/d17/d19/f24 d0/da/d14/f34 0 2024-03-16T10:45:11.267 INFO:tasks.workunit.client.0.smithi002.stdout:0/948: mkdir d3/d12f 0 2024-03-16T10:45:11.271 INFO:tasks.workunit.client.1.smithi026.stdout:9/167: dwrite d0/da/d16/d17/d19/f1e [0,4194304] 0 2024-03-16T10:45:11.272 INFO:tasks.workunit.client.1.smithi026.stdout:9/168: fdatasync d0/da/f27 0 2024-03-16T10:45:11.272 INFO:tasks.workunit.client.0.smithi002.stdout:5/868: chown d0/d16/dab/d2b/d5d/c67 5703454 1 2024-03-16T10:45:11.274 INFO:tasks.workunit.client.0.smithi002.stdout:6/974: creat d15/d2d/d73/d97/f151 x:0 0 0 2024-03-16T10:45:11.275 INFO:tasks.workunit.client.0.smithi002.stdout:3/974: truncate d1/fb0 929056 0 2024-03-16T10:45:11.276 INFO:tasks.workunit.client.1.smithi026.stdout:1/186: mknod d0/d33/d2e/c3d 0 2024-03-16T10:45:11.278 INFO:tasks.workunit.client.1.smithi026.stdout:3/211: mkdir d4/dc/dd/d47/d53 0 2024-03-16T10:45:11.279 INFO:tasks.workunit.client.1.smithi026.stdout:5/140: dwrite d1/f18 [0,4194304] 0 2024-03-16T10:45:11.280 INFO:tasks.workunit.client.0.smithi002.stdout:0/949: mkdir d3/d28/d65/d130 0 2024-03-16T10:45:11.281 INFO:tasks.workunit.client.1.smithi026.stdout:5/141: fsync d1/db/fc 0 2024-03-16T10:45:11.282 INFO:tasks.workunit.client.1.smithi026.stdout:5/142: readlink d1/db/l10 0 2024-03-16T10:45:11.285 INFO:tasks.workunit.client.0.smithi002.stdout:4/891: rename d2/d6/d32/d66/d7f/cbc to d2/d6/d32/d66/d7f/de7/d5e/c110 0 2024-03-16T10:45:11.287 INFO:tasks.workunit.client.1.smithi026.stdout:7/191: dwrite d6/f15 [0,4194304] 0 2024-03-16T10:45:11.291 INFO:tasks.workunit.client.1.smithi026.stdout:9/169: mknod d0/da/d16/c35 0 2024-03-16T10:45:11.298 INFO:tasks.workunit.client.1.smithi026.stdout:1/187: creat d0/d33/f3e x:0 0 0 2024-03-16T10:45:11.298 INFO:tasks.workunit.client.1.smithi026.stdout:9/170: dwrite d0/da/f11 [0,4194304] 0 2024-03-16T10:45:11.300 INFO:tasks.workunit.client.0.smithi002.stdout:7/841: dwrite d3/d4/d1d/d87/d5c/d63/f3c [0,4194304] 0 2024-03-16T10:45:11.300 INFO:tasks.workunit.client.0.smithi002.stdout:0/950: rename d3/d27/fc2 to d3/d31/d75/d11f/f131 0 2024-03-16T10:45:11.301 INFO:tasks.workunit.client.0.smithi002.stdout:0/951: fsync d3/d28/d65/f112 0 2024-03-16T10:45:11.304 INFO:tasks.workunit.client.0.smithi002.stdout:5/869: mknod d0/d16/dab/d33/c10e 0 2024-03-16T10:45:11.305 INFO:tasks.workunit.client.1.smithi026.stdout:1/188: dread d0/d17/f37 [0,4194304] 0 2024-03-16T10:45:11.306 INFO:tasks.workunit.client.0.smithi002.stdout:6/975: symlink d15/d2d/d73/d115/d127/l152 0 2024-03-16T10:45:11.308 INFO:tasks.workunit.client.0.smithi002.stdout:6/976: chown d15/d2d/d2b/d3f/d77/d9f/cf8 13809259 1 2024-03-16T10:45:11.309 INFO:tasks.workunit.client.1.smithi026.stdout:7/192: mknod d6/d1e/c37 0 2024-03-16T10:45:11.309 INFO:tasks.workunit.client.0.smithi002.stdout:6/977: truncate d15/d2d/d2b/d75/f13f 507446 0 2024-03-16T10:45:11.310 INFO:tasks.workunit.client.1.smithi026.stdout:2/194: write d2/d29/d17/d1a/d2b/d2e/f31 [810272,18502] 0 2024-03-16T10:45:11.310 INFO:tasks.workunit.client.0.smithi002.stdout:3/975: fsync d1/dd/d28/f6a 0 2024-03-16T10:45:11.313 INFO:tasks.workunit.client.1.smithi026.stdout:6/224: link d0/f6 d0/f51 0 2024-03-16T10:45:11.314 INFO:tasks.workunit.client.0.smithi002.stdout:2/886: getdents d7/db 0 2024-03-16T10:45:11.318 INFO:tasks.workunit.client.1.smithi026.stdout:6/225: dwrite d0/f1d [0,4194304] 0 2024-03-16T10:45:11.319 INFO:tasks.workunit.client.1.smithi026.stdout:9/171: symlink d0/da/d16/l36 0 2024-03-16T10:45:11.320 INFO:tasks.workunit.client.1.smithi026.stdout:3/212: getdents d4/d46/d4f 0 2024-03-16T10:45:11.321 INFO:tasks.workunit.client.1.smithi026.stdout:9/172: read d0/da/f1b [160364,46106] 0 2024-03-16T10:45:11.322 INFO:tasks.workunit.client.0.smithi002.stdout:3/976: dread d1/dd/d28/d7b/d2c/d63/f117 [0,4194304] 0 2024-03-16T10:45:11.324 INFO:tasks.workunit.client.0.smithi002.stdout:7/842: creat d3/d4/d1d/d87/d5c/d63/d3f/daa/d106/f132 x:0 0 0 2024-03-16T10:45:11.324 INFO:tasks.workunit.client.1.smithi026.stdout:9/173: truncate d0/da/d16/d17/d19/f24 265405 0 2024-03-16T10:45:11.324 INFO:tasks.workunit.client.0.smithi002.stdout:7/843: chown d3/ld9 46330047 1 2024-03-16T10:45:11.329 INFO:tasks.workunit.client.0.smithi002.stdout:5/870: symlink d0/d16/d5b/da0/dc4/l10f 0 2024-03-16T10:45:11.331 INFO:tasks.workunit.client.1.smithi026.stdout:6/226: dread d0/d28/f37 [0,4194304] 0 2024-03-16T10:45:11.338 INFO:tasks.workunit.client.0.smithi002.stdout:6/978: rename d15/d2d/da8 to d15/d2d/d2b/d75/d153 0 2024-03-16T10:45:11.338 INFO:tasks.workunit.client.0.smithi002.stdout:3/977: dwrite d1/dd/d28/d7b/d8a/d93/fc2 [0,4194304] 0 2024-03-16T10:45:11.342 INFO:tasks.workunit.client.1.smithi026.stdout:5/143: link d1/db/l14 d1/db/d1f/l2c 0 2024-03-16T10:45:11.342 INFO:tasks.workunit.client.0.smithi002.stdout:7/844: creat d3/d4/d1d/d87/d5c/dd0/f133 x:0 0 0 2024-03-16T10:45:11.342 INFO:tasks.workunit.client.0.smithi002.stdout:0/952: dread d3/d28/d65/f6a [0,4194304] 0 2024-03-16T10:45:11.345 INFO:tasks.workunit.client.1.smithi026.stdout:9/174: truncate d0/f9 841604 0 2024-03-16T10:45:11.352 INFO:tasks.workunit.client.1.smithi026.stdout:3/213: symlink d4/dc/dd/d47/d53/l54 0 2024-03-16T10:45:11.354 INFO:tasks.workunit.client.1.smithi026.stdout:5/144: read d1/f7 [797096,108312] 0 2024-03-16T10:45:11.357 INFO:tasks.workunit.client.1.smithi026.stdout:9/175: symlink d0/da/d14/l37 0 2024-03-16T10:45:11.360 INFO:tasks.workunit.client.1.smithi026.stdout:6/227: write d0/f3 [2639815,130571] 0 2024-03-16T10:45:11.361 INFO:tasks.workunit.client.1.smithi026.stdout:9/176: write d0/da/f11 [3120019,99806] 0 2024-03-16T10:45:11.363 INFO:tasks.workunit.client.1.smithi026.stdout:1/189: dwrite d0/fd [0,4194304] 0 2024-03-16T10:45:11.364 INFO:tasks.workunit.client.1.smithi026.stdout:3/214: read d4/dc/dd/d3d/f39 [3199983,18682] 0 2024-03-16T10:45:11.367 INFO:tasks.workunit.client.1.smithi026.stdout:7/193: dwrite d6/d1e/f29 [0,4194304] 0 2024-03-16T10:45:11.368 INFO:tasks.workunit.client.1.smithi026.stdout:7/194: fsync d6/f22 0 2024-03-16T10:45:11.378 INFO:tasks.workunit.client.1.smithi026.stdout:7/195: dwrite d6/f15 [4194304,4194304] 0 2024-03-16T10:45:11.381 INFO:tasks.workunit.client.1.smithi026.stdout:9/177: creat d0/da/d13/f38 x:0 0 0 2024-03-16T10:45:11.382 INFO:tasks.workunit.client.1.smithi026.stdout:5/145: dwrite d1/d1c/f1e [0,4194304] 0 2024-03-16T10:45:11.386 INFO:tasks.workunit.client.1.smithi026.stdout:6/228: creat d0/d28/d49/f52 x:0 0 0 2024-03-16T10:45:11.390 INFO:tasks.workunit.client.1.smithi026.stdout:3/215: mknod d4/dc/dd/d47/d53/c55 0 2024-03-16T10:45:11.402 INFO:tasks.workunit.client.1.smithi026.stdout:7/196: read d6/f31 [118256,44248] 0 2024-03-16T10:45:11.403 INFO:tasks.workunit.client.1.smithi026.stdout:9/178: rmdir d0/da/d13 39 2024-03-16T10:45:11.405 INFO:tasks.workunit.client.1.smithi026.stdout:5/146: rename d1/db/d1f/f2b to d1/db/f2d 0 2024-03-16T10:45:11.406 INFO:tasks.workunit.client.1.smithi026.stdout:1/190: write d0/d17/f37 [1385800,29839] 0 2024-03-16T10:45:11.410 INFO:tasks.workunit.client.1.smithi026.stdout:3/216: rename d4/dc/dd/c22 to d4/dc/dd/c56 0 2024-03-16T10:45:11.412 INFO:tasks.workunit.client.1.smithi026.stdout:7/197: creat d6/f38 x:0 0 0 2024-03-16T10:45:11.418 INFO:tasks.workunit.client.1.smithi026.stdout:1/191: write d0/f1e [370326,24168] 0 2024-03-16T10:45:11.419 INFO:tasks.workunit.client.1.smithi026.stdout:7/198: dwrite d6/f38 [0,4194304] 0 2024-03-16T10:45:11.420 INFO:tasks.workunit.client.1.smithi026.stdout:1/192: chown d0/d17/d1f/c15 4886 1 2024-03-16T10:45:11.427 INFO:tasks.workunit.client.1.smithi026.stdout:4/187: sync 2024-03-16T10:45:11.428 INFO:tasks.workunit.client.1.smithi026.stdout:7/199: creat d6/f39 x:0 0 0 2024-03-16T10:45:11.429 INFO:tasks.workunit.client.1.smithi026.stdout:4/188: write d1/f2d [121037,10135] 0 2024-03-16T10:45:11.433 INFO:tasks.workunit.client.1.smithi026.stdout:3/217: mkdir d4/d46/d4d/d57 0 2024-03-16T10:45:11.438 INFO:tasks.workunit.client.1.smithi026.stdout:6/229: dwrite d0/d10/f23 [0,4194304] 0 2024-03-16T10:45:11.438 INFO:tasks.workunit.client.1.smithi026.stdout:4/189: truncate d1/f6 1471556 0 2024-03-16T10:45:11.444 INFO:tasks.workunit.client.1.smithi026.stdout:5/147: dread - d1/db/f2d zero size 2024-03-16T10:45:11.445 INFO:tasks.workunit.client.1.smithi026.stdout:5/148: stat d1/d1c/f1e 0 2024-03-16T10:45:11.449 INFO:tasks.workunit.client.1.smithi026.stdout:7/200: mkdir d6/d35/d3a 0 2024-03-16T10:45:11.452 INFO:tasks.workunit.client.1.smithi026.stdout:6/230: symlink d0/d28/d49/l53 0 2024-03-16T10:45:11.453 INFO:tasks.workunit.client.1.smithi026.stdout:6/231: truncate d0/f45 499696 0 2024-03-16T10:45:11.453 INFO:tasks.workunit.client.1.smithi026.stdout:6/232: dread - d0/f35 zero size 2024-03-16T10:45:11.456 INFO:tasks.workunit.client.1.smithi026.stdout:3/218: creat d4/dc/d31/d48/f58 x:0 0 0 2024-03-16T10:45:11.459 INFO:tasks.workunit.client.1.smithi026.stdout:1/193: write d0/d33/d3b/f27 [2175341,100923] 0 2024-03-16T10:45:11.460 INFO:tasks.workunit.client.1.smithi026.stdout:3/219: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:11.468 INFO:tasks.workunit.client.1.smithi026.stdout:4/190: read d1/f27 [1362312,115890] 0 2024-03-16T10:45:11.471 INFO:tasks.workunit.client.1.smithi026.stdout:4/191: read d1/db/de/f2b [144332,85138] 0 2024-03-16T10:45:11.471 INFO:tasks.workunit.client.1.smithi026.stdout:1/194: dread d0/d33/d3b/f27 [0,4194304] 0 2024-03-16T10:45:11.472 INFO:tasks.workunit.client.0.smithi002.stdout:7/845: sync 2024-03-16T10:45:11.475 INFO:tasks.workunit.client.1.smithi026.stdout:7/201: link d6/cd d6/c3b 0 2024-03-16T10:45:11.480 INFO:tasks.workunit.client.1.smithi026.stdout:4/192: creat d1/db/d29/f3e x:0 0 0 2024-03-16T10:45:11.482 INFO:tasks.workunit.client.1.smithi026.stdout:1/195: creat d0/d17/d1f/f3f x:0 0 0 2024-03-16T10:45:11.483 INFO:tasks.workunit.client.1.smithi026.stdout:6/233: dwrite d0/d10/fd [0,4194304] 0 2024-03-16T10:45:11.484 INFO:tasks.workunit.client.1.smithi026.stdout:4/193: chown d1/db/d29/l38 178495284 1 2024-03-16T10:45:11.486 INFO:tasks.workunit.client.1.smithi026.stdout:5/149: dwrite d1/db/f12 [0,4194304] 0 2024-03-16T10:45:11.488 INFO:tasks.workunit.client.0.smithi002.stdout:2/887: creat d7/d2e/d1d/d38/df6/f117 x:0 0 0 2024-03-16T10:45:11.490 INFO:tasks.workunit.client.1.smithi026.stdout:6/234: dread d0/d10/f1a [0,4194304] 0 2024-03-16T10:45:11.494 INFO:tasks.workunit.client.0.smithi002.stdout:6/979: creat d15/d2d/d73/d115/d127/f154 x:0 0 0 2024-03-16T10:45:11.494 INFO:tasks.workunit.client.1.smithi026.stdout:7/202: mkdir d6/d35/d3a/d3c 0 2024-03-16T10:45:11.495 INFO:tasks.workunit.client.1.smithi026.stdout:7/203: readlink d6/l17 0 2024-03-16T10:45:11.497 INFO:tasks.workunit.client.1.smithi026.stdout:7/204: write d6/f22 [5023413,128164] 0 2024-03-16T10:45:11.498 INFO:tasks.workunit.client.0.smithi002.stdout:3/978: mkdir d1/db6/d15/d87/d159 0 2024-03-16T10:45:11.499 INFO:tasks.workunit.client.0.smithi002.stdout:3/979: fdatasync d1/dd/d28/d7b/dd5/dee/f134 0 2024-03-16T10:45:11.500 INFO:tasks.workunit.client.1.smithi026.stdout:3/220: getdents d4/dc/dd/d47/d53 0 2024-03-16T10:45:11.501 INFO:tasks.workunit.client.1.smithi026.stdout:3/221: chown d4/dc/dd 114594340 1 2024-03-16T10:45:11.502 INFO:tasks.workunit.client.0.smithi002.stdout:5/871: link d0/f6f d0/d24/dad/db6/f110 0 2024-03-16T10:45:11.503 INFO:tasks.workunit.client.0.smithi002.stdout:5/872: fsync d0/d1f/f2c 0 2024-03-16T10:45:11.505 INFO:tasks.workunit.client.1.smithi026.stdout:6/235: mkdir d0/d28/d2d/d54 0 2024-03-16T10:45:11.510 INFO:tasks.workunit.client.0.smithi002.stdout:2/888: rename d7/db/d70/d72/lb8 to d7/d2e/d28/d40/d50/l118 0 2024-03-16T10:45:11.511 INFO:tasks.workunit.client.0.smithi002.stdout:2/889: write d7/d27/d3f/df1/f113 [652478,111324] 0 2024-03-16T10:45:11.511 INFO:tasks.workunit.client.1.smithi026.stdout:7/205: rename d6/l8 to d6/d35/d3a/d3c/l3d 0 2024-03-16T10:45:11.512 INFO:tasks.workunit.client.0.smithi002.stdout:2/890: write d7/d2e/d28/d40/fa3 [2467650,12699] 0 2024-03-16T10:45:11.512 INFO:tasks.workunit.client.0.smithi002.stdout:5/873: write d0/f6f [2605615,18363] 0 2024-03-16T10:45:11.513 INFO:tasks.workunit.client.0.smithi002.stdout:2/891: dread - d7/db/d70/f111 zero size 2024-03-16T10:45:11.514 INFO:tasks.workunit.client.1.smithi026.stdout:1/196: creat d0/f40 x:0 0 0 2024-03-16T10:45:11.518 INFO:tasks.workunit.client.0.smithi002.stdout:4/892: dwrite d2/d6/d32/d66/d7f/de7/d5e/d93/ff4 [0,4194304] 0 2024-03-16T10:45:11.519 INFO:tasks.workunit.client.0.smithi002.stdout:0/953: write d3/d59/dca/fee [747314,62415] 0 2024-03-16T10:45:11.520 INFO:tasks.workunit.client.0.smithi002.stdout:0/954: chown d3/d28/d2a/d67 9760 1 2024-03-16T10:45:11.520 INFO:tasks.workunit.client.0.smithi002.stdout:5/874: chown d0/d24/dad/fae 1857100136 1 2024-03-16T10:45:11.522 INFO:tasks.workunit.client.1.smithi026.stdout:5/150: rename d1/d1c/l24 to d1/db/d27/l2e 0 2024-03-16T10:45:11.524 INFO:tasks.workunit.client.1.smithi026.stdout:7/206: dread d6/f24 [0,4194304] 0 2024-03-16T10:45:11.524 INFO:tasks.workunit.client.0.smithi002.stdout:0/955: dread d3/d59/dca/fdf [0,4194304] 0 2024-03-16T10:45:11.525 INFO:tasks.workunit.client.0.smithi002.stdout:4/893: dwrite d2/d33/fe3 [0,4194304] 0 2024-03-16T10:45:11.526 INFO:tasks.workunit.client.0.smithi002.stdout:2/892: rename d7/db/l30 to d7/db/d70/d72/l119 0 2024-03-16T10:45:11.526 INFO:tasks.workunit.client.1.smithi026.stdout:1/197: readlink d0/l16 0 2024-03-16T10:45:11.529 INFO:tasks.workunit.client.0.smithi002.stdout:7/846: write d3/d4/d1d/d87/d5c/d63/d34/f98 [2456403,22275] 0 2024-03-16T10:45:11.530 INFO:tasks.workunit.client.1.smithi026.stdout:4/194: link d1/db/l37 d1/db/de/l3f 0 2024-03-16T10:45:11.536 INFO:tasks.workunit.client.0.smithi002.stdout:6/980: dwrite d15/f5f [4194304,4194304] 0 2024-03-16T10:45:11.537 INFO:tasks.workunit.client.0.smithi002.stdout:6/981: chown d15/d2d/d73/d97 12800 1 2024-03-16T10:45:11.538 INFO:tasks.workunit.client.0.smithi002.stdout:6/982: readlink lf 0 2024-03-16T10:45:11.538 INFO:tasks.workunit.client.0.smithi002.stdout:6/983: chown d15/d2d/d2c/lff 451 1 2024-03-16T10:45:11.539 INFO:tasks.workunit.client.1.smithi026.stdout:4/195: dwrite d1/db/de/f3c [0,4194304] 0 2024-03-16T10:45:11.542 INFO:tasks.workunit.client.1.smithi026.stdout:6/236: rename d0/f45 to d0/d28/d2d/f55 0 2024-03-16T10:45:11.543 INFO:tasks.workunit.client.0.smithi002.stdout:0/956: symlink d3/d31/daa/ddb/l132 0 2024-03-16T10:45:11.545 INFO:tasks.workunit.client.1.smithi026.stdout:7/207: symlink d6/de/d16/l3e 0 2024-03-16T10:45:11.545 INFO:tasks.workunit.client.0.smithi002.stdout:0/957: dread - d3/d31/d75/db4/ddc/f118 zero size 2024-03-16T10:45:11.548 INFO:tasks.workunit.client.1.smithi026.stdout:1/198: rename d0/d33/d3b/l3a to d0/d33/d3b/l41 0 2024-03-16T10:45:11.554 INFO:tasks.workunit.client.1.smithi026.stdout:5/151: mknod d1/c2f 0 2024-03-16T10:45:11.557 INFO:tasks.workunit.client.0.smithi002.stdout:6/984: dread d15/d2d/d2b/f53 [0,4194304] 0 2024-03-16T10:45:11.557 INFO:tasks.workunit.client.0.smithi002.stdout:7/847: dread d3/d4/d1d/d87/d5c/d63/d34/f6b [4194304,4194304] 0 2024-03-16T10:45:11.558 INFO:tasks.workunit.client.1.smithi026.stdout:7/208: truncate d6/f31 956350 0 2024-03-16T10:45:11.559 INFO:tasks.workunit.client.1.smithi026.stdout:7/209: readlink d6/de/l2c 0 2024-03-16T10:45:11.560 INFO:tasks.workunit.client.1.smithi026.stdout:7/210: readlink d6/d1e/l2b 0 2024-03-16T10:45:11.562 INFO:tasks.workunit.client.0.smithi002.stdout:4/894: dread d2/d6/d32/d66/d7f/de7/d88/fab [0,4194304] 0 2024-03-16T10:45:11.563 INFO:tasks.workunit.client.1.smithi026.stdout:5/152: symlink d1/db/d27/l30 0 2024-03-16T10:45:11.563 INFO:tasks.workunit.client.1.smithi026.stdout:5/153: dread - d1/db/f2d zero size 2024-03-16T10:45:11.565 INFO:tasks.workunit.client.0.smithi002.stdout:2/893: link d7/d27/d3b/d95/da5/f102 d7/d27/d3b/f11a 0 2024-03-16T10:45:11.566 INFO:tasks.workunit.client.1.smithi026.stdout:4/196: mknod d1/db/c40 0 2024-03-16T10:45:11.571 INFO:tasks.workunit.client.0.smithi002.stdout:7/848: symlink d3/d37/d6a/l134 0 2024-03-16T10:45:11.571 INFO:tasks.workunit.client.0.smithi002.stdout:0/958: truncate d3/d31/fc6 1067385 0 2024-03-16T10:45:11.572 INFO:tasks.workunit.client.1.smithi026.stdout:4/197: unlink d1/db/l15 0 2024-03-16T10:45:11.573 INFO:tasks.workunit.client.1.smithi026.stdout:6/237: link d0/d10/cb d0/d41/d4f/c56 0 2024-03-16T10:45:11.573 INFO:tasks.workunit.client.0.smithi002.stdout:4/895: truncate d2/d33/f94 1513963 0 2024-03-16T10:45:11.574 INFO:tasks.workunit.client.1.smithi026.stdout:1/199: rmdir d0/d33/d3c 0 2024-03-16T10:45:11.575 INFO:tasks.workunit.client.1.smithi026.stdout:1/200: truncate d0/d33/f30 955271 0 2024-03-16T10:45:11.575 INFO:tasks.workunit.client.0.smithi002.stdout:2/894: creat d7/d27/d3f/f11b x:0 0 0 2024-03-16T10:45:11.576 INFO:tasks.workunit.client.0.smithi002.stdout:2/895: dread - d7/d27/d3b/d95/da5/ffa zero size 2024-03-16T10:45:11.576 INFO:tasks.workunit.client.0.smithi002.stdout:6/985: mkdir d15/d155 0 2024-03-16T10:45:11.578 INFO:tasks.workunit.client.0.smithi002.stdout:5/875: link d0/f6 d0/d24/dad/f111 0 2024-03-16T10:45:11.579 INFO:tasks.workunit.client.1.smithi026.stdout:5/154: mkdir d1/d1c/d2a/d31 0 2024-03-16T10:45:11.579 INFO:tasks.workunit.client.0.smithi002.stdout:0/959: rmdir d3/de 39 2024-03-16T10:45:11.582 INFO:tasks.workunit.client.0.smithi002.stdout:7/849: rename d3/d4/d1d/d87/d5c/d63/fa2 to d3/d5f/f135 0 2024-03-16T10:45:11.583 INFO:tasks.workunit.client.0.smithi002.stdout:7/850: chown d3/d4/d1d/d87/d96/d8c/c9a 47886969 1 2024-03-16T10:45:11.583 INFO:tasks.workunit.client.1.smithi026.stdout:6/238: rmdir d0/d41 39 2024-03-16T10:45:11.584 INFO:tasks.workunit.client.0.smithi002.stdout:7/851: write d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f11d [254407,35577] 0 2024-03-16T10:45:11.585 INFO:tasks.workunit.client.0.smithi002.stdout:4/896: mkdir d2/d6/d32/d66/d7f/de7/d2e/db0/d111 0 2024-03-16T10:45:11.588 INFO:tasks.workunit.client.0.smithi002.stdout:7/852: dread d3/f15 [4194304,4194304] 0 2024-03-16T10:45:11.588 INFO:tasks.workunit.client.1.smithi026.stdout:1/201: unlink d0/l6 0 2024-03-16T10:45:11.589 INFO:tasks.workunit.client.1.smithi026.stdout:6/239: dwrite d0/d28/d2d/f39 [0,4194304] 0 2024-03-16T10:45:11.590 INFO:tasks.workunit.client.1.smithi026.stdout:1/202: write d0/d33/d3b/f2f [9193251,99104] 0 2024-03-16T10:45:11.595 INFO:tasks.workunit.client.0.smithi002.stdout:5/876: chown d0/d16/dab/d2b/c72 81025735 1 2024-03-16T10:45:11.595 INFO:tasks.workunit.client.0.smithi002.stdout:5/877: stat d0/d16/d4a 0 2024-03-16T10:45:11.597 INFO:tasks.workunit.client.1.smithi026.stdout:4/198: symlink d1/l41 0 2024-03-16T10:45:11.599 INFO:tasks.workunit.client.1.smithi026.stdout:5/155: dwrite d1/db/fc [8388608,4194304] 0 2024-03-16T10:45:11.601 INFO:tasks.workunit.client.0.smithi002.stdout:0/960: creat d3/d28/d65/d108/f133 x:0 0 0 2024-03-16T10:45:11.602 INFO:tasks.workunit.client.0.smithi002.stdout:5/878: dwrite d0/d16/dab/d33/d4b/ff4 [0,4194304] 0 2024-03-16T10:45:11.605 INFO:tasks.workunit.client.0.smithi002.stdout:5/879: chown d0/d16/dab/d33/f101 195509 1 2024-03-16T10:45:11.605 INFO:tasks.workunit.client.1.smithi026.stdout:0/208: sync 2024-03-16T10:45:11.605 INFO:tasks.workunit.client.1.smithi026.stdout:7/211: truncate d6/de/d16/f1f 158241 0 2024-03-16T10:45:11.614 INFO:tasks.workunit.client.0.smithi002.stdout:2/896: dread d7/db/f57 [0,4194304] 0 2024-03-16T10:45:11.614 INFO:tasks.workunit.client.1.smithi026.stdout:1/203: rename d0/d33/c2b to d0/d17/d1f/d20/c42 0 2024-03-16T10:45:11.616 INFO:tasks.workunit.client.0.smithi002.stdout:6/986: mkdir d15/d2d/d73/d156 0 2024-03-16T10:45:11.616 INFO:tasks.workunit.client.0.smithi002.stdout:6/987: readlink d15/d2d/d2c/lff 0 2024-03-16T10:45:11.617 INFO:tasks.workunit.client.0.smithi002.stdout:3/980: sync 2024-03-16T10:45:11.621 INFO:tasks.workunit.client.1.smithi026.stdout:1/204: dwrite d0/d33/f30 [0,4194304] 0 2024-03-16T10:45:11.622 INFO:tasks.workunit.client.0.smithi002.stdout:0/961: rename d3/d59/fc4 to d3/d28/d65/f134 0 2024-03-16T10:45:11.632 INFO:tasks.workunit.client.1.smithi026.stdout:6/240: write d0/d10/f1a [1631632,85956] 0 2024-03-16T10:45:11.636 INFO:tasks.workunit.client.1.smithi026.stdout:5/156: dread d1/f9 [0,4194304] 0 2024-03-16T10:45:11.639 INFO:tasks.workunit.client.1.smithi026.stdout:1/205: rename d0/d33/d3b/f32 to d0/d17/d1f/d20/f43 0 2024-03-16T10:45:11.641 INFO:tasks.workunit.client.1.smithi026.stdout:0/209: symlink d1/d18/d1a/d24/l3f 0 2024-03-16T10:45:11.643 INFO:tasks.workunit.client.1.smithi026.stdout:4/199: creat d1/db/f42 x:0 0 0 2024-03-16T10:45:11.645 INFO:tasks.workunit.client.0.smithi002.stdout:7/853: rename d3/d12b/de6/c104 to d3/d4/d1d/d87/d5c/dd0/d109/c136 0 2024-03-16T10:45:11.645 INFO:tasks.workunit.client.1.smithi026.stdout:6/241: creat d0/d28/d2d/f57 x:0 0 0 2024-03-16T10:45:11.646 INFO:tasks.workunit.client.0.smithi002.stdout:0/962: mkdir d3/d5/d7c/d135 0 2024-03-16T10:45:11.647 INFO:tasks.workunit.client.0.smithi002.stdout:0/963: readlink d3/d28/ld4 0 2024-03-16T10:45:11.651 INFO:tasks.workunit.client.0.smithi002.stdout:5/880: mknod d0/d24/dad/db6/db9/c112 0 2024-03-16T10:45:11.652 INFO:tasks.workunit.client.1.smithi026.stdout:5/157: dwrite d1/f13 [0,4194304] 0 2024-03-16T10:45:11.653 INFO:tasks.workunit.client.0.smithi002.stdout:6/988: mkdir d15/d2d/d2b/d157 0 2024-03-16T10:45:11.658 INFO:tasks.workunit.client.0.smithi002.stdout:4/897: getdents d2/d33/d35 0 2024-03-16T10:45:11.660 INFO:tasks.workunit.client.0.smithi002.stdout:4/898: write d2/d33/d35/f40 [688818,40973] 0 2024-03-16T10:45:11.661 INFO:tasks.workunit.client.0.smithi002.stdout:0/964: rmdir d3/d31/d75/db4 39 2024-03-16T10:45:11.663 INFO:tasks.workunit.client.0.smithi002.stdout:5/881: readlink d0/l61 0 2024-03-16T10:45:11.666 INFO:tasks.workunit.client.1.smithi026.stdout:6/242: symlink d0/d10/l58 0 2024-03-16T10:45:11.668 INFO:tasks.workunit.client.0.smithi002.stdout:0/965: mknod d3/d31/d88/dc8/c136 0 2024-03-16T10:45:11.668 INFO:tasks.workunit.client.1.smithi026.stdout:4/200: dwrite d1/db/de/f2b [0,4194304] 0 2024-03-16T10:45:11.670 INFO:tasks.workunit.client.0.smithi002.stdout:6/989: symlink d15/dfd/d96/dbc/l158 0 2024-03-16T10:45:11.671 INFO:tasks.workunit.client.0.smithi002.stdout:7/854: write d3/d4/d1c/d5b/f6d [6101,6172] 0 2024-03-16T10:45:11.673 INFO:tasks.workunit.client.1.smithi026.stdout:6/243: rmdir d0 39 2024-03-16T10:45:11.674 INFO:tasks.workunit.client.0.smithi002.stdout:3/981: getdents d1/dd/d28 0 2024-03-16T10:45:11.675 INFO:tasks.workunit.client.1.smithi026.stdout:1/206: getdents d0/d17 0 2024-03-16T10:45:11.676 INFO:tasks.workunit.client.1.smithi026.stdout:0/210: getdents d1/d18/d1a 0 2024-03-16T10:45:11.678 INFO:tasks.workunit.client.0.smithi002.stdout:0/966: mknod d3/d27/d90/c137 0 2024-03-16T10:45:11.678 INFO:tasks.workunit.client.1.smithi026.stdout:4/201: creat d1/db/d29/f43 x:0 0 0 2024-03-16T10:45:11.683 INFO:tasks.workunit.client.1.smithi026.stdout:1/207: symlink d0/d17/d1f/l44 0 2024-03-16T10:45:11.684 INFO:tasks.workunit.client.0.smithi002.stdout:6/990: creat d15/d2d/d8a/d7c/d87/f159 x:0 0 0 2024-03-16T10:45:11.684 INFO:tasks.workunit.client.0.smithi002.stdout:6/991: write d15/dfd/d96/dbb/d107/f108 [1090765,108046] 0 2024-03-16T10:45:11.685 INFO:tasks.workunit.client.0.smithi002.stdout:7/855: rename d3/d4/d1d/d87/d5c/d63/l5a to d3/d4/d1c/d5b/d64/d99/d9c/l137 0 2024-03-16T10:45:11.688 INFO:tasks.workunit.client.1.smithi026.stdout:4/202: mkdir d1/db/d29/d44 0 2024-03-16T10:45:11.689 INFO:tasks.workunit.client.1.smithi026.stdout:1/208: dwrite d0/f40 [0,4194304] 0 2024-03-16T10:45:11.690 INFO:tasks.workunit.client.0.smithi002.stdout:5/882: dwrite d0/d16/dab/f85 [0,4194304] 0 2024-03-16T10:45:11.691 INFO:tasks.workunit.client.1.smithi026.stdout:6/244: rename d0/c22 to d0/d28/d2d/c59 0 2024-03-16T10:45:11.692 INFO:tasks.workunit.client.1.smithi026.stdout:6/245: write d0/d28/d2d/f4c [1802010,20422] 0 2024-03-16T10:45:11.694 INFO:tasks.workunit.client.1.smithi026.stdout:0/211: unlink d1/d18/d1a/d24/d11/c1f 0 2024-03-16T10:45:11.695 INFO:tasks.workunit.client.1.smithi026.stdout:1/209: dread d0/d33/f30 [0,4194304] 0 2024-03-16T10:45:11.696 INFO:tasks.workunit.client.1.smithi026.stdout:0/212: chown d1/d18/d1a/l33 2051 1 2024-03-16T10:45:11.696 INFO:tasks.workunit.client.0.smithi002.stdout:5/883: dwrite d0/d16/d5b/da0/dd5/fe5 [0,4194304] 0 2024-03-16T10:45:11.697 INFO:tasks.workunit.client.1.smithi026.stdout:1/210: dread - d0/d17/d1f/f3f zero size 2024-03-16T10:45:11.697 INFO:tasks.workunit.client.1.smithi026.stdout:1/211: stat d0/d33 0 2024-03-16T10:45:11.702 INFO:tasks.workunit.client.1.smithi026.stdout:0/213: dread d1/d18/d1a/d24/f3 [0,4194304] 0 2024-03-16T10:45:11.703 INFO:tasks.workunit.client.1.smithi026.stdout:0/214: readlink d1/d18/d1a/d24/l3f 0 2024-03-16T10:45:11.703 INFO:tasks.workunit.client.0.smithi002.stdout:0/967: dwrite d3/d59/dca/f53 [0,4194304] 0 2024-03-16T10:45:11.710 INFO:tasks.workunit.client.1.smithi026.stdout:6/246: creat d0/d28/d49/f5a x:0 0 0 2024-03-16T10:45:11.717 INFO:tasks.workunit.client.0.smithi002.stdout:6/992: truncate d15/d2d/f85 1146360 0 2024-03-16T10:45:11.720 INFO:tasks.workunit.client.0.smithi002.stdout:5/884: symlink d0/d16/dab/d43/dcc/l113 0 2024-03-16T10:45:11.721 INFO:tasks.workunit.client.0.smithi002.stdout:6/993: dread - d15/d2d/d8a/d7c/d87/fe3 zero size 2024-03-16T10:45:11.722 INFO:tasks.workunit.client.0.smithi002.stdout:3/982: getdents d1/dd/d28/d7b/d25 0 2024-03-16T10:45:11.723 INFO:tasks.workunit.client.1.smithi026.stdout:4/203: rename d1/db/de/c31 to d1/c45 0 2024-03-16T10:45:11.724 INFO:tasks.workunit.client.1.smithi026.stdout:4/204: fsync d1/db/de/f2b 0 2024-03-16T10:45:11.725 INFO:tasks.workunit.client.0.smithi002.stdout:5/885: write d0/d16/d4a/d55/ff0 [812213,98895] 0 2024-03-16T10:45:11.726 INFO:tasks.workunit.client.1.smithi026.stdout:0/215: rename d1/f1e to d1/f40 0 2024-03-16T10:45:11.729 INFO:tasks.workunit.client.0.smithi002.stdout:7/856: getdents d3/d4/d1d/d87/d5c/d63/d3f 0 2024-03-16T10:45:11.730 INFO:tasks.workunit.client.0.smithi002.stdout:3/983: dread d1/dd/d28/d7b/f131 [0,4194304] 0 2024-03-16T10:45:11.731 INFO:tasks.workunit.client.1.smithi026.stdout:1/212: getdents d0 0 2024-03-16T10:45:11.731 INFO:tasks.workunit.client.0.smithi002.stdout:3/984: fdatasync d1/db6/d15/d87/d11e/f136 0 2024-03-16T10:45:11.732 INFO:tasks.workunit.client.0.smithi002.stdout:3/985: write d1/db6/d15/d87/d11e/f11f [699328,15124] 0 2024-03-16T10:45:11.733 INFO:tasks.workunit.client.0.smithi002.stdout:5/886: read d0/d16/dab/d33/d8c/f98 [2546784,48166] 0 2024-03-16T10:45:11.733 INFO:tasks.workunit.client.0.smithi002.stdout:5/887: stat d0/d16/d4a/d7d 0 2024-03-16T10:45:11.734 INFO:tasks.workunit.client.0.smithi002.stdout:5/888: dread - d0/d16/dab/d33/d4b/fde zero size 2024-03-16T10:45:11.737 INFO:tasks.workunit.client.0.smithi002.stdout:7/857: dread d3/d61/d66/f4d [0,4194304] 0 2024-03-16T10:45:11.738 INFO:tasks.workunit.client.0.smithi002.stdout:5/889: unlink d0/d24/fe1 0 2024-03-16T10:45:11.739 INFO:tasks.workunit.client.1.smithi026.stdout:0/216: symlink d1/d18/d1a/d24/l41 0 2024-03-16T10:45:11.741 INFO:tasks.workunit.client.1.smithi026.stdout:6/247: rename d0/d10/d17 to d0/d5b 0 2024-03-16T10:45:11.743 INFO:tasks.workunit.client.1.smithi026.stdout:0/217: creat d1/d12/d2f/f42 x:0 0 0 2024-03-16T10:45:11.747 INFO:tasks.workunit.client.1.smithi026.stdout:4/205: link d1/l21 d1/db/d29/d44/l46 0 2024-03-16T10:45:11.750 INFO:tasks.workunit.client.1.smithi026.stdout:0/218: symlink d1/d12/l43 0 2024-03-16T10:45:11.752 INFO:tasks.workunit.client.1.smithi026.stdout:1/213: rename d0/d17/d1f/c1b to d0/d17/d1f/d20/c45 0 2024-03-16T10:45:11.752 INFO:tasks.workunit.client.1.smithi026.stdout:1/214: stat d0/d17/d1f/f3f 0 2024-03-16T10:45:11.753 INFO:tasks.workunit.client.1.smithi026.stdout:1/215: dread - d0/d33/f3e zero size 2024-03-16T10:45:11.753 INFO:tasks.workunit.client.1.smithi026.stdout:6/248: mknod d0/d41/d4f/c5c 0 2024-03-16T10:45:11.754 INFO:tasks.workunit.client.1.smithi026.stdout:6/249: chown d0/d10/ff 20243 1 2024-03-16T10:45:11.755 INFO:tasks.workunit.client.1.smithi026.stdout:6/250: read d0/d10/f23 [4040655,98701] 0 2024-03-16T10:45:11.756 INFO:tasks.workunit.client.1.smithi026.stdout:0/219: mknod d1/d12/c44 0 2024-03-16T10:45:11.757 INFO:tasks.workunit.client.1.smithi026.stdout:1/216: write d0/d33/d3b/f27 [835717,3619] 0 2024-03-16T10:45:11.758 INFO:tasks.workunit.client.1.smithi026.stdout:4/206: mkdir d1/db/d47 0 2024-03-16T10:45:11.761 INFO:tasks.workunit.client.1.smithi026.stdout:0/220: dread d1/f37 [0,4194304] 0 2024-03-16T10:45:11.761 INFO:tasks.workunit.client.1.smithi026.stdout:6/251: rename d0/d5b/f4b to d0/d28/f5d 0 2024-03-16T10:45:11.766 INFO:tasks.workunit.client.1.smithi026.stdout:0/221: dwrite d1/d18/f36 [0,4194304] 0 2024-03-16T10:45:11.768 INFO:tasks.workunit.client.1.smithi026.stdout:1/217: rename d0/d17/l24 to d0/d17/d1f/d20/l46 0 2024-03-16T10:45:11.769 INFO:tasks.workunit.client.1.smithi026.stdout:4/207: mkdir d1/db/d29/d44/d48 0 2024-03-16T10:45:11.772 INFO:tasks.workunit.client.1.smithi026.stdout:1/218: rename d0/d33/d3b/f36 to d0/d17/f47 0 2024-03-16T10:45:11.774 INFO:tasks.workunit.client.1.smithi026.stdout:4/208: symlink d1/l49 0 2024-03-16T10:45:11.774 INFO:tasks.workunit.client.1.smithi026.stdout:0/222: dwrite d1/d18/d1a/f2c [4194304,4194304] 0 2024-03-16T10:45:11.778 INFO:tasks.workunit.client.1.smithi026.stdout:0/223: creat d1/d12/f45 x:0 0 0 2024-03-16T10:45:11.779 INFO:tasks.workunit.client.1.smithi026.stdout:4/209: rename d1/db/d29/l38 to d1/db/d29/d44/d48/l4a 0 2024-03-16T10:45:11.780 INFO:tasks.workunit.client.1.smithi026.stdout:1/219: dwrite d0/f1e [0,4194304] 0 2024-03-16T10:45:11.781 INFO:tasks.workunit.client.1.smithi026.stdout:4/210: chown d1/db/de/l24 159 1 2024-03-16T10:45:11.784 INFO:tasks.workunit.client.1.smithi026.stdout:4/211: write d1/db/d29/f43 [76555,33840] 0 2024-03-16T10:45:11.785 INFO:tasks.workunit.client.1.smithi026.stdout:1/220: creat d0/d17/f48 x:0 0 0 2024-03-16T10:45:11.789 INFO:tasks.workunit.client.1.smithi026.stdout:0/224: dwrite d1/d18/d1a/d24/d6/f19 [0,4194304] 0 2024-03-16T10:45:11.791 INFO:tasks.workunit.client.1.smithi026.stdout:0/225: write d1/d18/d1a/d24/d6/f28 [5505639,60533] 0 2024-03-16T10:45:11.795 INFO:tasks.workunit.client.1.smithi026.stdout:0/226: creat d1/d18/d1a/f46 x:0 0 0 2024-03-16T10:45:11.795 INFO:tasks.workunit.client.1.smithi026.stdout:4/212: dread d1/f2d [0,4194304] 0 2024-03-16T10:45:11.797 INFO:tasks.workunit.client.1.smithi026.stdout:4/213: read d1/f3 [644818,43490] 0 2024-03-16T10:45:11.798 INFO:tasks.workunit.client.0.smithi002.stdout:5/890: mknod d0/d16/dab/d2b/d94/c114 0 2024-03-16T10:45:11.798 INFO:tasks.workunit.client.0.smithi002.stdout:3/986: creat d1/dd/d74/f15a x:0 0 0 2024-03-16T10:45:11.799 INFO:tasks.workunit.client.1.smithi026.stdout:8/157: sync 2024-03-16T10:45:11.801 INFO:tasks.workunit.client.0.smithi002.stdout:3/987: chown d1/dd/d28/d7b/d2c/d97/fc4 1 1 2024-03-16T10:45:11.802 INFO:tasks.workunit.client.1.smithi026.stdout:8/158: unlink d5/dc/f12 0 2024-03-16T10:45:11.803 INFO:tasks.workunit.client.0.smithi002.stdout:7/858: dwrite d3/d4/d1c/fd3 [0,4194304] 0 2024-03-16T10:45:11.805 INFO:tasks.workunit.client.0.smithi002.stdout:5/891: unlink d0/d16/dab/d33/d4b/fd3 0 2024-03-16T10:45:11.805 INFO:tasks.workunit.client.1.smithi026.stdout:4/214: dwrite d1/db/de/f2b [4194304,4194304] 0 2024-03-16T10:45:11.809 INFO:tasks.workunit.client.0.smithi002.stdout:5/892: dread d0/d16/d4a/d55/ff0 [0,4194304] 0 2024-03-16T10:45:11.812 INFO:tasks.workunit.client.1.smithi026.stdout:8/159: rename d5/f6 to d5/d7/d2c/f2d 0 2024-03-16T10:45:11.812 INFO:tasks.workunit.client.0.smithi002.stdout:5/893: dread d0/d16/dab/d33/d4b/ff4 [0,4194304] 0 2024-03-16T10:45:11.814 INFO:tasks.workunit.client.0.smithi002.stdout:5/894: write d0/d16/dab/d2b/d94/fbf [202527,130211] 0 2024-03-16T10:45:11.817 INFO:tasks.workunit.client.0.smithi002.stdout:7/859: mkdir d3/d4/d1c/d5b/d64/d138 0 2024-03-16T10:45:11.825 INFO:tasks.workunit.client.1.smithi026.stdout:1/221: dwrite d0/d17/d1f/fc [4194304,4194304] 0 2024-03-16T10:45:11.829 INFO:tasks.workunit.client.0.smithi002.stdout:5/895: dread d0/d1f/f58 [0,4194304] 0 2024-03-16T10:45:11.831 INFO:tasks.workunit.client.0.smithi002.stdout:7/860: fdatasync d3/d4/d1c/fee 0 2024-03-16T10:45:11.833 INFO:tasks.workunit.client.1.smithi026.stdout:8/160: dread d5/dc/f18 [0,4194304] 0 2024-03-16T10:45:11.835 INFO:tasks.workunit.client.1.smithi026.stdout:1/222: mkdir d0/d33/d3b/d49 0 2024-03-16T10:45:11.835 INFO:tasks.workunit.client.1.smithi026.stdout:1/223: chown d0/c19 696 1 2024-03-16T10:45:11.842 INFO:tasks.workunit.client.1.smithi026.stdout:4/215: truncate d1/db/de/f2b 8189586 0 2024-03-16T10:45:11.843 INFO:tasks.workunit.client.0.smithi002.stdout:3/988: dwrite d1/dd/f103 [0,4194304] 0 2024-03-16T10:45:11.846 INFO:tasks.workunit.client.1.smithi026.stdout:8/161: write d5/d7/f1a [1159002,32933] 0 2024-03-16T10:45:11.849 INFO:tasks.workunit.client.0.smithi002.stdout:5/896: write d0/d16/f19 [2010113,110043] 0 2024-03-16T10:45:11.851 INFO:tasks.workunit.client.0.smithi002.stdout:3/989: creat d1/dd/d28/d7b/d73/f15b x:0 0 0 2024-03-16T10:45:11.852 INFO:tasks.workunit.client.0.smithi002.stdout:3/990: write d1/d5a/f151 [572939,73737] 0 2024-03-16T10:45:11.852 INFO:tasks.workunit.client.1.smithi026.stdout:8/162: rmdir d5/dc 39 2024-03-16T10:45:11.853 INFO:tasks.workunit.client.1.smithi026.stdout:8/163: dread - d5/d7/f1b zero size 2024-03-16T10:45:11.854 INFO:tasks.workunit.client.1.smithi026.stdout:1/224: truncate d0/d33/d3b/f2f 6633985 0 2024-03-16T10:45:11.855 INFO:tasks.workunit.client.0.smithi002.stdout:5/897: dwrite d0/d16/dab/d2b/d5d/f92 [0,4194304] 0 2024-03-16T10:45:11.855 INFO:tasks.workunit.client.0.smithi002.stdout:7/861: mkdir d3/d4/d1d/d87/d5c/dd0/dd2/d139 0 2024-03-16T10:45:11.856 INFO:tasks.workunit.client.0.smithi002.stdout:5/898: dread - d0/d24/ff2 zero size 2024-03-16T10:45:11.857 INFO:tasks.workunit.client.1.smithi026.stdout:4/216: unlink d1/c45 0 2024-03-16T10:45:11.864 INFO:tasks.workunit.client.0.smithi002.stdout:3/991: dwrite d1/dd/d28/d7b/d22/f44 [0,4194304] 0 2024-03-16T10:45:11.865 INFO:tasks.workunit.client.0.smithi002.stdout:5/899: mknod d0/d16/c115 0 2024-03-16T10:45:11.865 INFO:tasks.workunit.client.1.smithi026.stdout:4/217: dwrite d1/f22 [0,4194304] 0 2024-03-16T10:45:11.867 INFO:tasks.workunit.client.0.smithi002.stdout:5/900: truncate d0/d16/dab/d33/fef 532982 0 2024-03-16T10:45:11.869 INFO:tasks.workunit.client.1.smithi026.stdout:8/164: dwrite d5/d23/f2b [0,4194304] 0 2024-03-16T10:45:11.870 INFO:tasks.workunit.client.0.smithi002.stdout:3/992: creat d1/db6/d15/d87/d11e/d149/f15c x:0 0 0 2024-03-16T10:45:11.870 INFO:tasks.workunit.client.1.smithi026.stdout:8/165: chown d5/d7/f1a 90 1 2024-03-16T10:45:11.881 INFO:tasks.workunit.client.0.smithi002.stdout:3/993: creat d1/dd/d28/d7b/d22/d158/f15d x:0 0 0 2024-03-16T10:45:11.883 INFO:tasks.workunit.client.1.smithi026.stdout:4/218: dwrite d1/db/de/f2b [0,4194304] 0 2024-03-16T10:45:11.886 INFO:tasks.workunit.client.1.smithi026.stdout:1/225: truncate d0/f40 1092342 0 2024-03-16T10:45:11.887 INFO:tasks.workunit.client.1.smithi026.stdout:8/166: creat d5/f2e x:0 0 0 2024-03-16T10:45:11.888 INFO:tasks.workunit.client.1.smithi026.stdout:1/226: chown d0/d17/f37 0 1 2024-03-16T10:45:11.889 INFO:tasks.workunit.client.0.smithi002.stdout:3/994: unlink d1/db6/d15/f43 0 2024-03-16T10:45:11.890 INFO:tasks.workunit.client.0.smithi002.stdout:3/995: chown d1/db6/d15/d87/d8c/l11d 1964902 1 2024-03-16T10:45:11.890 INFO:tasks.workunit.client.0.smithi002.stdout:5/901: dread d0/d16/d4a/d39/f93 [0,4194304] 0 2024-03-16T10:45:11.891 INFO:tasks.workunit.client.1.smithi026.stdout:1/227: symlink d0/l4a 0 2024-03-16T10:45:11.891 INFO:tasks.workunit.client.0.smithi002.stdout:3/996: chown d1/dd/d28/d37/df9/c102 3 1 2024-03-16T10:45:11.892 INFO:tasks.workunit.client.1.smithi026.stdout:4/219: dwrite d1/f8 [0,4194304] 0 2024-03-16T10:45:11.895 INFO:tasks.workunit.client.1.smithi026.stdout:8/167: dwrite d5/d7/f22 [0,4194304] 0 2024-03-16T10:45:11.895 INFO:tasks.workunit.client.0.smithi002.stdout:3/997: mknod d1/db6/d15/dcc/c15e 0 2024-03-16T10:45:11.895 INFO:tasks.workunit.client.0.smithi002.stdout:3/998: readlink d1/db6/d15/d87/d8c/l144 0 2024-03-16T10:45:11.896 INFO:tasks.workunit.client.0.smithi002.stdout:3/999: chown d1/d5a/leb 82507 1 2024-03-16T10:45:11.897 INFO:tasks.workunit.client.1.smithi026.stdout:4/220: write d1/f2d [2816237,91789] 0 2024-03-16T10:45:11.898 INFO:tasks.workunit.client.1.smithi026.stdout:8/168: symlink d5/d23/l2f 0 2024-03-16T10:45:11.902 INFO:tasks.workunit.client.1.smithi026.stdout:8/169: truncate d5/f21 3936370 0 2024-03-16T10:45:11.905 INFO:tasks.workunit.client.1.smithi026.stdout:8/170: mknod d5/d7/d2c/c30 0 2024-03-16T10:45:11.908 INFO:tasks.workunit.client.0.smithi002.stdout:5/902: write d0/d16/d4a/d39/f53 [406349,113458] 0 2024-03-16T10:45:11.914 INFO:tasks.workunit.client.0.smithi002.stdout:5/903: symlink d0/d16/l116 0 2024-03-16T10:45:11.915 INFO:tasks.workunit.client.1.smithi026.stdout:1/228: symlink d0/d17/d1f/d38/l4b 0 2024-03-16T10:45:11.915 INFO:tasks.workunit.client.1.smithi026.stdout:8/171: link d5/dc/f18 d5/d28/f31 0 2024-03-16T10:45:11.919 INFO:tasks.workunit.client.0.smithi002.stdout:5/904: rename d0/d16/d5b/da0/dc4 to d0/d16/dab/d43/dcc/d117 0 2024-03-16T10:45:11.919 INFO:tasks.workunit.client.1.smithi026.stdout:4/221: dwrite d1/db/de/f3d [0,4194304] 0 2024-03-16T10:45:11.922 INFO:tasks.workunit.client.1.smithi026.stdout:1/229: mknod d0/d17/c4c 0 2024-03-16T10:45:11.925 INFO:tasks.workunit.client.0.smithi002.stdout:5/905: mkdir d0/d16/d5b/d118 0 2024-03-16T10:45:11.926 INFO:tasks.workunit.client.0.smithi002.stdout:5/906: chown d0/d16/d4a/d55/l68 352211 1 2024-03-16T10:45:11.934 INFO:tasks.workunit.client.0.smithi002.stdout:5/907: mkdir d0/d16/dab/d2b/d119 0 2024-03-16T10:45:11.937 INFO:tasks.workunit.client.1.smithi026.stdout:8/172: link f4 d5/d7/d2c/f32 0 2024-03-16T10:45:11.940 INFO:tasks.workunit.client.0.smithi002.stdout:5/908: link d0/d16/dab/d33/d8c/ffc d0/d16/dab/d43/dcc/f11a 0 2024-03-16T10:45:11.941 INFO:tasks.workunit.client.1.smithi026.stdout:8/173: mknod d5/d23/c33 0 2024-03-16T10:45:11.942 INFO:tasks.workunit.client.1.smithi026.stdout:8/174: write d5/d7/f22 [1770760,72810] 0 2024-03-16T10:45:11.943 INFO:tasks.workunit.client.1.smithi026.stdout:4/222: dwrite d1/f3 [0,4194304] 0 2024-03-16T10:45:11.944 INFO:tasks.workunit.client.1.smithi026.stdout:1/230: write d0/d33/f30 [319592,104360] 0 2024-03-16T10:45:11.945 INFO:tasks.workunit.client.0.smithi002.stdout:5/909: dwrite d0/d16/dab/d33/d8c/fa8 [0,4194304] 0 2024-03-16T10:45:11.947 INFO:tasks.workunit.client.0.smithi002.stdout:2/897: sync 2024-03-16T10:45:11.954 INFO:tasks.workunit.client.1.smithi026.stdout:8/175: truncate d5/dc/f18 3306063 0 2024-03-16T10:45:11.955 INFO:tasks.workunit.client.1.smithi026.stdout:2/195: sync 2024-03-16T10:45:11.956 INFO:tasks.workunit.client.1.smithi026.stdout:3/222: sync 2024-03-16T10:45:11.956 INFO:tasks.workunit.client.1.smithi026.stdout:9/179: sync 2024-03-16T10:45:11.956 INFO:tasks.workunit.client.1.smithi026.stdout:1/231: truncate d0/d33/d3b/f2f 3928718 0 2024-03-16T10:45:11.957 INFO:tasks.workunit.client.0.smithi002.stdout:2/898: mkdir d7/d27/d3b/d95/dd2/d11c 0 2024-03-16T10:45:11.958 INFO:tasks.workunit.client.0.smithi002.stdout:2/899: chown d7/d2e/d28/d40/lda 2471955 1 2024-03-16T10:45:11.959 INFO:tasks.workunit.client.1.smithi026.stdout:8/176: rmdir d5/dc/d1f 39 2024-03-16T10:45:11.959 INFO:tasks.workunit.client.0.smithi002.stdout:5/910: write d0/d16/dab/d33/fef [1048724,78944] 0 2024-03-16T10:45:11.963 INFO:tasks.workunit.client.1.smithi026.stdout:9/180: creat d0/da/d14/f39 x:0 0 0 2024-03-16T10:45:11.965 INFO:tasks.workunit.client.0.smithi002.stdout:2/900: write d7/db/f58 [2625190,14876] 0 2024-03-16T10:45:11.967 INFO:tasks.workunit.client.1.smithi026.stdout:4/223: rename d1/db/l37 to d1/db/l4b 0 2024-03-16T10:45:11.971 INFO:tasks.workunit.client.1.smithi026.stdout:2/196: creat d2/d29/d17/d1a/d2b/d2e/f3f x:0 0 0 2024-03-16T10:45:11.972 INFO:tasks.workunit.client.1.smithi026.stdout:3/223: write d4/d29/f30 [230727,69498] 0 2024-03-16T10:45:11.973 INFO:tasks.workunit.client.1.smithi026.stdout:8/177: unlink d5/d7/f22 0 2024-03-16T10:45:11.974 INFO:tasks.workunit.client.1.smithi026.stdout:2/197: chown d2/d29/d17/d1a/d2b/c16 1035533 1 2024-03-16T10:45:11.974 INFO:tasks.workunit.client.1.smithi026.stdout:4/224: dwrite d1/db/d29/f3e [0,4194304] 0 2024-03-16T10:45:11.976 INFO:tasks.workunit.client.0.smithi002.stdout:5/911: rename d0/d16/dab/d33/d4b/fe8 to d0/d24/dad/f11b 0 2024-03-16T10:45:11.976 INFO:tasks.workunit.client.1.smithi026.stdout:4/225: fsync d1/f27 0 2024-03-16T10:45:11.976 INFO:tasks.workunit.client.0.smithi002.stdout:5/912: chown d0/d24/dad/db6/db9/fc2 7 1 2024-03-16T10:45:11.977 INFO:tasks.workunit.client.1.smithi026.stdout:2/198: chown d2/d29/d17/d1a/f1b 15188277 1 2024-03-16T10:45:11.978 INFO:tasks.workunit.client.0.smithi002.stdout:2/901: mkdir d7/db/d11d 0 2024-03-16T10:45:11.978 INFO:tasks.workunit.client.0.smithi002.stdout:2/902: chown d7/d2e/d1d/d2f/d4b/d6e/fc6 3852707 1 2024-03-16T10:45:11.980 INFO:tasks.workunit.client.1.smithi026.stdout:1/232: fsync d0/fd 0 2024-03-16T10:45:11.985 INFO:tasks.workunit.client.0.smithi002.stdout:2/903: rename d7/d2e/d28/d40/fba to d7/d2e/d28/d44/f11e 0 2024-03-16T10:45:11.988 INFO:tasks.workunit.client.0.smithi002.stdout:2/904: mkdir d7/d2e/d1d/d107/d11f 0 2024-03-16T10:45:11.990 INFO:tasks.workunit.client.1.smithi026.stdout:3/224: rename d4/dc/l23 to d4/dc/dd/d47/d53/l59 0 2024-03-16T10:45:11.994 INFO:tasks.workunit.client.1.smithi026.stdout:4/226: unlink d1/db/de/f2a 0 2024-03-16T10:45:11.994 INFO:tasks.workunit.client.0.smithi002.stdout:2/905: read d7/db/f2a [763268,127487] 0 2024-03-16T10:45:11.995 INFO:tasks.workunit.client.1.smithi026.stdout:2/199: chown d2/d29/d17/c3d 18 1 2024-03-16T10:45:11.997 INFO:tasks.workunit.client.0.smithi002.stdout:5/913: dwrite d0/d16/dab/d2b/d94/f7a [0,4194304] 0 2024-03-16T10:45:11.997 INFO:tasks.workunit.client.0.smithi002.stdout:5/914: fdatasync d0/d24/ff2 0 2024-03-16T10:45:11.999 INFO:tasks.workunit.client.0.smithi002.stdout:5/915: write d0/d16/dab/d2b/d94/f7a [1746207,118191] 0 2024-03-16T10:45:12.001 INFO:tasks.workunit.client.0.smithi002.stdout:4/899: sync 2024-03-16T10:45:12.001 INFO:tasks.workunit.client.0.smithi002.stdout:0/968: sync 2024-03-16T10:45:12.002 INFO:tasks.workunit.client.1.smithi026.stdout:4/227: mkdir d1/db/de/d4c 0 2024-03-16T10:45:12.003 INFO:tasks.workunit.client.1.smithi026.stdout:4/228: write d1/f32 [4928425,100396] 0 2024-03-16T10:45:12.004 INFO:tasks.workunit.client.0.smithi002.stdout:5/916: fsync d0/d16/dab/d43/dcc/f11a 0 2024-03-16T10:45:12.004 INFO:tasks.workunit.client.1.smithi026.stdout:4/229: chown d1/db/d47 643 1 2024-03-16T10:45:12.007 INFO:tasks.workunit.client.1.smithi026.stdout:2/200: creat d2/d29/d17/f40 x:0 0 0 2024-03-16T10:45:12.013 INFO:tasks.workunit.client.1.smithi026.stdout:8/178: write d5/f21 [2604191,33980] 0 2024-03-16T10:45:12.013 INFO:tasks.workunit.client.1.smithi026.stdout:3/225: mkdir d4/d46/d4f/d5a 0 2024-03-16T10:45:12.015 INFO:tasks.workunit.client.1.smithi026.stdout:3/226: dread d4/dc/f19 [0,4194304] 0 2024-03-16T10:45:12.017 INFO:tasks.workunit.client.0.smithi002.stdout:4/900: symlink d2/d6/d32/d66/d7f/de7/d2e/db0/dfe/l112 0 2024-03-16T10:45:12.017 INFO:tasks.workunit.client.0.smithi002.stdout:0/969: write d3/d5/f1c [815426,27697] 0 2024-03-16T10:45:12.019 INFO:tasks.workunit.client.1.smithi026.stdout:8/179: creat d5/d7/f34 x:0 0 0 2024-03-16T10:45:12.019 INFO:tasks.workunit.client.1.smithi026.stdout:8/180: stat d5/d7/d2c 0 2024-03-16T10:45:12.020 INFO:tasks.workunit.client.0.smithi002.stdout:5/917: creat d0/d16/dab/d33/f11c x:0 0 0 2024-03-16T10:45:12.021 INFO:tasks.workunit.client.0.smithi002.stdout:5/918: write d0/d16/dab/d33/d8c/f95 [872443,130084] 0 2024-03-16T10:45:12.021 INFO:tasks.workunit.client.1.smithi026.stdout:4/230: creat d1/db/de/d4c/f4d x:0 0 0 2024-03-16T10:45:12.023 INFO:tasks.workunit.client.1.smithi026.stdout:1/233: dwrite d0/f40 [0,4194304] 0 2024-03-16T10:45:12.025 INFO:tasks.workunit.client.1.smithi026.stdout:2/201: mkdir d2/d29/d3b/d41 0 2024-03-16T10:45:12.025 INFO:tasks.workunit.client.0.smithi002.stdout:0/970: mkdir d3/d12f/d138 0 2024-03-16T10:45:12.034 INFO:tasks.workunit.client.1.smithi026.stdout:3/227: rename d4/f1b to d4/dc/dd/d4b/f5b 0 2024-03-16T10:45:12.035 INFO:tasks.workunit.client.1.smithi026.stdout:3/228: write d4/dc/dd/f24 [1047996,17351] 0 2024-03-16T10:45:12.036 INFO:tasks.workunit.client.1.smithi026.stdout:3/229: stat d4/d29/f3a 0 2024-03-16T10:45:12.039 INFO:tasks.workunit.client.0.smithi002.stdout:4/901: dwrite d2/d6/d32/d66/df7/fff [0,4194304] 0 2024-03-16T10:45:12.041 INFO:tasks.workunit.client.0.smithi002.stdout:0/971: write d3/d5/f23 [4652343,72987] 0 2024-03-16T10:45:12.042 INFO:tasks.workunit.client.1.smithi026.stdout:1/234: truncate d0/fd 1151266 0 2024-03-16T10:45:12.044 INFO:tasks.workunit.client.1.smithi026.stdout:3/230: creat d4/dc/dd/d47/d53/f5c x:0 0 0 2024-03-16T10:45:12.045 INFO:tasks.workunit.client.0.smithi002.stdout:5/919: dwrite d0/d16/f4d [0,4194304] 0 2024-03-16T10:45:12.045 INFO:tasks.workunit.client.1.smithi026.stdout:8/181: truncate d5/dc/f1e 33071 0 2024-03-16T10:45:12.046 INFO:tasks.workunit.client.1.smithi026.stdout:3/231: write d4/dc/dd/d47/d53/f5c [451640,43090] 0 2024-03-16T10:45:12.048 INFO:tasks.workunit.client.0.smithi002.stdout:5/920: stat d0/d16/dab/d33/cfa 0 2024-03-16T10:45:12.049 INFO:tasks.workunit.client.1.smithi026.stdout:1/235: mkdir d0/d17/d4d 0 2024-03-16T10:45:12.051 INFO:tasks.workunit.client.1.smithi026.stdout:1/236: fdatasync d0/d17/d1f/f3f 0 2024-03-16T10:45:12.052 INFO:tasks.workunit.client.1.smithi026.stdout:8/182: dwrite d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:12.054 INFO:tasks.workunit.client.0.smithi002.stdout:5/921: rmdir d0/d16/dab/d33 39 2024-03-16T10:45:12.059 INFO:tasks.workunit.client.0.smithi002.stdout:0/972: getdents d3/d12f 0 2024-03-16T10:45:12.063 INFO:tasks.workunit.client.1.smithi026.stdout:8/183: unlink d5/f21 0 2024-03-16T10:45:12.065 INFO:tasks.workunit.client.1.smithi026.stdout:2/202: link d2/d29/fe d2/d29/d17/d1a/d2b/d2e/f42 0 2024-03-16T10:45:12.068 INFO:tasks.workunit.client.1.smithi026.stdout:1/237: creat d0/d17/d1f/d38/f4e x:0 0 0 2024-03-16T10:45:12.071 INFO:tasks.workunit.client.1.smithi026.stdout:7/212: sync 2024-03-16T10:45:12.071 INFO:tasks.workunit.client.1.smithi026.stdout:5/158: sync 2024-03-16T10:45:12.072 INFO:tasks.workunit.client.1.smithi026.stdout:0/227: sync 2024-03-16T10:45:12.072 INFO:tasks.workunit.client.1.smithi026.stdout:6/252: sync 2024-03-16T10:45:12.073 INFO:tasks.workunit.client.1.smithi026.stdout:3/232: dread d4/dc/f25 [0,4194304] 0 2024-03-16T10:45:12.073 INFO:tasks.workunit.client.1.smithi026.stdout:7/213: chown d6/d2f/f34 0 1 2024-03-16T10:45:12.074 INFO:tasks.workunit.client.1.smithi026.stdout:4/231: rename d1/db/l4b to d1/l4e 0 2024-03-16T10:45:12.074 INFO:tasks.workunit.client.0.smithi002.stdout:5/922: write d0/d16/dab/daa/faf [665161,127743] 0 2024-03-16T10:45:12.075 INFO:tasks.workunit.client.1.smithi026.stdout:3/233: chown d4/d29/l2a 0 1 2024-03-16T10:45:12.077 INFO:tasks.workunit.client.1.smithi026.stdout:1/238: symlink d0/d17/l4f 0 2024-03-16T10:45:12.080 INFO:tasks.workunit.client.1.smithi026.stdout:6/253: mknod d0/d28/d2d/c5e 0 2024-03-16T10:45:12.082 INFO:tasks.workunit.client.1.smithi026.stdout:5/159: fdatasync d1/db/d1f/f26 0 2024-03-16T10:45:12.082 INFO:tasks.workunit.client.1.smithi026.stdout:7/214: mkdir d6/d35/d3a/d3c/d3f 0 2024-03-16T10:45:12.083 INFO:tasks.workunit.client.0.smithi002.stdout:0/973: dread d3/de/f51 [0,4194304] 0 2024-03-16T10:45:12.084 INFO:tasks.workunit.client.1.smithi026.stdout:4/232: rename d1/db/d47 to d1/db/d29/d44/d48/d4f 0 2024-03-16T10:45:12.084 INFO:tasks.workunit.client.1.smithi026.stdout:7/215: chown d6/de 7422 1 2024-03-16T10:45:12.088 INFO:tasks.workunit.client.0.smithi002.stdout:0/974: dwrite d3/d31/daa/ddb/f120 [0,4194304] 0 2024-03-16T10:45:12.089 INFO:tasks.workunit.client.1.smithi026.stdout:9/181: sync 2024-03-16T10:45:12.090 INFO:tasks.workunit.client.0.smithi002.stdout:0/975: stat d3/d28/l38 0 2024-03-16T10:45:12.090 INFO:tasks.workunit.client.1.smithi026.stdout:7/216: readlink d6/de/d16/l3e 0 2024-03-16T10:45:12.090 INFO:tasks.workunit.client.0.smithi002.stdout:0/976: stat d3/de/f7b 0 2024-03-16T10:45:12.092 INFO:tasks.workunit.client.0.smithi002.stdout:5/923: dwrite d0/d1f/f36 [0,4194304] 0 2024-03-16T10:45:12.092 INFO:tasks.workunit.client.1.smithi026.stdout:5/160: creat d1/d1c/d2a/f32 x:0 0 0 2024-03-16T10:45:12.096 INFO:tasks.workunit.client.0.smithi002.stdout:6/994: sync 2024-03-16T10:45:12.097 INFO:tasks.workunit.client.0.smithi002.stdout:7/862: sync 2024-03-16T10:45:12.097 INFO:tasks.workunit.client.1.smithi026.stdout:3/234: dwrite d4/f18 [0,4194304] 0 2024-03-16T10:45:12.100 INFO:tasks.workunit.client.1.smithi026.stdout:2/203: getdents d2/d29/d17/d1a/d2b 0 2024-03-16T10:45:12.101 INFO:tasks.workunit.client.1.smithi026.stdout:3/235: fsync d4/dc/dd/f24 0 2024-03-16T10:45:12.101 INFO:tasks.workunit.client.0.smithi002.stdout:0/977: mkdir d3/d31/daa/dff/d139 0 2024-03-16T10:45:12.103 INFO:tasks.workunit.client.1.smithi026.stdout:8/184: dwrite d5/d7/d2c/f2d [8388608,4194304] 0 2024-03-16T10:45:12.106 INFO:tasks.workunit.client.1.smithi026.stdout:0/228: link d1/ca d1/d18/c47 0 2024-03-16T10:45:12.107 INFO:tasks.workunit.client.0.smithi002.stdout:6/995: symlink d15/d2d/d2b/d3f/d77/d112/l15a 0 2024-03-16T10:45:12.107 INFO:tasks.workunit.client.1.smithi026.stdout:1/239: dwrite d0/d17/d1f/f35 [0,4194304] 0 2024-03-16T10:45:12.108 INFO:tasks.workunit.client.0.smithi002.stdout:6/996: stat d15/d2d/d2b/d75/d153 0 2024-03-16T10:45:12.108 INFO:tasks.workunit.client.0.smithi002.stdout:6/997: chown d15/d80/dbe/fd0 346638 1 2024-03-16T10:45:12.109 INFO:tasks.workunit.client.1.smithi026.stdout:6/254: link d0/f35 d0/f5f 0 2024-03-16T10:45:12.112 INFO:tasks.workunit.client.1.smithi026.stdout:7/217: rmdir d6/d35/d3a 39 2024-03-16T10:45:12.113 INFO:tasks.workunit.client.0.smithi002.stdout:7/863: rename d3/d61/d66/l112 to d3/d4/d1d/d87/d5c/ddb/def/l13a 0 2024-03-16T10:45:12.114 INFO:tasks.workunit.client.1.smithi026.stdout:4/233: dwrite d1/db/f26 [0,4194304] 0 2024-03-16T10:45:12.117 INFO:tasks.workunit.client.0.smithi002.stdout:7/864: read d3/d4/d1d/d87/d96/d8c/f95 [384062,45070] 0 2024-03-16T10:45:12.119 INFO:tasks.workunit.client.0.smithi002.stdout:6/998: creat d15/d2d/d73/d115/d127/f15b x:0 0 0 2024-03-16T10:45:12.122 INFO:tasks.workunit.client.0.smithi002.stdout:0/978: fsync d3/f49 0 2024-03-16T10:45:12.122 INFO:tasks.workunit.client.1.smithi026.stdout:4/234: dwrite d1/f3 [4194304,4194304] 0 2024-03-16T10:45:12.125 INFO:tasks.workunit.client.0.smithi002.stdout:2/906: sync 2024-03-16T10:45:12.125 INFO:tasks.workunit.client.1.smithi026.stdout:6/255: dread d0/f2c [0,4194304] 0 2024-03-16T10:45:12.127 INFO:tasks.workunit.client.1.smithi026.stdout:5/161: creat d1/db/d1f/f33 x:0 0 0 2024-03-16T10:45:12.128 INFO:tasks.workunit.client.1.smithi026.stdout:8/185: chown d5/dc/d1f/c24 174 1 2024-03-16T10:45:12.128 INFO:tasks.workunit.client.1.smithi026.stdout:3/236: symlink d4/d46/d4f/l5d 0 2024-03-16T10:45:12.130 INFO:tasks.workunit.client.0.smithi002.stdout:6/999: rename d15/f1c to d15/d80/df5/f15c 0 2024-03-16T10:45:12.131 INFO:tasks.workunit.client.0.smithi002.stdout:5/924: dread d0/d16/dab/daa/f9c [0,4194304] 0 2024-03-16T10:45:12.133 INFO:tasks.workunit.client.1.smithi026.stdout:2/204: mknod d2/d29/d17/c43 0 2024-03-16T10:45:12.134 INFO:tasks.workunit.client.1.smithi026.stdout:0/229: read - d1/d18/d1a/f46 zero size 2024-03-16T10:45:12.135 INFO:tasks.workunit.client.1.smithi026.stdout:0/230: chown d1/d18 5078 1 2024-03-16T10:45:12.136 INFO:tasks.workunit.client.1.smithi026.stdout:1/240: mknod d0/d17/d1f/d20/c50 0 2024-03-16T10:45:12.137 INFO:tasks.workunit.client.1.smithi026.stdout:6/256: dwrite d0/d28/f40 [0,4194304] 0 2024-03-16T10:45:12.138 INFO:tasks.workunit.client.0.smithi002.stdout:2/907: creat d7/d2e/d28/d40/d50/db4/dcd/f120 x:0 0 0 2024-03-16T10:45:12.138 INFO:tasks.workunit.client.1.smithi026.stdout:6/257: chown d0/d28/d2d 9075050 1 2024-03-16T10:45:12.139 INFO:tasks.workunit.client.1.smithi026.stdout:7/218: rmdir d6/de/d16 39 2024-03-16T10:45:12.141 INFO:tasks.workunit.client.1.smithi026.stdout:7/219: fdatasync d6/fc 0 2024-03-16T10:45:12.142 INFO:tasks.workunit.client.1.smithi026.stdout:1/241: dread d0/d33/d3b/f27 [0,4194304] 0 2024-03-16T10:45:12.145 INFO:tasks.workunit.client.0.smithi002.stdout:7/865: creat d3/f13b x:0 0 0 2024-03-16T10:45:12.151 INFO:tasks.workunit.client.0.smithi002.stdout:7/866: mkdir d3/d4/d1d/d87/d5c/d63/d3f/dba/d13c 0 2024-03-16T10:45:12.151 INFO:tasks.workunit.client.0.smithi002.stdout:0/979: dread d3/de/f13 [4194304,4194304] 0 2024-03-16T10:45:12.153 INFO:tasks.workunit.client.1.smithi026.stdout:4/235: unlink d1/db/d29/f2c 0 2024-03-16T10:45:12.156 INFO:tasks.workunit.client.1.smithi026.stdout:8/186: read d5/d7/f1c [320079,35546] 0 2024-03-16T10:45:12.157 INFO:tasks.workunit.client.0.smithi002.stdout:5/925: dread d0/f57 [0,4194304] 0 2024-03-16T10:45:12.157 INFO:tasks.workunit.client.0.smithi002.stdout:7/867: mknod d3/d4/d1d/d87/d96/d5e/c13d 0 2024-03-16T10:45:12.158 INFO:tasks.workunit.client.0.smithi002.stdout:7/868: fsync d3/d4/d1d/d87/d5c/d63/d34/f98 0 2024-03-16T10:45:12.158 INFO:tasks.workunit.client.1.smithi026.stdout:3/237: chown d4/lb 44476 1 2024-03-16T10:45:12.159 INFO:tasks.workunit.client.0.smithi002.stdout:0/980: creat d3/d12f/d138/f13a x:0 0 0 2024-03-16T10:45:12.161 INFO:tasks.workunit.client.0.smithi002.stdout:0/981: chown d3/d28/d2a/dc7/cf3 57402 1 2024-03-16T10:45:12.163 INFO:tasks.workunit.client.0.smithi002.stdout:5/926: dwrite d0/d16/dab/d2b/d5d/f92 [0,4194304] 0 2024-03-16T10:45:12.166 INFO:tasks.workunit.client.1.smithi026.stdout:0/231: mkdir d1/d12/d2f/d48 0 2024-03-16T10:45:12.166 INFO:tasks.workunit.client.1.smithi026.stdout:9/182: truncate d0/da/f12 636991 0 2024-03-16T10:45:12.169 INFO:tasks.workunit.client.0.smithi002.stdout:0/982: mkdir d3/d31/d88/dc8/d13b 0 2024-03-16T10:45:12.169 INFO:tasks.workunit.client.0.smithi002.stdout:2/908: dread d7/d2e/d28/d44/fb1 [0,4194304] 0 2024-03-16T10:45:12.173 INFO:tasks.workunit.client.1.smithi026.stdout:6/258: creat d0/d28/d2d/f60 x:0 0 0 2024-03-16T10:45:12.174 INFO:tasks.workunit.client.1.smithi026.stdout:6/259: chown d0/d5b/f30 293 1 2024-03-16T10:45:12.175 INFO:tasks.workunit.client.0.smithi002.stdout:7/869: rename d3/d5f/f135 to d3/d5f/f13e 0 2024-03-16T10:45:12.175 INFO:tasks.workunit.client.0.smithi002.stdout:7/870: chown d3/d4/d1d/d87/d5c/ddb 1578469332 1 2024-03-16T10:45:12.176 INFO:tasks.workunit.client.0.smithi002.stdout:7/871: dread - d3/d12b/f102 zero size 2024-03-16T10:45:12.177 INFO:tasks.workunit.client.0.smithi002.stdout:0/983: read d3/f2d [90812,94380] 0 2024-03-16T10:45:12.178 INFO:tasks.workunit.client.0.smithi002.stdout:2/909: symlink d7/d2e/d1d/d38/d49/d8c/l121 0 2024-03-16T10:45:12.181 INFO:tasks.workunit.client.0.smithi002.stdout:5/927: link d0/d16/d4a/d39/ff7 d0/d24/dad/db6/db9/f11d 0 2024-03-16T10:45:12.183 INFO:tasks.workunit.client.1.smithi026.stdout:5/162: mknod d1/c34 0 2024-03-16T10:45:12.183 INFO:tasks.workunit.client.1.smithi026.stdout:5/163: stat d1/db/d27/f1b 0 2024-03-16T10:45:12.184 INFO:tasks.workunit.client.1.smithi026.stdout:4/236: mknod d1/db/de/c50 0 2024-03-16T10:45:12.189 INFO:tasks.workunit.client.0.smithi002.stdout:0/984: creat d3/d28/d65/d108/f13c x:0 0 0 2024-03-16T10:45:12.189 INFO:tasks.workunit.client.0.smithi002.stdout:0/985: dread - d3/d28/d65/d108/f133 zero size 2024-03-16T10:45:12.190 INFO:tasks.workunit.client.1.smithi026.stdout:2/205: truncate d2/d29/fe 50957 0 2024-03-16T10:45:12.190 INFO:tasks.workunit.client.0.smithi002.stdout:2/910: readlink d7/db/l91 0 2024-03-16T10:45:12.191 INFO:tasks.workunit.client.0.smithi002.stdout:2/911: write d7/d2e/d28/d40/fa3 [2136092,123166] 0 2024-03-16T10:45:12.193 INFO:tasks.workunit.client.1.smithi026.stdout:3/238: dwrite d4/dc/dd/d3d/f4a [0,4194304] 0 2024-03-16T10:45:12.195 INFO:tasks.workunit.client.0.smithi002.stdout:7/872: creat d3/d4/d1d/d87/d5c/d63/d3f/dba/d13c/f13f x:0 0 0 2024-03-16T10:45:12.196 INFO:tasks.workunit.client.1.smithi026.stdout:6/260: rename d0/d28/d2d/f39 to d0/d28/d2d/f61 0 2024-03-16T10:45:12.197 INFO:tasks.workunit.client.0.smithi002.stdout:4/902: sync 2024-03-16T10:45:12.198 INFO:tasks.workunit.client.1.smithi026.stdout:7/220: creat d6/d35/d3a/d3c/f40 x:0 0 0 2024-03-16T10:45:12.202 INFO:tasks.workunit.client.0.smithi002.stdout:0/986: truncate d3/d28/d65/f112 613151 0 2024-03-16T10:45:12.203 INFO:tasks.workunit.client.0.smithi002.stdout:2/912: dwrite d7/d2e/d1d/d38/f3a [0,4194304] 0 2024-03-16T10:45:12.203 INFO:tasks.workunit.client.1.smithi026.stdout:4/237: symlink d1/db/de/l51 0 2024-03-16T10:45:12.204 INFO:tasks.workunit.client.1.smithi026.stdout:4/238: readlink d1/l35 0 2024-03-16T10:45:12.204 INFO:tasks.workunit.client.0.smithi002.stdout:5/928: dwrite d0/d16/dab/d33/d8c/fea [0,4194304] 0 2024-03-16T10:45:12.208 INFO:tasks.workunit.client.1.smithi026.stdout:8/187: mknod d5/c35 0 2024-03-16T10:45:12.208 INFO:tasks.workunit.client.1.smithi026.stdout:8/188: chown d5/d7/f11 44 1 2024-03-16T10:45:12.209 INFO:tasks.workunit.client.0.smithi002.stdout:4/903: dwrite d2/d6/ddb/fe2 [0,4194304] 0 2024-03-16T10:45:12.210 INFO:tasks.workunit.client.0.smithi002.stdout:7/873: symlink d3/d4/d1d/d87/d96/d5e/l140 0 2024-03-16T10:45:12.213 INFO:tasks.workunit.client.0.smithi002.stdout:7/874: truncate d3/d12b/f114 133284 0 2024-03-16T10:45:12.221 INFO:tasks.workunit.client.0.smithi002.stdout:7/875: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f41 [4194304,4194304] 0 2024-03-16T10:45:12.222 INFO:tasks.workunit.client.0.smithi002.stdout:7/876: chown d3/d37/d6a/d127 3176 1 2024-03-16T10:45:12.223 INFO:tasks.workunit.client.1.smithi026.stdout:2/206: write d2/d29/d17/d1a/f1b [4554559,33100] 0 2024-03-16T10:45:12.225 INFO:tasks.workunit.client.1.smithi026.stdout:9/183: unlink d0/da/d13/c2c 0 2024-03-16T10:45:12.226 INFO:tasks.workunit.client.1.smithi026.stdout:9/184: fsync d0/da/d13/f38 0 2024-03-16T10:45:12.226 INFO:tasks.workunit.client.0.smithi002.stdout:4/904: fdatasync d2/d6/f13 0 2024-03-16T10:45:12.228 INFO:tasks.workunit.client.1.smithi026.stdout:3/239: mknod d4/dc/dd/d3d/c5e 0 2024-03-16T10:45:12.232 INFO:tasks.workunit.client.0.smithi002.stdout:0/987: dwrite d3/d31/d75/db4/f11e [0,4194304] 0 2024-03-16T10:45:12.234 INFO:tasks.workunit.client.1.smithi026.stdout:7/221: rename d6/d1e/c23 to d6/d35/d3a/c41 0 2024-03-16T10:45:12.235 INFO:tasks.workunit.client.1.smithi026.stdout:4/239: dread d1/db/de/f1d [0,4194304] 0 2024-03-16T10:45:12.235 INFO:tasks.workunit.client.1.smithi026.stdout:1/242: dwrite d0/d17/d1f/d20/f43 [0,4194304] 0 2024-03-16T10:45:12.237 INFO:tasks.workunit.client.1.smithi026.stdout:7/222: chown d6/f24 24186 1 2024-03-16T10:45:12.237 INFO:tasks.workunit.client.0.smithi002.stdout:2/913: unlink d7/db/d70/d72/lcb 0 2024-03-16T10:45:12.237 INFO:tasks.workunit.client.1.smithi026.stdout:7/223: chown d6/d1e/c37 21300717 1 2024-03-16T10:45:12.241 INFO:tasks.workunit.client.1.smithi026.stdout:5/164: mknod d1/db/c35 0 2024-03-16T10:45:12.243 INFO:tasks.workunit.client.1.smithi026.stdout:7/224: write d6/d35/d3a/d3c/f40 [577864,112464] 0 2024-03-16T10:45:12.247 INFO:tasks.workunit.client.1.smithi026.stdout:4/240: dwrite d1/db/de/f2b [0,4194304] 0 2024-03-16T10:45:12.248 INFO:tasks.workunit.client.1.smithi026.stdout:4/241: chown d1/db/d29/f43 0 1 2024-03-16T10:45:12.248 INFO:tasks.workunit.client.1.smithi026.stdout:8/189: mkdir d5/d7/d2c/d36 0 2024-03-16T10:45:12.249 INFO:tasks.workunit.client.1.smithi026.stdout:4/242: stat d1/f18 0 2024-03-16T10:45:12.250 INFO:tasks.workunit.client.1.smithi026.stdout:4/243: write d1/f6 [1121916,99691] 0 2024-03-16T10:45:12.252 INFO:tasks.workunit.client.1.smithi026.stdout:0/232: getdents d1/d18/d1a 0 2024-03-16T10:45:12.254 INFO:tasks.workunit.client.1.smithi026.stdout:8/190: dread d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:12.255 INFO:tasks.workunit.client.1.smithi026.stdout:2/207: creat d2/d29/d3b/f44 x:0 0 0 2024-03-16T10:45:12.257 INFO:tasks.workunit.client.1.smithi026.stdout:3/240: rmdir d4/d46/d4f 39 2024-03-16T10:45:12.264 INFO:tasks.workunit.client.1.smithi026.stdout:9/185: rename d0/c5 to d0/da/d16/d17/d19/c3a 0 2024-03-16T10:45:12.265 INFO:tasks.workunit.client.1.smithi026.stdout:3/241: dwrite d4/dc/dd/f4c [0,4194304] 0 2024-03-16T10:45:12.266 INFO:tasks.workunit.client.1.smithi026.stdout:3/242: chown d4/dc/dd/d3d/l45 15746037 1 2024-03-16T10:45:12.266 INFO:tasks.workunit.client.1.smithi026.stdout:1/243: unlink d0/d17/d1f/cb 0 2024-03-16T10:45:12.267 INFO:tasks.workunit.client.1.smithi026.stdout:1/244: stat d0/d33/l29 0 2024-03-16T10:45:12.270 INFO:tasks.workunit.client.0.smithi002.stdout:5/929: truncate d0/d24/dad/db6/db9/ffd 2561999 0 2024-03-16T10:45:12.273 INFO:tasks.workunit.client.1.smithi026.stdout:7/225: symlink d6/d35/d3a/d3c/l42 0 2024-03-16T10:45:12.274 INFO:tasks.workunit.client.0.smithi002.stdout:2/914: symlink d7/d27/d3b/d95/dd2/d11c/l122 0 2024-03-16T10:45:12.283 INFO:tasks.workunit.client.1.smithi026.stdout:4/244: dwrite d1/f32 [0,4194304] 0 2024-03-16T10:45:12.284 INFO:tasks.workunit.client.1.smithi026.stdout:0/233: rename d1/d18/d1a/l35 to d1/d18/d1a/l49 0 2024-03-16T10:45:12.285 INFO:tasks.workunit.client.0.smithi002.stdout:0/988: creat d3/de/f13d x:0 0 0 2024-03-16T10:45:12.285 INFO:tasks.workunit.client.1.smithi026.stdout:4/245: chown d1/f2d 3542 1 2024-03-16T10:45:12.286 INFO:tasks.workunit.client.1.smithi026.stdout:5/165: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:12.293 INFO:tasks.workunit.client.1.smithi026.stdout:2/208: unlink d2/d29/d17/d1a/f1b 0 2024-03-16T10:45:12.295 INFO:tasks.workunit.client.0.smithi002.stdout:7/877: getdents d3/d4/d1d/d87/d96 0 2024-03-16T10:45:12.297 INFO:tasks.workunit.client.0.smithi002.stdout:4/905: link d2/d6/d32/d66/d7f/de7/d88/f9b d2/d6/d32/d68/f113 0 2024-03-16T10:45:12.301 INFO:tasks.workunit.client.0.smithi002.stdout:0/989: symlink d3/d31/daa/dff/l13e 0 2024-03-16T10:45:12.302 INFO:tasks.workunit.client.1.smithi026.stdout:1/245: mknod d0/d33/d3b/c51 0 2024-03-16T10:45:12.303 INFO:tasks.workunit.client.1.smithi026.stdout:1/246: dread - d0/d17/d1f/f3f zero size 2024-03-16T10:45:12.304 INFO:tasks.workunit.client.1.smithi026.stdout:1/247: write d0/d17/d1f/d38/f4e [605875,57780] 0 2024-03-16T10:45:12.305 INFO:tasks.workunit.client.0.smithi002.stdout:5/930: dwrite d0/d16/dab/d33/d8c/ffc [0,4194304] 0 2024-03-16T10:45:12.305 INFO:tasks.workunit.client.0.smithi002.stdout:2/915: creat d7/d27/d3b/dd4/de1/f123 x:0 0 0 2024-03-16T10:45:12.305 INFO:tasks.workunit.client.1.smithi026.stdout:1/248: dread - d0/d33/f3e zero size 2024-03-16T10:45:12.306 INFO:tasks.workunit.client.1.smithi026.stdout:1/249: write d0/d17/d1f/fc [34170,20798] 0 2024-03-16T10:45:12.307 INFO:tasks.workunit.client.0.smithi002.stdout:7/878: symlink d3/d4/d1d/d87/d5c/dd0/dd2/l141 0 2024-03-16T10:45:12.308 INFO:tasks.workunit.client.0.smithi002.stdout:7/879: stat d3/d4/d1d/d87/d5c/dd0/f133 0 2024-03-16T10:45:12.311 INFO:tasks.workunit.client.1.smithi026.stdout:7/226: chown d6/de/d16/f19 0 1 2024-03-16T10:45:12.313 INFO:tasks.workunit.client.1.smithi026.stdout:3/243: write d4/dc/f25 [1724589,89883] 0 2024-03-16T10:45:12.313 INFO:tasks.workunit.client.1.smithi026.stdout:7/227: chown d6/d1e/f32 864 1 2024-03-16T10:45:12.316 INFO:tasks.workunit.client.1.smithi026.stdout:8/191: dwrite f4 [0,4194304] 0 2024-03-16T10:45:12.318 INFO:tasks.workunit.client.1.smithi026.stdout:7/228: write d6/de/d16/f2e [1442163,6651] 0 2024-03-16T10:45:12.319 INFO:tasks.workunit.client.1.smithi026.stdout:1/250: dwrite d0/fd [0,4194304] 0 2024-03-16T10:45:12.319 INFO:tasks.workunit.client.1.smithi026.stdout:3/244: chown d4/dc/dd/d47/d53/f5c 753055 1 2024-03-16T10:45:12.323 INFO:tasks.workunit.client.1.smithi026.stdout:5/166: symlink d1/d1c/l36 0 2024-03-16T10:45:12.324 INFO:tasks.workunit.client.0.smithi002.stdout:5/931: rename d0/d16/dab/d33/d8c/fbc to d0/d16/dab/daa/f11e 0 2024-03-16T10:45:12.329 INFO:tasks.workunit.client.1.smithi026.stdout:7/229: truncate d6/d1e/f20 1076418 0 2024-03-16T10:45:12.329 INFO:tasks.workunit.client.1.smithi026.stdout:7/230: stat d6/c9 0 2024-03-16T10:45:12.330 INFO:tasks.workunit.client.1.smithi026.stdout:2/209: rmdir d2 39 2024-03-16T10:45:12.331 INFO:tasks.workunit.client.0.smithi002.stdout:0/990: mknod d3/d31/d75/db4/d109/d9c/d12e/d119/c13f 0 2024-03-16T10:45:12.332 INFO:tasks.workunit.client.0.smithi002.stdout:5/932: mkdir d0/d24/dad/db6/d11f 0 2024-03-16T10:45:12.334 INFO:tasks.workunit.client.0.smithi002.stdout:5/933: write d0/d1f/f2c [4908716,45046] 0 2024-03-16T10:45:12.335 INFO:tasks.workunit.client.1.smithi026.stdout:1/251: dwrite d0/fd [4194304,4194304] 0 2024-03-16T10:45:12.336 INFO:tasks.workunit.client.1.smithi026.stdout:1/252: dread - d0/d17/d1f/f3f zero size 2024-03-16T10:45:12.339 INFO:tasks.workunit.client.1.smithi026.stdout:8/192: dwrite f4 [0,4194304] 0 2024-03-16T10:45:12.342 INFO:tasks.workunit.client.0.smithi002.stdout:5/934: truncate d0/d24/dad/db6/db9/f11d 4308795 0 2024-03-16T10:45:12.343 INFO:tasks.workunit.client.0.smithi002.stdout:0/991: dwrite d3/d59/dca/f52 [0,4194304] 0 2024-03-16T10:45:12.343 INFO:tasks.workunit.client.0.smithi002.stdout:5/935: fsync d0/d16/d4a/f107 0 2024-03-16T10:45:12.347 INFO:tasks.workunit.client.1.smithi026.stdout:7/231: creat d6/de/d16/f43 x:0 0 0 2024-03-16T10:45:12.349 INFO:tasks.workunit.client.1.smithi026.stdout:9/186: creat d0/da/f3b x:0 0 0 2024-03-16T10:45:12.349 INFO:tasks.workunit.client.1.smithi026.stdout:7/232: chown d6/de/d16 24140725 1 2024-03-16T10:45:12.349 INFO:tasks.workunit.client.1.smithi026.stdout:9/187: chown d0/da/d16 154819922 1 2024-03-16T10:45:12.351 INFO:tasks.workunit.client.0.smithi002.stdout:5/936: read d0/d16/dab/d2b/d5d/fa7 [695751,120827] 0 2024-03-16T10:45:12.352 INFO:tasks.workunit.client.0.smithi002.stdout:2/916: write d7/d27/d3b/d95/da5/f102 [469054,54206] 0 2024-03-16T10:45:12.353 INFO:tasks.workunit.client.0.smithi002.stdout:2/917: chown d7/d2e/d1d/d38/da8/lc3 3903 1 2024-03-16T10:45:12.354 INFO:tasks.workunit.client.0.smithi002.stdout:2/918: write d7/d2e/d1d/d38/f3a [3486748,9849] 0 2024-03-16T10:45:12.354 INFO:tasks.workunit.client.1.smithi026.stdout:0/234: dread d1/d18/d1a/d24/d6/f28 [4194304,4194304] 0 2024-03-16T10:45:12.356 INFO:tasks.workunit.client.1.smithi026.stdout:1/253: rmdir d0/d17 39 2024-03-16T10:45:12.357 INFO:tasks.workunit.client.0.smithi002.stdout:7/880: dread d3/d4/d1d/d87/d96/f56 [0,4194304] 0 2024-03-16T10:45:12.357 INFO:tasks.workunit.client.0.smithi002.stdout:4/906: write d2/d6/d32/f106 [4518907,88383] 0 2024-03-16T10:45:12.359 INFO:tasks.workunit.client.0.smithi002.stdout:4/907: write d2/d6/d32/d66/d7f/fb5 [2224521,7112] 0 2024-03-16T10:45:12.361 INFO:tasks.workunit.client.1.smithi026.stdout:8/193: mkdir d5/dc/d1f/d37 0 2024-03-16T10:45:12.363 INFO:tasks.workunit.client.1.smithi026.stdout:4/246: creat d1/f52 x:0 0 0 2024-03-16T10:45:12.363 INFO:tasks.workunit.client.1.smithi026.stdout:4/247: fdatasync d1/f2d 0 2024-03-16T10:45:12.368 INFO:tasks.workunit.client.1.smithi026.stdout:2/210: dread d2/d29/d17/d1a/d2b/d2e/f3c [0,4194304] 0 2024-03-16T10:45:12.374 INFO:tasks.workunit.client.1.smithi026.stdout:2/211: dwrite d2/d29/d17/f2d [0,4194304] 0 2024-03-16T10:45:12.375 INFO:tasks.workunit.client.1.smithi026.stdout:0/235: fdatasync d1/d18/d1a/d24/d6/f28 0 2024-03-16T10:45:12.377 INFO:tasks.workunit.client.1.smithi026.stdout:2/212: truncate d2/d29/d17/f23 633295 0 2024-03-16T10:45:12.378 INFO:tasks.workunit.client.1.smithi026.stdout:2/213: write d2/d29/d17/f23 [336850,44901] 0 2024-03-16T10:45:12.386 INFO:tasks.workunit.client.1.smithi026.stdout:7/233: write d6/de/d16/f1a [2304841,99695] 0 2024-03-16T10:45:12.386 INFO:tasks.workunit.client.1.smithi026.stdout:2/214: dread d2/d29/d17/f23 [0,4194304] 0 2024-03-16T10:45:12.387 INFO:tasks.workunit.client.1.smithi026.stdout:4/248: rename d1/db/de/d3b to d1/db/de/d4c/d53 0 2024-03-16T10:45:12.388 INFO:tasks.workunit.client.1.smithi026.stdout:7/234: read d6/d1e/f20 [516802,20419] 0 2024-03-16T10:45:12.389 INFO:tasks.workunit.client.1.smithi026.stdout:2/215: dread - d2/d29/d17/d1a/d2b/d2e/f3f zero size 2024-03-16T10:45:12.390 INFO:tasks.workunit.client.1.smithi026.stdout:3/245: truncate d4/dc/dd/d47/d53/f5c 390469 0 2024-03-16T10:45:12.393 INFO:tasks.workunit.client.1.smithi026.stdout:2/216: dread d2/d29/d17/d1a/d2b/d2e/f3c [4194304,4194304] 0 2024-03-16T10:45:12.397 INFO:tasks.workunit.client.1.smithi026.stdout:0/236: dread d1/f15 [0,4194304] 0 2024-03-16T10:45:12.398 INFO:tasks.workunit.client.1.smithi026.stdout:7/235: dwrite d6/d35/d3a/d3c/f40 [0,4194304] 0 2024-03-16T10:45:12.399 INFO:tasks.workunit.client.1.smithi026.stdout:7/236: chown d6/de/d16/c1c 66066 1 2024-03-16T10:45:12.401 INFO:tasks.workunit.client.1.smithi026.stdout:1/254: write d0/d17/d1f/f3f [336571,123353] 0 2024-03-16T10:45:12.402 INFO:tasks.workunit.client.1.smithi026.stdout:7/237: write d6/d1e/f20 [1150042,59152] 0 2024-03-16T10:45:12.411 INFO:tasks.workunit.client.1.smithi026.stdout:1/255: dwrite d0/d17/f37 [0,4194304] 0 2024-03-16T10:45:12.418 INFO:tasks.workunit.client.1.smithi026.stdout:4/249: stat d1/db/l39 0 2024-03-16T10:45:12.419 INFO:tasks.workunit.client.1.smithi026.stdout:7/238: dread d6/f22 [0,4194304] 0 2024-03-16T10:45:12.426 INFO:tasks.workunit.client.1.smithi026.stdout:7/239: dwrite d6/de/f11 [0,4194304] 0 2024-03-16T10:45:12.429 INFO:tasks.workunit.client.1.smithi026.stdout:2/217: creat d2/d29/d17/d1a/d2b/d2e/f45 x:0 0 0 2024-03-16T10:45:12.429 INFO:tasks.workunit.client.1.smithi026.stdout:7/240: dread d6/f22 [4194304,4194304] 0 2024-03-16T10:45:12.436 INFO:tasks.workunit.client.1.smithi026.stdout:8/194: rmdir d5/dc/d1f/d37 0 2024-03-16T10:45:12.439 INFO:tasks.workunit.client.1.smithi026.stdout:0/237: dread d1/f40 [0,4194304] 0 2024-03-16T10:45:12.441 INFO:tasks.workunit.client.1.smithi026.stdout:8/195: dread f4 [0,4194304] 0 2024-03-16T10:45:12.442 INFO:tasks.workunit.client.1.smithi026.stdout:8/196: stat d5/d7/d2c 0 2024-03-16T10:45:12.442 INFO:tasks.workunit.client.1.smithi026.stdout:8/197: dread - d5/d7/f34 zero size 2024-03-16T10:45:12.443 INFO:tasks.workunit.client.1.smithi026.stdout:8/198: chown d5/c27 36016358 1 2024-03-16T10:45:12.444 INFO:tasks.workunit.client.1.smithi026.stdout:9/188: getdents d0/da/d13 0 2024-03-16T10:45:12.449 INFO:tasks.workunit.client.1.smithi026.stdout:7/241: mkdir d6/de/d16/d44 0 2024-03-16T10:45:12.454 INFO:tasks.workunit.client.1.smithi026.stdout:2/218: rmdir d2/d29/d17/d1a 39 2024-03-16T10:45:12.463 INFO:tasks.workunit.client.0.smithi002.stdout:7/881: sync 2024-03-16T10:45:12.464 INFO:tasks.workunit.client.0.smithi002.stdout:7/882: chown d3/d4/d1c/d5b/d64/d99/d9c/cd1 11039432 1 2024-03-16T10:45:12.465 INFO:tasks.workunit.client.1.smithi026.stdout:9/189: creat d0/f3c x:0 0 0 2024-03-16T10:45:12.466 INFO:tasks.workunit.client.1.smithi026.stdout:9/190: stat d0/da/d16/l18 0 2024-03-16T10:45:12.467 INFO:tasks.workunit.client.1.smithi026.stdout:3/246: rename d4/dc/dd/f4c to d4/dc/f5f 0 2024-03-16T10:45:12.468 INFO:tasks.workunit.client.1.smithi026.stdout:1/256: rename d0/d17 to d0/d17/d4d/d52 22 2024-03-16T10:45:12.468 INFO:tasks.workunit.client.1.smithi026.stdout:3/247: readlink d4/dc/dd/d47/d53/l54 0 2024-03-16T10:45:12.470 INFO:tasks.workunit.client.1.smithi026.stdout:7/242: symlink d6/l45 0 2024-03-16T10:45:12.473 INFO:tasks.workunit.client.1.smithi026.stdout:6/261: sync 2024-03-16T10:45:12.473 INFO:tasks.workunit.client.1.smithi026.stdout:5/167: sync 2024-03-16T10:45:12.474 INFO:tasks.workunit.client.1.smithi026.stdout:6/262: chown d0/d10/c14 3 1 2024-03-16T10:45:12.474 INFO:tasks.workunit.client.1.smithi026.stdout:6/263: write d0/d28/d49/f52 [154451,21204] 0 2024-03-16T10:45:12.476 INFO:tasks.workunit.client.1.smithi026.stdout:6/264: write d0/d28/d49/f5a [838737,38738] 0 2024-03-16T10:45:12.476 INFO:tasks.workunit.client.1.smithi026.stdout:6/265: fsync d0/f3 0 2024-03-16T10:45:12.477 INFO:tasks.workunit.client.0.smithi002.stdout:0/992: creat d3/d31/d88/dc8/f140 x:0 0 0 2024-03-16T10:45:12.478 INFO:tasks.workunit.client.1.smithi026.stdout:9/191: dread d0/f1d [0,4194304] 0 2024-03-16T10:45:12.483 INFO:tasks.workunit.client.0.smithi002.stdout:5/937: rename d0/d16/f19 to d0/d16/dab/d2b/d119/f120 0 2024-03-16T10:45:12.483 INFO:tasks.workunit.client.0.smithi002.stdout:5/938: readlink d0/d1f/l71 0 2024-03-16T10:45:12.483 INFO:tasks.workunit.client.0.smithi002.stdout:5/939: chown d0/d16/dab/d2b 127983506 1 2024-03-16T10:45:12.484 INFO:tasks.workunit.client.0.smithi002.stdout:5/940: stat d0/d16/dab/daa/ld1 0 2024-03-16T10:45:12.485 INFO:tasks.workunit.client.1.smithi026.stdout:4/250: truncate d1/db/d29/f3e 2979092 0 2024-03-16T10:45:12.487 INFO:tasks.workunit.client.1.smithi026.stdout:3/248: mkdir d4/dc/dd/d60 0 2024-03-16T10:45:12.487 INFO:tasks.workunit.client.1.smithi026.stdout:4/251: write d1/f18 [4233437,111667] 0 2024-03-16T10:45:12.489 INFO:tasks.workunit.client.1.smithi026.stdout:0/238: dwrite d1/f40 [0,4194304] 0 2024-03-16T10:45:12.489 INFO:tasks.workunit.client.1.smithi026.stdout:3/249: write d4/fe [5171882,70371] 0 2024-03-16T10:45:12.495 INFO:tasks.workunit.client.1.smithi026.stdout:5/168: fdatasync d1/fd 0 2024-03-16T10:45:12.496 INFO:tasks.workunit.client.1.smithi026.stdout:5/169: stat d1/f21 0 2024-03-16T10:45:12.499 INFO:tasks.workunit.client.1.smithi026.stdout:3/250: dwrite d4/dc/dd/f21 [0,4194304] 0 2024-03-16T10:45:12.507 INFO:tasks.workunit.client.1.smithi026.stdout:9/192: creat d0/da/d14/f3d x:0 0 0 2024-03-16T10:45:12.508 INFO:tasks.workunit.client.1.smithi026.stdout:3/251: dwrite d4/dc/dd/f2e [0,4194304] 0 2024-03-16T10:45:12.510 INFO:tasks.workunit.client.0.smithi002.stdout:2/919: mkdir d7/db/d11d/d124 0 2024-03-16T10:45:12.516 INFO:tasks.workunit.client.0.smithi002.stdout:0/993: write d3/d31/d75/f8c [1618758,70350] 0 2024-03-16T10:45:12.519 INFO:tasks.workunit.client.1.smithi026.stdout:7/243: mknod d6/de/c46 0 2024-03-16T10:45:12.520 INFO:tasks.workunit.client.1.smithi026.stdout:6/266: write d0/f2c [1910864,108080] 0 2024-03-16T10:45:12.521 INFO:tasks.workunit.client.1.smithi026.stdout:0/239: creat d1/d12/f4a x:0 0 0 2024-03-16T10:45:12.522 INFO:tasks.workunit.client.1.smithi026.stdout:0/240: write d1/d18/d1a/d24/f29 [1054075,124795] 0 2024-03-16T10:45:12.523 INFO:tasks.workunit.client.1.smithi026.stdout:2/219: fsync d2/d29/d17/d1a/d2b/d2e/f42 0 2024-03-16T10:45:12.524 INFO:tasks.workunit.client.1.smithi026.stdout:0/241: chown d1/d18/d1a/d24/d11/f2d 3970239 1 2024-03-16T10:45:12.526 INFO:tasks.workunit.client.1.smithi026.stdout:0/242: write d1/d18/d1a/d24/f29 [4917869,78577] 0 2024-03-16T10:45:12.530 INFO:tasks.workunit.client.0.smithi002.stdout:7/883: creat d3/d4/d1d/d87/d5c/f142 x:0 0 0 2024-03-16T10:45:12.532 INFO:tasks.workunit.client.1.smithi026.stdout:8/199: getdents d5/d7 0 2024-03-16T10:45:12.533 INFO:tasks.workunit.client.1.smithi026.stdout:2/220: dwrite d2/d29/d17/f2d [0,4194304] 0 2024-03-16T10:45:12.535 INFO:tasks.workunit.client.0.smithi002.stdout:5/941: creat d0/d16/dab/d2b/f121 x:0 0 0 2024-03-16T10:45:12.536 INFO:tasks.workunit.client.1.smithi026.stdout:5/170: truncate d1/db/d1f/f26 101616 0 2024-03-16T10:45:12.538 INFO:tasks.workunit.client.1.smithi026.stdout:0/243: dwrite d1/d18/d1a/d24/d6/f28 [0,4194304] 0 2024-03-16T10:45:12.539 INFO:tasks.workunit.client.0.smithi002.stdout:2/920: truncate d7/d2e/d28/d40/d50/fa1 642715 0 2024-03-16T10:45:12.540 INFO:tasks.workunit.client.0.smithi002.stdout:4/908: getdents d2/d33/d1c 0 2024-03-16T10:45:12.541 INFO:tasks.workunit.client.1.smithi026.stdout:8/200: dread d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:12.546 INFO:tasks.workunit.client.0.smithi002.stdout:7/884: rename d3/d4/d1d/d87/d96/d8c/c9f to d3/d4/d1d/d87/d5c/dc3/dc7/c143 0 2024-03-16T10:45:12.549 INFO:tasks.workunit.client.0.smithi002.stdout:5/942: mkdir d0/d16/d5b/d122 0 2024-03-16T10:45:12.551 INFO:tasks.workunit.client.1.smithi026.stdout:9/193: rename d0/da/d13/c2d to d0/da/d16/d17/c3e 0 2024-03-16T10:45:12.552 INFO:tasks.workunit.client.1.smithi026.stdout:9/194: dread - d0/f3c zero size 2024-03-16T10:45:12.553 INFO:tasks.workunit.client.1.smithi026.stdout:9/195: dread d0/da/d14/f34 [0,4194304] 0 2024-03-16T10:45:12.554 INFO:tasks.workunit.client.0.smithi002.stdout:2/921: rmdir d7/d27/d3f 39 2024-03-16T10:45:12.557 INFO:tasks.workunit.client.1.smithi026.stdout:1/257: truncate d0/fd 5322887 0 2024-03-16T10:45:12.558 INFO:tasks.workunit.client.0.smithi002.stdout:4/909: mkdir d2/d6/d114 0 2024-03-16T10:45:12.560 INFO:tasks.workunit.client.1.smithi026.stdout:8/201: dread d5/d7/f11 [0,4194304] 0 2024-03-16T10:45:12.561 INFO:tasks.workunit.client.0.smithi002.stdout:7/885: unlink d3/d61/d66/f4f 0 2024-03-16T10:45:12.562 INFO:tasks.workunit.client.0.smithi002.stdout:7/886: chown d3/d37/f130 387160 1 2024-03-16T10:45:12.563 INFO:tasks.workunit.client.0.smithi002.stdout:0/994: link d3/d28/d2a/d67/c87 d3/d59/c141 0 2024-03-16T10:45:12.570 INFO:tasks.workunit.client.0.smithi002.stdout:2/922: symlink d7/d27/d3b/d95/dd2/d11c/l125 0 2024-03-16T10:45:12.572 INFO:tasks.workunit.client.0.smithi002.stdout:4/910: creat d2/d33/d1c/d45/f115 x:0 0 0 2024-03-16T10:45:12.573 INFO:tasks.workunit.client.1.smithi026.stdout:5/171: rmdir d1/db/d1f 39 2024-03-16T10:45:12.575 INFO:tasks.workunit.client.0.smithi002.stdout:7/887: chown d3/d5f/f13e 347563 1 2024-03-16T10:45:12.577 INFO:tasks.workunit.client.1.smithi026.stdout:7/244: mknod d6/de/d16/d44/c47 0 2024-03-16T10:45:12.579 INFO:tasks.workunit.client.0.smithi002.stdout:5/943: dwrite d0/d16/d4a/d55/ff0 [0,4194304] 0 2024-03-16T10:45:12.580 INFO:tasks.workunit.client.0.smithi002.stdout:4/911: dread - d2/d6/fcc zero size 2024-03-16T10:45:12.580 INFO:tasks.workunit.client.0.smithi002.stdout:5/944: write d0/f6f [266342,45528] 0 2024-03-16T10:45:12.581 INFO:tasks.workunit.client.1.smithi026.stdout:0/244: dwrite d1/f15 [0,4194304] 0 2024-03-16T10:45:12.584 INFO:tasks.workunit.client.1.smithi026.stdout:6/267: mkdir d0/d62 0 2024-03-16T10:45:12.584 INFO:tasks.workunit.client.0.smithi002.stdout:0/995: dwrite d3/d31/d88/f8b [4194304,4194304] 0 2024-03-16T10:45:12.589 INFO:tasks.workunit.client.0.smithi002.stdout:0/996: dread d3/d31/d75/d11f/f131 [4194304,4194304] 0 2024-03-16T10:45:12.596 INFO:tasks.workunit.client.0.smithi002.stdout:7/888: dwrite d3/d4/ff4 [0,4194304] 0 2024-03-16T10:45:12.596 INFO:tasks.workunit.client.1.smithi026.stdout:9/196: dread d0/f9 [0,4194304] 0 2024-03-16T10:45:12.598 INFO:tasks.workunit.client.0.smithi002.stdout:2/923: truncate d7/d27/d3b/d95/da5/f102 258433 0 2024-03-16T10:45:12.598 INFO:tasks.workunit.client.0.smithi002.stdout:7/889: write d3/d4/d1d/d87/d5c/dc3/f11f [206015,99633] 0 2024-03-16T10:45:12.598 INFO:tasks.workunit.client.0.smithi002.stdout:5/945: symlink d0/d16/d5b/da0/l123 0 2024-03-16T10:45:12.602 INFO:tasks.workunit.client.1.smithi026.stdout:3/252: symlink d4/d46/d4f/d5a/l61 0 2024-03-16T10:45:12.605 INFO:tasks.workunit.client.0.smithi002.stdout:7/890: dwrite d3/d4/d1d/d87/d5c/d63/d3f/f89 [0,4194304] 0 2024-03-16T10:45:12.605 INFO:tasks.workunit.client.1.smithi026.stdout:7/245: symlink d6/de/d16/d44/l48 0 2024-03-16T10:45:12.612 INFO:tasks.workunit.client.0.smithi002.stdout:4/912: write d2/d6/d32/d66/d7f/de7/f5f [3071777,24973] 0 2024-03-16T10:45:12.614 INFO:tasks.workunit.client.0.smithi002.stdout:4/913: write d2/d6/f89 [3382248,53691] 0 2024-03-16T10:45:12.618 INFO:tasks.workunit.client.0.smithi002.stdout:0/997: write d3/f49 [2528,80539] 0 2024-03-16T10:45:12.620 INFO:tasks.workunit.client.1.smithi026.stdout:8/202: fsync d5/dc/f18 0 2024-03-16T10:45:12.620 INFO:tasks.workunit.client.1.smithi026.stdout:0/245: dwrite d1/d18/d1a/f25 [0,4194304] 0 2024-03-16T10:45:12.622 INFO:tasks.workunit.client.0.smithi002.stdout:5/946: mkdir d0/d1f/da1/d124 0 2024-03-16T10:45:12.623 INFO:tasks.workunit.client.1.smithi026.stdout:2/221: link d2/d29/d17/c3d d2/d29/d3b/d41/c46 0 2024-03-16T10:45:12.627 INFO:tasks.workunit.client.1.smithi026.stdout:5/172: symlink d1/db/l37 0 2024-03-16T10:45:12.630 INFO:tasks.workunit.client.1.smithi026.stdout:8/203: dwrite d5/d7/d2c/f2d [8388608,4194304] 0 2024-03-16T10:45:12.630 INFO:tasks.workunit.client.1.smithi026.stdout:3/253: fdatasync d4/fa 0 2024-03-16T10:45:12.631 INFO:tasks.workunit.client.0.smithi002.stdout:4/914: creat d2/d6/d32/d68/f116 x:0 0 0 2024-03-16T10:45:12.632 INFO:tasks.workunit.client.0.smithi002.stdout:4/915: stat d2/d6/d32/d66/d7f/de7/d2e/db4/ff3 0 2024-03-16T10:45:12.633 INFO:tasks.workunit.client.1.smithi026.stdout:1/258: dread d0/d33/d3b/f2f [0,4194304] 0 2024-03-16T10:45:12.635 INFO:tasks.workunit.client.1.smithi026.stdout:3/254: write d4/dc/d31/d48/f58 [220712,10144] 0 2024-03-16T10:45:12.635 INFO:tasks.workunit.client.1.smithi026.stdout:6/268: creat d0/d62/f63 x:0 0 0 2024-03-16T10:45:12.636 INFO:tasks.workunit.client.1.smithi026.stdout:6/269: chown d0/d5b/f25 115143 1 2024-03-16T10:45:12.638 INFO:tasks.workunit.client.0.smithi002.stdout:5/947: rename d0/d1f/c44 to d0/d24/dad/db6/c125 0 2024-03-16T10:45:12.642 INFO:tasks.workunit.client.1.smithi026.stdout:6/270: dwrite d0/d10/f36 [0,4194304] 0 2024-03-16T10:45:12.644 INFO:tasks.workunit.client.1.smithi026.stdout:6/271: chown d0/d28/f4d 1624 1 2024-03-16T10:45:12.644 INFO:tasks.workunit.client.1.smithi026.stdout:6/272: stat d0/f51 0 2024-03-16T10:45:12.646 INFO:tasks.workunit.client.1.smithi026.stdout:6/273: write d0/f32 [649598,24020] 0 2024-03-16T10:45:12.647 INFO:tasks.workunit.client.0.smithi002.stdout:4/916: symlink d2/d6/d32/d66/d7f/de7/d5e/d93/l117 0 2024-03-16T10:45:12.649 INFO:tasks.workunit.client.0.smithi002.stdout:4/917: write d2/d33/d1c/f83 [1514381,78182] 0 2024-03-16T10:45:12.650 INFO:tasks.workunit.client.1.smithi026.stdout:0/246: dwrite d1/d18/d1a/f2c [0,4194304] 0 2024-03-16T10:45:12.651 INFO:tasks.workunit.client.1.smithi026.stdout:2/222: creat d2/d29/d3b/f47 x:0 0 0 2024-03-16T10:45:12.652 INFO:tasks.workunit.client.0.smithi002.stdout:5/948: fdatasync d0/d16/d4a/d39/f80 0 2024-03-16T10:45:12.652 INFO:tasks.workunit.client.1.smithi026.stdout:2/223: write d2/d29/d3b/f44 [765648,49035] 0 2024-03-16T10:45:12.653 INFO:tasks.workunit.client.0.smithi002.stdout:2/924: link d7/d2e/d28/d44/f11e d7/d2e/d1d/d38/df6/f126 0 2024-03-16T10:45:12.656 INFO:tasks.workunit.client.1.smithi026.stdout:2/224: dread - d2/d29/d36/f3e zero size 2024-03-16T10:45:12.658 INFO:tasks.workunit.client.1.smithi026.stdout:5/173: readlink d1/l6 0 2024-03-16T10:45:12.659 INFO:tasks.workunit.client.1.smithi026.stdout:0/247: dread d1/d18/d1a/d24/d11/f2d [4194304,4194304] 0 2024-03-16T10:45:12.660 INFO:tasks.workunit.client.1.smithi026.stdout:8/204: truncate d5/d7/d2c/f32 8658934 0 2024-03-16T10:45:12.662 INFO:tasks.workunit.client.1.smithi026.stdout:1/259: symlink d0/d17/d1f/l53 0 2024-03-16T10:45:12.662 INFO:tasks.workunit.client.0.smithi002.stdout:0/998: write d3/d31/fc6 [1496644,116040] 0 2024-03-16T10:45:12.665 INFO:tasks.workunit.client.1.smithi026.stdout:3/255: rename d4/d46 to d4/dc/dd/d60/d62 0 2024-03-16T10:45:12.667 INFO:tasks.workunit.client.1.smithi026.stdout:7/246: write d6/f31 [1391697,108575] 0 2024-03-16T10:45:12.668 INFO:tasks.workunit.client.0.smithi002.stdout:5/949: dread d0/d16/dab/d33/d4b/ff4 [0,4194304] 0 2024-03-16T10:45:12.672 INFO:tasks.workunit.client.0.smithi002.stdout:4/918: rename d2/d6/d32/d66/d7f/de7/d5e/d93/ff4 to d2/d6/d32/d66/d7f/de7/d88/db2/f118 0 2024-03-16T10:45:12.674 INFO:tasks.workunit.client.0.smithi002.stdout:0/999: dread - d3/d28/d2a/f104 zero size 2024-03-16T10:45:12.675 INFO:tasks.workunit.client.1.smithi026.stdout:6/274: write d0/d28/d2d/f61 [2183781,113209] 0 2024-03-16T10:45:12.676 INFO:tasks.workunit.client.0.smithi002.stdout:2/925: dread d7/d2e/f67 [0,4194304] 0 2024-03-16T10:45:12.677 INFO:tasks.workunit.client.0.smithi002.stdout:2/926: stat d7/d2e/d28/d40/d6c/f7a 0 2024-03-16T10:45:12.680 INFO:tasks.workunit.client.0.smithi002.stdout:4/919: creat d2/d33/d1c/d45/f119 x:0 0 0 2024-03-16T10:45:12.681 INFO:tasks.workunit.client.1.smithi026.stdout:2/225: creat d2/d29/d17/d1a/d2b/d2e/f48 x:0 0 0 2024-03-16T10:45:12.681 INFO:tasks.workunit.client.0.smithi002.stdout:4/920: fsync d2/d6/d32/d66/d7f/de7/d2e/db4/ff3 0 2024-03-16T10:45:12.682 INFO:tasks.workunit.client.1.smithi026.stdout:5/174: dread - d1/db/f22 zero size 2024-03-16T10:45:12.682 INFO:tasks.workunit.client.0.smithi002.stdout:4/921: write d2/d6/d32/d66/d7f/de7/d2e/db4/fd2 [3763479,44958] 0 2024-03-16T10:45:12.683 INFO:tasks.workunit.client.1.smithi026.stdout:5/175: write d1/db/f22 [509568,100629] 0 2024-03-16T10:45:12.683 INFO:tasks.workunit.client.0.smithi002.stdout:5/950: rename d0/d16/d4a/c2d to d0/d16/d4a/d55/c126 0 2024-03-16T10:45:12.684 INFO:tasks.workunit.client.1.smithi026.stdout:5/176: readlink d1/d1c/l36 0 2024-03-16T10:45:12.684 INFO:tasks.workunit.client.0.smithi002.stdout:5/951: stat d0/d16/dab/d33/d8c/fa8 0 2024-03-16T10:45:12.684 INFO:tasks.workunit.client.0.smithi002.stdout:5/952: chown d0/d16/l35 2810988 1 2024-03-16T10:45:12.685 INFO:tasks.workunit.client.1.smithi026.stdout:0/248: creat d1/d12/f4b x:0 0 0 2024-03-16T10:45:12.688 INFO:tasks.workunit.client.1.smithi026.stdout:1/260: mkdir d0/d33/d2e/d54 0 2024-03-16T10:45:12.689 INFO:tasks.workunit.client.0.smithi002.stdout:2/927: link d7/d27/d3b/c53 d7/d2e/d28/d40/d50/c127 0 2024-03-16T10:45:12.689 INFO:tasks.workunit.client.1.smithi026.stdout:1/261: write d0/d17/d1f/fc [2489134,107712] 0 2024-03-16T10:45:12.690 INFO:tasks.workunit.client.0.smithi002.stdout:5/953: mknod d0/d16/d4a/d55/c127 0 2024-03-16T10:45:12.693 INFO:tasks.workunit.client.0.smithi002.stdout:4/922: mkdir d2/d6/d32/d66/d7f/de7/d11a 0 2024-03-16T10:45:12.694 INFO:tasks.workunit.client.1.smithi026.stdout:7/247: creat d6/de/d16/f49 x:0 0 0 2024-03-16T10:45:12.695 INFO:tasks.workunit.client.0.smithi002.stdout:2/928: dwrite d7/d2e/d1d/d2f/d4b/f114 [0,4194304] 0 2024-03-16T10:45:12.697 INFO:tasks.workunit.client.1.smithi026.stdout:6/275: creat d0/d5b/f64 x:0 0 0 2024-03-16T10:45:12.697 INFO:tasks.workunit.client.0.smithi002.stdout:7/891: sync 2024-03-16T10:45:12.699 INFO:tasks.workunit.client.1.smithi026.stdout:9/197: getdents d0 0 2024-03-16T10:45:12.701 INFO:tasks.workunit.client.0.smithi002.stdout:5/954: unlink d0/d16/dab/d2b/d5d/l9a 0 2024-03-16T10:45:12.702 INFO:tasks.workunit.client.0.smithi002.stdout:5/955: dread - d0/d16/dab/d33/d4b/fde zero size 2024-03-16T10:45:12.702 INFO:tasks.workunit.client.1.smithi026.stdout:7/248: dwrite d6/de/d16/f19 [0,4194304] 0 2024-03-16T10:45:12.704 INFO:tasks.workunit.client.0.smithi002.stdout:2/929: truncate d7/d2e/d1d/d107/d10e/f22 450775 0 2024-03-16T10:45:12.709 INFO:tasks.workunit.client.0.smithi002.stdout:7/892: symlink d3/d4/d1c/d5b/d64/ddf/l144 0 2024-03-16T10:45:12.710 INFO:tasks.workunit.client.1.smithi026.stdout:8/205: fsync f4 0 2024-03-16T10:45:12.710 INFO:tasks.workunit.client.0.smithi002.stdout:7/893: write d3/d4/f12e [433840,50634] 0 2024-03-16T10:45:12.711 INFO:tasks.workunit.client.0.smithi002.stdout:7/894: chown d3/d4/d1d/d87/d5c/d63/d3f/f41 83577556 1 2024-03-16T10:45:12.712 INFO:tasks.workunit.client.0.smithi002.stdout:2/930: creat d7/d27/d3b/d95/f128 x:0 0 0 2024-03-16T10:45:12.713 INFO:tasks.workunit.client.0.smithi002.stdout:7/895: stat d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/f53 0 2024-03-16T10:45:12.715 INFO:tasks.workunit.client.0.smithi002.stdout:7/896: write d3/d4/d1d/d87/d5c/dc3/f11f [1016724,49412] 0 2024-03-16T10:45:12.715 INFO:tasks.workunit.client.0.smithi002.stdout:5/956: mkdir d0/d24/dad/d128 0 2024-03-16T10:45:12.716 INFO:tasks.workunit.client.0.smithi002.stdout:4/923: link d2/d33/c55 d2/d6/d114/c11b 0 2024-03-16T10:45:12.717 INFO:tasks.workunit.client.0.smithi002.stdout:5/957: write d0/d16/dab/d2b/d94/f7a [4081516,63902] 0 2024-03-16T10:45:12.718 INFO:tasks.workunit.client.1.smithi026.stdout:6/276: write d0/d5b/f30 [3843317,26296] 0 2024-03-16T10:45:12.718 INFO:tasks.workunit.client.1.smithi026.stdout:1/262: dwrite d0/f40 [0,4194304] 0 2024-03-16T10:45:12.719 INFO:tasks.workunit.client.0.smithi002.stdout:7/897: write d3/d4/d1d/d87/d5c/d63/d3f/dba/d13c/f13f [531859,115020] 0 2024-03-16T10:45:12.719 INFO:tasks.workunit.client.0.smithi002.stdout:2/931: creat d7/d2e/d46/f129 x:0 0 0 2024-03-16T10:45:12.720 INFO:tasks.workunit.client.1.smithi026.stdout:6/277: dread d0/f44 [0,4194304] 0 2024-03-16T10:45:12.722 INFO:tasks.workunit.client.0.smithi002.stdout:2/932: dread d7/d2e/d1d/d2f/d4b/fc8 [0,4194304] 0 2024-03-16T10:45:12.724 INFO:tasks.workunit.client.1.smithi026.stdout:9/198: creat d0/da/d16/f3f x:0 0 0 2024-03-16T10:45:12.725 INFO:tasks.workunit.client.1.smithi026.stdout:1/263: dread d0/d17/d1f/fc [4194304,4194304] 0 2024-03-16T10:45:12.730 INFO:tasks.workunit.client.1.smithi026.stdout:3/256: write d4/dc/dd/d47/d53/f5c [1042705,20976] 0 2024-03-16T10:45:12.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:12 smithi026 ceph-mon[37041]: pgmap v39: 65 pgs: 65 active+clean; 2.3 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 100 MiB/s rd, 266 MiB/s wr, 584 op/s 2024-03-16T10:45:12.732 INFO:tasks.workunit.client.0.smithi002.stdout:5/958: creat d0/d16/d5b/da0/f129 x:0 0 0 2024-03-16T10:45:12.733 INFO:tasks.workunit.client.1.smithi026.stdout:4/252: sync 2024-03-16T10:45:12.733 INFO:tasks.workunit.client.1.smithi026.stdout:4/253: chown d1/db/d29/d44/d48 108174 1 2024-03-16T10:45:12.734 INFO:tasks.workunit.client.0.smithi002.stdout:2/933: mknod d7/d2e/d1d/d38/df6/c12a 0 2024-03-16T10:45:12.737 INFO:tasks.workunit.client.1.smithi026.stdout:4/254: dread d1/db/f26 [0,4194304] 0 2024-03-16T10:45:12.742 INFO:tasks.workunit.client.0.smithi002.stdout:5/959: unlink d0/d16/dab/d33/c10e 0 2024-03-16T10:45:12.743 INFO:tasks.workunit.client.0.smithi002.stdout:5/960: dread - d0/d16/dab/d2b/f121 zero size 2024-03-16T10:45:12.744 INFO:tasks.workunit.client.0.smithi002.stdout:2/934: rmdir d7/d2e/d1d/d38/d49 39 2024-03-16T10:45:12.748 INFO:tasks.workunit.client.1.smithi026.stdout:6/278: creat d0/d41/d4f/f65 x:0 0 0 2024-03-16T10:45:12.749 INFO:tasks.workunit.client.0.smithi002.stdout:5/961: dwrite d0/d16/dab/d2b/d94/f10d [0,4194304] 0 2024-03-16T10:45:12.750 INFO:tasks.workunit.client.0.smithi002.stdout:4/924: link d2/d6/f7e d2/d6/d32/d68/f11c 0 2024-03-16T10:45:12.752 INFO:tasks.workunit.client.1.smithi026.stdout:9/199: creat d0/da/d16/d17/f40 x:0 0 0 2024-03-16T10:45:12.753 INFO:tasks.workunit.client.1.smithi026.stdout:9/200: fdatasync d0/da/d13/f31 0 2024-03-16T10:45:12.754 INFO:tasks.workunit.client.0.smithi002.stdout:5/962: dwrite d0/d1f/f36 [0,4194304] 0 2024-03-16T10:45:12.755 INFO:tasks.workunit.client.1.smithi026.stdout:9/201: write d0/da/f3b [135811,117929] 0 2024-03-16T10:45:12.755 INFO:tasks.workunit.client.0.smithi002.stdout:5/963: fdatasync d0/d16/dab/d33/d4b/fde 0 2024-03-16T10:45:12.758 INFO:tasks.workunit.client.1.smithi026.stdout:2/226: rename d2/d29/d3b/f44 to d2/d29/d17/f49 0 2024-03-16T10:45:12.760 INFO:tasks.workunit.client.0.smithi002.stdout:2/935: creat d7/d2e/d1d/d38/d49/d8c/f12b x:0 0 0 2024-03-16T10:45:12.761 INFO:tasks.workunit.client.1.smithi026.stdout:5/177: creat d1/db/f38 x:0 0 0 2024-03-16T10:45:12.761 INFO:tasks.workunit.client.0.smithi002.stdout:2/936: truncate d7/d27/d3b/dd4/f106 393202 0 2024-03-16T10:45:12.762 INFO:tasks.workunit.client.1.smithi026.stdout:5/178: chown d1/f21 0 1 2024-03-16T10:45:12.762 INFO:tasks.workunit.client.0.smithi002.stdout:2/937: dread - d7/d2e/d28/d40/d50/db4/dcd/ff4 zero size 2024-03-16T10:45:12.762 INFO:tasks.workunit.client.1.smithi026.stdout:5/179: readlink d1/l6 0 2024-03-16T10:45:12.763 INFO:tasks.workunit.client.1.smithi026.stdout:5/180: chown d1/d1c/f1e 5283172 1 2024-03-16T10:45:12.770 INFO:tasks.workunit.client.0.smithi002.stdout:5/964: rmdir d0/d1f/da1/d124 0 2024-03-16T10:45:12.770 INFO:tasks.workunit.client.1.smithi026.stdout:5/181: dread d1/db/f22 [0,4194304] 0 2024-03-16T10:45:12.771 INFO:tasks.workunit.client.1.smithi026.stdout:7/249: dwrite d6/de/d16/f1f [0,4194304] 0 2024-03-16T10:45:12.772 INFO:tasks.workunit.client.1.smithi026.stdout:4/255: creat d1/db/d29/f54 x:0 0 0 2024-03-16T10:45:12.774 INFO:tasks.workunit.client.0.smithi002.stdout:2/938: creat d7/d27/d3f/d7f/f12c x:0 0 0 2024-03-16T10:45:12.775 INFO:tasks.workunit.client.0.smithi002.stdout:5/965: creat d0/d16/dab/d33/d8c/f12a x:0 0 0 2024-03-16T10:45:12.776 INFO:tasks.workunit.client.0.smithi002.stdout:5/966: readlink d0/d16/dab/d2b/d5d/la9 0 2024-03-16T10:45:12.777 INFO:tasks.workunit.client.1.smithi026.stdout:6/279: rmdir d0/d28 39 2024-03-16T10:45:12.778 INFO:tasks.workunit.client.0.smithi002.stdout:2/939: write d7/d27/d3b/f94 [3491076,124095] 0 2024-03-16T10:45:12.778 INFO:tasks.workunit.client.1.smithi026.stdout:7/250: dwrite d6/d35/d3a/d3c/f40 [0,4194304] 0 2024-03-16T10:45:12.778 INFO:tasks.workunit.client.1.smithi026.stdout:6/280: chown d0/f11 366727289 1 2024-03-16T10:45:12.779 INFO:tasks.workunit.client.1.smithi026.stdout:1/264: unlink d0/l2a 0 2024-03-16T10:45:12.780 INFO:tasks.workunit.client.1.smithi026.stdout:7/251: write d6/d1e/f32 [881531,13934] 0 2024-03-16T10:45:12.782 INFO:tasks.workunit.client.1.smithi026.stdout:9/202: rmdir d0/da/d16 39 2024-03-16T10:45:12.787 INFO:tasks.workunit.client.0.smithi002.stdout:5/967: mkdir d0/d24/dad/d128/d12b 0 2024-03-16T10:45:12.789 INFO:tasks.workunit.client.1.smithi026.stdout:6/281: dwrite d0/d5b/f30 [4194304,4194304] 0 2024-03-16T10:45:12.790 INFO:tasks.workunit.client.1.smithi026.stdout:0/249: rename d1/f3c to d1/d12/d2f/d48/f4c 0 2024-03-16T10:45:12.793 INFO:tasks.workunit.client.1.smithi026.stdout:0/250: stat d1/f15 0 2024-03-16T10:45:12.793 INFO:tasks.workunit.client.1.smithi026.stdout:7/252: dwrite d6/de/d16/f43 [0,4194304] 0 2024-03-16T10:45:12.794 INFO:tasks.workunit.client.1.smithi026.stdout:2/227: mknod d2/d29/d17/d1a/d2b/c4a 0 2024-03-16T10:45:12.795 INFO:tasks.workunit.client.1.smithi026.stdout:7/253: read - d6/f39 zero size 2024-03-16T10:45:12.796 INFO:tasks.workunit.client.0.smithi002.stdout:5/968: unlink d0/d16/d5b/f102 0 2024-03-16T10:45:12.797 INFO:tasks.workunit.client.0.smithi002.stdout:5/969: readlink d0/d16/dab/d2b/d5d/la9 0 2024-03-16T10:45:12.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:12 smithi002 ceph-mon[29248]: pgmap v39: 65 pgs: 65 active+clean; 2.3 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 100 MiB/s rd, 266 MiB/s wr, 584 op/s 2024-03-16T10:45:12.803 INFO:tasks.workunit.client.0.smithi002.stdout:2/940: write d7/d2e/d1d/d38/fa4 [192898,119881] 0 2024-03-16T10:45:12.804 INFO:tasks.workunit.client.0.smithi002.stdout:2/941: write d7/d27/d3b/d95/da5/fbb [3407011,26059] 0 2024-03-16T10:45:12.805 INFO:tasks.workunit.client.0.smithi002.stdout:5/970: symlink d0/d16/dab/d33/l12c 0 2024-03-16T10:45:12.805 INFO:tasks.workunit.client.0.smithi002.stdout:2/942: fdatasync d7/d2e/d46/f10f 0 2024-03-16T10:45:12.806 INFO:tasks.workunit.client.0.smithi002.stdout:5/971: fdatasync d0/d24/dad/db6/fff 0 2024-03-16T10:45:12.807 INFO:tasks.workunit.client.1.smithi026.stdout:8/206: creat d5/dc/f38 x:0 0 0 2024-03-16T10:45:12.807 INFO:tasks.workunit.client.1.smithi026.stdout:3/257: truncate d4/d29/f30 4094170 0 2024-03-16T10:45:12.808 INFO:tasks.workunit.client.1.smithi026.stdout:3/258: readlink d4/dc/dd/d47/d53/l54 0 2024-03-16T10:45:12.810 INFO:tasks.workunit.client.0.smithi002.stdout:2/943: mkdir d7/d27/d3f/d12d 0 2024-03-16T10:45:12.812 INFO:tasks.workunit.client.1.smithi026.stdout:4/256: rmdir d1/db/d29/d44/d48 39 2024-03-16T10:45:12.814 INFO:tasks.workunit.client.0.smithi002.stdout:2/944: symlink d7/d2e/d28/d40/d50/l12e 0 2024-03-16T10:45:12.816 INFO:tasks.workunit.client.1.smithi026.stdout:4/257: dread d1/f3 [4194304,4194304] 0 2024-03-16T10:45:12.818 INFO:tasks.workunit.client.1.smithi026.stdout:4/258: dread d1/f6 [4194304,4194304] 0 2024-03-16T10:45:12.822 INFO:tasks.workunit.client.0.smithi002.stdout:2/945: symlink d7/db/d70/l12f 0 2024-03-16T10:45:12.823 INFO:tasks.workunit.client.0.smithi002.stdout:5/972: truncate d0/d16/dab/d43/d48/f8f 6474943 0 2024-03-16T10:45:12.824 INFO:tasks.workunit.client.1.smithi026.stdout:6/282: mknod d0/d41/c66 0 2024-03-16T10:45:12.825 INFO:tasks.workunit.client.1.smithi026.stdout:2/228: mkdir d2/d29/d17/d1a/d2b/d4b 0 2024-03-16T10:45:12.826 INFO:tasks.workunit.client.1.smithi026.stdout:7/254: creat d6/d2f/f4a x:0 0 0 2024-03-16T10:45:12.827 INFO:tasks.workunit.client.1.smithi026.stdout:7/255: stat d6/c9 0 2024-03-16T10:45:12.827 INFO:tasks.workunit.client.1.smithi026.stdout:7/256: stat d6/d1e/f20 0 2024-03-16T10:45:12.832 INFO:tasks.workunit.client.1.smithi026.stdout:3/259: rename d4/l7 to d4/dc/dd/d47/l63 0 2024-03-16T10:45:12.835 INFO:tasks.workunit.client.0.smithi002.stdout:2/946: dread d7/d2e/d1d/d2f/d4b/d6e/fc6 [0,4194304] 0 2024-03-16T10:45:12.835 INFO:tasks.workunit.client.0.smithi002.stdout:2/947: dread - d7/d27/d3f/f11b zero size 2024-03-16T10:45:12.836 INFO:tasks.workunit.client.0.smithi002.stdout:2/948: write d7/d27/d3f/f69 [4442749,71159] 0 2024-03-16T10:45:12.837 INFO:tasks.workunit.client.1.smithi026.stdout:1/265: symlink d0/d33/d3b/d49/l55 0 2024-03-16T10:45:12.837 INFO:tasks.workunit.client.0.smithi002.stdout:2/949: readlink d7/d2e/d1d/d107/d10e/l14 0 2024-03-16T10:45:12.837 INFO:tasks.workunit.client.1.smithi026.stdout:1/266: chown d0/d17/c4c 625621 1 2024-03-16T10:45:12.838 INFO:tasks.workunit.client.1.smithi026.stdout:0/251: write d1/d18/d1a/d24/d11/f2d [7624622,73565] 0 2024-03-16T10:45:12.839 INFO:tasks.workunit.client.1.smithi026.stdout:1/267: chown d0/d33/l29 50 1 2024-03-16T10:45:12.840 INFO:tasks.workunit.client.0.smithi002.stdout:5/973: truncate d0/d24/dad/db6/f110 1370245 0 2024-03-16T10:45:12.840 INFO:tasks.workunit.client.1.smithi026.stdout:1/268: fsync d0/d33/f30 0 2024-03-16T10:45:12.841 INFO:tasks.workunit.client.1.smithi026.stdout:9/203: creat d0/da/d16/d17/d19/f41 x:0 0 0 2024-03-16T10:45:12.841 INFO:tasks.workunit.client.1.smithi026.stdout:4/259: dread d1/db/d29/f3e [0,4194304] 0 2024-03-16T10:45:12.842 INFO:tasks.workunit.client.1.smithi026.stdout:9/204: chown d0/c21 2841 1 2024-03-16T10:45:12.842 INFO:tasks.workunit.client.0.smithi002.stdout:2/950: symlink d7/d2e/d1d/d38/da8/l130 0 2024-03-16T10:45:12.845 INFO:tasks.workunit.client.0.smithi002.stdout:5/974: fsync d0/d16/dab/d2b/d5d/fa7 0 2024-03-16T10:45:12.847 INFO:tasks.workunit.client.1.smithi026.stdout:4/260: dwrite d1/db/de/f3c [0,4194304] 0 2024-03-16T10:45:12.848 INFO:tasks.workunit.client.0.smithi002.stdout:2/951: dwrite d7/d2e/f10d [0,4194304] 0 2024-03-16T10:45:12.848 INFO:tasks.workunit.client.1.smithi026.stdout:4/261: stat d1/db/de/l51 0 2024-03-16T10:45:12.855 INFO:tasks.workunit.client.1.smithi026.stdout:8/207: creat d5/d28/f39 x:0 0 0 2024-03-16T10:45:12.858 INFO:tasks.workunit.client.1.smithi026.stdout:3/260: chown d4/dc/dd/d47/d53/l59 211063 1 2024-03-16T10:45:12.863 INFO:tasks.workunit.client.1.smithi026.stdout:0/252: mknod d1/d12/d2f/c4d 0 2024-03-16T10:45:12.863 INFO:tasks.workunit.client.1.smithi026.stdout:0/253: dread - d1/d12/f4a zero size 2024-03-16T10:45:12.866 INFO:tasks.workunit.client.0.smithi002.stdout:5/975: write d0/d16/dab/d43/f99 [2110536,77158] 0 2024-03-16T10:45:12.867 INFO:tasks.workunit.client.1.smithi026.stdout:9/205: rmdir d0/da/d16 39 2024-03-16T10:45:12.867 INFO:tasks.workunit.client.0.smithi002.stdout:2/952: dread d7/d2e/d1d/f21 [4194304,4194304] 0 2024-03-16T10:45:12.868 INFO:tasks.workunit.client.1.smithi026.stdout:2/229: mknod d2/d29/d17/c4c 0 2024-03-16T10:45:12.871 INFO:tasks.workunit.client.1.smithi026.stdout:4/262: write d1/f6 [4025048,72995] 0 2024-03-16T10:45:12.873 INFO:tasks.workunit.client.1.smithi026.stdout:7/257: creat d6/d35/d3a/d3c/d3f/f4b x:0 0 0 2024-03-16T10:45:12.873 INFO:tasks.workunit.client.0.smithi002.stdout:5/976: dread - d0/d16/d4a/d39/fe9 zero size 2024-03-16T10:45:12.875 INFO:tasks.workunit.client.0.smithi002.stdout:5/977: write d0/d24/dad/db6/db9/fc2 [293875,95184] 0 2024-03-16T10:45:12.875 INFO:tasks.workunit.client.0.smithi002.stdout:2/953: symlink d7/db/d11d/d124/l131 0 2024-03-16T10:45:12.877 INFO:tasks.workunit.client.1.smithi026.stdout:5/182: rename d1/db/f2d to d1/db/f39 0 2024-03-16T10:45:12.880 INFO:tasks.workunit.client.0.smithi002.stdout:5/978: mknod d0/d16/d5b/da0/dd5/c12d 0 2024-03-16T10:45:12.880 INFO:tasks.workunit.client.0.smithi002.stdout:5/979: dread - d0/d16/d4a/f107 zero size 2024-03-16T10:45:12.883 INFO:tasks.workunit.client.1.smithi026.stdout:1/269: unlink d0/d33/d3b/l41 0 2024-03-16T10:45:12.885 INFO:tasks.workunit.client.1.smithi026.stdout:6/283: getdents d0/d28/d2d/d54 0 2024-03-16T10:45:12.885 INFO:tasks.workunit.client.1.smithi026.stdout:3/261: write d4/dc/f19 [278112,54632] 0 2024-03-16T10:45:12.886 INFO:tasks.workunit.client.1.smithi026.stdout:6/284: write d0/d41/d4f/f65 [756884,127624] 0 2024-03-16T10:45:12.888 INFO:tasks.workunit.client.1.smithi026.stdout:4/263: write d1/f3 [3489365,116859] 0 2024-03-16T10:45:12.889 INFO:tasks.workunit.client.1.smithi026.stdout:7/258: unlink d6/c9 0 2024-03-16T10:45:12.890 INFO:tasks.workunit.client.1.smithi026.stdout:8/208: truncate d5/dc/f1e 375612 0 2024-03-16T10:45:12.892 INFO:tasks.workunit.client.0.smithi002.stdout:2/954: truncate d7/d2e/d28/d40/d6c/f7a 4132548 0 2024-03-16T10:45:12.893 INFO:tasks.workunit.client.1.smithi026.stdout:5/183: dread - d1/db/d1f/f33 zero size 2024-03-16T10:45:12.893 INFO:tasks.workunit.client.0.smithi002.stdout:5/980: dread d0/d24/dad/fae [0,4194304] 0 2024-03-16T10:45:12.894 INFO:tasks.workunit.client.0.smithi002.stdout:7/898: sync 2024-03-16T10:45:12.894 INFO:tasks.workunit.client.0.smithi002.stdout:5/981: fdatasync d0/d16/d4a/d7d/fc5 0 2024-03-16T10:45:12.895 INFO:tasks.workunit.client.1.smithi026.stdout:0/254: creat d1/d18/d1a/d2b/f4e x:0 0 0 2024-03-16T10:45:12.895 INFO:tasks.workunit.client.1.smithi026.stdout:8/209: dread d5/dc/f29 [0,4194304] 0 2024-03-16T10:45:12.896 INFO:tasks.workunit.client.1.smithi026.stdout:1/270: unlink d0/d17/f47 0 2024-03-16T10:45:12.897 INFO:tasks.workunit.client.1.smithi026.stdout:4/264: dwrite d1/db/d29/f43 [0,4194304] 0 2024-03-16T10:45:12.897 INFO:tasks.workunit.client.1.smithi026.stdout:0/255: readlink d1/d18/d1a/d24/l3f 0 2024-03-16T10:45:12.898 INFO:tasks.workunit.client.1.smithi026.stdout:1/271: read d0/d17/f37 [783307,22626] 0 2024-03-16T10:45:12.899 INFO:tasks.workunit.client.0.smithi002.stdout:2/955: rename d7/d27/d3f to d7/d2e/d28/d40/d50/d132 0 2024-03-16T10:45:12.900 INFO:tasks.workunit.client.1.smithi026.stdout:3/262: creat d4/dc/dd/d60/f64 x:0 0 0 2024-03-16T10:45:12.907 INFO:tasks.workunit.client.0.smithi002.stdout:2/956: unlink d7/db/d70/l12f 0 2024-03-16T10:45:12.910 INFO:tasks.workunit.client.1.smithi026.stdout:7/259: rename d6/c2a to d6/d1e/c4c 0 2024-03-16T10:45:12.912 INFO:tasks.workunit.client.1.smithi026.stdout:5/184: rmdir d1/d1c/d2a 39 2024-03-16T10:45:12.914 INFO:tasks.workunit.client.0.smithi002.stdout:2/957: mknod d7/db/d11d/d124/c133 0 2024-03-16T10:45:12.918 INFO:tasks.workunit.client.0.smithi002.stdout:7/899: creat d3/d4/d1d/d87/d5c/d63/f145 x:0 0 0 2024-03-16T10:45:12.919 INFO:tasks.workunit.client.1.smithi026.stdout:4/265: creat d1/db/de/f55 x:0 0 0 2024-03-16T10:45:12.919 INFO:tasks.workunit.client.0.smithi002.stdout:5/982: truncate d0/d16/d4a/d39/ff7 784411 0 2024-03-16T10:45:12.921 INFO:tasks.workunit.client.1.smithi026.stdout:1/272: creat d0/d33/d2e/f56 x:0 0 0 2024-03-16T10:45:12.923 INFO:tasks.workunit.client.0.smithi002.stdout:2/958: unlink d7/d2e/d1d/d2f/d4b/fa0 0 2024-03-16T10:45:12.923 INFO:tasks.workunit.client.1.smithi026.stdout:0/256: mkdir d1/d18/d1a/d2b/d4f 0 2024-03-16T10:45:12.924 INFO:tasks.workunit.client.0.smithi002.stdout:2/959: readlink d7/d2e/d1d/d38/l89 0 2024-03-16T10:45:12.924 INFO:tasks.workunit.client.0.smithi002.stdout:5/983: dwrite d0/d16/dab/feb [0,4194304] 0 2024-03-16T10:45:12.925 INFO:tasks.workunit.client.1.smithi026.stdout:3/263: mknod d4/d29/c65 0 2024-03-16T10:45:12.927 INFO:tasks.workunit.client.1.smithi026.stdout:2/230: creat d2/f4d x:0 0 0 2024-03-16T10:45:12.928 INFO:tasks.workunit.client.1.smithi026.stdout:2/231: write d2/d29/d17/d1a/d2b/d2e/f33 [756730,10302] 0 2024-03-16T10:45:12.930 INFO:tasks.workunit.client.1.smithi026.stdout:7/260: mknod d6/de/d16/d44/c4d 0 2024-03-16T10:45:12.931 INFO:tasks.workunit.client.1.smithi026.stdout:7/261: write d6/d1e/f29 [2440490,13280] 0 2024-03-16T10:45:12.932 INFO:tasks.workunit.client.0.smithi002.stdout:5/984: dwrite d0/d16/dab/d2b/d94/f10d [4194304,4194304] 0 2024-03-16T10:45:12.933 INFO:tasks.workunit.client.0.smithi002.stdout:5/985: stat d0/d16/dab/le7 0 2024-03-16T10:45:12.934 INFO:tasks.workunit.client.0.smithi002.stdout:7/900: rmdir d3/d4/d1d/d87/d5c/d63/d3f/dba/dcc/d119 0 2024-03-16T10:45:12.936 INFO:tasks.workunit.client.1.smithi026.stdout:8/210: symlink d5/dc/l3a 0 2024-03-16T10:45:12.941 INFO:tasks.workunit.client.0.smithi002.stdout:5/986: fsync d0/d16/dab/d33/d8c/f98 0 2024-03-16T10:45:12.941 INFO:tasks.workunit.client.0.smithi002.stdout:5/987: fsync d0/d16/dab/f85 0 2024-03-16T10:45:12.943 INFO:tasks.workunit.client.0.smithi002.stdout:5/988: write d0/d24/dad/db6/db9/fd4 [381352,67785] 0 2024-03-16T10:45:12.944 INFO:tasks.workunit.client.1.smithi026.stdout:1/273: stat d0/d33/d2e/l39 0 2024-03-16T10:45:12.945 INFO:tasks.workunit.client.0.smithi002.stdout:2/960: link d7/d2e/d28/d40/f5a d7/d2e/d28/f134 0 2024-03-16T10:45:12.947 INFO:tasks.workunit.client.1.smithi026.stdout:0/257: unlink d1/f37 0 2024-03-16T10:45:12.952 INFO:tasks.workunit.client.1.smithi026.stdout:6/285: creat d0/d10/f67 x:0 0 0 2024-03-16T10:45:12.954 INFO:tasks.workunit.client.1.smithi026.stdout:2/232: rename d2/d29/d3b/d41 to d2/d29/d17/d1a/d2b/d4e 0 2024-03-16T10:45:12.955 INFO:tasks.workunit.client.0.smithi002.stdout:5/989: chown d0/d1f/lb7 0 1 2024-03-16T10:45:12.957 INFO:tasks.workunit.client.0.smithi002.stdout:2/961: symlink d7/d27/d3b/d95/da5/l135 0 2024-03-16T10:45:12.957 INFO:tasks.workunit.client.0.smithi002.stdout:2/962: chown d7/d27/d3b/d95/f128 67021468 1 2024-03-16T10:45:12.958 INFO:tasks.workunit.client.1.smithi026.stdout:7/262: mkdir d6/d35/d3a/d4e 0 2024-03-16T10:45:12.959 INFO:tasks.workunit.client.0.smithi002.stdout:7/901: rename d3/d4/dde/l121 to d3/d4/d1d/l146 0 2024-03-16T10:45:12.960 INFO:tasks.workunit.client.1.smithi026.stdout:5/185: dwrite d1/db/d27/f23 [0,4194304] 0 2024-03-16T10:45:12.962 INFO:tasks.workunit.client.1.smithi026.stdout:1/274: chown d0/l1d 13 1 2024-03-16T10:45:12.963 INFO:tasks.workunit.client.0.smithi002.stdout:2/963: dwrite d7/d2e/d1d/d107/d10e/f5d [0,4194304] 0 2024-03-16T10:45:12.965 INFO:tasks.workunit.client.0.smithi002.stdout:5/990: mknod d0/d16/dab/d2b/c12e 0 2024-03-16T10:45:12.966 INFO:tasks.workunit.client.0.smithi002.stdout:5/991: fsync d0/d1f/fbd 0 2024-03-16T10:45:12.966 INFO:tasks.workunit.client.1.smithi026.stdout:3/264: truncate d4/d29/f30 456297 0 2024-03-16T10:45:12.967 INFO:tasks.workunit.client.0.smithi002.stdout:5/992: chown d0/d24/dad/db6 9799420 1 2024-03-16T10:45:12.968 INFO:tasks.workunit.client.0.smithi002.stdout:2/964: dread - d7/d2e/d1d/fd3 zero size 2024-03-16T10:45:12.968 INFO:tasks.workunit.client.0.smithi002.stdout:5/993: stat d0/d16/d4a/f108 0 2024-03-16T10:45:12.968 INFO:tasks.workunit.client.1.smithi026.stdout:3/265: write d4/dc/d31/d48/f58 [282143,123472] 0 2024-03-16T10:45:12.970 INFO:tasks.workunit.client.0.smithi002.stdout:2/965: truncate d7/d2e/d28/d40/d50/d132/df1/f113 854716 0 2024-03-16T10:45:12.972 INFO:tasks.workunit.client.1.smithi026.stdout:4/266: creat d1/f56 x:0 0 0 2024-03-16T10:45:12.973 INFO:tasks.workunit.client.0.smithi002.stdout:4/925: sync 2024-03-16T10:45:12.974 INFO:tasks.workunit.client.1.smithi026.stdout:4/267: write d1/f8 [2314933,109679] 0 2024-03-16T10:45:12.974 INFO:tasks.workunit.client.0.smithi002.stdout:7/902: rename d3/d4/d1d/d87/d5c/d63/f7f to d3/d12b/f147 0 2024-03-16T10:45:12.976 INFO:tasks.workunit.client.0.smithi002.stdout:2/966: creat d7/d2e/d28/d40/d50/db4/f136 x:0 0 0 2024-03-16T10:45:12.976 INFO:tasks.workunit.client.1.smithi026.stdout:1/275: mknod d0/d33/d2e/c57 0 2024-03-16T10:45:12.977 INFO:tasks.workunit.client.0.smithi002.stdout:4/926: symlink d2/d6/l11d 0 2024-03-16T10:45:12.978 INFO:tasks.workunit.client.1.smithi026.stdout:0/258: mknod d1/d18/c50 0 2024-03-16T10:45:12.980 INFO:tasks.workunit.client.0.smithi002.stdout:7/903: dread d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f11a [0,4194304] 0 2024-03-16T10:45:12.980 INFO:tasks.workunit.client.0.smithi002.stdout:2/967: creat d7/d2e/d46/f137 x:0 0 0 2024-03-16T10:45:12.981 INFO:tasks.workunit.client.0.smithi002.stdout:7/904: chown d3/d4/d1d/d87/d5c/d63/d3f/dba/dcc 11 1 2024-03-16T10:45:12.982 INFO:tasks.workunit.client.0.smithi002.stdout:4/927: mkdir d2/d33/d35/d11e 0 2024-03-16T10:45:12.982 INFO:tasks.workunit.client.0.smithi002.stdout:7/905: fsync d3/f15 0 2024-03-16T10:45:12.983 INFO:tasks.workunit.client.0.smithi002.stdout:4/928: truncate d2/d33/d1c/d45/f119 809376 0 2024-03-16T10:45:12.984 INFO:tasks.workunit.client.1.smithi026.stdout:3/266: rmdir d4/dc/dd/d60/d62/d4f 39 2024-03-16T10:45:12.984 INFO:tasks.workunit.client.1.smithi026.stdout:3/267: fdatasync d4/dc/f4e 0 2024-03-16T10:45:12.985 INFO:tasks.workunit.client.0.smithi002.stdout:4/929: chown d2/d6/d32/d66/d7f/de7/f49 10344927 1 2024-03-16T10:45:12.989 INFO:tasks.workunit.client.0.smithi002.stdout:7/906: unlink d3/d4/d1d/d87/d5c/lbb 0 2024-03-16T10:45:12.994 INFO:tasks.workunit.client.0.smithi002.stdout:2/968: dread d7/d2e/d1d/d38/d49/fe5 [0,4194304] 0 2024-03-16T10:45:12.995 INFO:tasks.workunit.client.0.smithi002.stdout:7/907: unlink d3/d4/d1d/d87/d5c/d63/ffa 0 2024-03-16T10:45:12.996 INFO:tasks.workunit.client.0.smithi002.stdout:7/908: write d3/d4/f12e [232296,58091] 0 2024-03-16T10:45:12.997 INFO:tasks.workunit.client.0.smithi002.stdout:7/909: fsync d3/d12b/f110 0 2024-03-16T10:45:12.997 INFO:tasks.workunit.client.1.smithi026.stdout:4/268: mknod d1/db/de/c57 0 2024-03-16T10:45:13.003 INFO:tasks.workunit.client.1.smithi026.stdout:6/286: truncate d0/f1d 2555449 0 2024-03-16T10:45:13.004 INFO:tasks.workunit.client.1.smithi026.stdout:1/276: dwrite d0/f1e [0,4194304] 0 2024-03-16T10:45:13.006 INFO:tasks.workunit.client.1.smithi026.stdout:2/233: link d2/d29/d17/c43 d2/d29/d17/d1a/d2b/d4e/c4f 0 2024-03-16T10:45:13.009 INFO:tasks.workunit.client.1.smithi026.stdout:6/287: dread d0/d28/d49/f5a [0,4194304] 0 2024-03-16T10:45:13.010 INFO:tasks.workunit.client.1.smithi026.stdout:8/211: getdents d5/dc/d1f 0 2024-03-16T10:45:13.015 INFO:tasks.workunit.client.1.smithi026.stdout:4/269: creat d1/db/de/f58 x:0 0 0 2024-03-16T10:45:13.015 INFO:tasks.workunit.client.1.smithi026.stdout:4/270: stat d1/db/de/d4c 0 2024-03-16T10:45:13.018 INFO:tasks.workunit.client.0.smithi002.stdout:4/930: mkdir d2/d33/d35/d59/d11f 0 2024-03-16T10:45:13.019 INFO:tasks.workunit.client.1.smithi026.stdout:0/259: mkdir d1/d18/d1a/d24/d51 0 2024-03-16T10:45:13.022 INFO:tasks.workunit.client.1.smithi026.stdout:4/271: dwrite d1/db/de/f3d [4194304,4194304] 0 2024-03-16T10:45:13.023 INFO:tasks.workunit.client.0.smithi002.stdout:7/910: write d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/f82 [766022,26828] 0 2024-03-16T10:45:13.024 INFO:tasks.workunit.client.1.smithi026.stdout:4/272: chown d1/db/de/c50 8154 1 2024-03-16T10:45:13.025 INFO:tasks.workunit.client.1.smithi026.stdout:4/273: readlink d1/db/de/l25 0 2024-03-16T10:45:13.026 INFO:tasks.workunit.client.1.smithi026.stdout:1/277: mkdir d0/d33/d3b/d49/d58 0 2024-03-16T10:45:13.026 INFO:tasks.workunit.client.1.smithi026.stdout:4/274: write d1/db/de/f58 [438290,78975] 0 2024-03-16T10:45:13.027 INFO:tasks.workunit.client.1.smithi026.stdout:4/275: chown d1/db/d29/d44 471305426 1 2024-03-16T10:45:13.029 INFO:tasks.workunit.client.0.smithi002.stdout:2/969: dwrite d7/db/f35 [0,4194304] 0 2024-03-16T10:45:13.029 INFO:tasks.workunit.client.0.smithi002.stdout:4/931: rename d2/d6/d32/d66/cee to d2/d6/d32/d66/d7f/de7/d11a/c120 0 2024-03-16T10:45:13.030 INFO:tasks.workunit.client.1.smithi026.stdout:2/234: creat d2/d29/d17/d1a/d2b/d2e/f50 x:0 0 0 2024-03-16T10:45:13.030 INFO:tasks.workunit.client.0.smithi002.stdout:2/970: dread - d7/d2e/d28/f112 zero size 2024-03-16T10:45:13.031 INFO:tasks.workunit.client.0.smithi002.stdout:7/911: read - d3/d61/faf zero size 2024-03-16T10:45:13.031 INFO:tasks.workunit.client.1.smithi026.stdout:2/235: fdatasync d2/d29/d17/d1a/d2b/d2e/f33 0 2024-03-16T10:45:13.032 INFO:tasks.workunit.client.1.smithi026.stdout:2/236: chown c1 332 1 2024-03-16T10:45:13.032 INFO:tasks.workunit.client.1.smithi026.stdout:2/237: stat d2/d29/d17/c3d 0 2024-03-16T10:45:13.034 INFO:tasks.workunit.client.1.smithi026.stdout:6/288: rename d0/f11 to d0/d28/d2d/f68 0 2024-03-16T10:45:13.036 INFO:tasks.workunit.client.1.smithi026.stdout:3/268: truncate d4/dc/f25 2145631 0 2024-03-16T10:45:13.037 INFO:tasks.workunit.client.1.smithi026.stdout:8/212: unlink d5/d7/d2c/c30 0 2024-03-16T10:45:13.045 INFO:tasks.workunit.client.1.smithi026.stdout:3/269: dwrite d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:13.054 INFO:tasks.workunit.client.1.smithi026.stdout:3/270: dwrite d4/dc/dd/d3d/f49 [0,4194304] 0 2024-03-16T10:45:13.063 INFO:tasks.workunit.client.1.smithi026.stdout:3/271: dwrite d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:13.072 INFO:tasks.workunit.client.1.smithi026.stdout:8/213: dread d5/dc/f15 [0,4194304] 0 2024-03-16T10:45:13.081 INFO:tasks.workunit.client.0.smithi002.stdout:5/994: sync 2024-03-16T10:45:13.082 INFO:tasks.workunit.client.0.smithi002.stdout:5/995: truncate d0/d24/dad/db6/db9/fda 674903 0 2024-03-16T10:45:13.083 INFO:tasks.workunit.client.0.smithi002.stdout:5/996: dread - d0/d16/d4a/d7d/f100 zero size 2024-03-16T10:45:13.084 INFO:tasks.workunit.client.0.smithi002.stdout:5/997: truncate d0/d24/fee 270144 0 2024-03-16T10:45:13.090 INFO:tasks.workunit.client.1.smithi026.stdout:5/186: getdents d1 0 2024-03-16T10:45:13.151 INFO:tasks.workunit.client.0.smithi002.stdout:4/932: unlink d2/d33/c55 0 2024-03-16T10:45:13.151 INFO:tasks.workunit.client.0.smithi002.stdout:4/933: chown d2/f77 425352 1 2024-03-16T10:45:13.154 INFO:tasks.workunit.client.0.smithi002.stdout:5/998: readlink d0/d16/dab/daa/l104 0 2024-03-16T10:45:13.155 INFO:tasks.workunit.client.0.smithi002.stdout:5/999: write d0/d1f/f2c [69793,112730] 0 2024-03-16T10:45:13.175 INFO:tasks.workunit.client.1.smithi026.stdout:9/206: sync 2024-03-16T10:45:13.175 INFO:tasks.workunit.client.1.smithi026.stdout:7/263: sync 2024-03-16T10:45:13.176 INFO:tasks.workunit.client.0.smithi002.stdout:4/934: fdatasync d2/d33/f94 0 2024-03-16T10:45:13.179 INFO:tasks.workunit.client.1.smithi026.stdout:6/289: creat d0/d41/d4f/f69 x:0 0 0 2024-03-16T10:45:13.180 INFO:tasks.workunit.client.1.smithi026.stdout:6/290: stat d0/d10/d47 0 2024-03-16T10:45:13.184 INFO:tasks.workunit.client.0.smithi002.stdout:4/935: fsync d2/d6/fd6 0 2024-03-16T10:45:13.187 INFO:tasks.workunit.client.1.smithi026.stdout:2/238: dread d2/d29/d17/f27 [0,4194304] 0 2024-03-16T10:45:13.188 INFO:tasks.workunit.client.1.smithi026.stdout:2/239: read d2/d29/d17/d1a/d2b/f1f [5313528,23676] 0 2024-03-16T10:45:13.190 INFO:tasks.workunit.client.0.smithi002.stdout:4/936: dwrite d2/d33/d1c/d45/f115 [0,4194304] 0 2024-03-16T10:45:13.191 INFO:tasks.workunit.client.0.smithi002.stdout:4/937: write d2/d33/d1c/d45/f115 [3753411,26524] 0 2024-03-16T10:45:13.196 INFO:tasks.workunit.client.1.smithi026.stdout:8/214: dread d5/d7/f1c [0,4194304] 0 2024-03-16T10:45:13.197 INFO:tasks.workunit.client.1.smithi026.stdout:0/260: creat d1/d18/d1a/d2b/d4f/f52 x:0 0 0 2024-03-16T10:45:13.198 INFO:tasks.workunit.client.1.smithi026.stdout:8/215: dread d5/dc/f29 [0,4194304] 0 2024-03-16T10:45:13.199 INFO:tasks.workunit.client.1.smithi026.stdout:8/216: write d5/d7/f1b [421132,34921] 0 2024-03-16T10:45:13.202 INFO:tasks.workunit.client.1.smithi026.stdout:8/217: truncate d5/f2e 566895 0 2024-03-16T10:45:13.204 INFO:tasks.workunit.client.0.smithi002.stdout:4/938: mknod d2/d6/d32/d66/d7f/de7/d2e/c121 0 2024-03-16T10:45:13.206 INFO:tasks.workunit.client.1.smithi026.stdout:0/261: dwrite d1/d12/d2f/d39/f3e [0,4194304] 0 2024-03-16T10:45:13.207 INFO:tasks.workunit.client.1.smithi026.stdout:7/264: mknod d6/d35/d3a/c4f 0 2024-03-16T10:45:13.208 INFO:tasks.workunit.client.1.smithi026.stdout:0/262: truncate d1/d12/d2f/d48/f4c 475407 0 2024-03-16T10:45:13.217 INFO:tasks.workunit.client.1.smithi026.stdout:4/276: rmdir d1/db/de/d4c/d53 0 2024-03-16T10:45:13.222 INFO:tasks.workunit.client.1.smithi026.stdout:6/291: dread d0/d10/f23 [0,4194304] 0 2024-03-16T10:45:13.222 INFO:tasks.workunit.client.1.smithi026.stdout:6/292: chown d0/d28/f40 2 1 2024-03-16T10:45:13.223 INFO:tasks.workunit.client.1.smithi026.stdout:9/207: creat d0/da/d16/d17/d19/f42 x:0 0 0 2024-03-16T10:45:13.223 INFO:tasks.workunit.client.1.smithi026.stdout:9/208: fdatasync d0/da/d13/f38 0 2024-03-16T10:45:13.225 INFO:tasks.workunit.client.1.smithi026.stdout:0/263: creat d1/d18/d1a/d2b/d4f/f53 x:0 0 0 2024-03-16T10:45:13.226 INFO:tasks.workunit.client.1.smithi026.stdout:0/264: dread - d1/d18/d1a/d2b/d4f/f52 zero size 2024-03-16T10:45:13.227 INFO:tasks.workunit.client.1.smithi026.stdout:0/265: write d1/d12/d2f/d39/f3e [2240575,5168] 0 2024-03-16T10:45:13.229 INFO:tasks.workunit.client.1.smithi026.stdout:5/187: creat d1/d1c/d2a/d31/f3a x:0 0 0 2024-03-16T10:45:13.233 INFO:tasks.workunit.client.1.smithi026.stdout:5/188: dread d1/db/fc [4194304,4194304] 0 2024-03-16T10:45:13.235 INFO:tasks.workunit.client.1.smithi026.stdout:0/266: rmdir d1/d12/d2f/d48 39 2024-03-16T10:45:13.236 INFO:tasks.workunit.client.1.smithi026.stdout:0/267: dread - d1/d18/d1a/d2b/d4f/f52 zero size 2024-03-16T10:45:13.241 INFO:tasks.workunit.client.1.smithi026.stdout:5/189: unlink d1/db/d27/f1b 0 2024-03-16T10:45:13.244 INFO:tasks.workunit.client.1.smithi026.stdout:9/209: link d0/c6 d0/da/d16/c43 0 2024-03-16T10:45:13.246 INFO:tasks.workunit.client.1.smithi026.stdout:9/210: write d0/da/d16/d17/d19/f2b [636084,57837] 0 2024-03-16T10:45:13.247 INFO:tasks.workunit.client.1.smithi026.stdout:5/190: dwrite d1/f13 [0,4194304] 0 2024-03-16T10:45:13.248 INFO:tasks.workunit.client.1.smithi026.stdout:9/211: stat d0/da/f27 0 2024-03-16T10:45:13.252 INFO:tasks.workunit.client.1.smithi026.stdout:5/191: creat d1/d1c/d2a/f3b x:0 0 0 2024-03-16T10:45:13.256 INFO:tasks.workunit.client.1.smithi026.stdout:0/268: dwrite d1/d12/d2f/d48/f4c [0,4194304] 0 2024-03-16T10:45:13.256 INFO:tasks.workunit.client.1.smithi026.stdout:5/192: dread d1/db/f22 [0,4194304] 0 2024-03-16T10:45:13.261 INFO:tasks.workunit.client.1.smithi026.stdout:0/269: write d1/d18/d1a/f26 [4049536,80371] 0 2024-03-16T10:45:13.266 INFO:tasks.workunit.client.1.smithi026.stdout:0/270: write d1/d18/d1a/d24/d6/f31 [2875473,29548] 0 2024-03-16T10:45:13.271 INFO:tasks.workunit.client.1.smithi026.stdout:0/271: rename d1/d12/d2f/d48/f4c to d1/d18/d1a/d24/d11/f54 0 2024-03-16T10:45:13.273 INFO:tasks.workunit.client.1.smithi026.stdout:0/272: mkdir d1/d12/d2f/d48/d55 0 2024-03-16T10:45:13.275 INFO:tasks.workunit.client.1.smithi026.stdout:0/273: rmdir d1/d18/d1a/d24/d6 39 2024-03-16T10:45:13.277 INFO:tasks.workunit.client.1.smithi026.stdout:0/274: unlink d1/d18/d1a/d24/d11/c14 0 2024-03-16T10:45:13.279 INFO:tasks.workunit.client.1.smithi026.stdout:0/275: write d1/d18/d1a/f46 [583424,41503] 0 2024-03-16T10:45:13.284 INFO:tasks.workunit.client.1.smithi026.stdout:0/276: link d1/f15 d1/d18/d1a/d2b/d4f/f56 0 2024-03-16T10:45:13.284 INFO:tasks.workunit.client.1.smithi026.stdout:0/277: dread - d1/d18/d1a/d2b/f4e zero size 2024-03-16T10:45:13.285 INFO:tasks.workunit.client.1.smithi026.stdout:0/278: chown d1/d18/d1a/d24/d11/f2d 40263 1 2024-03-16T10:45:13.292 INFO:tasks.workunit.client.1.smithi026.stdout:0/279: rename d1/d12/d2f/d39 to d1/d18/d1a/d24/d57 0 2024-03-16T10:45:13.292 INFO:tasks.workunit.client.1.smithi026.stdout:0/280: readlink d1/d18/d1a/d24/l3f 0 2024-03-16T10:45:13.294 INFO:tasks.workunit.client.1.smithi026.stdout:0/281: truncate d1/d18/d1a/d2b/d4f/f53 115616 0 2024-03-16T10:45:13.298 INFO:tasks.workunit.client.1.smithi026.stdout:0/282: mknod d1/d18/d1a/d24/d6/c58 0 2024-03-16T10:45:13.298 INFO:tasks.workunit.client.1.smithi026.stdout:0/283: stat d1/d12/f4a 0 2024-03-16T10:45:13.303 INFO:tasks.workunit.client.1.smithi026.stdout:0/284: link d1/d12/f4a d1/d18/d1a/d24/f59 0 2024-03-16T10:45:13.306 INFO:tasks.workunit.client.1.smithi026.stdout:0/285: write d1/d12/f4a [495529,65115] 0 2024-03-16T10:45:13.311 INFO:tasks.workunit.client.1.smithi026.stdout:0/286: creat d1/d18/d1a/d24/d6/f5a x:0 0 0 2024-03-16T10:45:13.311 INFO:tasks.workunit.client.0.smithi002.stdout:2/971: write d7/d27/d3b/f11a [821313,73675] 0 2024-03-16T10:45:13.312 INFO:tasks.workunit.client.0.smithi002.stdout:2/972: dread - d7/d27/d3b/d95/f128 zero size 2024-03-16T10:45:13.313 INFO:tasks.workunit.client.0.smithi002.stdout:2/973: dread - d7/d2e/d46/f10f zero size 2024-03-16T10:45:13.313 INFO:tasks.workunit.client.1.smithi026.stdout:0/287: fdatasync d1/d18/d1a/d2b/d4f/f56 0 2024-03-16T10:45:13.314 INFO:tasks.workunit.client.0.smithi002.stdout:7/912: truncate d3/d4/d1d/d87/d5c/ddb/def/ffc 3987818 0 2024-03-16T10:45:13.316 INFO:tasks.workunit.client.1.smithi026.stdout:1/278: dwrite d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:13.318 INFO:tasks.workunit.client.1.smithi026.stdout:1/279: dread - d0/d33/d2e/f56 zero size 2024-03-16T10:45:13.320 INFO:tasks.workunit.client.0.smithi002.stdout:7/913: dwrite d3/d4/d1d/d87/d5c/dd0/f133 [0,4194304] 0 2024-03-16T10:45:13.322 INFO:tasks.workunit.client.1.smithi026.stdout:1/280: rmdir d0/d33/d2e 39 2024-03-16T10:45:13.323 INFO:tasks.workunit.client.1.smithi026.stdout:0/288: dwrite d1/d18/d1a/d24/d6/f28 [0,4194304] 0 2024-03-16T10:45:13.334 INFO:tasks.workunit.client.1.smithi026.stdout:0/289: dwrite d1/d18/d1a/d2b/f32 [0,4194304] 0 2024-03-16T10:45:13.339 INFO:tasks.workunit.client.1.smithi026.stdout:0/290: mknod d1/d18/d1a/d24/d11/c5b 0 2024-03-16T10:45:13.340 INFO:tasks.workunit.client.1.smithi026.stdout:1/281: creat d0/d33/f59 x:0 0 0 2024-03-16T10:45:13.342 INFO:tasks.workunit.client.1.smithi026.stdout:1/282: write d0/d17/d1f/f35 [2511811,109842] 0 2024-03-16T10:45:13.343 INFO:tasks.workunit.client.1.smithi026.stdout:1/283: read d0/d17/f37 [2525285,24980] 0 2024-03-16T10:45:13.347 INFO:tasks.workunit.client.0.smithi002.stdout:7/914: fsync d3/d4/d1d/d87/d5c/dd0/f133 0 2024-03-16T10:45:13.356 INFO:tasks.workunit.client.0.smithi002.stdout:4/939: dwrite d2/d6/d32/d66/d7f/f96 [0,4194304] 0 2024-03-16T10:45:13.359 INFO:tasks.workunit.client.1.smithi026.stdout:9/212: rename d0/da/d16/d17/d19 to d0/da/d16/d17/d44 0 2024-03-16T10:45:13.367 INFO:tasks.workunit.client.1.smithi026.stdout:8/218: truncate d5/d7/d2c/f2d 7081206 0 2024-03-16T10:45:13.368 INFO:tasks.workunit.client.1.smithi026.stdout:6/293: write d0/d5b/f18 [2028714,4668] 0 2024-03-16T10:45:13.376 INFO:tasks.workunit.client.1.smithi026.stdout:8/219: link d5/c27 d5/d7/d2c/d36/c3b 0 2024-03-16T10:45:13.377 INFO:tasks.workunit.client.1.smithi026.stdout:8/220: stat d5/d7/d2c/d36 0 2024-03-16T10:45:13.380 INFO:tasks.workunit.client.1.smithi026.stdout:6/294: dread d0/d28/f37 [0,4194304] 0 2024-03-16T10:45:13.383 INFO:tasks.workunit.client.1.smithi026.stdout:8/221: truncate d5/dc/f15 1351489 0 2024-03-16T10:45:13.383 INFO:tasks.workunit.client.1.smithi026.stdout:8/222: fdatasync d5/dc/f38 0 2024-03-16T10:45:13.384 INFO:tasks.workunit.client.1.smithi026.stdout:0/291: rename d1/d18/d1a/d2b/d4f/f52 to d1/d18/f5c 0 2024-03-16T10:45:13.385 INFO:tasks.workunit.client.1.smithi026.stdout:8/223: truncate d5/d7/f10 9090609 0 2024-03-16T10:45:13.386 INFO:tasks.workunit.client.1.smithi026.stdout:0/292: write d1/d12/f45 [165856,62622] 0 2024-03-16T10:45:13.388 INFO:tasks.workunit.client.1.smithi026.stdout:8/224: dread d5/d7/f1b [0,4194304] 0 2024-03-16T10:45:13.389 INFO:tasks.workunit.client.1.smithi026.stdout:6/295: rename d0/d41/d4f/c5c to d0/d41/c6a 0 2024-03-16T10:45:13.390 INFO:tasks.workunit.client.1.smithi026.stdout:5/193: truncate d1/db/f12 436747 0 2024-03-16T10:45:13.390 INFO:tasks.workunit.client.1.smithi026.stdout:8/225: stat d5/d7/d2c 0 2024-03-16T10:45:13.394 INFO:tasks.workunit.client.1.smithi026.stdout:0/293: dwrite d1/d18/d1a/d2b/d4f/f53 [0,4194304] 0 2024-03-16T10:45:13.402 INFO:tasks.workunit.client.1.smithi026.stdout:5/194: dread - d1/db/f17 zero size 2024-03-16T10:45:13.407 INFO:tasks.workunit.client.1.smithi026.stdout:8/226: rename d5/dc/fe to d5/d28/f3c 0 2024-03-16T10:45:13.408 INFO:tasks.workunit.client.1.smithi026.stdout:8/227: truncate d5/dc/f38 618874 0 2024-03-16T10:45:13.411 INFO:tasks.workunit.client.1.smithi026.stdout:1/284: creat d0/f5a x:0 0 0 2024-03-16T10:45:13.414 INFO:tasks.workunit.client.1.smithi026.stdout:8/228: dwrite d5/f2e [0,4194304] 0 2024-03-16T10:45:13.417 INFO:tasks.workunit.client.1.smithi026.stdout:7/265: symlink d6/l50 0 2024-03-16T10:45:13.418 INFO:tasks.workunit.client.1.smithi026.stdout:0/294: creat d1/d18/d1a/d24/d51/f5d x:0 0 0 2024-03-16T10:45:13.423 INFO:tasks.workunit.client.1.smithi026.stdout:2/240: sync 2024-03-16T10:45:13.424 INFO:tasks.workunit.client.1.smithi026.stdout:3/272: creat d4/dc/f66 x:0 0 0 2024-03-16T10:45:13.426 INFO:tasks.workunit.client.1.smithi026.stdout:6/296: rmdir d0/d10/d47 0 2024-03-16T10:45:13.432 INFO:tasks.workunit.client.1.smithi026.stdout:3/273: dwrite d4/dc/d31/d48/f58 [0,4194304] 0 2024-03-16T10:45:13.433 INFO:tasks.workunit.client.1.smithi026.stdout:7/266: creat d6/d35/d3a/d3c/f51 x:0 0 0 2024-03-16T10:45:13.434 INFO:tasks.workunit.client.1.smithi026.stdout:3/274: chown d4/d29/c44 102628 1 2024-03-16T10:45:13.436 INFO:tasks.workunit.client.1.smithi026.stdout:3/275: truncate d4/dc/dd/d4b/f5b 106920 0 2024-03-16T10:45:13.437 INFO:tasks.workunit.client.0.smithi002.stdout:2/974: write f6 [4775911,48525] 0 2024-03-16T10:45:13.442 INFO:tasks.workunit.client.1.smithi026.stdout:6/297: creat d0/d41/d4f/f6b x:0 0 0 2024-03-16T10:45:13.443 INFO:tasks.workunit.client.1.smithi026.stdout:6/298: stat d0/d41/d4f/c50 0 2024-03-16T10:45:13.444 INFO:tasks.workunit.client.1.smithi026.stdout:5/195: creat d1/d1c/f3c x:0 0 0 2024-03-16T10:45:13.451 INFO:tasks.workunit.client.1.smithi026.stdout:0/295: rename d1/f15 to d1/d18/d1a/d24/d51/f5e 0 2024-03-16T10:45:13.452 INFO:tasks.workunit.client.1.smithi026.stdout:0/296: dread - d1/d18/d1a/d2b/f4e zero size 2024-03-16T10:45:13.453 INFO:tasks.workunit.client.1.smithi026.stdout:3/276: mknod d4/dc/dd/d60/c67 0 2024-03-16T10:45:13.454 INFO:tasks.workunit.client.1.smithi026.stdout:3/277: truncate d4/dc/f20 93924 0 2024-03-16T10:45:13.456 INFO:tasks.workunit.client.1.smithi026.stdout:2/241: mkdir d2/d29/d51 0 2024-03-16T10:45:13.458 INFO:tasks.workunit.client.1.smithi026.stdout:9/213: write d0/f1f [1093420,112770] 0 2024-03-16T10:45:13.459 INFO:tasks.workunit.client.1.smithi026.stdout:9/214: write d0/da/d13/f31 [1030943,41292] 0 2024-03-16T10:45:13.461 INFO:tasks.workunit.client.1.smithi026.stdout:9/215: truncate d0/da/d16/d17/f40 195567 0 2024-03-16T10:45:13.462 INFO:tasks.workunit.client.1.smithi026.stdout:9/216: write d0/da/f11 [3141754,7612] 0 2024-03-16T10:45:13.466 INFO:tasks.workunit.client.1.smithi026.stdout:0/297: mkdir d1/d18/d1a/d24/d51/d5f 0 2024-03-16T10:45:13.469 INFO:tasks.workunit.client.1.smithi026.stdout:3/278: symlink d4/dc/dd/d3d/l68 0 2024-03-16T10:45:13.470 INFO:tasks.workunit.client.1.smithi026.stdout:2/242: dread d2/d29/d17/d1a/d2b/d2e/f31 [0,4194304] 0 2024-03-16T10:45:13.473 INFO:tasks.workunit.client.1.smithi026.stdout:4/277: sync 2024-03-16T10:45:13.479 INFO:tasks.workunit.client.1.smithi026.stdout:7/267: creat d6/d35/f52 x:0 0 0 2024-03-16T10:45:13.480 INFO:tasks.workunit.client.1.smithi026.stdout:3/279: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:13.481 INFO:tasks.workunit.client.1.smithi026.stdout:7/268: write d6/de/d16/f19 [3480648,95400] 0 2024-03-16T10:45:13.483 INFO:tasks.workunit.client.1.smithi026.stdout:7/269: write d6/de/d16/f18 [1970333,21480] 0 2024-03-16T10:45:13.484 INFO:tasks.workunit.client.1.smithi026.stdout:7/270: write d6/d35/f52 [783603,77772] 0 2024-03-16T10:45:13.486 INFO:tasks.workunit.client.1.smithi026.stdout:0/298: rename d1/d18/d1a/d24/d6 to d1/d12/d2f/d48/d55/d60 0 2024-03-16T10:45:13.488 INFO:tasks.workunit.client.1.smithi026.stdout:2/243: symlink d2/d29/d3b/l52 0 2024-03-16T10:45:13.492 INFO:tasks.workunit.client.1.smithi026.stdout:0/299: dwrite d1/d18/d1a/f2c [8388608,4194304] 0 2024-03-16T10:45:13.494 INFO:tasks.workunit.client.1.smithi026.stdout:6/299: creat d0/d10/f6c x:0 0 0 2024-03-16T10:45:13.495 INFO:tasks.workunit.client.1.smithi026.stdout:4/278: mkdir d1/db/de/d4c/d59 0 2024-03-16T10:45:13.497 INFO:tasks.workunit.client.1.smithi026.stdout:4/279: read d1/f3 [4408994,73496] 0 2024-03-16T10:45:13.500 INFO:tasks.workunit.client.0.smithi002.stdout:4/940: getdents d2/d6/d32/d68/d108 0 2024-03-16T10:45:13.501 INFO:tasks.workunit.client.1.smithi026.stdout:6/300: dwrite d0/f2c [0,4194304] 0 2024-03-16T10:45:13.502 INFO:tasks.workunit.client.1.smithi026.stdout:6/301: dread - d0/f5f zero size 2024-03-16T10:45:13.504 INFO:tasks.workunit.client.0.smithi002.stdout:7/915: link d3/d4/d1c/l51 d3/d4/d1d/d87/d96/d5e/db9/l148 0 2024-03-16T10:45:13.505 INFO:tasks.workunit.client.0.smithi002.stdout:4/941: chown d2/d33/f6a 35431 1 2024-03-16T10:45:13.508 INFO:tasks.workunit.client.1.smithi026.stdout:3/280: mkdir d4/dc/d31/d48/d69 0 2024-03-16T10:45:13.509 INFO:tasks.workunit.client.1.smithi026.stdout:3/281: fdatasync d4/dc/dd/d60/f64 0 2024-03-16T10:45:13.512 INFO:tasks.workunit.client.1.smithi026.stdout:7/271: creat d6/f53 x:0 0 0 2024-03-16T10:45:13.515 INFO:tasks.workunit.client.1.smithi026.stdout:3/282: dread d4/dc/dd/d60/d62/f52 [0,4194304] 0 2024-03-16T10:45:13.521 INFO:tasks.workunit.client.1.smithi026.stdout:5/196: getdents d1/db/d27 0 2024-03-16T10:45:13.522 INFO:tasks.workunit.client.1.smithi026.stdout:5/197: rename d1 to d1/d1c/d2a/d3d 22 2024-03-16T10:45:13.522 INFO:tasks.workunit.client.1.smithi026.stdout:5/198: stat d1/c3 0 2024-03-16T10:45:13.523 INFO:tasks.workunit.client.1.smithi026.stdout:9/217: creat d0/da/f45 x:0 0 0 2024-03-16T10:45:13.525 INFO:tasks.workunit.client.1.smithi026.stdout:6/302: creat d0/d62/f6d x:0 0 0 2024-03-16T10:45:13.526 INFO:tasks.workunit.client.1.smithi026.stdout:6/303: chown d0/d28/d49/f52 1320738 1 2024-03-16T10:45:13.527 INFO:tasks.workunit.client.1.smithi026.stdout:7/272: mkdir d6/d35/d3a/d3c/d3f/d54 0 2024-03-16T10:45:13.527 INFO:tasks.workunit.client.1.smithi026.stdout:1/285: write d0/d33/d3b/f27 [2526536,37886] 0 2024-03-16T10:45:13.528 INFO:tasks.workunit.client.1.smithi026.stdout:3/283: creat d4/dc/dd/d3d/f6a x:0 0 0 2024-03-16T10:45:13.529 INFO:tasks.workunit.client.1.smithi026.stdout:3/284: fdatasync d4/dc/f43 0 2024-03-16T10:45:13.530 INFO:tasks.workunit.client.1.smithi026.stdout:0/300: mkdir d1/d18/d61 0 2024-03-16T10:45:13.531 INFO:tasks.workunit.client.1.smithi026.stdout:0/301: chown d1/d12/d2f/f42 353090 1 2024-03-16T10:45:13.533 INFO:tasks.workunit.client.1.smithi026.stdout:3/285: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:13.534 INFO:tasks.workunit.client.1.smithi026.stdout:5/199: creat d1/db/d27/f3e x:0 0 0 2024-03-16T10:45:13.535 INFO:tasks.workunit.client.1.smithi026.stdout:9/218: mknod d0/da/d16/c46 0 2024-03-16T10:45:13.536 INFO:tasks.workunit.client.1.smithi026.stdout:6/304: mknod d0/d41/c6e 0 2024-03-16T10:45:13.538 INFO:tasks.workunit.client.1.smithi026.stdout:7/273: mknod d6/de/d16/d44/c55 0 2024-03-16T10:45:13.538 INFO:tasks.workunit.client.1.smithi026.stdout:0/302: dwrite d1/d18/d1a/d24/d11/f2d [8388608,4194304] 0 2024-03-16T10:45:13.539 INFO:tasks.workunit.client.1.smithi026.stdout:7/274: readlink d6/d1e/l2b 0 2024-03-16T10:45:13.545 INFO:tasks.workunit.client.1.smithi026.stdout:4/280: creat d1/db/d29/d44/f5a x:0 0 0 2024-03-16T10:45:13.545 INFO:tasks.workunit.client.1.smithi026.stdout:5/200: dwrite d1/d1c/d2a/f32 [0,4194304] 0 2024-03-16T10:45:13.546 INFO:tasks.workunit.client.1.smithi026.stdout:9/219: truncate d0/da/f1b 422185 0 2024-03-16T10:45:13.547 INFO:tasks.workunit.client.1.smithi026.stdout:5/201: write d1/d1c/f3c [709587,86014] 0 2024-03-16T10:45:13.548 INFO:tasks.workunit.client.1.smithi026.stdout:6/305: creat d0/d28/f6f x:0 0 0 2024-03-16T10:45:13.569 INFO:tasks.workunit.client.0.smithi002.stdout:2/975: write d7/d2e/d1d/d38/d49/fe0 [580692,62783] 0 2024-03-16T10:45:13.573 INFO:tasks.workunit.client.0.smithi002.stdout:2/976: creat d7/d2e/f138 x:0 0 0 2024-03-16T10:45:13.574 INFO:tasks.workunit.client.0.smithi002.stdout:4/942: write d2/d6/d32/d66/d7f/de7/d5e/f75 [1154076,5284] 0 2024-03-16T10:45:13.575 INFO:tasks.workunit.client.0.smithi002.stdout:2/977: read d7/d2e/d1d/f21 [3775572,54724] 0 2024-03-16T10:45:13.576 INFO:tasks.workunit.client.0.smithi002.stdout:7/916: dwrite d3/d4/d1d/d87/d5c/d63/d34/f78 [0,4194304] 0 2024-03-16T10:45:13.580 INFO:tasks.workunit.client.0.smithi002.stdout:2/978: truncate d7/d2e/d1d/f9e 815910 0 2024-03-16T10:45:13.583 INFO:tasks.workunit.client.0.smithi002.stdout:7/917: creat d3/d12b/f149 x:0 0 0 2024-03-16T10:45:13.585 INFO:tasks.workunit.client.0.smithi002.stdout:7/918: fsync d3/d4/d1d/d87/f7e 0 2024-03-16T10:45:13.590 INFO:tasks.workunit.client.0.smithi002.stdout:7/919: creat d3/d4/d1d/d87/d5c/d63/d3f/d75/f14a x:0 0 0 2024-03-16T10:45:13.591 INFO:tasks.workunit.client.0.smithi002.stdout:2/979: getdents d7/d2e/d1d/d2f/d4b/d6e 0 2024-03-16T10:45:13.593 INFO:tasks.workunit.client.0.smithi002.stdout:2/980: creat d7/d2e/d1d/d38/d49/f139 x:0 0 0 2024-03-16T10:45:13.594 INFO:tasks.workunit.client.0.smithi002.stdout:2/981: stat d7/f90 0 2024-03-16T10:45:13.594 INFO:tasks.workunit.client.0.smithi002.stdout:2/982: dread - d7/d2e/d1d/d38/d49/f139 zero size 2024-03-16T10:45:13.594 INFO:tasks.workunit.client.0.smithi002.stdout:7/920: creat d3/d61/f14b x:0 0 0 2024-03-16T10:45:13.602 INFO:tasks.workunit.client.0.smithi002.stdout:2/983: dwrite d7/d2e/d28/ffb [0,4194304] 0 2024-03-16T10:45:13.603 INFO:tasks.workunit.client.0.smithi002.stdout:4/943: dwrite d2/d6/fb8 [0,4194304] 0 2024-03-16T10:45:13.603 INFO:tasks.workunit.client.0.smithi002.stdout:7/921: link d3/cf d3/d4/d1d/d87/d5c/d63/d3f/dba/c14c 0 2024-03-16T10:45:13.607 INFO:tasks.workunit.client.0.smithi002.stdout:7/922: mkdir d3/d4/d1d/d87/d96/d5e/db9/d14d 0 2024-03-16T10:45:13.607 INFO:tasks.workunit.client.0.smithi002.stdout:4/944: creat d2/d6/d32/d66/d7f/f122 x:0 0 0 2024-03-16T10:45:13.608 INFO:tasks.workunit.client.0.smithi002.stdout:2/984: symlink d7/d27/d3b/dd4/l13a 0 2024-03-16T10:45:13.608 INFO:tasks.workunit.client.0.smithi002.stdout:2/985: stat d7/d27/lc7 0 2024-03-16T10:45:13.609 INFO:tasks.workunit.client.0.smithi002.stdout:4/945: symlink d2/d33/d10a/l123 0 2024-03-16T10:45:13.610 INFO:tasks.workunit.client.0.smithi002.stdout:7/923: creat d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f14e x:0 0 0 2024-03-16T10:45:13.613 INFO:tasks.workunit.client.0.smithi002.stdout:7/924: fdatasync d3/d4/d1d/d87/d96/d8c/f9b 0 2024-03-16T10:45:13.618 INFO:tasks.workunit.client.0.smithi002.stdout:2/986: link d7/d2e/d28/d44/d6d/c8a d7/d2e/d28/d40/d50/c13b 0 2024-03-16T10:45:13.621 INFO:tasks.workunit.client.0.smithi002.stdout:7/925: write d3/f52 [12222858,54424] 0 2024-03-16T10:45:13.624 INFO:tasks.workunit.client.0.smithi002.stdout:7/926: mkdir d3/d4/d1d/d14f 0 2024-03-16T10:45:13.625 INFO:tasks.workunit.client.0.smithi002.stdout:4/946: dread d2/d33/d1c/fba [0,4194304] 0 2024-03-16T10:45:13.625 INFO:tasks.workunit.client.0.smithi002.stdout:2/987: dread d7/d2e/d1d/d2f/d4b/d6e/fab [0,4194304] 0 2024-03-16T10:45:13.627 INFO:tasks.workunit.client.0.smithi002.stdout:4/947: chown d2/d33/d35/cc1 238 1 2024-03-16T10:45:13.627 INFO:tasks.workunit.client.0.smithi002.stdout:2/988: write d7/d2e/d1d/f21 [4111189,62841] 0 2024-03-16T10:45:13.630 INFO:tasks.workunit.client.0.smithi002.stdout:7/927: link d3/d4/d1c/d5b/d64/lad d3/d4/d1d/d87/d5c/d63/d3f/daa/l150 0 2024-03-16T10:45:13.631 INFO:tasks.workunit.client.0.smithi002.stdout:7/928: stat d3/d4/d1d/d87/d5c/dd0/f133 0 2024-03-16T10:45:13.632 INFO:tasks.workunit.client.0.smithi002.stdout:4/948: link d2/d6/d32/d66/d7f/de7/d2e/db4/fd2 d2/d6/d32/d66/d7f/de7/d5e/d93/f124 0 2024-03-16T10:45:13.633 INFO:tasks.workunit.client.0.smithi002.stdout:2/989: link d7/d2e/d1d/d2f/d4b/d6e/fab d7/db/f13c 0 2024-03-16T10:45:13.634 INFO:tasks.workunit.client.0.smithi002.stdout:7/929: mkdir d3/d4/d1d/d14f/d151 0 2024-03-16T10:45:13.636 INFO:tasks.workunit.client.0.smithi002.stdout:7/930: creat d3/d37/d6a/f152 x:0 0 0 2024-03-16T10:45:13.636 INFO:tasks.workunit.client.0.smithi002.stdout:2/990: truncate d7/f2b 13102 0 2024-03-16T10:45:13.638 INFO:tasks.workunit.client.0.smithi002.stdout:7/931: rmdir d3/d4/d1d/d87/d5c/d63/d3f/dba 39 2024-03-16T10:45:13.638 INFO:tasks.workunit.client.0.smithi002.stdout:7/932: readlink d3/d37/d6a/l134 0 2024-03-16T10:45:13.639 INFO:tasks.workunit.client.0.smithi002.stdout:4/949: dread d2/d33/f94 [0,4194304] 0 2024-03-16T10:45:13.639 INFO:tasks.workunit.client.0.smithi002.stdout:2/991: unlink d7/d2e/d1d/d2f/d4b/ff8 0 2024-03-16T10:45:13.639 INFO:tasks.workunit.client.0.smithi002.stdout:7/933: write d3/d12b/f102 [843900,107935] 0 2024-03-16T10:45:13.642 INFO:tasks.workunit.client.0.smithi002.stdout:4/950: symlink d2/d6/d32/l125 0 2024-03-16T10:45:13.643 INFO:tasks.workunit.client.0.smithi002.stdout:7/934: mkdir d3/d4/d1d/d87/d5c/dd0/dd2/d153 0 2024-03-16T10:45:13.645 INFO:tasks.workunit.client.0.smithi002.stdout:7/935: dread - d3/d4/d1d/d87/d5c/d63/d3f/daa/d106/f132 zero size 2024-03-16T10:45:13.645 INFO:tasks.workunit.client.0.smithi002.stdout:2/992: getdents d7/d2e/d1d/d38 0 2024-03-16T10:45:13.645 INFO:tasks.workunit.client.0.smithi002.stdout:4/951: creat d2/d33/d35/d59/d11f/f126 x:0 0 0 2024-03-16T10:45:13.645 INFO:tasks.workunit.client.0.smithi002.stdout:2/993: chown d7/d2e/d28 24 1 2024-03-16T10:45:13.647 INFO:tasks.workunit.client.0.smithi002.stdout:2/994: write d7/d2e/d28/d40/d50/db4/f136 [173706,96383] 0 2024-03-16T10:45:13.648 INFO:tasks.workunit.client.0.smithi002.stdout:7/936: rename d3/d4/d1c/l49 to d3/d4/d1d/d87/l154 0 2024-03-16T10:45:13.649 INFO:tasks.workunit.client.0.smithi002.stdout:4/952: mkdir d2/d6/d32/d66/d7f/de7/d88/db2/d127 0 2024-03-16T10:45:13.652 INFO:tasks.workunit.client.0.smithi002.stdout:2/995: link d7/d2e/d1d/d38/f3a d7/d2e/d28/d40/d6c/f13d 0 2024-03-16T10:45:13.653 INFO:tasks.workunit.client.0.smithi002.stdout:4/953: creat d2/d6/d32/d66/d7f/de7/d88/db2/d127/f128 x:0 0 0 2024-03-16T10:45:13.655 INFO:tasks.workunit.client.0.smithi002.stdout:2/996: creat d7/d2e/d1d/d2f/f13e x:0 0 0 2024-03-16T10:45:13.655 INFO:tasks.workunit.client.1.smithi026.stdout:5/202: write d1/db/d27/f23 [2686596,51749] 0 2024-03-16T10:45:13.656 INFO:tasks.workunit.client.0.smithi002.stdout:2/997: dread - d7/d2e/d46/f137 zero size 2024-03-16T10:45:13.656 INFO:tasks.workunit.client.0.smithi002.stdout:4/954: mknod d2/d6/ddb/c129 0 2024-03-16T10:45:13.656 INFO:tasks.workunit.client.1.smithi026.stdout:4/281: creat d1/db/de/f5b x:0 0 0 2024-03-16T10:45:13.657 INFO:tasks.workunit.client.1.smithi026.stdout:9/220: mknod d0/da/d16/d17/c47 0 2024-03-16T10:45:13.657 INFO:tasks.workunit.client.1.smithi026.stdout:6/306: read - d0/d62/f63 zero size 2024-03-16T10:45:13.657 INFO:tasks.workunit.client.1.smithi026.stdout:9/221: readlink d0/da/d14/l37 0 2024-03-16T10:45:13.658 INFO:tasks.workunit.client.1.smithi026.stdout:6/307: chown d0/d28/f5d 735820 1 2024-03-16T10:45:13.661 INFO:tasks.workunit.client.1.smithi026.stdout:6/308: write d0/d28/d49/f52 [586564,67871] 0 2024-03-16T10:45:13.663 INFO:tasks.workunit.client.1.smithi026.stdout:2/244: write d2/d29/fe [445823,100997] 0 2024-03-16T10:45:13.670 INFO:tasks.workunit.client.1.smithi026.stdout:8/229: dwrite d5/dc/f15 [0,4194304] 0 2024-03-16T10:45:13.670 INFO:tasks.workunit.client.0.smithi002.stdout:4/955: dread d2/d6/f17 [0,4194304] 0 2024-03-16T10:45:13.671 INFO:tasks.workunit.client.1.smithi026.stdout:9/222: dwrite d0/da/d13/f26 [0,4194304] 0 2024-03-16T10:45:13.673 INFO:tasks.workunit.client.1.smithi026.stdout:7/275: symlink d6/d35/d3a/d4e/l56 0 2024-03-16T10:45:13.675 INFO:tasks.workunit.client.1.smithi026.stdout:3/286: creat d4/dc/dd/d60/d62/f6b x:0 0 0 2024-03-16T10:45:13.677 INFO:tasks.workunit.client.1.smithi026.stdout:2/245: rmdir d2 39 2024-03-16T10:45:13.677 INFO:tasks.workunit.client.1.smithi026.stdout:5/203: write d1/db/fc [8497419,104466] 0 2024-03-16T10:45:13.677 INFO:tasks.workunit.client.1.smithi026.stdout:4/282: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:13.681 INFO:tasks.workunit.client.1.smithi026.stdout:8/230: mknod d5/d7/c3d 0 2024-03-16T10:45:13.682 INFO:tasks.workunit.client.1.smithi026.stdout:4/283: dread d1/db/de/f58 [0,4194304] 0 2024-03-16T10:45:13.686 INFO:tasks.workunit.client.1.smithi026.stdout:9/223: mknod d0/da/d14/c48 0 2024-03-16T10:45:13.690 INFO:tasks.workunit.client.1.smithi026.stdout:4/284: dwrite d1/db/d29/f54 [0,4194304] 0 2024-03-16T10:45:13.690 INFO:tasks.workunit.client.0.smithi002.stdout:2/998: rmdir d7/d2e/d1d/d2f/d4b 39 2024-03-16T10:45:13.692 INFO:tasks.workunit.client.0.smithi002.stdout:2/999: truncate d7/d27/d3b/dd4/de1/f123 949200 0 2024-03-16T10:45:13.693 INFO:tasks.workunit.client.0.smithi002.stdout:4/956: getdents d2/d6 0 2024-03-16T10:45:13.694 INFO:tasks.workunit.client.1.smithi026.stdout:3/287: stat d4/dc/dd/d60/d62/d4f/d5a 0 2024-03-16T10:45:13.697 INFO:tasks.workunit.client.1.smithi026.stdout:7/276: truncate d6/f22 3815008 0 2024-03-16T10:45:13.698 INFO:tasks.workunit.client.1.smithi026.stdout:7/277: fsync d6/de/d16/f49 0 2024-03-16T10:45:13.699 INFO:tasks.workunit.client.1.smithi026.stdout:7/278: read - d6/f53 zero size 2024-03-16T10:45:13.700 INFO:tasks.workunit.client.1.smithi026.stdout:7/279: truncate d6/f39 755977 0 2024-03-16T10:45:13.703 INFO:tasks.workunit.client.1.smithi026.stdout:2/246: dread d2/d29/d17/d1a/d2b/f1f [4194304,4194304] 0 2024-03-16T10:45:13.703 INFO:tasks.workunit.client.1.smithi026.stdout:2/247: readlink l0 0 2024-03-16T10:45:13.705 INFO:tasks.workunit.client.0.smithi002.stdout:7/937: dwrite d3/d4/d1d/d87/d5c/d63/d3f/fac [4194304,4194304] 0 2024-03-16T10:45:13.707 INFO:tasks.workunit.client.1.smithi026.stdout:1/286: getdents d0/d33/d2e 0 2024-03-16T10:45:13.709 INFO:tasks.workunit.client.1.smithi026.stdout:9/224: write d0/f1d [3777574,18624] 0 2024-03-16T10:45:13.715 INFO:tasks.workunit.client.1.smithi026.stdout:3/288: dread d4/dc/f13 [0,4194304] 0 2024-03-16T10:45:13.715 INFO:tasks.workunit.client.0.smithi002.stdout:4/957: getdents d2/d6/d32/d66/d7f/de7/d2e/db0/dfe 0 2024-03-16T10:45:13.717 INFO:tasks.workunit.client.1.smithi026.stdout:6/309: link d0/d10/ff d0/d41/f70 0 2024-03-16T10:45:13.721 INFO:tasks.workunit.client.0.smithi002.stdout:4/958: dwrite d2/d6/d32/d66/d7f/de7/d88/db2/fd0 [0,4194304] 0 2024-03-16T10:45:13.722 INFO:tasks.workunit.client.1.smithi026.stdout:2/248: unlink c1 0 2024-03-16T10:45:13.730 INFO:tasks.workunit.client.0.smithi002.stdout:4/959: mkdir d2/d6/d32/d66/d7f/de7/d5e/d93/d12a 0 2024-03-16T10:45:13.730 INFO:tasks.workunit.client.0.smithi002.stdout:4/960: write d2/d33/f64 [2645226,31702] 0 2024-03-16T10:45:13.731 INFO:tasks.workunit.client.1.smithi026.stdout:1/287: symlink d0/d17/d1f/d20/l5b 0 2024-03-16T10:45:13.732 INFO:tasks.workunit.client.1.smithi026.stdout:1/288: fdatasync d0/d17/d1f/d38/f4e 0 2024-03-16T10:45:13.735 INFO:tasks.workunit.client.0.smithi002.stdout:7/938: write d3/d61/faf [77261,64551] 0 2024-03-16T10:45:13.735 INFO:tasks.workunit.client.1.smithi026.stdout:4/285: mknod d1/db/c5c 0 2024-03-16T10:45:13.737 INFO:tasks.workunit.client.1.smithi026.stdout:4/286: write d1/db/f3a [744759,92126] 0 2024-03-16T10:45:13.737 INFO:tasks.workunit.client.0.smithi002.stdout:4/961: truncate d2/d33/f6a 28845 0 2024-03-16T10:45:13.737 INFO:tasks.workunit.client.1.smithi026.stdout:4/287: fsync d1/f18 0 2024-03-16T10:45:13.742 INFO:tasks.workunit.client.1.smithi026.stdout:8/231: write d5/dc/f1e [754308,80245] 0 2024-03-16T10:45:13.743 INFO:tasks.workunit.client.1.smithi026.stdout:4/288: dread d1/db/d29/f3e [0,4194304] 0 2024-03-16T10:45:13.744 INFO:tasks.workunit.client.1.smithi026.stdout:6/310: mkdir d0/d41/d71 0 2024-03-16T10:45:13.744 INFO:tasks.workunit.client.1.smithi026.stdout:6/311: symlink d0/d5b/l72 2 2024-03-16T10:45:13.745 INFO:tasks.workunit.client.1.smithi026.stdout:4/289: readlink d1/db/l39 0 2024-03-16T10:45:13.745 INFO:tasks.workunit.client.1.smithi026.stdout:3/289: dwrite d4/f41 [0,4194304] 0 2024-03-16T10:45:13.748 INFO:tasks.workunit.client.1.smithi026.stdout:6/312: write d0/d10/f67 [925930,54973] 0 2024-03-16T10:45:13.749 INFO:tasks.workunit.client.1.smithi026.stdout:4/290: truncate d1/db/de/d4c/f4d 869284 0 2024-03-16T10:45:13.750 INFO:tasks.workunit.client.1.smithi026.stdout:4/291: chown d1/db/de/f5b 11165 1 2024-03-16T10:45:13.750 INFO:tasks.workunit.client.1.smithi026.stdout:5/204: rename d1/db/f17 to d1/d1c/f3f 0 2024-03-16T10:45:13.751 INFO:tasks.workunit.client.1.smithi026.stdout:8/232: dread d5/f2e [0,4194304] 0 2024-03-16T10:45:13.753 INFO:tasks.workunit.client.1.smithi026.stdout:5/205: truncate d1/f21 688303 0 2024-03-16T10:45:13.754 INFO:tasks.workunit.client.0.smithi002.stdout:7/939: dread d3/d4/d1d/d87/d5c/d63/f3c [0,4194304] 0 2024-03-16T10:45:13.756 INFO:tasks.workunit.client.1.smithi026.stdout:4/292: dread d1/db/d29/f3e [0,4194304] 0 2024-03-16T10:45:13.757 INFO:tasks.workunit.client.1.smithi026.stdout:9/225: dwrite d0/da/d16/d17/d44/f24 [0,4194304] 0 2024-03-16T10:45:13.758 INFO:tasks.workunit.client.0.smithi002.stdout:4/962: write d2/d6/d32/d66/d7f/de7/f7d [1793801,65614] 0 2024-03-16T10:45:13.758 INFO:tasks.workunit.client.0.smithi002.stdout:7/940: chown d3/d4/d1d/d87/d5c/d63/d3f/dba/d100 76148905 1 2024-03-16T10:45:13.758 INFO:tasks.workunit.client.1.smithi026.stdout:9/226: read - d0/da/f27 zero size 2024-03-16T10:45:13.759 INFO:tasks.workunit.client.1.smithi026.stdout:9/227: readlink d0/da/d16/d17/l22 0 2024-03-16T10:45:13.765 INFO:tasks.workunit.client.1.smithi026.stdout:7/280: creat d6/de/f57 x:0 0 0 2024-03-16T10:45:13.767 INFO:tasks.workunit.client.1.smithi026.stdout:3/290: stat d4/d29/l2f 0 2024-03-16T10:45:13.769 INFO:tasks.workunit.client.1.smithi026.stdout:3/291: write d4/dc/d31/f51 [581762,81626] 0 2024-03-16T10:45:13.771 INFO:tasks.workunit.client.1.smithi026.stdout:7/281: dread d6/de/d16/f1f [0,4194304] 0 2024-03-16T10:45:13.772 INFO:tasks.workunit.client.1.smithi026.stdout:2/249: creat d2/d29/d17/d38/f53 x:0 0 0 2024-03-16T10:45:13.773 INFO:tasks.workunit.client.1.smithi026.stdout:2/250: dread - d2/d29/d17/d38/f53 zero size 2024-03-16T10:45:13.773 INFO:tasks.workunit.client.0.smithi002.stdout:4/963: rmdir d2/d6/d32/d66/d7f/de7/d5e/d93/d12a 0 2024-03-16T10:45:13.774 INFO:tasks.workunit.client.1.smithi026.stdout:2/251: write d2/d29/d3b/f47 [834775,25492] 0 2024-03-16T10:45:13.775 INFO:tasks.workunit.client.1.smithi026.stdout:1/289: getdents d0/d33/d3b/d49/d58 0 2024-03-16T10:45:13.776 INFO:tasks.workunit.client.1.smithi026.stdout:2/252: readlink d2/d29/d17/d1a/d2b/l34 0 2024-03-16T10:45:13.776 INFO:tasks.workunit.client.0.smithi002.stdout:7/941: dwrite d3/d4/d1d/d87/d96/d5e/f60 [0,4194304] 0 2024-03-16T10:45:13.777 INFO:tasks.workunit.client.1.smithi026.stdout:2/253: dread - d2/f4d zero size 2024-03-16T10:45:13.777 INFO:tasks.workunit.client.1.smithi026.stdout:4/293: chown d1/db/de/l3f 1 1 2024-03-16T10:45:13.778 INFO:tasks.workunit.client.0.smithi002.stdout:4/964: rmdir d2/d33/d10a 39 2024-03-16T10:45:13.778 INFO:tasks.workunit.client.1.smithi026.stdout:9/228: creat d0/da/d13/f49 x:0 0 0 2024-03-16T10:45:13.779 INFO:tasks.workunit.client.0.smithi002.stdout:4/965: chown d2/d6/d32/d66/d7f/de7/la1 1 1 2024-03-16T10:45:13.779 INFO:tasks.workunit.client.1.smithi026.stdout:9/229: fsync d0/f1d 0 2024-03-16T10:45:13.781 INFO:tasks.workunit.client.1.smithi026.stdout:9/230: write d0/da/d16/d17/d44/f24 [1223391,76413] 0 2024-03-16T10:45:13.783 INFO:tasks.workunit.client.0.smithi002.stdout:4/966: rmdir d2/d6/d32/d66/d7f/de7/d2e/db0 39 2024-03-16T10:45:13.785 INFO:tasks.workunit.client.1.smithi026.stdout:3/292: fdatasync d4/dc/dd/d4b/f5b 0 2024-03-16T10:45:13.787 INFO:tasks.workunit.client.1.smithi026.stdout:6/313: truncate d0/f1d 1435816 0 2024-03-16T10:45:13.788 INFO:tasks.workunit.client.0.smithi002.stdout:4/967: symlink d2/d6/d32/d66/d7f/de7/d88/db2/d127/l12b 0 2024-03-16T10:45:13.790 INFO:tasks.workunit.client.1.smithi026.stdout:7/282: symlink d6/d35/d3a/d4e/l58 0 2024-03-16T10:45:13.790 INFO:tasks.workunit.client.0.smithi002.stdout:4/968: write d2/d33/d1c/d45/f119 [616619,95966] 0 2024-03-16T10:45:13.791 INFO:tasks.workunit.client.1.smithi026.stdout:5/206: mkdir d1/db/d40 0 2024-03-16T10:45:13.791 INFO:tasks.workunit.client.1.smithi026.stdout:6/314: dread d0/d5b/f30 [4194304,4194304] 0 2024-03-16T10:45:13.791 INFO:tasks.workunit.client.1.smithi026.stdout:7/283: write d6/de/d16/f49 [1006373,70243] 0 2024-03-16T10:45:13.793 INFO:tasks.workunit.client.0.smithi002.stdout:7/942: dwrite d3/d4/d1d/d87/d96/d8c/fa4 [0,4194304] 0 2024-03-16T10:45:13.795 INFO:tasks.workunit.client.0.smithi002.stdout:4/969: symlink d2/d33/d35/d59/d11f/l12c 0 2024-03-16T10:45:13.796 INFO:tasks.workunit.client.1.smithi026.stdout:9/231: symlink d0/da/d16/d17/l4a 0 2024-03-16T10:45:13.796 INFO:tasks.workunit.client.0.smithi002.stdout:7/943: write d3/d4/d1d/d87/d5c/d63/f145 [1041167,129904] 0 2024-03-16T10:45:13.797 INFO:tasks.workunit.client.0.smithi002.stdout:7/944: stat d3/d4/d1c 0 2024-03-16T10:45:13.797 INFO:tasks.workunit.client.1.smithi026.stdout:3/293: creat d4/dc/dd/d60/d62/d4d/f6c x:0 0 0 2024-03-16T10:45:13.799 INFO:tasks.workunit.client.0.smithi002.stdout:4/970: dread d2/d6/f13 [4194304,4194304] 0 2024-03-16T10:45:13.803 INFO:tasks.workunit.client.1.smithi026.stdout:8/233: rename d5/f2e to d5/f3e 0 2024-03-16T10:45:13.804 INFO:tasks.workunit.client.1.smithi026.stdout:9/232: dwrite d0/f2f [0,4194304] 0 2024-03-16T10:45:13.805 INFO:tasks.workunit.client.1.smithi026.stdout:9/233: readlink d0/da/d16/d17/l22 0 2024-03-16T10:45:13.809 INFO:tasks.workunit.client.1.smithi026.stdout:8/234: dread d5/dc/f1e [0,4194304] 0 2024-03-16T10:45:13.810 INFO:tasks.workunit.client.0.smithi002.stdout:4/971: mkdir d2/d33/d1c/d12d 0 2024-03-16T10:45:13.814 INFO:tasks.workunit.client.1.smithi026.stdout:5/207: creat d1/db/d27/f41 x:0 0 0 2024-03-16T10:45:13.814 INFO:tasks.workunit.client.1.smithi026.stdout:7/284: dwrite d6/f24 [4194304,4194304] 0 2024-03-16T10:45:13.816 INFO:tasks.workunit.client.1.smithi026.stdout:0/303: sync 2024-03-16T10:45:13.817 INFO:tasks.workunit.client.1.smithi026.stdout:6/315: rename d0/d28/l46 to d0/d62/l73 0 2024-03-16T10:45:13.818 INFO:tasks.workunit.client.1.smithi026.stdout:3/294: chown d4/l38 472 1 2024-03-16T10:45:13.822 INFO:tasks.workunit.client.0.smithi002.stdout:7/945: write d3/d4/d1d/d87/d96/d8c/f95 [921964,35880] 0 2024-03-16T10:45:13.822 INFO:tasks.workunit.client.1.smithi026.stdout:1/290: link d0/d17/l4f d0/d17/d1f/d20/l5c 0 2024-03-16T10:45:13.824 INFO:tasks.workunit.client.1.smithi026.stdout:5/208: symlink d1/db/d27/l42 0 2024-03-16T10:45:13.825 INFO:tasks.workunit.client.1.smithi026.stdout:7/285: mkdir d6/d35/d3a/d59 0 2024-03-16T10:45:13.826 INFO:tasks.workunit.client.1.smithi026.stdout:7/286: chown d6/de/d16/c26 36900 1 2024-03-16T10:45:13.827 INFO:tasks.workunit.client.1.smithi026.stdout:0/304: chown d1/d18/d1a/l33 12 1 2024-03-16T10:45:13.828 INFO:tasks.workunit.client.1.smithi026.stdout:7/287: read - d6/d35/d3a/d3c/d3f/f4b zero size 2024-03-16T10:45:13.829 INFO:tasks.workunit.client.0.smithi002.stdout:7/946: unlink d3/d4/d1d/d87/d5c/d63/d3f/daa/lb3 0 2024-03-16T10:45:13.830 INFO:tasks.workunit.client.0.smithi002.stdout:4/972: dread d2/d6/d32/d66/d7f/de7/d88/f9b [0,4194304] 0 2024-03-16T10:45:13.833 INFO:tasks.workunit.client.1.smithi026.stdout:7/288: creat d6/d35/d3a/d4e/f5a x:0 0 0 2024-03-16T10:45:13.834 INFO:tasks.workunit.client.1.smithi026.stdout:7/289: write d6/de/d16/f2e [899775,10507] 0 2024-03-16T10:45:13.835 INFO:tasks.workunit.client.1.smithi026.stdout:3/295: symlink d4/l6d 0 2024-03-16T10:45:13.836 INFO:tasks.workunit.client.1.smithi026.stdout:3/296: chown d4/dc/dd/d60/d62/d4f/d5a 29252842 1 2024-03-16T10:45:13.838 INFO:tasks.workunit.client.0.smithi002.stdout:4/973: rename d2/d33/c12 to d2/d33/d35/d59/c12e 0 2024-03-16T10:45:13.839 INFO:tasks.workunit.client.0.smithi002.stdout:4/974: write d2/d6/d32/d66/d7f/de7/f7d [257151,130435] 0 2024-03-16T10:45:13.840 INFO:tasks.workunit.client.1.smithi026.stdout:9/234: link d0/da/d16/d17/f2e d0/da/f4b 0 2024-03-16T10:45:13.843 INFO:tasks.workunit.client.1.smithi026.stdout:8/235: rename d5/dc/d1f to d5/dc/d3f 0 2024-03-16T10:45:13.843 INFO:tasks.workunit.client.1.smithi026.stdout:8/236: dread - d5/d7/f34 zero size 2024-03-16T10:45:13.846 INFO:tasks.workunit.client.1.smithi026.stdout:1/291: mknod d0/d33/c5d 0 2024-03-16T10:45:13.847 INFO:tasks.workunit.client.1.smithi026.stdout:3/297: dread d4/dc/d31/f34 [0,4194304] 0 2024-03-16T10:45:13.848 INFO:tasks.workunit.client.0.smithi002.stdout:4/975: creat d2/d6/d32/d66/d7f/de7/d11a/f12f x:0 0 0 2024-03-16T10:45:13.848 INFO:tasks.workunit.client.1.smithi026.stdout:3/298: read d4/dc/dd/f21 [1385117,91854] 0 2024-03-16T10:45:13.852 INFO:tasks.workunit.client.1.smithi026.stdout:3/299: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:13.853 INFO:tasks.workunit.client.1.smithi026.stdout:3/300: read d4/dc/d31/f3b [780010,28077] 0 2024-03-16T10:45:13.854 INFO:tasks.workunit.client.1.smithi026.stdout:3/301: chown d4/dc/d31/l37 0 1 2024-03-16T10:45:13.857 INFO:tasks.workunit.client.1.smithi026.stdout:3/302: dread d4/dc/dd/d3d/f49 [0,4194304] 0 2024-03-16T10:45:13.858 INFO:tasks.workunit.client.0.smithi002.stdout:4/976: creat d2/f130 x:0 0 0 2024-03-16T10:45:13.861 INFO:tasks.workunit.client.1.smithi026.stdout:7/290: symlink d6/d35/d3a/d59/l5b 0 2024-03-16T10:45:13.863 INFO:tasks.workunit.client.1.smithi026.stdout:7/291: dread - d6/d35/d3a/d3c/d3f/f4b zero size 2024-03-16T10:45:13.863 INFO:tasks.workunit.client.1.smithi026.stdout:3/303: dwrite d4/dc/dd/f24 [0,4194304] 0 2024-03-16T10:45:13.864 INFO:tasks.workunit.client.0.smithi002.stdout:7/947: truncate d3/d4/d1d/d87/d96/d5e/f60 3836028 0 2024-03-16T10:45:13.865 INFO:tasks.workunit.client.0.smithi002.stdout:7/948: dread d3/d4/d1d/d87/d5c/dc3/df7/f108 [0,4194304] 0 2024-03-16T10:45:13.869 INFO:tasks.workunit.client.0.smithi002.stdout:4/977: unlink d2/d33/d35/f6f 0 2024-03-16T10:45:13.870 INFO:tasks.workunit.client.1.smithi026.stdout:9/235: symlink d0/da/l4c 0 2024-03-16T10:45:13.873 INFO:tasks.workunit.client.1.smithi026.stdout:0/305: truncate d1/d18/d1a/d24/d11/f2d 8737052 0 2024-03-16T10:45:13.874 INFO:tasks.workunit.client.1.smithi026.stdout:0/306: write d1/d18/d1a/d24/f29 [5833621,87311] 0 2024-03-16T10:45:13.874 INFO:tasks.workunit.client.1.smithi026.stdout:8/237: mkdir d5/dc/d40 0 2024-03-16T10:45:13.875 INFO:tasks.workunit.client.1.smithi026.stdout:8/238: stat d5/d7/l17 0 2024-03-16T10:45:13.878 INFO:tasks.workunit.client.1.smithi026.stdout:1/292: creat d0/d33/d2e/d54/f5e x:0 0 0 2024-03-16T10:45:13.878 INFO:tasks.workunit.client.1.smithi026.stdout:1/293: chown d0/d33/d3b/d49/d58 7184 1 2024-03-16T10:45:13.879 INFO:tasks.workunit.client.0.smithi002.stdout:7/949: dread d3/d4/d1c/f33 [0,4194304] 0 2024-03-16T10:45:13.881 INFO:tasks.workunit.client.1.smithi026.stdout:7/292: creat d6/d35/d3a/d3c/f5c x:0 0 0 2024-03-16T10:45:13.885 INFO:tasks.workunit.client.1.smithi026.stdout:7/293: chown d6/f38 906343328 1 2024-03-16T10:45:13.885 INFO:tasks.workunit.client.1.smithi026.stdout:3/304: creat d4/d29/f6e x:0 0 0 2024-03-16T10:45:13.886 INFO:tasks.workunit.client.1.smithi026.stdout:3/305: write d4/dc/d31/f51 [10933,113300] 0 2024-03-16T10:45:13.891 INFO:tasks.workunit.client.1.smithi026.stdout:1/294: mknod d0/d33/d3b/c5f 0 2024-03-16T10:45:13.893 INFO:tasks.workunit.client.1.smithi026.stdout:7/294: fsync d6/d35/d3a/d3c/f5c 0 2024-03-16T10:45:13.894 INFO:tasks.workunit.client.1.smithi026.stdout:9/236: mkdir d0/da/d4d 0 2024-03-16T10:45:13.895 INFO:tasks.workunit.client.0.smithi002.stdout:7/950: write d3/d61/f94 [353152,76176] 0 2024-03-16T10:45:13.897 INFO:tasks.workunit.client.1.smithi026.stdout:0/307: creat d1/d18/d61/f62 x:0 0 0 2024-03-16T10:45:13.898 INFO:tasks.workunit.client.0.smithi002.stdout:7/951: truncate d3/d4/d1d/d87/d5c/d63/d3f/daa/d106/f132 363927 0 2024-03-16T10:45:13.901 INFO:tasks.workunit.client.1.smithi026.stdout:0/308: creat d1/d18/d1a/d24/d51/f63 x:0 0 0 2024-03-16T10:45:13.905 INFO:tasks.workunit.client.1.smithi026.stdout:9/237: rename d0/c6 to d0/c4e 0 2024-03-16T10:45:13.907 INFO:tasks.workunit.client.1.smithi026.stdout:0/309: symlink d1/d18/d1a/d2b/l64 0 2024-03-16T10:45:13.909 INFO:tasks.workunit.client.1.smithi026.stdout:8/239: truncate d5/d7/f1d 2139661 0 2024-03-16T10:45:13.910 INFO:tasks.workunit.client.1.smithi026.stdout:9/238: dread d0/da/d16/d17/d44/f24 [0,4194304] 0 2024-03-16T10:45:13.911 INFO:tasks.workunit.client.1.smithi026.stdout:1/295: dread d0/fd [0,4194304] 0 2024-03-16T10:45:13.912 INFO:tasks.workunit.client.0.smithi002.stdout:7/952: link d3/d4/d1d/d87/d5c/dc3/dc7/c143 d3/d4/d1d/d87/d5c/d63/d3f/c155 0 2024-03-16T10:45:13.914 INFO:tasks.workunit.client.1.smithi026.stdout:1/296: symlink d0/d17/d1f/l60 0 2024-03-16T10:45:13.916 INFO:tasks.workunit.client.0.smithi002.stdout:7/953: creat d3/d4/d1d/d87/d5c/d63/d55/f156 x:0 0 0 2024-03-16T10:45:13.917 INFO:tasks.workunit.client.1.smithi026.stdout:8/240: dwrite d5/d23/f2b [4194304,4194304] 0 2024-03-16T10:45:13.920 INFO:tasks.workunit.client.1.smithi026.stdout:9/239: getdents d0/da/d16/d17/d44 0 2024-03-16T10:45:13.921 INFO:tasks.workunit.client.1.smithi026.stdout:9/240: fsync d0/da/d14/f25 0 2024-03-16T10:45:13.922 INFO:tasks.workunit.client.1.smithi026.stdout:9/241: write d0/da/d14/f3d [536700,122075] 0 2024-03-16T10:45:13.923 INFO:tasks.workunit.client.1.smithi026.stdout:9/242: chown d0/lf 0 1 2024-03-16T10:45:13.925 INFO:tasks.workunit.client.1.smithi026.stdout:9/243: write d0/da/f11 [281576,104453] 0 2024-03-16T10:45:13.925 INFO:tasks.workunit.client.1.smithi026.stdout:1/297: creat d0/d17/d4d/f61 x:0 0 0 2024-03-16T10:45:13.926 INFO:tasks.workunit.client.1.smithi026.stdout:9/244: read - d0/da/f4b zero size 2024-03-16T10:45:13.929 INFO:tasks.workunit.client.1.smithi026.stdout:8/241: symlink d5/l41 0 2024-03-16T10:45:13.932 INFO:tasks.workunit.client.1.smithi026.stdout:1/298: unlink d0/d33/d2e/c31 0 2024-03-16T10:45:13.933 INFO:tasks.workunit.client.0.smithi002.stdout:7/954: truncate d3/d4/d1d/d87/d5c/d63/f145 821325 0 2024-03-16T10:45:13.935 INFO:tasks.workunit.client.1.smithi026.stdout:8/242: read d5/d7/d2c/f32 [3007787,104419] 0 2024-03-16T10:45:13.937 INFO:tasks.workunit.client.1.smithi026.stdout:9/245: mkdir d0/da/d4d/d4f 0 2024-03-16T10:45:13.937 INFO:tasks.workunit.client.0.smithi002.stdout:7/955: creat d3/d4/d1d/d14f/d151/f157 x:0 0 0 2024-03-16T10:45:13.939 INFO:tasks.workunit.client.1.smithi026.stdout:9/246: symlink d0/da/d4d/d4f/l50 0 2024-03-16T10:45:13.941 INFO:tasks.workunit.client.1.smithi026.stdout:8/243: getdents d5/d7 0 2024-03-16T10:45:13.941 INFO:tasks.workunit.client.1.smithi026.stdout:8/244: fsync d5/dc/f15 0 2024-03-16T10:45:13.942 INFO:tasks.workunit.client.0.smithi002.stdout:7/956: mknod d3/d4/d1d/d87/d5c/dd0/dd2/d153/c158 0 2024-03-16T10:45:13.944 INFO:tasks.workunit.client.1.smithi026.stdout:8/245: symlink d5/dc/d40/l42 0 2024-03-16T10:45:13.944 INFO:tasks.workunit.client.0.smithi002.stdout:7/957: dread - d3/d61/d66/f74 zero size 2024-03-16T10:45:13.947 INFO:tasks.workunit.client.0.smithi002.stdout:7/958: creat d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f159 x:0 0 0 2024-03-16T10:45:13.950 INFO:tasks.workunit.client.1.smithi026.stdout:8/246: dwrite d5/dc/f15 [0,4194304] 0 2024-03-16T10:45:13.950 INFO:tasks.workunit.client.0.smithi002.stdout:7/959: unlink d3/d4/d1c/d5b/f123 0 2024-03-16T10:45:13.951 INFO:tasks.workunit.client.1.smithi026.stdout:8/247: read d5/d7/f1c [6733853,117678] 0 2024-03-16T10:45:13.952 INFO:tasks.workunit.client.1.smithi026.stdout:8/248: stat d5/c16 0 2024-03-16T10:45:13.956 INFO:tasks.workunit.client.1.smithi026.stdout:8/249: truncate f4 3991349 0 2024-03-16T10:45:13.958 INFO:tasks.workunit.client.1.smithi026.stdout:9/247: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:13.959 INFO:tasks.workunit.client.1.smithi026.stdout:9/248: dread - d0/da/d16/d17/d44/f42 zero size 2024-03-16T10:45:13.960 INFO:tasks.workunit.client.1.smithi026.stdout:8/250: chown f4 122 1 2024-03-16T10:45:13.962 INFO:tasks.workunit.client.1.smithi026.stdout:9/249: symlink d0/da/d16/d17/d44/l51 0 2024-03-16T10:45:13.964 INFO:tasks.workunit.client.1.smithi026.stdout:8/251: fsync d5/d7/d2c/f2d 0 2024-03-16T10:45:13.965 INFO:tasks.workunit.client.1.smithi026.stdout:9/250: rename d0/da/d16/d17/d44/l51 to d0/da/d14/l52 0 2024-03-16T10:45:13.966 INFO:tasks.workunit.client.1.smithi026.stdout:9/251: write d0/da/f2a [765608,68477] 0 2024-03-16T10:45:13.971 INFO:tasks.workunit.client.1.smithi026.stdout:9/252: dwrite d0/f1f [0,4194304] 0 2024-03-16T10:45:13.977 INFO:tasks.workunit.client.1.smithi026.stdout:9/253: creat d0/da/d13/f53 x:0 0 0 2024-03-16T10:45:13.981 INFO:tasks.workunit.client.1.smithi026.stdout:9/254: rename d0/da/d16/c43 to d0/c54 0 2024-03-16T10:45:13.984 INFO:tasks.workunit.client.1.smithi026.stdout:9/255: creat d0/da/d13/f55 x:0 0 0 2024-03-16T10:45:13.985 INFO:tasks.workunit.client.1.smithi026.stdout:8/252: dwrite d5/d7/f1b [0,4194304] 0 2024-03-16T10:45:13.987 INFO:tasks.workunit.client.1.smithi026.stdout:9/256: read d0/da/d14/f3d [442626,116420] 0 2024-03-16T10:45:13.990 INFO:tasks.workunit.client.1.smithi026.stdout:9/257: symlink d0/da/d16/d17/d44/l56 0 2024-03-16T10:45:13.993 INFO:tasks.workunit.client.1.smithi026.stdout:9/258: mknod d0/da/d16/c57 0 2024-03-16T10:45:13.994 INFO:tasks.workunit.client.1.smithi026.stdout:9/259: truncate d0/da/d13/f38 801388 0 2024-03-16T10:45:13.995 INFO:tasks.workunit.client.1.smithi026.stdout:9/260: write d0/f1f [1898526,15685] 0 2024-03-16T10:45:13.997 INFO:tasks.workunit.client.1.smithi026.stdout:8/253: write d5/d7/f1c [4644569,7326] 0 2024-03-16T10:45:13.998 INFO:tasks.workunit.client.1.smithi026.stdout:8/254: chown d5/dc/f18 195 1 2024-03-16T10:45:13.999 INFO:tasks.workunit.client.1.smithi026.stdout:9/261: rename d0/da/f4b to d0/da/d4d/f58 0 2024-03-16T10:45:14.001 INFO:tasks.workunit.client.1.smithi026.stdout:8/255: rename d5/dc/f15 to d5/d7/f43 0 2024-03-16T10:45:14.002 INFO:tasks.workunit.client.1.smithi026.stdout:9/262: mknod d0/da/d16/c59 0 2024-03-16T10:45:14.002 INFO:tasks.workunit.client.1.smithi026.stdout:8/256: chown d5/dc/f18 224677 1 2024-03-16T10:45:14.003 INFO:tasks.workunit.client.1.smithi026.stdout:8/257: write d5/dc/f38 [1633477,124448] 0 2024-03-16T10:45:14.007 INFO:tasks.workunit.client.1.smithi026.stdout:8/258: unlink d5/c27 0 2024-03-16T10:45:14.010 INFO:tasks.workunit.client.1.smithi026.stdout:9/263: getdents d0/da/d13 0 2024-03-16T10:45:14.015 INFO:tasks.workunit.client.1.smithi026.stdout:8/259: link d5/d7/ca d5/dc/d3f/c44 0 2024-03-16T10:45:14.015 INFO:tasks.workunit.client.1.smithi026.stdout:9/264: dwrite d0/da/f2a [0,4194304] 0 2024-03-16T10:45:14.019 INFO:tasks.workunit.client.1.smithi026.stdout:8/260: mknod d5/c45 0 2024-03-16T10:45:14.024 INFO:tasks.workunit.client.1.smithi026.stdout:9/265: dwrite d0/da/f27 [0,4194304] 0 2024-03-16T10:45:14.029 INFO:tasks.workunit.client.1.smithi026.stdout:9/266: getdents d0/da/d13 0 2024-03-16T10:45:14.029 INFO:tasks.workunit.client.1.smithi026.stdout:8/261: dwrite d5/dc/f38 [0,4194304] 0 2024-03-16T10:45:14.035 INFO:tasks.workunit.client.1.smithi026.stdout:9/267: dread d0/da/d13/f26 [0,4194304] 0 2024-03-16T10:45:14.038 INFO:tasks.workunit.client.1.smithi026.stdout:9/268: creat d0/da/d13/f5a x:0 0 0 2024-03-16T10:45:14.041 INFO:tasks.workunit.client.1.smithi026.stdout:9/269: mkdir d0/da/d5b 0 2024-03-16T10:45:14.043 INFO:tasks.workunit.client.1.smithi026.stdout:9/270: creat d0/da/d4d/f5c x:0 0 0 2024-03-16T10:45:14.045 INFO:tasks.workunit.client.1.smithi026.stdout:9/271: mkdir d0/da/d16/d5d 0 2024-03-16T10:45:14.048 INFO:tasks.workunit.client.1.smithi026.stdout:9/272: getdents d0/da/d16/d5d 0 2024-03-16T10:45:14.048 INFO:tasks.workunit.client.1.smithi026.stdout:8/262: dwrite d5/d7/f1d [0,4194304] 0 2024-03-16T10:45:14.048 INFO:tasks.workunit.client.1.smithi026.stdout:9/273: chown d0 14592916 1 2024-03-16T10:45:14.051 INFO:tasks.workunit.client.1.smithi026.stdout:8/263: creat d5/dc/d3f/f46 x:0 0 0 2024-03-16T10:45:14.053 INFO:tasks.workunit.client.1.smithi026.stdout:9/274: dread d0/da/f2a [0,4194304] 0 2024-03-16T10:45:14.058 INFO:tasks.workunit.client.1.smithi026.stdout:9/275: dwrite d0/f3c [0,4194304] 0 2024-03-16T10:45:14.064 INFO:tasks.workunit.client.1.smithi026.stdout:9/276: rename d0/da/d4d/f58 to d0/da/d4d/f5e 0 2024-03-16T10:45:14.065 INFO:tasks.workunit.client.1.smithi026.stdout:9/277: fdatasync d0/da/d13/f5a 0 2024-03-16T10:45:14.065 INFO:tasks.workunit.client.1.smithi026.stdout:8/264: write d5/d28/f31 [2470379,86039] 0 2024-03-16T10:45:14.067 INFO:tasks.workunit.client.1.smithi026.stdout:9/278: write d0/da/d16/d17/d44/f24 [1894550,129201] 0 2024-03-16T10:45:14.069 INFO:tasks.workunit.client.1.smithi026.stdout:8/265: symlink d5/d7/d2c/d36/l47 0 2024-03-16T10:45:14.098 INFO:tasks.workunit.client.1.smithi026.stdout:8/266: dread d5/d28/f3c [0,4194304] 0 2024-03-16T10:45:14.098 INFO:tasks.workunit.client.1.smithi026.stdout:2/254: sync 2024-03-16T10:45:14.103 INFO:tasks.workunit.client.1.smithi026.stdout:8/267: rename d5/d7/ca to d5/d28/c48 0 2024-03-16T10:45:14.104 INFO:tasks.workunit.client.1.smithi026.stdout:8/268: readlink d5/l41 0 2024-03-16T10:45:14.104 INFO:tasks.workunit.client.1.smithi026.stdout:9/279: dwrite d0/da/f12 [0,4194304] 0 2024-03-16T10:45:14.107 INFO:tasks.workunit.client.1.smithi026.stdout:2/255: dwrite d2/d29/d17/d1a/d2b/f18 [4194304,4194304] 0 2024-03-16T10:45:14.108 INFO:tasks.workunit.client.1.smithi026.stdout:8/269: creat d5/d7/f49 x:0 0 0 2024-03-16T10:45:14.110 INFO:tasks.workunit.client.1.smithi026.stdout:9/280: mkdir d0/d5f 0 2024-03-16T10:45:14.112 INFO:tasks.workunit.client.1.smithi026.stdout:9/281: write d0/da/d13/f31 [43423,49643] 0 2024-03-16T10:45:14.114 INFO:tasks.workunit.client.1.smithi026.stdout:8/270: creat d5/f4a x:0 0 0 2024-03-16T10:45:14.115 INFO:tasks.workunit.client.1.smithi026.stdout:9/282: symlink d0/da/d16/d17/l60 0 2024-03-16T10:45:14.116 INFO:tasks.workunit.client.1.smithi026.stdout:9/283: readlink d0/da/d16/d17/l28 0 2024-03-16T10:45:14.119 INFO:tasks.workunit.client.1.smithi026.stdout:8/271: dread d5/d7/f1b [0,4194304] 0 2024-03-16T10:45:14.120 INFO:tasks.workunit.client.1.smithi026.stdout:9/284: mkdir d0/da/d16/d17/d61 0 2024-03-16T10:45:14.120 INFO:tasks.workunit.client.1.smithi026.stdout:8/272: stat d5/d7/f1b 0 2024-03-16T10:45:14.121 INFO:tasks.workunit.client.1.smithi026.stdout:2/256: link d2/d29/d17/d1a/l2f d2/d29/l54 0 2024-03-16T10:45:14.123 INFO:tasks.workunit.client.1.smithi026.stdout:8/273: write d5/f3e [3757983,2897] 0 2024-03-16T10:45:14.124 INFO:tasks.workunit.client.1.smithi026.stdout:2/257: mkdir d2/d29/d51/d55 0 2024-03-16T10:45:14.126 INFO:tasks.workunit.client.1.smithi026.stdout:8/274: truncate d5/dc/d3f/f46 614671 0 2024-03-16T10:45:14.128 INFO:tasks.workunit.client.1.smithi026.stdout:8/275: fdatasync d5/d28/f39 0 2024-03-16T10:45:14.129 INFO:tasks.workunit.client.1.smithi026.stdout:9/285: dwrite d0/da/d16/d17/d44/f41 [0,4194304] 0 2024-03-16T10:45:14.131 INFO:tasks.workunit.client.1.smithi026.stdout:8/276: readlink d5/d7/d2c/d36/l47 0 2024-03-16T10:45:14.132 INFO:tasks.workunit.client.1.smithi026.stdout:8/277: chown d5/d7/d2c/f32 48489 1 2024-03-16T10:45:14.133 INFO:tasks.workunit.client.1.smithi026.stdout:2/258: rename d2/d29/d17/d1a/d2b/d2e/f48 to d2/d29/f56 0 2024-03-16T10:45:14.135 INFO:tasks.workunit.client.1.smithi026.stdout:2/259: write d2/d29/d36/f3e [955592,20810] 0 2024-03-16T10:45:14.137 INFO:tasks.workunit.client.0.smithi002.stdout:7/960: sync 2024-03-16T10:45:14.137 INFO:tasks.workunit.client.0.smithi002.stdout:4/978: sync 2024-03-16T10:45:14.137 INFO:tasks.workunit.client.1.smithi026.stdout:9/286: rename d0/da/d16/d17/d61 to d0/da/d4d/d4f/d62 0 2024-03-16T10:45:14.138 INFO:tasks.workunit.client.0.smithi002.stdout:4/979: fdatasync d2/d6/d32/d66/d7f/de7/f7d 0 2024-03-16T10:45:14.140 INFO:tasks.workunit.client.1.smithi026.stdout:2/260: rename d2/d29/d17/d1a/d2b/d2e/f50 to d2/d29/d17/d1a/d2b/d4b/f57 0 2024-03-16T10:45:14.142 INFO:tasks.workunit.client.0.smithi002.stdout:7/961: fsync d3/d4/d1d/d87/d5c/d63/d55/f113 0 2024-03-16T10:45:14.144 INFO:tasks.workunit.client.1.smithi026.stdout:2/261: creat d2/d29/d36/f58 x:0 0 0 2024-03-16T10:45:14.147 INFO:tasks.workunit.client.1.smithi026.stdout:9/287: dwrite d0/da/d14/f3d [0,4194304] 0 2024-03-16T10:45:14.148 INFO:tasks.workunit.client.0.smithi002.stdout:7/962: dwrite d3/f14 [0,4194304] 0 2024-03-16T10:45:14.149 INFO:tasks.workunit.client.1.smithi026.stdout:8/278: truncate d5/f3e 3012947 0 2024-03-16T10:45:14.156 INFO:tasks.workunit.client.1.smithi026.stdout:8/279: write d5/d7/f1b [4860757,83181] 0 2024-03-16T10:45:14.158 INFO:tasks.workunit.client.0.smithi002.stdout:4/980: dread d2/fad [0,4194304] 0 2024-03-16T10:45:14.158 INFO:tasks.workunit.client.1.smithi026.stdout:8/280: truncate d5/d28/f39 193825 0 2024-03-16T10:45:14.158 INFO:tasks.workunit.client.1.smithi026.stdout:9/288: getdents d0/da/d13 0 2024-03-16T10:45:14.160 INFO:tasks.workunit.client.1.smithi026.stdout:9/289: mkdir d0/d63 0 2024-03-16T10:45:14.161 INFO:tasks.workunit.client.1.smithi026.stdout:8/281: creat d5/d23/f4b x:0 0 0 2024-03-16T10:45:14.161 INFO:tasks.workunit.client.1.smithi026.stdout:9/290: write d0/da/f27 [2941839,8555] 0 2024-03-16T10:45:14.163 INFO:tasks.workunit.client.1.smithi026.stdout:8/282: dread d5/dc/d3f/f46 [0,4194304] 0 2024-03-16T10:45:14.164 INFO:tasks.workunit.client.1.smithi026.stdout:8/283: stat d5/ld 0 2024-03-16T10:45:14.167 INFO:tasks.workunit.client.1.smithi026.stdout:8/284: fdatasync d5/d7/f43 0 2024-03-16T10:45:14.168 INFO:tasks.workunit.client.0.smithi002.stdout:7/963: write d3/d4/d1c/f81 [2513864,101371] 0 2024-03-16T10:45:14.170 INFO:tasks.workunit.client.0.smithi002.stdout:7/964: chown d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/lb2 248445900 1 2024-03-16T10:45:14.171 INFO:tasks.workunit.client.1.smithi026.stdout:8/285: unlink l2 0 2024-03-16T10:45:14.172 INFO:tasks.workunit.client.1.smithi026.stdout:9/291: link d0/c21 d0/da/d4d/d4f/c64 0 2024-03-16T10:45:14.173 INFO:tasks.workunit.client.1.smithi026.stdout:9/292: write d0/da/d13/f38 [343079,51501] 0 2024-03-16T10:45:14.176 INFO:tasks.workunit.client.0.smithi002.stdout:4/981: dwrite d2/d6/d32/fa6 [0,4194304] 0 2024-03-16T10:45:14.176 INFO:tasks.workunit.client.0.smithi002.stdout:7/965: dwrite d3/d4/d1d/d87/d5c/d63/d34/f98 [0,4194304] 0 2024-03-16T10:45:14.177 INFO:tasks.workunit.client.1.smithi026.stdout:9/293: creat d0/da/d4d/d4f/d62/f65 x:0 0 0 2024-03-16T10:45:14.184 INFO:tasks.workunit.client.1.smithi026.stdout:9/294: creat d0/f66 x:0 0 0 2024-03-16T10:45:14.185 INFO:tasks.workunit.client.0.smithi002.stdout:7/966: mkdir d3/d4/d1d/d87/d5c/d63/d3f/daa/d15a 0 2024-03-16T10:45:14.189 INFO:tasks.workunit.client.1.smithi026.stdout:8/286: truncate d5/f3e 2482725 0 2024-03-16T10:45:14.190 INFO:tasks.workunit.client.1.smithi026.stdout:8/287: fsync d5/d7/f1d 0 2024-03-16T10:45:14.191 INFO:tasks.workunit.client.0.smithi002.stdout:7/967: fdatasync d3/d4/d1d/d87/d5c/d63/d3f/dba/d100/f53 0 2024-03-16T10:45:14.192 INFO:tasks.workunit.client.0.smithi002.stdout:4/982: write d2/d33/fcd [1754202,85261] 0 2024-03-16T10:45:14.200 INFO:tasks.workunit.client.1.smithi026.stdout:9/295: dread d0/da/f11 [0,4194304] 0 2024-03-16T10:45:14.201 INFO:tasks.workunit.client.1.smithi026.stdout:8/288: creat d5/d7/d2c/d36/f4c x:0 0 0 2024-03-16T10:45:14.202 INFO:tasks.workunit.client.1.smithi026.stdout:8/289: chown d5/d23/c33 45200819 1 2024-03-16T10:45:14.203 INFO:tasks.workunit.client.0.smithi002.stdout:7/968: creat d3/d4/d1d/d87/d5c/f15b x:0 0 0 2024-03-16T10:45:14.204 INFO:tasks.workunit.client.0.smithi002.stdout:7/969: chown d3/d4/d1d/d87/d5c/d63/d3f 2850779 1 2024-03-16T10:45:14.204 INFO:tasks.workunit.client.0.smithi002.stdout:4/983: rmdir d2/d33/d1c/d45 39 2024-03-16T10:45:14.207 INFO:tasks.workunit.client.1.smithi026.stdout:9/296: dwrite d0/da/d14/f39 [0,4194304] 0 2024-03-16T10:45:14.207 INFO:tasks.workunit.client.0.smithi002.stdout:4/984: unlink d2/d6/l11d 0 2024-03-16T10:45:14.210 INFO:tasks.workunit.client.0.smithi002.stdout:7/970: mkdir d3/d4/d1d/d87/d5c/dd0/d109/d15c 0 2024-03-16T10:45:14.211 INFO:tasks.workunit.client.1.smithi026.stdout:9/297: dread d0/da/d13/f26 [0,4194304] 0 2024-03-16T10:45:14.213 INFO:tasks.workunit.client.1.smithi026.stdout:9/298: write d0/da/f12 [965360,81602] 0 2024-03-16T10:45:14.215 INFO:tasks.workunit.client.0.smithi002.stdout:7/971: dread d3/f91 [0,4194304] 0 2024-03-16T10:45:14.217 INFO:tasks.workunit.client.0.smithi002.stdout:7/972: chown d3/d4/d1c/d5b/d64/d99/d9c/ffb 403 1 2024-03-16T10:45:14.218 INFO:tasks.workunit.client.1.smithi026.stdout:9/299: rename d0/da/d14/c48 to d0/da/d5b/c67 0 2024-03-16T10:45:14.219 INFO:tasks.workunit.client.1.smithi026.stdout:9/300: write d0/da/d16/d17/f40 [479687,119976] 0 2024-03-16T10:45:14.220 INFO:tasks.workunit.client.1.smithi026.stdout:9/301: fsync d0/da/d16/d17/d44/f42 0 2024-03-16T10:45:14.222 INFO:tasks.workunit.client.1.smithi026.stdout:9/302: write d0/da/d16/d17/f40 [431909,107826] 0 2024-03-16T10:45:14.223 INFO:tasks.workunit.client.0.smithi002.stdout:4/985: link d2/d33/d10a/f10f d2/d33/d1c/d45/f131 0 2024-03-16T10:45:14.224 INFO:tasks.workunit.client.1.smithi026.stdout:8/290: truncate d5/d7/f1b 1354438 0 2024-03-16T10:45:14.227 INFO:tasks.workunit.client.1.smithi026.stdout:9/303: chown d0/da/d4d/f5e 112 1 2024-03-16T10:45:14.229 INFO:tasks.workunit.client.1.smithi026.stdout:9/304: write d0/da/d13/f5a [1043490,85464] 0 2024-03-16T10:45:14.230 INFO:tasks.workunit.client.0.smithi002.stdout:7/973: write d3/d4/d1d/d87/d5c/dd0/dd2/fe9 [2635643,29453] 0 2024-03-16T10:45:14.231 INFO:tasks.workunit.client.1.smithi026.stdout:8/291: dwrite d5/d28/f31 [0,4194304] 0 2024-03-16T10:45:14.232 INFO:tasks.workunit.client.0.smithi002.stdout:4/986: unlink d2/d6/d32/d66/d7f/de7/d5e/c110 0 2024-03-16T10:45:14.232 INFO:tasks.workunit.client.0.smithi002.stdout:4/987: chown d2/d6/f17 29300 1 2024-03-16T10:45:14.232 INFO:tasks.workunit.client.1.smithi026.stdout:9/305: creat d0/f68 x:0 0 0 2024-03-16T10:45:14.233 INFO:tasks.workunit.client.0.smithi002.stdout:7/974: mknod d3/d4/dde/c15d 0 2024-03-16T10:45:14.237 INFO:tasks.workunit.client.1.smithi026.stdout:9/306: creat d0/da/d13/f69 x:0 0 0 2024-03-16T10:45:14.237 INFO:tasks.workunit.client.1.smithi026.stdout:8/292: dread d5/f3e [0,4194304] 0 2024-03-16T10:45:14.237 INFO:tasks.workunit.client.1.smithi026.stdout:9/307: readlink d0/da/l4c 0 2024-03-16T10:45:14.237 INFO:tasks.workunit.client.0.smithi002.stdout:7/975: mknod d3/d61/d85/c15e 0 2024-03-16T10:45:14.238 INFO:tasks.workunit.client.0.smithi002.stdout:4/988: mknod d2/c132 0 2024-03-16T10:45:14.239 INFO:tasks.workunit.client.1.smithi026.stdout:9/308: creat d0/da/d14/f6a x:0 0 0 2024-03-16T10:45:14.241 INFO:tasks.workunit.client.0.smithi002.stdout:7/976: mknod d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/c15f 0 2024-03-16T10:45:14.241 INFO:tasks.workunit.client.1.smithi026.stdout:8/293: dread d5/d7/f1c [0,4194304] 0 2024-03-16T10:45:14.242 INFO:tasks.workunit.client.0.smithi002.stdout:4/989: creat d2/d6/d32/d66/d7f/f133 x:0 0 0 2024-03-16T10:45:14.244 INFO:tasks.workunit.client.1.smithi026.stdout:9/309: unlink d0/f1f 0 2024-03-16T10:45:14.245 INFO:tasks.workunit.client.1.smithi026.stdout:8/294: dread d5/d7/d2c/f32 [0,4194304] 0 2024-03-16T10:45:14.246 INFO:tasks.workunit.client.1.smithi026.stdout:9/310: write d0/da/d13/f38 [844772,60792] 0 2024-03-16T10:45:14.246 INFO:tasks.workunit.client.0.smithi002.stdout:7/977: mkdir d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/d160 0 2024-03-16T10:45:14.247 INFO:tasks.workunit.client.1.smithi026.stdout:8/295: dread d5/d28/f39 [0,4194304] 0 2024-03-16T10:45:14.248 INFO:tasks.workunit.client.1.smithi026.stdout:8/296: readlink d5/d23/l2f 0 2024-03-16T10:45:14.248 INFO:tasks.workunit.client.1.smithi026.stdout:9/311: dread d0/da/d14/f25 [0,4194304] 0 2024-03-16T10:45:14.249 INFO:tasks.workunit.client.1.smithi026.stdout:5/209: sync 2024-03-16T10:45:14.249 INFO:tasks.workunit.client.1.smithi026.stdout:4/294: sync 2024-03-16T10:45:14.249 INFO:tasks.workunit.client.1.smithi026.stdout:6/316: sync 2024-03-16T10:45:14.249 INFO:tasks.workunit.client.1.smithi026.stdout:8/297: write d5/f4a [923928,112565] 0 2024-03-16T10:45:14.250 INFO:tasks.workunit.client.1.smithi026.stdout:9/312: readlink d0/da/d4d/d4f/l50 0 2024-03-16T10:45:14.251 INFO:tasks.workunit.client.0.smithi002.stdout:4/990: creat d2/f134 x:0 0 0 2024-03-16T10:45:14.252 INFO:tasks.workunit.client.1.smithi026.stdout:6/317: write d0/d28/d49/f52 [1182670,65139] 0 2024-03-16T10:45:14.252 INFO:tasks.workunit.client.1.smithi026.stdout:8/298: write d5/d7/f49 [827232,87811] 0 2024-03-16T10:45:14.253 INFO:tasks.workunit.client.1.smithi026.stdout:9/313: truncate d0/da/d13/f69 476379 0 2024-03-16T10:45:14.254 INFO:tasks.workunit.client.1.smithi026.stdout:8/299: chown d5/d7/d2c/f2d 185 1 2024-03-16T10:45:14.258 INFO:tasks.workunit.client.1.smithi026.stdout:6/318: mknod d0/d5b/c74 0 2024-03-16T10:45:14.263 INFO:tasks.workunit.client.1.smithi026.stdout:4/295: creat d1/db/f5d x:0 0 0 2024-03-16T10:45:14.270 INFO:tasks.workunit.client.1.smithi026.stdout:5/210: link d1/db/d27/l2e d1/db/d27/l43 0 2024-03-16T10:45:14.271 INFO:tasks.workunit.client.1.smithi026.stdout:9/314: mkdir d0/d63/d6b 0 2024-03-16T10:45:14.272 INFO:tasks.workunit.client.1.smithi026.stdout:9/315: dread - d0/da/d4d/f5c zero size 2024-03-16T10:45:14.273 INFO:tasks.workunit.client.1.smithi026.stdout:6/319: symlink d0/d41/d71/l75 0 2024-03-16T10:45:14.277 INFO:tasks.workunit.client.1.smithi026.stdout:4/296: dread d1/f27 [4194304,4194304] 0 2024-03-16T10:45:14.277 INFO:tasks.workunit.client.1.smithi026.stdout:9/316: unlink d0/da/d16/d17/d44/f41 0 2024-03-16T10:45:14.278 INFO:tasks.workunit.client.0.smithi002.stdout:7/978: write d3/d37/d6a/f72 [5201538,41022] 0 2024-03-16T10:45:14.283 INFO:tasks.workunit.client.1.smithi026.stdout:8/300: truncate d5/d28/f39 988904 0 2024-03-16T10:45:14.284 INFO:tasks.workunit.client.0.smithi002.stdout:7/979: creat d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f161 x:0 0 0 2024-03-16T10:45:14.286 INFO:tasks.workunit.client.1.smithi026.stdout:5/211: creat d1/db/f44 x:0 0 0 2024-03-16T10:45:14.290 INFO:tasks.workunit.client.0.smithi002.stdout:4/991: write d2/d6/d32/d66/f82 [166581,61339] 0 2024-03-16T10:45:14.290 INFO:tasks.workunit.client.0.smithi002.stdout:7/980: dwrite d3/d4/d1d/d87/d5c/d63/d34/f98 [0,4194304] 0 2024-03-16T10:45:14.298 INFO:tasks.workunit.client.1.smithi026.stdout:5/212: symlink d1/d1c/l45 0 2024-03-16T10:45:14.299 INFO:tasks.workunit.client.1.smithi026.stdout:5/213: write d1/fd [4992153,55519] 0 2024-03-16T10:45:14.300 INFO:tasks.workunit.client.1.smithi026.stdout:4/297: link d1/db/l1f d1/db/de/l5e 0 2024-03-16T10:45:14.302 INFO:tasks.workunit.client.1.smithi026.stdout:5/214: truncate d1/db/d1f/f33 393194 0 2024-03-16T10:45:14.303 INFO:tasks.workunit.client.1.smithi026.stdout:8/301: creat d5/dc/f4d x:0 0 0 2024-03-16T10:45:14.304 INFO:tasks.workunit.client.0.smithi002.stdout:4/992: dread d2/d6/d32/d66/d7f/de7/f49 [0,4194304] 0 2024-03-16T10:45:14.305 INFO:tasks.workunit.client.1.smithi026.stdout:9/317: dread d0/da/d13/f69 [0,4194304] 0 2024-03-16T10:45:14.305 INFO:tasks.workunit.client.1.smithi026.stdout:4/298: creat d1/db/d29/f5f x:0 0 0 2024-03-16T10:45:14.308 INFO:tasks.workunit.client.1.smithi026.stdout:9/318: write d0/da/d16/f3f [329055,19281] 0 2024-03-16T10:45:14.309 INFO:tasks.workunit.client.0.smithi002.stdout:4/993: rename d2/f130 to d2/d6/d32/d68/f135 0 2024-03-16T10:45:14.310 INFO:tasks.workunit.client.1.smithi026.stdout:8/302: dwrite d5/d7/f1d [0,4194304] 0 2024-03-16T10:45:14.311 INFO:tasks.workunit.client.1.smithi026.stdout:8/303: read d5/d7/f1c [3354519,47920] 0 2024-03-16T10:45:14.311 INFO:tasks.workunit.client.0.smithi002.stdout:7/981: dwrite d3/d5f/f8d [0,4194304] 0 2024-03-16T10:45:14.314 INFO:tasks.workunit.client.0.smithi002.stdout:4/994: mkdir d2/d6/d32/d68/d136 0 2024-03-16T10:45:14.314 INFO:tasks.workunit.client.0.smithi002.stdout:7/982: write d3/d4/d1d/d87/d5c/f15b [96703,71225] 0 2024-03-16T10:45:14.314 INFO:tasks.workunit.client.1.smithi026.stdout:9/319: write d0/da/f11 [1473715,93638] 0 2024-03-16T10:45:14.316 INFO:tasks.workunit.client.0.smithi002.stdout:7/983: write d3/d4/d1d/d87/d5c/dc3/fdc [143997,37357] 0 2024-03-16T10:45:14.317 INFO:tasks.workunit.client.1.smithi026.stdout:8/304: chown d5/d7/d2c/d36 3728145 1 2024-03-16T10:45:14.319 INFO:tasks.workunit.client.0.smithi002.stdout:4/995: dread d2/d6/f13 [0,4194304] 0 2024-03-16T10:45:14.320 INFO:tasks.workunit.client.1.smithi026.stdout:4/299: creat d1/db/d29/d44/d48/f60 x:0 0 0 2024-03-16T10:45:14.321 INFO:tasks.workunit.client.1.smithi026.stdout:5/215: getdents d1 0 2024-03-16T10:45:14.321 INFO:tasks.workunit.client.1.smithi026.stdout:5/216: chown d1/d1c 199 1 2024-03-16T10:45:14.324 INFO:tasks.workunit.client.1.smithi026.stdout:8/305: dread d5/d7/f49 [0,4194304] 0 2024-03-16T10:45:14.324 INFO:tasks.workunit.client.0.smithi002.stdout:7/984: unlink d3/d4/d1c/f33 0 2024-03-16T10:45:14.325 INFO:tasks.workunit.client.1.smithi026.stdout:9/320: mkdir d0/da/d16/d17/d6c 0 2024-03-16T10:45:14.326 INFO:tasks.workunit.client.1.smithi026.stdout:9/321: write d0/da/d13/f53 [167361,121110] 0 2024-03-16T10:45:14.331 INFO:tasks.workunit.client.1.smithi026.stdout:8/306: unlink d5/dc/f29 0 2024-03-16T10:45:14.334 INFO:tasks.workunit.client.0.smithi002.stdout:4/996: rename d2/d6/d114 to d2/d137 0 2024-03-16T10:45:14.338 INFO:tasks.workunit.client.1.smithi026.stdout:4/300: write d1/db/d29/f43 [5063459,121654] 0 2024-03-16T10:45:14.344 INFO:tasks.workunit.client.0.smithi002.stdout:4/997: write d2/d6/d32/d68/f11c [33596,118933] 0 2024-03-16T10:45:14.345 INFO:tasks.workunit.client.1.smithi026.stdout:8/307: mkdir d5/d23/d4e 0 2024-03-16T10:45:14.346 INFO:tasks.workunit.client.1.smithi026.stdout:5/217: truncate d1/db/fc 11766773 0 2024-03-16T10:45:14.349 INFO:tasks.workunit.client.1.smithi026.stdout:4/301: mkdir d1/db/d29/d61 0 2024-03-16T10:45:14.351 INFO:tasks.workunit.client.0.smithi002.stdout:7/985: dwrite d3/ff8 [0,4194304] 0 2024-03-16T10:45:14.352 INFO:tasks.workunit.client.1.smithi026.stdout:5/218: mknod d1/db/d27/c46 0 2024-03-16T10:45:14.358 INFO:tasks.workunit.client.1.smithi026.stdout:5/219: creat d1/d1c/d2a/f47 x:0 0 0 2024-03-16T10:45:14.359 INFO:tasks.workunit.client.1.smithi026.stdout:9/322: link d0/da/l4c d0/l6d 0 2024-03-16T10:45:14.365 INFO:tasks.workunit.client.1.smithi026.stdout:5/220: creat d1/f48 x:0 0 0 2024-03-16T10:45:14.367 INFO:tasks.workunit.client.1.smithi026.stdout:4/302: write d1/f3 [9068145,103182] 0 2024-03-16T10:45:14.369 INFO:tasks.workunit.client.1.smithi026.stdout:5/221: creat d1/d1c/d2a/d31/f49 x:0 0 0 2024-03-16T10:45:14.370 INFO:tasks.workunit.client.1.smithi026.stdout:8/308: truncate f4 2276759 0 2024-03-16T10:45:14.370 INFO:tasks.workunit.client.1.smithi026.stdout:5/222: dread - d1/d1c/d2a/f3b zero size 2024-03-16T10:45:14.371 INFO:tasks.workunit.client.1.smithi026.stdout:8/309: write d5/d7/f1d [1203630,37276] 0 2024-03-16T10:45:14.373 INFO:tasks.workunit.client.1.smithi026.stdout:9/323: rename d0/da/d5b to d0/da/d13/d6e 0 2024-03-16T10:45:14.382 INFO:tasks.workunit.client.1.smithi026.stdout:4/303: creat d1/db/de/d4c/d59/f62 x:0 0 0 2024-03-16T10:45:14.385 INFO:tasks.workunit.client.1.smithi026.stdout:8/310: link d5/dc/d3f/f20 d5/d7/f4f 0 2024-03-16T10:45:14.387 INFO:tasks.workunit.client.1.smithi026.stdout:9/324: mknod d0/d63/c6f 0 2024-03-16T10:45:14.390 INFO:tasks.workunit.client.1.smithi026.stdout:8/311: fdatasync d5/d7/f1a 0 2024-03-16T10:45:14.392 INFO:tasks.workunit.client.1.smithi026.stdout:4/304: mkdir d1/db/d29/d61/d63 0 2024-03-16T10:45:14.401 INFO:tasks.workunit.client.1.smithi026.stdout:5/223: getdents d1/d1c/d2a/d31 0 2024-03-16T10:45:14.406 INFO:tasks.workunit.client.1.smithi026.stdout:8/312: creat d5/d7/d2c/f50 x:0 0 0 2024-03-16T10:45:14.413 INFO:tasks.workunit.client.1.smithi026.stdout:8/313: dread d5/d28/f39 [0,4194304] 0 2024-03-16T10:45:14.416 INFO:tasks.workunit.client.1.smithi026.stdout:5/224: mknod d1/db/c4a 0 2024-03-16T10:45:14.418 INFO:tasks.workunit.client.1.smithi026.stdout:8/314: mknod d5/dc/d40/c51 0 2024-03-16T10:45:14.419 INFO:tasks.workunit.client.1.smithi026.stdout:4/305: rmdir d1/db/d29/d61/d63 0 2024-03-16T10:45:14.420 INFO:tasks.workunit.client.1.smithi026.stdout:4/306: write d1/f56 [760465,84496] 0 2024-03-16T10:45:14.422 INFO:tasks.workunit.client.1.smithi026.stdout:4/307: write d1/f2d [3287689,81836] 0 2024-03-16T10:45:14.425 INFO:tasks.workunit.client.1.smithi026.stdout:8/315: creat d5/dc/d40/f52 x:0 0 0 2024-03-16T10:45:14.426 INFO:tasks.workunit.client.1.smithi026.stdout:8/316: truncate d5/d7/f43 4306959 0 2024-03-16T10:45:14.428 INFO:tasks.workunit.client.1.smithi026.stdout:8/317: truncate d5/d7/d2c/f50 438031 0 2024-03-16T10:45:14.428 INFO:tasks.workunit.client.1.smithi026.stdout:5/225: dread d1/db/fc [8388608,4194304] 0 2024-03-16T10:45:14.431 INFO:tasks.workunit.client.1.smithi026.stdout:8/318: mkdir d5/dc/d40/d53 0 2024-03-16T10:45:14.434 INFO:tasks.workunit.client.1.smithi026.stdout:4/308: creat d1/db/f64 x:0 0 0 2024-03-16T10:45:14.435 INFO:tasks.workunit.client.1.smithi026.stdout:3/306: sync 2024-03-16T10:45:14.435 INFO:tasks.workunit.client.1.smithi026.stdout:5/226: mknod d1/db/d40/c4b 0 2024-03-16T10:45:14.437 INFO:tasks.workunit.client.1.smithi026.stdout:4/309: creat d1/db/de/f65 x:0 0 0 2024-03-16T10:45:14.439 INFO:tasks.workunit.client.1.smithi026.stdout:3/307: read d4/dc/d31/f34 [2821401,54504] 0 2024-03-16T10:45:14.439 INFO:tasks.workunit.client.1.smithi026.stdout:4/310: dread - d1/db/d29/f5f zero size 2024-03-16T10:45:14.453 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:14 smithi026 ceph-mon[37041]: pgmap v40: 65 pgs: 65 active+clean; 2.3 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 73 MiB/s rd, 185 MiB/s wr, 399 op/s 2024-03-16T10:45:14.468 INFO:tasks.workunit.client.0.smithi002.stdout:4/998: mkdir d2/d6/d32/d66/d7f/de7/d5e/d138 0 2024-03-16T10:45:14.469 INFO:tasks.workunit.client.0.smithi002.stdout:7/986: fsync d3/d4/d1c/d5b/d64/ddf/fe8 0 2024-03-16T10:45:14.472 INFO:tasks.workunit.client.1.smithi026.stdout:5/227: dwrite d1/db/f44 [0,4194304] 0 2024-03-16T10:45:14.472 INFO:tasks.workunit.client.1.smithi026.stdout:9/325: write d0/da/d16/d17/f2e [387148,128924] 0 2024-03-16T10:45:14.472 INFO:tasks.workunit.client.0.smithi002.stdout:4/999: dread d2/d6/fb8 [0,4194304] 0 2024-03-16T10:45:14.474 INFO:tasks.workunit.client.1.smithi026.stdout:3/308: dread d4/dc/d31/f51 [0,4194304] 0 2024-03-16T10:45:14.475 INFO:tasks.workunit.client.1.smithi026.stdout:3/309: readlink d4/dc/l2d 0 2024-03-16T10:45:14.477 INFO:tasks.workunit.client.1.smithi026.stdout:3/310: write d4/dc/dd/d60/f64 [826197,113864] 0 2024-03-16T10:45:14.478 INFO:tasks.workunit.client.1.smithi026.stdout:5/228: mkdir d1/d1c/d4c 0 2024-03-16T10:45:14.479 INFO:tasks.workunit.client.1.smithi026.stdout:9/326: creat d0/d63/d6b/f70 x:0 0 0 2024-03-16T10:45:14.490 INFO:tasks.workunit.client.0.smithi002.stdout:7/987: dread d3/d4/d1d/d87/d96/d5e/fbd [0,4194304] 0 2024-03-16T10:45:14.490 INFO:tasks.workunit.client.0.smithi002.stdout:7/988: stat d3/d4/dde/c15d 0 2024-03-16T10:45:14.601 INFO:tasks.workunit.client.1.smithi026.stdout:3/311: creat d4/dc/d31/d48/d69/f6f x:0 0 0 2024-03-16T10:45:14.602 INFO:tasks.workunit.client.1.smithi026.stdout:5/229: mkdir d1/d1c/d4c/d4d 0 2024-03-16T10:45:14.604 INFO:tasks.workunit.client.1.smithi026.stdout:5/230: write d1/db/f44 [3227141,111442] 0 2024-03-16T10:45:14.604 INFO:tasks.workunit.client.0.smithi002.stdout:7/989: symlink d3/d4/d1d/d87/d5c/dd0/d109/d15c/l162 0 2024-03-16T10:45:14.604 INFO:tasks.workunit.client.1.smithi026.stdout:3/312: fsync d4/dc/dd/f21 0 2024-03-16T10:45:14.610 INFO:tasks.workunit.client.1.smithi026.stdout:3/313: rmdir d4/dc/dd/d47/d53 39 2024-03-16T10:45:14.611 INFO:tasks.workunit.client.1.smithi026.stdout:3/314: fdatasync d4/dc/f43 0 2024-03-16T10:45:14.612 INFO:tasks.workunit.client.1.smithi026.stdout:5/231: dwrite d1/db/d1f/f33 [0,4194304] 0 2024-03-16T10:45:14.617 INFO:tasks.workunit.client.1.smithi026.stdout:5/232: creat d1/d1c/d4c/d4d/f4e x:0 0 0 2024-03-16T10:45:14.621 INFO:tasks.workunit.client.1.smithi026.stdout:3/315: link d4/d29/f30 d4/dc/dd/d60/d62/d4f/f70 0 2024-03-16T10:45:14.622 INFO:tasks.workunit.client.1.smithi026.stdout:3/316: write d4/dc/dd/d4b/f5b [397260,121753] 0 2024-03-16T10:45:14.624 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:14 smithi002 ceph-mon[29248]: pgmap v40: 65 pgs: 65 active+clean; 2.3 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 73 MiB/s rd, 185 MiB/s wr, 399 op/s 2024-03-16T10:45:14.629 INFO:tasks.workunit.client.1.smithi026.stdout:3/317: rename d4/dc/dd/d47/d53/l54 to d4/dc/dd/d60/d62/l71 0 2024-03-16T10:45:14.630 INFO:tasks.workunit.client.1.smithi026.stdout:3/318: read d4/dc/dd/d3d/f49 [3086954,62460] 0 2024-03-16T10:45:14.630 INFO:tasks.workunit.client.0.smithi002.stdout:7/990: write d3/d4/d1d/d87/d96/fb8 [2457457,113508] 0 2024-03-16T10:45:14.631 INFO:tasks.workunit.client.1.smithi026.stdout:3/319: write d4/dc/dd/d3d/f6a [366433,35495] 0 2024-03-16T10:45:14.631 INFO:tasks.workunit.client.1.smithi026.stdout:5/233: getdents d1/d1c/d4c/d4d 0 2024-03-16T10:45:14.633 INFO:tasks.workunit.client.1.smithi026.stdout:5/234: write d1/f9 [446598,14072] 0 2024-03-16T10:45:14.635 INFO:tasks.workunit.client.1.smithi026.stdout:5/235: symlink d1/db/d40/l4f 0 2024-03-16T10:45:14.637 INFO:tasks.workunit.client.0.smithi002.stdout:7/991: link d3/d4/d1d/d87/d96/f56 d3/d4/d1d/d87/d5c/dc3/f163 0 2024-03-16T10:45:14.638 INFO:tasks.workunit.client.1.smithi026.stdout:3/320: getdents d4/dc/dd/d4b 0 2024-03-16T10:45:14.639 INFO:tasks.workunit.client.0.smithi002.stdout:7/992: unlink d3/d61/faf 0 2024-03-16T10:45:14.640 INFO:tasks.workunit.client.1.smithi026.stdout:5/236: creat d1/d1c/f50 x:0 0 0 2024-03-16T10:45:14.641 INFO:tasks.workunit.client.1.smithi026.stdout:3/321: truncate d4/dc/dd/d3d/f39 3233196 0 2024-03-16T10:45:14.642 INFO:tasks.workunit.client.1.smithi026.stdout:5/237: mkdir d1/db/d40/d51 0 2024-03-16T10:45:14.643 INFO:tasks.workunit.client.0.smithi002.stdout:7/993: rename d3/d4/d1d/d87/d5c/d63/d3f/d75/dd7/f159 to d3/d4/d1c/d5b/d64/d138/f164 0 2024-03-16T10:45:14.644 INFO:tasks.workunit.client.0.smithi002.stdout:7/994: readlink d3/d4/d1d/d87/d5c/dc3/le4 0 2024-03-16T10:45:14.644 INFO:tasks.workunit.client.0.smithi002.stdout:7/995: stat d3/d61/d85/fb6 0 2024-03-16T10:45:14.645 INFO:tasks.workunit.client.1.smithi026.stdout:5/238: chown d1/db/l1d 9053 1 2024-03-16T10:45:14.646 INFO:tasks.workunit.client.0.smithi002.stdout:7/996: read d3/d4/d1c/d5b/d64/fc1 [199170,82628] 0 2024-03-16T10:45:14.647 INFO:tasks.workunit.client.1.smithi026.stdout:3/322: dread d4/dc/dd/d60/d62/d4f/f70 [0,4194304] 0 2024-03-16T10:45:14.650 INFO:tasks.workunit.client.0.smithi002.stdout:7/997: mkdir d3/d61/d165 0 2024-03-16T10:45:14.651 INFO:tasks.workunit.client.1.smithi026.stdout:5/239: dwrite d1/db/d1f/f33 [4194304,4194304] 0 2024-03-16T10:45:14.652 INFO:tasks.workunit.client.1.smithi026.stdout:5/240: chown d1/db/f22 738740785 1 2024-03-16T10:45:14.652 INFO:tasks.workunit.client.1.smithi026.stdout:3/323: chown d4/dc/f25 15 1 2024-03-16T10:45:14.653 INFO:tasks.workunit.client.1.smithi026.stdout:5/241: chown d1/d1c/d2a/d31 34050236 1 2024-03-16T10:45:14.654 INFO:tasks.workunit.client.1.smithi026.stdout:3/324: readlink d4/l8 0 2024-03-16T10:45:14.655 INFO:tasks.workunit.client.1.smithi026.stdout:3/325: write d4/dc/dd/f21 [200764,71549] 0 2024-03-16T10:45:14.655 INFO:tasks.workunit.client.1.smithi026.stdout:5/242: creat d1/d1c/d2a/d31/f52 x:0 0 0 2024-03-16T10:45:14.657 INFO:tasks.workunit.client.0.smithi002.stdout:7/998: dwrite d3/d4/d1d/d87/d5c/f15b [0,4194304] 0 2024-03-16T10:45:14.658 INFO:tasks.workunit.client.0.smithi002.stdout:7/999: chown d3/d4/d1c/cc5 0 1 2024-03-16T10:45:14.665 INFO:tasks.workunit.client.0.smithi002.stderr:+ rm -rf -- ./tmp.QX75oORedg 2024-03-16T10:45:14.668 INFO:tasks.workunit.client.1.smithi026.stdout:7/295: sync 2024-03-16T10:45:14.668 INFO:tasks.workunit.client.1.smithi026.stdout:0/310: sync 2024-03-16T10:45:14.672 INFO:tasks.workunit.client.1.smithi026.stdout:7/296: write d6/f39 [1288061,45324] 0 2024-03-16T10:45:14.673 INFO:tasks.workunit.client.1.smithi026.stdout:0/311: read d1/d18/d1a/d24/f59 [522205,24473] 0 2024-03-16T10:45:14.677 INFO:tasks.workunit.client.1.smithi026.stdout:3/326: dwrite d4/f18 [4194304,4194304] 0 2024-03-16T10:45:14.680 INFO:tasks.workunit.client.1.smithi026.stdout:5/243: dwrite d1/d1c/d2a/d31/f3a [0,4194304] 0 2024-03-16T10:45:14.682 INFO:tasks.workunit.client.1.smithi026.stdout:0/312: creat d1/d18/d1a/d24/f65 x:0 0 0 2024-03-16T10:45:14.684 INFO:tasks.workunit.client.1.smithi026.stdout:3/327: rmdir d4/dc/dd/d3d 39 2024-03-16T10:45:14.685 INFO:tasks.workunit.client.1.smithi026.stdout:7/297: dwrite d6/d35/d3a/d3c/f40 [0,4194304] 0 2024-03-16T10:45:14.687 INFO:tasks.workunit.client.1.smithi026.stdout:0/313: chown d1/d18/d1a/d24/f29 41604582 1 2024-03-16T10:45:14.688 INFO:tasks.workunit.client.1.smithi026.stdout:5/244: truncate d1/f7 937601 0 2024-03-16T10:45:14.690 INFO:tasks.workunit.client.1.smithi026.stdout:5/245: truncate d1/d1c/f50 555685 0 2024-03-16T10:45:14.690 INFO:tasks.workunit.client.1.smithi026.stdout:5/246: stat d1/c3 0 2024-03-16T10:45:14.691 INFO:tasks.workunit.client.1.smithi026.stdout:5/247: chown l0 12 1 2024-03-16T10:45:14.692 INFO:tasks.workunit.client.1.smithi026.stdout:5/248: read d1/db/f44 [3653882,102354] 0 2024-03-16T10:45:14.693 INFO:tasks.workunit.client.1.smithi026.stdout:7/298: mkdir d6/d35/d3a/d5d 0 2024-03-16T10:45:14.702 INFO:tasks.workunit.client.1.smithi026.stdout:7/299: creat d6/de/d16/f5e x:0 0 0 2024-03-16T10:45:14.706 INFO:tasks.workunit.client.1.smithi026.stdout:7/300: creat d6/d2f/f5f x:0 0 0 2024-03-16T10:45:14.707 INFO:tasks.workunit.client.1.smithi026.stdout:0/314: getdents d1/d12/d2f/d48 0 2024-03-16T10:45:14.707 INFO:tasks.workunit.client.1.smithi026.stdout:3/328: dread d4/dc/dd/f2e [0,4194304] 0 2024-03-16T10:45:14.708 INFO:tasks.workunit.client.1.smithi026.stdout:3/329: chown d4/dc/f16 30 1 2024-03-16T10:45:14.708 INFO:tasks.workunit.client.1.smithi026.stdout:5/249: link d1/db/f12 d1/db/d27/f53 0 2024-03-16T10:45:14.710 INFO:tasks.workunit.client.1.smithi026.stdout:7/301: mknod d6/d35/d3a/d5d/c60 0 2024-03-16T10:45:14.713 INFO:tasks.workunit.client.1.smithi026.stdout:7/302: chown d6/c30 5 1 2024-03-16T10:45:14.713 INFO:tasks.workunit.client.1.smithi026.stdout:3/330: dread d4/dc/dd/d60/f64 [0,4194304] 0 2024-03-16T10:45:14.714 INFO:tasks.workunit.client.1.smithi026.stdout:0/315: mknod d1/d12/d2f/c66 0 2024-03-16T10:45:14.717 INFO:tasks.workunit.client.1.smithi026.stdout:5/250: dwrite d1/f48 [0,4194304] 0 2024-03-16T10:45:14.720 INFO:tasks.workunit.client.1.smithi026.stdout:3/331: dread d4/f18 [4194304,4194304] 0 2024-03-16T10:45:14.721 INFO:tasks.workunit.client.1.smithi026.stdout:3/332: chown d4/fe 202957762 1 2024-03-16T10:45:14.723 INFO:tasks.workunit.client.1.smithi026.stdout:7/303: rename d6/de/d16/c1c to d6/d35/d3a/d3c/c61 0 2024-03-16T10:45:14.724 INFO:tasks.workunit.client.1.smithi026.stdout:5/251: chown d1/db/d27/l43 403 1 2024-03-16T10:45:14.728 INFO:tasks.workunit.client.1.smithi026.stdout:7/304: mknod d6/d2f/c62 0 2024-03-16T10:45:14.729 INFO:tasks.workunit.client.1.smithi026.stdout:7/305: fdatasync d6/d35/f52 0 2024-03-16T10:45:14.730 INFO:tasks.workunit.client.1.smithi026.stdout:0/316: creat d1/d18/d1a/d24/f67 x:0 0 0 2024-03-16T10:45:14.731 INFO:tasks.workunit.client.1.smithi026.stdout:5/252: creat d1/db/d40/d51/f54 x:0 0 0 2024-03-16T10:45:14.732 INFO:tasks.workunit.client.1.smithi026.stdout:5/253: write d1/f5 [3101419,6814] 0 2024-03-16T10:45:14.732 INFO:tasks.workunit.client.1.smithi026.stdout:7/306: fdatasync d6/de/d16/f1f 0 2024-03-16T10:45:14.734 INFO:tasks.workunit.client.1.smithi026.stdout:7/307: write d6/de/d16/f19 [336071,3322] 0 2024-03-16T10:45:14.735 INFO:tasks.workunit.client.1.smithi026.stdout:0/317: symlink d1/d18/d1a/d2b/l68 0 2024-03-16T10:45:14.738 INFO:tasks.workunit.client.1.smithi026.stdout:5/254: rmdir d1/d1c/d2a/d31 39 2024-03-16T10:45:14.738 INFO:tasks.workunit.client.1.smithi026.stdout:0/318: write d1/d18/d1a/d24/d51/f63 [746696,124011] 0 2024-03-16T10:45:14.739 INFO:tasks.workunit.client.1.smithi026.stdout:5/255: chown d1/c3 4 1 2024-03-16T10:45:14.742 INFO:tasks.workunit.client.1.smithi026.stdout:7/308: dwrite d6/d35/d3a/d3c/f5c [0,4194304] 0 2024-03-16T10:45:14.743 INFO:tasks.workunit.client.1.smithi026.stdout:5/256: dread - d1/d1c/d4c/d4d/f4e zero size 2024-03-16T10:45:14.743 INFO:tasks.workunit.client.1.smithi026.stdout:0/319: write d1/d18/d1a/d24/d57/f3e [4288928,39475] 0 2024-03-16T10:45:14.743 INFO:tasks.workunit.client.1.smithi026.stdout:3/333: link d4/l28 d4/dc/dd/d47/d53/l72 0 2024-03-16T10:45:14.752 INFO:tasks.workunit.client.1.smithi026.stdout:3/334: dwrite d4/dc/f66 [0,4194304] 0 2024-03-16T10:45:14.753 INFO:tasks.workunit.client.1.smithi026.stdout:7/309: dread d6/d35/d3a/d3c/f40 [0,4194304] 0 2024-03-16T10:45:14.754 INFO:tasks.workunit.client.1.smithi026.stdout:7/310: fsync d6/d35/f52 0 2024-03-16T10:45:14.756 INFO:tasks.workunit.client.1.smithi026.stdout:0/320: unlink d1/d12/d2f/d48/d55/d60/f5a 0 2024-03-16T10:45:14.773 INFO:tasks.workunit.client.1.smithi026.stdout:5/257: truncate d1/f9 3400215 0 2024-03-16T10:45:14.778 INFO:tasks.workunit.client.1.smithi026.stdout:7/311: unlink d6/l50 0 2024-03-16T10:45:14.782 INFO:tasks.workunit.client.1.smithi026.stdout:0/321: stat d1/d18/c38 0 2024-03-16T10:45:14.786 INFO:tasks.workunit.client.1.smithi026.stdout:3/335: dwrite d4/fe [0,4194304] 0 2024-03-16T10:45:14.787 INFO:tasks.workunit.client.1.smithi026.stdout:5/258: dread d1/f5 [0,4194304] 0 2024-03-16T10:45:14.790 INFO:tasks.workunit.client.1.smithi026.stdout:7/312: dwrite d6/f15 [0,4194304] 0 2024-03-16T10:45:14.796 INFO:tasks.workunit.client.1.smithi026.stdout:3/336: dwrite d4/fe [0,4194304] 0 2024-03-16T10:45:14.798 INFO:tasks.workunit.client.1.smithi026.stdout:3/337: write d4/dc/d31/d48/f58 [2408554,89349] 0 2024-03-16T10:45:14.800 INFO:tasks.workunit.client.1.smithi026.stdout:0/322: unlink d1/d18/d1a/d24/l4 0 2024-03-16T10:45:14.803 INFO:tasks.workunit.client.1.smithi026.stdout:7/313: symlink d6/d35/d3a/d5d/l63 0 2024-03-16T10:45:14.806 INFO:tasks.workunit.client.1.smithi026.stdout:0/323: mknod d1/d18/d61/c69 0 2024-03-16T10:45:14.808 INFO:tasks.workunit.client.1.smithi026.stdout:5/259: mkdir d1/d1c/d2a/d55 0 2024-03-16T10:45:14.809 INFO:tasks.workunit.client.1.smithi026.stdout:5/260: write d1/f13 [2345391,106687] 0 2024-03-16T10:45:14.810 INFO:tasks.workunit.client.1.smithi026.stdout:5/261: write d1/d1c/d2a/f32 [1814091,128977] 0 2024-03-16T10:45:14.811 INFO:tasks.workunit.client.1.smithi026.stdout:3/338: mkdir d4/dc/d73 0 2024-03-16T10:45:14.814 INFO:tasks.workunit.client.1.smithi026.stdout:5/262: dread d1/db/d1f/f33 [4194304,4194304] 0 2024-03-16T10:45:14.814 INFO:tasks.workunit.client.1.smithi026.stdout:1/299: sync 2024-03-16T10:45:14.815 INFO:tasks.workunit.client.1.smithi026.stdout:7/314: unlink d6/d1e/c4c 0 2024-03-16T10:45:14.816 INFO:tasks.workunit.client.1.smithi026.stdout:1/300: write d0/d33/f30 [5135302,68055] 0 2024-03-16T10:45:14.819 INFO:tasks.workunit.client.1.smithi026.stdout:3/339: symlink d4/d29/l74 0 2024-03-16T10:45:14.819 INFO:tasks.workunit.client.1.smithi026.stdout:3/340: readlink d4/dc/l2d 0 2024-03-16T10:45:14.821 INFO:tasks.workunit.client.1.smithi026.stdout:0/324: symlink d1/d18/l6a 0 2024-03-16T10:45:14.822 INFO:tasks.workunit.client.1.smithi026.stdout:7/315: dwrite d6/f31 [0,4194304] 0 2024-03-16T10:45:14.827 INFO:tasks.workunit.client.1.smithi026.stdout:1/301: mkdir d0/d33/d2e/d54/d62 0 2024-03-16T10:45:14.828 INFO:tasks.workunit.client.1.smithi026.stdout:7/316: dread d6/d35/d3a/d3c/f40 [0,4194304] 0 2024-03-16T10:45:14.829 INFO:tasks.workunit.client.1.smithi026.stdout:1/302: dread - d0/d33/d2e/d54/f5e zero size 2024-03-16T10:45:14.829 INFO:tasks.workunit.client.1.smithi026.stdout:5/263: symlink d1/d1c/d2a/l56 0 2024-03-16T10:45:14.834 INFO:tasks.workunit.client.1.smithi026.stdout:5/264: write d1/d1c/d2a/d31/f49 [160370,75389] 0 2024-03-16T10:45:14.835 INFO:tasks.workunit.client.1.smithi026.stdout:5/265: write d1/db/d40/d51/f54 [852761,885] 0 2024-03-16T10:45:14.836 INFO:tasks.workunit.client.1.smithi026.stdout:5/266: stat d1/db/f44 0 2024-03-16T10:45:14.836 INFO:tasks.workunit.client.1.smithi026.stdout:1/303: rename d0/d33/d3b/d49 to d0/d17/d1f/d63 0 2024-03-16T10:45:14.837 INFO:tasks.workunit.client.1.smithi026.stdout:1/304: chown d0/fd 4 1 2024-03-16T10:45:14.837 INFO:tasks.workunit.client.1.smithi026.stdout:5/267: chown d1/d1c/d4c/d4d/f4e 120466416 1 2024-03-16T10:45:14.840 INFO:tasks.workunit.client.1.smithi026.stdout:0/325: link d1/d12/d2f/d48/d55/d60/f31 d1/d18/d61/f6b 0 2024-03-16T10:45:14.841 INFO:tasks.workunit.client.1.smithi026.stdout:3/341: creat d4/dc/dd/f75 x:0 0 0 2024-03-16T10:45:14.842 INFO:tasks.workunit.client.1.smithi026.stdout:1/305: dwrite d0/f40 [0,4194304] 0 2024-03-16T10:45:14.848 INFO:tasks.workunit.client.1.smithi026.stdout:3/342: dread d4/dc/dd/f24 [0,4194304] 0 2024-03-16T10:45:14.853 INFO:tasks.workunit.client.1.smithi026.stdout:7/317: creat d6/de/f64 x:0 0 0 2024-03-16T10:45:14.855 INFO:tasks.workunit.client.1.smithi026.stdout:5/268: dwrite d1/d1c/f3f [0,4194304] 0 2024-03-16T10:45:14.855 INFO:tasks.workunit.client.1.smithi026.stdout:0/326: symlink d1/d18/d1a/d24/l6c 0 2024-03-16T10:45:14.856 INFO:tasks.workunit.client.1.smithi026.stdout:5/269: fdatasync d1/db/f38 0 2024-03-16T10:45:14.859 INFO:tasks.workunit.client.1.smithi026.stdout:3/343: mknod d4/dc/d31/d48/c76 0 2024-03-16T10:45:14.860 INFO:tasks.workunit.client.1.smithi026.stdout:1/306: dread d0/d17/d1f/f35 [0,4194304] 0 2024-03-16T10:45:14.860 INFO:tasks.workunit.client.1.smithi026.stdout:3/344: truncate d4/dc/f20 725323 0 2024-03-16T10:45:14.870 INFO:tasks.workunit.client.1.smithi026.stdout:1/307: mkdir d0/d33/d3b/d64 0 2024-03-16T10:45:14.878 INFO:tasks.workunit.client.1.smithi026.stdout:1/308: chown d0/d17/d1f/d20/l5c 3 1 2024-03-16T10:45:14.883 INFO:tasks.workunit.client.1.smithi026.stdout:7/318: dwrite d6/d1e/f20 [0,4194304] 0 2024-03-16T10:45:14.884 INFO:tasks.workunit.client.1.smithi026.stdout:3/345: creat d4/f77 x:0 0 0 2024-03-16T10:45:14.886 INFO:tasks.workunit.client.1.smithi026.stdout:8/319: sync 2024-03-16T10:45:14.886 INFO:tasks.workunit.client.1.smithi026.stdout:6/320: sync 2024-03-16T10:45:14.886 INFO:tasks.workunit.client.1.smithi026.stdout:2/262: sync 2024-03-16T10:45:14.889 INFO:tasks.workunit.client.1.smithi026.stdout:0/327: getdents d1/d18/d1a/d24 0 2024-03-16T10:45:14.889 INFO:tasks.workunit.client.1.smithi026.stdout:1/309: mkdir d0/d17/d1f/d20/d65 0 2024-03-16T10:45:14.890 INFO:tasks.workunit.client.1.smithi026.stdout:5/270: getdents d1/db/d40/d51 0 2024-03-16T10:45:14.891 INFO:tasks.workunit.client.1.smithi026.stdout:8/320: dread - d5/d23/f4b zero size 2024-03-16T10:45:14.892 INFO:tasks.workunit.client.1.smithi026.stdout:7/319: mkdir d6/d35/d3a/d3c/d3f/d65 0 2024-03-16T10:45:14.898 INFO:tasks.workunit.client.1.smithi026.stdout:1/310: dwrite d0/f1e [0,4194304] 0 2024-03-16T10:45:14.900 INFO:tasks.workunit.client.1.smithi026.stdout:3/346: mkdir d4/dc/d31/d48/d69/d78 0 2024-03-16T10:45:14.900 INFO:tasks.workunit.client.1.smithi026.stdout:1/311: write d0/d33/f3e [700092,28418] 0 2024-03-16T10:45:14.903 INFO:tasks.workunit.client.1.smithi026.stdout:2/263: creat d2/d29/d17/d38/f59 x:0 0 0 2024-03-16T10:45:14.906 INFO:tasks.workunit.client.1.smithi026.stdout:2/264: write d2/f4d [280612,48232] 0 2024-03-16T10:45:14.907 INFO:tasks.workunit.client.1.smithi026.stdout:0/328: dread d1/d18/d1a/d24/d57/f3e [0,4194304] 0 2024-03-16T10:45:14.909 INFO:tasks.workunit.client.1.smithi026.stdout:0/329: chown d1/d18/d1a/d2b/d4f/f56 7 1 2024-03-16T10:45:14.911 INFO:tasks.workunit.client.1.smithi026.stdout:0/330: readlink d1/d18/d1a/l33 0 2024-03-16T10:45:14.913 INFO:tasks.workunit.client.1.smithi026.stdout:1/312: symlink d0/l66 0 2024-03-16T10:45:14.917 INFO:tasks.workunit.client.1.smithi026.stdout:8/321: dread d5/d7/d2c/f2d [0,4194304] 0 2024-03-16T10:45:14.918 INFO:tasks.workunit.client.1.smithi026.stdout:2/265: dwrite d2/d29/f3a [0,4194304] 0 2024-03-16T10:45:14.919 INFO:tasks.workunit.client.1.smithi026.stdout:2/266: dread - d2/d29/f56 zero size 2024-03-16T10:45:14.920 INFO:tasks.workunit.client.1.smithi026.stdout:3/347: dread d4/dc/dd/d3d/f49 [0,4194304] 0 2024-03-16T10:45:14.920 INFO:tasks.workunit.client.1.smithi026.stdout:0/331: dread d1/d18/d1a/f46 [0,4194304] 0 2024-03-16T10:45:14.923 INFO:tasks.workunit.client.1.smithi026.stdout:7/320: getdents d6/d35/d3a/d59 0 2024-03-16T10:45:14.925 INFO:tasks.workunit.client.1.smithi026.stdout:8/322: symlink d5/d23/l54 0 2024-03-16T10:45:14.925 INFO:tasks.workunit.client.1.smithi026.stdout:0/332: read d1/f40 [4667322,39968] 0 2024-03-16T10:45:14.925 INFO:tasks.workunit.client.1.smithi026.stdout:8/323: stat d5/d7 0 2024-03-16T10:45:14.929 INFO:tasks.workunit.client.1.smithi026.stdout:3/348: dwrite d4/dc/f4e [0,4194304] 0 2024-03-16T10:45:14.929 INFO:tasks.workunit.client.1.smithi026.stdout:5/271: link d1/db/d27/f23 d1/d1c/d2a/f57 0 2024-03-16T10:45:14.932 INFO:tasks.workunit.client.1.smithi026.stdout:2/267: creat d2/d29/d17/d1a/d2b/d4b/f5a x:0 0 0 2024-03-16T10:45:14.934 INFO:tasks.workunit.client.1.smithi026.stdout:5/272: dread d1/f5 [0,4194304] 0 2024-03-16T10:45:14.935 INFO:tasks.workunit.client.1.smithi026.stdout:5/273: fdatasync d1/d1c/f3f 0 2024-03-16T10:45:14.944 INFO:tasks.workunit.client.1.smithi026.stdout:5/274: write d1/db/f39 [65988,18366] 0 2024-03-16T10:45:14.947 INFO:tasks.workunit.client.1.smithi026.stdout:7/321: creat d6/d35/f66 x:0 0 0 2024-03-16T10:45:14.950 INFO:tasks.workunit.client.1.smithi026.stdout:2/268: write d2/d29/d17/d1a/d2b/f15 [822668,104311] 0 2024-03-16T10:45:14.955 INFO:tasks.workunit.client.1.smithi026.stdout:3/349: dwrite d4/dc/dd/f24 [0,4194304] 0 2024-03-16T10:45:14.956 INFO:tasks.workunit.client.1.smithi026.stdout:7/322: dread d6/d35/d3a/d3c/f5c [0,4194304] 0 2024-03-16T10:45:14.957 INFO:tasks.workunit.client.1.smithi026.stdout:2/269: creat d2/d29/d17/d1a/d2b/d4b/f5b x:0 0 0 2024-03-16T10:45:14.958 INFO:tasks.workunit.client.1.smithi026.stdout:2/270: readlink d2/d29/d17/d1a/d2b/l25 0 2024-03-16T10:45:14.959 INFO:tasks.workunit.client.1.smithi026.stdout:3/350: creat d4/dc/dd/d3d/f79 x:0 0 0 2024-03-16T10:45:14.959 INFO:tasks.workunit.client.1.smithi026.stdout:5/275: dread d1/d1c/f1e [0,4194304] 0 2024-03-16T10:45:14.960 INFO:tasks.workunit.client.1.smithi026.stdout:5/276: chown d1/db/d1f 14565575 1 2024-03-16T10:45:14.960 INFO:tasks.workunit.client.1.smithi026.stdout:7/323: mknod d6/d35/d3a/d3c/d3f/c67 0 2024-03-16T10:45:14.961 INFO:tasks.workunit.client.1.smithi026.stdout:7/324: fsync d6/de/f57 0 2024-03-16T10:45:14.962 INFO:tasks.workunit.client.1.smithi026.stdout:3/351: symlink d4/dc/d31/l7a 0 2024-03-16T10:45:14.962 INFO:tasks.workunit.client.1.smithi026.stdout:3/352: read - d4/f77 zero size 2024-03-16T10:45:14.963 INFO:tasks.workunit.client.1.smithi026.stdout:3/353: write d4/dc/f20 [258377,30391] 0 2024-03-16T10:45:14.967 INFO:tasks.workunit.client.1.smithi026.stdout:5/277: creat d1/d1c/d2a/d55/f58 x:0 0 0 2024-03-16T10:45:14.969 INFO:tasks.workunit.client.1.smithi026.stdout:5/278: write d1/db/d40/d51/f54 [529613,113660] 0 2024-03-16T10:45:14.970 INFO:tasks.workunit.client.1.smithi026.stdout:7/325: mkdir d6/d35/d3a/d3c/d3f/d65/d68 0 2024-03-16T10:45:14.975 INFO:tasks.workunit.client.1.smithi026.stdout:3/354: mknod d4/dc/dd/c7b 0 2024-03-16T10:45:14.975 INFO:tasks.workunit.client.1.smithi026.stdout:7/326: dwrite d6/d1e/f20 [0,4194304] 0 2024-03-16T10:45:14.977 INFO:tasks.workunit.client.1.smithi026.stdout:7/327: fsync d6/de/d16/f19 0 2024-03-16T10:45:14.978 INFO:tasks.workunit.client.1.smithi026.stdout:2/271: truncate d2/d29/d3b/f47 736751 0 2024-03-16T10:45:14.978 INFO:tasks.workunit.client.1.smithi026.stdout:7/328: dread - d6/d2f/f5f zero size 2024-03-16T10:45:14.982 INFO:tasks.workunit.client.1.smithi026.stdout:2/272: fdatasync d2/d29/d17/d1a/d2b/d2e/f42 0 2024-03-16T10:45:14.982 INFO:tasks.workunit.client.1.smithi026.stdout:7/329: read d6/de/f11 [3928704,74069] 0 2024-03-16T10:45:14.985 INFO:tasks.workunit.client.1.smithi026.stdout:3/355: dwrite d4/dc/dd/d4b/f5b [0,4194304] 0 2024-03-16T10:45:14.986 INFO:tasks.workunit.client.1.smithi026.stdout:5/279: creat d1/db/f59 x:0 0 0 2024-03-16T10:45:14.987 INFO:tasks.workunit.client.1.smithi026.stdout:3/356: write d4/fa [641402,47966] 0 2024-03-16T10:45:14.993 INFO:tasks.workunit.client.1.smithi026.stdout:7/330: rename d6/de/d16/l3e to d6/l69 0 2024-03-16T10:45:14.995 INFO:tasks.workunit.client.1.smithi026.stdout:3/357: dwrite d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:14.995 INFO:tasks.workunit.client.1.smithi026.stdout:2/273: creat d2/d29/d17/d1a/d2b/d4b/f5c x:0 0 0 2024-03-16T10:45:14.996 INFO:tasks.workunit.client.1.smithi026.stdout:2/274: chown d2/d29 445450 1 2024-03-16T10:45:14.997 INFO:tasks.workunit.client.1.smithi026.stdout:3/358: chown d4/dc/l32 12 1 2024-03-16T10:45:15.005 INFO:tasks.workunit.client.1.smithi026.stdout:7/331: creat d6/d1e/f6a x:0 0 0 2024-03-16T10:45:15.007 INFO:tasks.workunit.client.1.smithi026.stdout:2/275: readlink d2/d29/d17/d1a/l2f 0 2024-03-16T10:45:15.012 INFO:tasks.workunit.client.1.smithi026.stdout:2/276: creat d2/d29/d17/d1a/d2b/d4b/f5d x:0 0 0 2024-03-16T10:45:15.015 INFO:tasks.workunit.client.1.smithi026.stdout:5/280: rename d1/db/d27/f53 to d1/d1c/d2a/f5a 0 2024-03-16T10:45:15.015 INFO:tasks.workunit.client.1.smithi026.stdout:5/281: stat d1/db/d1f/f26 0 2024-03-16T10:45:15.017 INFO:tasks.workunit.client.1.smithi026.stdout:7/332: getdents d6/d35/d3a/d5d 0 2024-03-16T10:45:15.019 INFO:tasks.workunit.client.1.smithi026.stdout:3/359: rename d4/dc/dd/d4b to d4/dc/dd/d60/d62/d4d/d7c 0 2024-03-16T10:45:15.020 INFO:tasks.workunit.client.1.smithi026.stdout:5/282: creat d1/db/d1f/f5b x:0 0 0 2024-03-16T10:45:15.021 INFO:tasks.workunit.client.1.smithi026.stdout:5/283: write d1/d1c/d2a/d31/f3a [349218,6265] 0 2024-03-16T10:45:15.023 INFO:tasks.workunit.client.1.smithi026.stdout:2/277: truncate d2/d29/fe 218310 0 2024-03-16T10:45:15.023 INFO:tasks.workunit.client.1.smithi026.stdout:5/284: stat d1/db/d1f/f5b 0 2024-03-16T10:45:15.024 INFO:tasks.workunit.client.1.smithi026.stdout:7/333: symlink d6/de/d16/l6b 0 2024-03-16T10:45:15.025 INFO:tasks.workunit.client.1.smithi026.stdout:7/334: write d6/d35/f66 [75099,91681] 0 2024-03-16T10:45:15.030 INFO:tasks.workunit.client.1.smithi026.stdout:3/360: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:15.036 INFO:tasks.workunit.client.1.smithi026.stdout:2/278: dread d2/d29/fe [0,4194304] 0 2024-03-16T10:45:15.037 INFO:tasks.workunit.client.1.smithi026.stdout:2/279: dread - d2/d29/d17/d1a/d2b/d4b/f5b zero size 2024-03-16T10:45:15.041 INFO:tasks.workunit.client.1.smithi026.stdout:2/280: symlink d2/d29/d17/d1a/d2b/l5e 0 2024-03-16T10:45:15.042 INFO:tasks.workunit.client.1.smithi026.stdout:3/361: write d4/dc/f16 [110434,60030] 0 2024-03-16T10:45:15.044 INFO:tasks.workunit.client.1.smithi026.stdout:2/281: write d2/d29/d36/f58 [579751,69010] 0 2024-03-16T10:45:15.046 INFO:tasks.workunit.client.1.smithi026.stdout:5/285: dwrite d1/db/f22 [0,4194304] 0 2024-03-16T10:45:15.048 INFO:tasks.workunit.client.1.smithi026.stdout:3/362: mkdir d4/d29/d7d 0 2024-03-16T10:45:15.049 INFO:tasks.workunit.client.1.smithi026.stdout:3/363: read d4/dc/d31/f34 [1564811,67471] 0 2024-03-16T10:45:15.050 INFO:tasks.workunit.client.1.smithi026.stdout:2/282: rmdir d2/d29/d17/d1a 39 2024-03-16T10:45:15.051 INFO:tasks.workunit.client.1.smithi026.stdout:2/283: dread - d2/d29/d17/f40 zero size 2024-03-16T10:45:15.052 INFO:tasks.workunit.client.1.smithi026.stdout:3/364: read d4/dc/dd/d60/d62/d4f/f70 [4057877,114442] 0 2024-03-16T10:45:15.056 INFO:tasks.workunit.client.1.smithi026.stdout:3/365: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:15.056 INFO:tasks.workunit.client.1.smithi026.stdout:2/284: creat d2/d29/d51/f5f x:0 0 0 2024-03-16T10:45:15.058 INFO:tasks.workunit.client.1.smithi026.stdout:5/286: fdatasync d1/db/f12 0 2024-03-16T10:45:15.062 INFO:tasks.workunit.client.1.smithi026.stdout:3/366: rename d4/dc/f43 to d4/dc/dd/d60/d62/d4f/f7e 0 2024-03-16T10:45:15.066 INFO:tasks.workunit.client.1.smithi026.stdout:5/287: rename d1/d1c/d2a/f5a to d1/db/d1f/f5c 0 2024-03-16T10:45:15.067 INFO:tasks.workunit.client.1.smithi026.stdout:5/288: readlink l0 0 2024-03-16T10:45:15.067 INFO:tasks.workunit.client.1.smithi026.stdout:3/367: mkdir d4/dc/d31/d48/d7f 0 2024-03-16T10:45:15.071 INFO:tasks.workunit.client.1.smithi026.stdout:5/289: unlink d1/d1c/f1e 0 2024-03-16T10:45:15.072 INFO:tasks.workunit.client.1.smithi026.stdout:2/285: dread d2/d29/d17/d1a/d2b/d2e/f3c [0,4194304] 0 2024-03-16T10:45:15.075 INFO:tasks.workunit.client.1.smithi026.stdout:5/290: chown d1/f7 19064 1 2024-03-16T10:45:15.077 INFO:tasks.workunit.client.1.smithi026.stdout:2/286: mkdir d2/d29/d17/d1a/d2b/d2e/d60 0 2024-03-16T10:45:15.080 INFO:tasks.workunit.client.1.smithi026.stdout:9/327: sync 2024-03-16T10:45:15.080 INFO:tasks.workunit.client.1.smithi026.stdout:4/311: sync 2024-03-16T10:45:15.080 INFO:tasks.workunit.client.1.smithi026.stdout:5/291: creat d1/db/d40/d51/f5d x:0 0 0 2024-03-16T10:45:15.083 INFO:tasks.workunit.client.1.smithi026.stdout:2/287: rmdir d2/d29/d17/d1a/d2b/d4b 39 2024-03-16T10:45:15.084 INFO:tasks.workunit.client.1.smithi026.stdout:3/368: write d4/dc/f25 [1708976,78793] 0 2024-03-16T10:45:15.085 INFO:tasks.workunit.client.1.smithi026.stdout:9/328: creat d0/da/d4d/f71 x:0 0 0 2024-03-16T10:45:15.086 INFO:tasks.workunit.client.1.smithi026.stdout:9/329: readlink d0/da/l29 0 2024-03-16T10:45:15.087 INFO:tasks.workunit.client.1.smithi026.stdout:9/330: write d0/da/d13/f53 [1063670,89749] 0 2024-03-16T10:45:15.088 INFO:tasks.workunit.client.1.smithi026.stdout:5/292: creat d1/db/d40/d51/f5e x:0 0 0 2024-03-16T10:45:15.091 INFO:tasks.workunit.client.1.smithi026.stdout:4/312: rename d1/db/de/c57 to d1/db/d29/d44/d48/c66 0 2024-03-16T10:45:15.093 INFO:tasks.workunit.client.1.smithi026.stdout:3/369: dwrite d4/dc/dd/f75 [0,4194304] 0 2024-03-16T10:45:15.093 INFO:tasks.workunit.client.1.smithi026.stdout:2/288: mknod d2/d29/d17/d1a/c61 0 2024-03-16T10:45:15.097 INFO:tasks.workunit.client.1.smithi026.stdout:9/331: dwrite d0/da/d16/d17/d44/f24 [4194304,4194304] 0 2024-03-16T10:45:15.104 INFO:tasks.workunit.client.1.smithi026.stdout:5/293: rename d1/db/l10 to d1/d1c/d2a/d31/l5f 0 2024-03-16T10:45:15.107 INFO:tasks.workunit.client.1.smithi026.stdout:3/370: truncate d4/dc/dd/d3d/f39 808563 0 2024-03-16T10:45:15.107 INFO:tasks.workunit.client.1.smithi026.stdout:9/332: write d0/da/f2a [1418708,115870] 0 2024-03-16T10:45:15.109 INFO:tasks.workunit.client.1.smithi026.stdout:4/313: creat d1/db/f67 x:0 0 0 2024-03-16T10:45:15.112 INFO:tasks.workunit.client.1.smithi026.stdout:5/294: link d1/f13 d1/db/d27/f60 0 2024-03-16T10:45:15.144 INFO:tasks.workunit.client.1.smithi026.stdout:4/314: unlink d1/f32 0 2024-03-16T10:45:15.147 INFO:tasks.workunit.client.1.smithi026.stdout:2/289: rename d2/d29/d17/d1a/d2b/l34 to d2/d29/l62 0 2024-03-16T10:45:15.147 INFO:tasks.workunit.client.1.smithi026.stdout:3/371: getdents d4/dc/dd/d60/d62/d4d/d57 0 2024-03-16T10:45:15.149 INFO:tasks.workunit.client.1.smithi026.stdout:5/295: mkdir d1/db/d1f/d61 0 2024-03-16T10:45:15.152 INFO:tasks.workunit.client.1.smithi026.stdout:4/315: fdatasync d1/db/d29/f3e 0 2024-03-16T10:45:15.153 INFO:tasks.workunit.client.1.smithi026.stdout:4/316: read d1/f8 [4122184,13738] 0 2024-03-16T10:45:15.154 INFO:tasks.workunit.client.1.smithi026.stdout:9/333: rename d0/da/d4d/d4f to d0/da/d16/d5d/d72 0 2024-03-16T10:45:15.163 INFO:tasks.workunit.client.1.smithi026.stdout:9/334: dwrite d0/da/d16/d17/f40 [0,4194304] 0 2024-03-16T10:45:15.165 INFO:tasks.workunit.client.1.smithi026.stdout:9/335: readlink d0/da/d16/d17/l28 0 2024-03-16T10:45:15.165 INFO:tasks.workunit.client.1.smithi026.stdout:2/290: dwrite d2/d29/d17/d1a/d2b/d4b/f5a [0,4194304] 0 2024-03-16T10:45:15.171 INFO:tasks.workunit.client.1.smithi026.stdout:5/296: mknod d1/d1c/d2a/c62 0 2024-03-16T10:45:15.176 INFO:tasks.workunit.client.1.smithi026.stdout:4/317: mknod d1/db/d29/d44/c68 0 2024-03-16T10:45:15.178 INFO:tasks.workunit.client.1.smithi026.stdout:3/372: write d4/d29/f30 [4148039,76731] 0 2024-03-16T10:45:15.182 INFO:tasks.workunit.client.1.smithi026.stdout:2/291: dread d2/d29/d17/f2d [0,4194304] 0 2024-03-16T10:45:15.188 INFO:tasks.workunit.client.1.smithi026.stdout:4/318: dread d1/db/d29/f3e [0,4194304] 0 2024-03-16T10:45:15.193 INFO:tasks.workunit.client.1.smithi026.stdout:9/336: rename d0/da/d16/d5d/d72/c64 to d0/da/d13/d6e/c73 0 2024-03-16T10:45:15.194 INFO:tasks.workunit.client.1.smithi026.stdout:3/373: dwrite d4/dc/dd/d60/d62/d4f/f7e [0,4194304] 0 2024-03-16T10:45:15.202 INFO:tasks.workunit.client.1.smithi026.stdout:9/337: write d0/f68 [611118,11607] 0 2024-03-16T10:45:15.203 INFO:tasks.workunit.client.1.smithi026.stdout:5/297: rename d1/db/d27/l43 to d1/db/d40/l63 0 2024-03-16T10:45:15.210 INFO:tasks.workunit.client.1.smithi026.stdout:9/338: dwrite d0/da/d4d/f5e [0,4194304] 0 2024-03-16T10:45:15.211 INFO:tasks.workunit.client.1.smithi026.stdout:2/292: creat d2/d29/f63 x:0 0 0 2024-03-16T10:45:15.213 INFO:tasks.workunit.client.1.smithi026.stdout:5/298: symlink d1/l64 0 2024-03-16T10:45:15.214 INFO:tasks.workunit.client.1.smithi026.stdout:5/299: fsync d1/db/d27/f3e 0 2024-03-16T10:45:15.214 INFO:tasks.workunit.client.1.smithi026.stdout:3/374: mkdir d4/dc/dd/d80 0 2024-03-16T10:45:15.216 INFO:tasks.workunit.client.1.smithi026.stdout:5/300: truncate d1/d1c/f50 736214 0 2024-03-16T10:45:15.216 INFO:tasks.workunit.client.1.smithi026.stdout:4/319: symlink d1/db/de/d4c/l69 0 2024-03-16T10:45:15.216 INFO:tasks.workunit.client.1.smithi026.stdout:9/339: dread d0/da/d13/f53 [0,4194304] 0 2024-03-16T10:45:15.220 INFO:tasks.workunit.client.1.smithi026.stdout:3/375: mknod d4/dc/dd/d3d/c81 0 2024-03-16T10:45:15.220 INFO:tasks.workunit.client.1.smithi026.stdout:3/376: stat d4/dc/dd/f21 0 2024-03-16T10:45:15.221 INFO:tasks.workunit.client.1.smithi026.stdout:3/377: chown d4/fa 7267 1 2024-03-16T10:45:15.222 INFO:tasks.workunit.client.1.smithi026.stdout:9/340: chown d0/da/d13/d6e/c73 2625230 1 2024-03-16T10:45:15.223 INFO:tasks.workunit.client.1.smithi026.stdout:4/320: mkdir d1/db/de/d4c/d59/d6a 0 2024-03-16T10:45:15.225 INFO:tasks.workunit.client.1.smithi026.stdout:2/293: dread d2/d29/d17/d1a/d2b/f15 [0,4194304] 0 2024-03-16T10:45:15.226 INFO:tasks.workunit.client.1.smithi026.stdout:2/294: write d2/d29/d17/f40 [573902,117669] 0 2024-03-16T10:45:15.226 INFO:tasks.workunit.client.1.smithi026.stdout:5/301: mkdir d1/db/d1f/d61/d65 0 2024-03-16T10:45:15.228 INFO:tasks.workunit.client.1.smithi026.stdout:3/378: write d4/dc/dd/d3d/f49 [4245167,88789] 0 2024-03-16T10:45:15.229 INFO:tasks.workunit.client.1.smithi026.stdout:9/341: creat d0/da/d13/d6e/f74 x:0 0 0 2024-03-16T10:45:15.230 INFO:tasks.workunit.client.1.smithi026.stdout:9/342: stat d0/da/d4d 0 2024-03-16T10:45:15.230 INFO:tasks.workunit.client.1.smithi026.stdout:4/321: readlink d1/db/de/l24 0 2024-03-16T10:45:15.231 INFO:tasks.workunit.client.1.smithi026.stdout:3/379: write d4/dc/f5f [796109,34741] 0 2024-03-16T10:45:15.232 INFO:tasks.workunit.client.1.smithi026.stdout:4/322: chown d1/f27 73132892 1 2024-03-16T10:45:15.233 INFO:tasks.workunit.client.1.smithi026.stdout:5/302: getdents d1/db/d1f/d61/d65 0 2024-03-16T10:45:15.233 INFO:tasks.workunit.client.1.smithi026.stdout:9/343: truncate d0/da/d13/f49 77549 0 2024-03-16T10:45:15.233 INFO:tasks.workunit.client.1.smithi026.stdout:2/295: dread d2/d29/d17/f2d [0,4194304] 0 2024-03-16T10:45:15.235 INFO:tasks.workunit.client.1.smithi026.stdout:4/323: mkdir d1/db/de/d4c/d6b 0 2024-03-16T10:45:15.237 INFO:tasks.workunit.client.1.smithi026.stdout:4/324: truncate d1/db/f42 1008738 0 2024-03-16T10:45:15.237 INFO:tasks.workunit.client.1.smithi026.stdout:5/303: creat d1/db/d27/f66 x:0 0 0 2024-03-16T10:45:15.238 INFO:tasks.workunit.client.1.smithi026.stdout:2/296: mkdir d2/d29/d17/d38/d64 0 2024-03-16T10:45:15.239 INFO:tasks.workunit.client.1.smithi026.stdout:3/380: mknod d4/dc/d31/d48/d7f/c82 0 2024-03-16T10:45:15.240 INFO:tasks.workunit.client.1.smithi026.stdout:9/344: creat d0/da/f75 x:0 0 0 2024-03-16T10:45:15.240 INFO:tasks.workunit.client.1.smithi026.stdout:3/381: write d4/dc/dd/f12 [1632130,22763] 0 2024-03-16T10:45:15.241 INFO:tasks.workunit.client.1.smithi026.stdout:4/325: write d1/db/f26 [4525088,103495] 0 2024-03-16T10:45:15.242 INFO:tasks.workunit.client.1.smithi026.stdout:5/304: creat d1/db/d40/f67 x:0 0 0 2024-03-16T10:45:15.245 INFO:tasks.workunit.client.1.smithi026.stdout:9/345: symlink d0/da/d4d/l76 0 2024-03-16T10:45:15.247 INFO:tasks.workunit.client.1.smithi026.stdout:9/346: fsync d0/d63/d6b/f70 0 2024-03-16T10:45:15.249 INFO:tasks.workunit.client.1.smithi026.stdout:5/305: unlink d1/l64 0 2024-03-16T10:45:15.249 INFO:tasks.workunit.client.1.smithi026.stdout:3/382: dread d4/dc/dd/d3d/f49 [0,4194304] 0 2024-03-16T10:45:15.250 INFO:tasks.workunit.client.1.smithi026.stdout:2/297: dwrite d2/d29/d17/d1a/d2b/d4b/f57 [0,4194304] 0 2024-03-16T10:45:15.252 INFO:tasks.workunit.client.1.smithi026.stdout:3/383: dread - d4/dc/dd/d3d/f79 zero size 2024-03-16T10:45:15.258 INFO:tasks.workunit.client.1.smithi026.stdout:9/347: dwrite d0/da/d16/d17/d44/f24 [0,4194304] 0 2024-03-16T10:45:15.260 INFO:tasks.workunit.client.1.smithi026.stdout:2/298: unlink d2/d29/d17/d1a/c26 0 2024-03-16T10:45:15.263 INFO:tasks.workunit.client.1.smithi026.stdout:3/384: dread d4/dc/dd/d60/d62/d4d/d7c/f5b [0,4194304] 0 2024-03-16T10:45:15.264 INFO:tasks.workunit.client.1.smithi026.stdout:4/326: getdents d1/db/d29/d44 0 2024-03-16T10:45:15.267 INFO:tasks.workunit.client.1.smithi026.stdout:5/306: dread d1/d1c/d2a/f32 [0,4194304] 0 2024-03-16T10:45:15.269 INFO:tasks.workunit.client.1.smithi026.stdout:6/321: sync 2024-03-16T10:45:15.271 INFO:tasks.workunit.client.1.smithi026.stdout:6/322: write d0/d41/d4f/f6b [740401,35028] 0 2024-03-16T10:45:15.272 INFO:tasks.workunit.client.1.smithi026.stdout:2/299: mkdir d2/d29/d17/d1a/d2b/d4e/d65 0 2024-03-16T10:45:15.273 INFO:tasks.workunit.client.1.smithi026.stdout:2/300: fsync d2/f4d 0 2024-03-16T10:45:15.276 INFO:tasks.workunit.client.1.smithi026.stdout:2/301: dread d2/f4d [0,4194304] 0 2024-03-16T10:45:15.277 INFO:tasks.workunit.client.1.smithi026.stdout:2/302: chown d2/d29/d17/d1a/d2b/d4b 44 1 2024-03-16T10:45:15.281 INFO:tasks.workunit.client.1.smithi026.stdout:3/385: dread d4/dc/dd/d3d/f4a [0,4194304] 0 2024-03-16T10:45:15.281 INFO:tasks.workunit.client.1.smithi026.stdout:4/327: dwrite d1/f22 [0,4194304] 0 2024-03-16T10:45:15.283 INFO:tasks.workunit.client.1.smithi026.stdout:9/348: link d0/da/f3b d0/d63/d6b/f77 0 2024-03-16T10:45:15.286 INFO:tasks.workunit.client.1.smithi026.stdout:6/323: rename d0/f12 to d0/d28/d2d/f76 0 2024-03-16T10:45:15.291 INFO:tasks.workunit.client.1.smithi026.stdout:2/303: creat d2/d29/d3b/f66 x:0 0 0 2024-03-16T10:45:15.292 INFO:tasks.workunit.client.1.smithi026.stdout:5/307: mkdir d1/d1c/d68 0 2024-03-16T10:45:15.296 INFO:tasks.workunit.client.1.smithi026.stdout:4/328: creat d1/db/d29/d61/f6c x:0 0 0 2024-03-16T10:45:15.300 INFO:tasks.workunit.client.1.smithi026.stdout:3/386: rename d4/dc/dd/d60/d62/d4d to d4/dc/d73/d83 0 2024-03-16T10:45:15.302 INFO:tasks.workunit.client.1.smithi026.stdout:2/304: mkdir d2/d29/d17/d1a/d2b/d4e/d67 0 2024-03-16T10:45:15.303 INFO:tasks.workunit.client.1.smithi026.stdout:5/308: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:15.304 INFO:tasks.workunit.client.1.smithi026.stdout:2/305: write d2/d29/d17/d38/f59 [648834,53867] 0 2024-03-16T10:45:15.305 INFO:tasks.workunit.client.1.smithi026.stdout:4/329: readlink d1/db/de/l51 0 2024-03-16T10:45:15.306 INFO:tasks.workunit.client.1.smithi026.stdout:5/309: write d1/db/f38 [517536,113409] 0 2024-03-16T10:45:15.307 INFO:tasks.workunit.client.1.smithi026.stdout:2/306: fsync d2/d29/d17/d1a/d2b/d4b/f57 0 2024-03-16T10:45:15.307 INFO:tasks.workunit.client.1.smithi026.stdout:4/330: chown d1/db/f64 3 1 2024-03-16T10:45:15.308 INFO:tasks.workunit.client.1.smithi026.stdout:3/387: symlink d4/dc/d31/d48/d7f/l84 0 2024-03-16T10:45:15.310 INFO:tasks.workunit.client.1.smithi026.stdout:2/307: truncate d2/d29/f56 686503 0 2024-03-16T10:45:15.320 INFO:tasks.workunit.client.1.smithi026.stdout:5/310: dwrite d1/db/d27/f3e [0,4194304] 0 2024-03-16T10:45:15.321 INFO:tasks.workunit.client.1.smithi026.stdout:3/388: dwrite d4/dc/f4e [0,4194304] 0 2024-03-16T10:45:15.322 INFO:tasks.workunit.client.1.smithi026.stdout:9/349: write d0/f2f [871888,61573] 0 2024-03-16T10:45:15.323 INFO:tasks.workunit.client.1.smithi026.stdout:5/311: fdatasync d1/db/d40/d51/f5e 0 2024-03-16T10:45:15.324 INFO:tasks.workunit.client.1.smithi026.stdout:3/389: readlink d4/lb 0 2024-03-16T10:45:15.325 INFO:tasks.workunit.client.1.smithi026.stdout:3/390: write d4/dc/f66 [2221068,119283] 0 2024-03-16T10:45:15.330 INFO:tasks.workunit.client.1.smithi026.stdout:2/308: dwrite d2/d29/d17/d1a/d2b/d4b/f5c [0,4194304] 0 2024-03-16T10:45:15.331 INFO:tasks.workunit.client.1.smithi026.stdout:5/312: rename d1/db/d40 to d1/d1c/d2a/d31/d69 0 2024-03-16T10:45:15.340 INFO:tasks.workunit.client.1.smithi026.stdout:4/331: dwrite d1/db/de/f58 [0,4194304] 0 2024-03-16T10:45:15.341 INFO:tasks.workunit.client.1.smithi026.stdout:2/309: mkdir d2/d29/d68 0 2024-03-16T10:45:15.344 INFO:tasks.workunit.client.1.smithi026.stdout:3/391: dwrite d4/d29/f3a [0,4194304] 0 2024-03-16T10:45:15.346 INFO:tasks.workunit.client.1.smithi026.stdout:9/350: dwrite d0/d63/d6b/f70 [0,4194304] 0 2024-03-16T10:45:15.355 INFO:tasks.workunit.client.1.smithi026.stdout:4/332: dwrite d1/db/f67 [0,4194304] 0 2024-03-16T10:45:15.356 INFO:tasks.workunit.client.1.smithi026.stdout:4/333: write d1/db/f26 [3291696,78490] 0 2024-03-16T10:45:15.362 INFO:tasks.workunit.client.1.smithi026.stdout:9/351: rename d0/da/d4d/f5c to d0/da/d13/f78 0 2024-03-16T10:45:15.368 INFO:tasks.workunit.client.1.smithi026.stdout:9/352: symlink d0/da/d16/l79 0 2024-03-16T10:45:15.369 INFO:tasks.workunit.client.1.smithi026.stdout:9/353: read - d0/da/d13/f55 zero size 2024-03-16T10:45:15.374 INFO:tasks.workunit.client.1.smithi026.stdout:9/354: fdatasync d0/f4 0 2024-03-16T10:45:15.376 INFO:tasks.workunit.client.1.smithi026.stdout:9/355: readlink d0/da/d16/d17/l28 0 2024-03-16T10:45:15.377 INFO:tasks.workunit.client.1.smithi026.stdout:9/356: write d0/da/f11 [4160500,62479] 0 2024-03-16T10:45:15.378 INFO:tasks.workunit.client.1.smithi026.stdout:4/334: getdents d1/db/d29/d61 0 2024-03-16T10:45:15.379 INFO:tasks.workunit.client.1.smithi026.stdout:2/310: dwrite d2/d29/d17/d1a/d2b/d2e/f31 [0,4194304] 0 2024-03-16T10:45:15.381 INFO:tasks.workunit.client.1.smithi026.stdout:9/357: dread - d0/da/d13/f78 zero size 2024-03-16T10:45:15.381 INFO:tasks.workunit.client.1.smithi026.stdout:2/311: chown d2/d29/d36 2 1 2024-03-16T10:45:15.382 INFO:tasks.workunit.client.1.smithi026.stdout:2/312: dread - d2/d29/f63 zero size 2024-03-16T10:45:15.383 INFO:tasks.workunit.client.1.smithi026.stdout:2/313: dread - d2/d29/d51/f5f zero size 2024-03-16T10:45:15.394 INFO:tasks.workunit.client.1.smithi026.stdout:2/314: mkdir d2/d29/d17/d1a/d2b/d69 0 2024-03-16T10:45:15.397 INFO:tasks.workunit.client.1.smithi026.stdout:9/358: creat d0/da/f7a x:0 0 0 2024-03-16T10:45:15.400 INFO:tasks.workunit.client.1.smithi026.stdout:4/335: getdents d1/db/d29 0 2024-03-16T10:45:15.402 INFO:tasks.workunit.client.1.smithi026.stdout:4/336: chown d1/db/d29/d44/d48 488848 1 2024-03-16T10:45:15.404 INFO:tasks.workunit.client.1.smithi026.stdout:2/315: rename d2/d29/d17/d1a/d2b/l25 to d2/d29/d17/d38/d64/l6a 0 2024-03-16T10:45:15.407 INFO:tasks.workunit.client.1.smithi026.stdout:9/359: write d0/f1d [3435631,61477] 0 2024-03-16T10:45:15.409 INFO:tasks.workunit.client.1.smithi026.stdout:9/360: write d0/da/d13/f49 [1026553,5158] 0 2024-03-16T10:45:15.410 INFO:tasks.workunit.client.1.smithi026.stdout:9/361: truncate d0/da/d13/f38 1867320 0 2024-03-16T10:45:15.412 INFO:tasks.workunit.client.1.smithi026.stdout:2/316: symlink d2/d29/d17/d1a/l6b 0 2024-03-16T10:45:15.415 INFO:tasks.workunit.client.1.smithi026.stdout:4/337: mknod d1/db/de/c6d 0 2024-03-16T10:45:15.416 INFO:tasks.workunit.client.1.smithi026.stdout:9/362: symlink d0/da/d16/d17/d44/l7b 0 2024-03-16T10:45:15.418 INFO:tasks.workunit.client.1.smithi026.stdout:4/338: dread d1/db/d29/f3e [0,4194304] 0 2024-03-16T10:45:15.419 INFO:tasks.workunit.client.1.smithi026.stdout:2/317: unlink d2/d29/d17/d38/f53 0 2024-03-16T10:45:15.423 INFO:tasks.workunit.client.1.smithi026.stdout:4/339: dwrite d1/db/f26 [0,4194304] 0 2024-03-16T10:45:15.424 INFO:tasks.workunit.client.1.smithi026.stdout:9/363: rename d0/da/d14/f25 to d0/da/d16/d5d/f7c 0 2024-03-16T10:45:15.428 INFO:tasks.workunit.client.1.smithi026.stdout:9/364: chown d0/f66 16 1 2024-03-16T10:45:15.429 INFO:tasks.workunit.client.1.smithi026.stdout:9/365: truncate d0/f4 9306162 0 2024-03-16T10:45:15.431 INFO:tasks.workunit.client.1.smithi026.stdout:2/318: creat d2/d29/d17/d1a/d2b/d69/f6c x:0 0 0 2024-03-16T10:45:15.433 INFO:tasks.workunit.client.1.smithi026.stdout:2/319: dread - d2/d29/d17/d1a/d2b/d69/f6c zero size 2024-03-16T10:45:15.433 INFO:tasks.workunit.client.1.smithi026.stdout:4/340: mkdir d1/db/de/d6e 0 2024-03-16T10:45:15.435 INFO:tasks.workunit.client.1.smithi026.stdout:4/341: stat d1/db/d29/d44/d48 0 2024-03-16T10:45:15.436 INFO:tasks.workunit.client.1.smithi026.stdout:2/320: chown d2/d29/d17/d1a/d2b/d4b/f5d 635077 1 2024-03-16T10:45:15.436 INFO:tasks.workunit.client.1.smithi026.stdout:9/366: dwrite d0/da/d13/f38 [0,4194304] 0 2024-03-16T10:45:15.438 INFO:tasks.workunit.client.1.smithi026.stdout:2/321: dread - d2/d29/d3b/f66 zero size 2024-03-16T10:45:15.440 INFO:tasks.workunit.client.1.smithi026.stdout:9/367: chown d0/c21 238 1 2024-03-16T10:45:15.441 INFO:tasks.workunit.client.1.smithi026.stdout:9/368: fsync d0/da/f2a 0 2024-03-16T10:45:15.443 INFO:tasks.workunit.client.1.smithi026.stdout:9/369: truncate d0/da/f75 377865 0 2024-03-16T10:45:15.444 INFO:tasks.workunit.client.1.smithi026.stdout:2/322: unlink d2/l2a 0 2024-03-16T10:45:15.448 INFO:tasks.workunit.client.1.smithi026.stdout:4/342: rename d1/db/d29/d44/d48/d4f to d1/db/de/d4c/d6f 0 2024-03-16T10:45:15.449 INFO:tasks.workunit.client.1.smithi026.stdout:4/343: truncate d1/db/f64 795045 0 2024-03-16T10:45:15.450 INFO:tasks.workunit.client.1.smithi026.stdout:4/344: readlink d1/db/d29/d44/l46 0 2024-03-16T10:45:15.457 INFO:tasks.workunit.client.1.smithi026.stdout:2/323: dwrite d2/d29/d17/d1a/d2b/f15 [0,4194304] 0 2024-03-16T10:45:15.463 INFO:tasks.workunit.client.1.smithi026.stdout:2/324: rmdir d2/d29/d17/d1a/d2b/d4b 39 2024-03-16T10:45:15.465 INFO:tasks.workunit.client.1.smithi026.stdout:4/345: creat d1/db/de/f70 x:0 0 0 2024-03-16T10:45:15.466 INFO:tasks.workunit.client.1.smithi026.stdout:4/346: readlink d1/l35 0 2024-03-16T10:45:15.471 INFO:tasks.workunit.client.1.smithi026.stdout:4/347: symlink d1/db/d29/d44/d48/l71 0 2024-03-16T10:45:15.472 INFO:tasks.workunit.client.1.smithi026.stdout:1/313: sync 2024-03-16T10:45:15.473 INFO:tasks.workunit.client.1.smithi026.stdout:2/325: mkdir d2/d29/d51/d55/d6d 0 2024-03-16T10:45:15.474 INFO:tasks.workunit.client.1.smithi026.stdout:2/326: chown d2/d29/d17/d1a/c61 5428737 1 2024-03-16T10:45:15.474 INFO:tasks.workunit.client.1.smithi026.stdout:2/327: fsync d2/f35 0 2024-03-16T10:45:15.475 INFO:tasks.workunit.client.1.smithi026.stdout:2/328: chown d2/d29/d17/d1a/d2b/d69 27489 1 2024-03-16T10:45:15.476 INFO:tasks.workunit.client.1.smithi026.stdout:4/348: rename d1/db/l17 to d1/db/de/d4c/d6f/l72 0 2024-03-16T10:45:15.478 INFO:tasks.workunit.client.1.smithi026.stdout:4/349: read d1/f56 [638336,38978] 0 2024-03-16T10:45:15.479 INFO:tasks.workunit.client.1.smithi026.stdout:1/314: mknod d0/d17/d1f/c67 0 2024-03-16T10:45:15.480 INFO:tasks.workunit.client.1.smithi026.stdout:1/315: read d0/fd [747837,40664] 0 2024-03-16T10:45:15.484 INFO:tasks.workunit.client.1.smithi026.stdout:2/329: mknod d2/d29/d17/d1a/d2b/d2e/c6e 0 2024-03-16T10:45:15.487 INFO:tasks.workunit.client.1.smithi026.stdout:4/350: dread d1/f2d [0,4194304] 0 2024-03-16T10:45:15.487 INFO:tasks.workunit.client.1.smithi026.stdout:4/351: chown d1/db/f26 2 1 2024-03-16T10:45:15.488 INFO:tasks.workunit.client.1.smithi026.stdout:4/352: fsync d1/db/de/d4c/d59/f62 0 2024-03-16T10:45:15.491 INFO:tasks.workunit.client.1.smithi026.stdout:1/316: rename d0 to d0/d33/d68 22 2024-03-16T10:45:15.491 INFO:tasks.workunit.client.1.smithi026.stdout:1/317: fsync d0/d33/f3e 0 2024-03-16T10:45:15.492 INFO:tasks.workunit.client.1.smithi026.stdout:2/330: mkdir d2/d29/d17/d1a/d2b/d4e/d6f 0 2024-03-16T10:45:15.493 INFO:tasks.workunit.client.1.smithi026.stdout:1/318: write d0/d33/d3b/f27 [1908580,130148] 0 2024-03-16T10:45:15.497 INFO:tasks.workunit.client.1.smithi026.stdout:1/319: dread d0/d17/d1f/d38/f4e [0,4194304] 0 2024-03-16T10:45:15.515 INFO:tasks.workunit.client.1.smithi026.stdout:4/353: link d1/db/d29/f5f d1/db/d29/d44/d48/f73 0 2024-03-16T10:45:15.516 INFO:tasks.workunit.client.1.smithi026.stdout:1/320: mknod d0/d17/d1f/d63/c69 0 2024-03-16T10:45:15.517 INFO:tasks.workunit.client.1.smithi026.stdout:9/370: truncate d0/da/f2a 1379346 0 2024-03-16T10:45:15.518 INFO:tasks.workunit.client.1.smithi026.stdout:2/331: creat d2/d29/d17/d1a/d2b/d4e/d6f/f70 x:0 0 0 2024-03-16T10:45:15.519 INFO:tasks.workunit.client.1.smithi026.stdout:1/321: mkdir d0/d17/d1f/d63/d6a 0 2024-03-16T10:45:15.520 INFO:tasks.workunit.client.1.smithi026.stdout:1/322: chown d0/d33/d2e/c57 2 1 2024-03-16T10:45:15.521 INFO:tasks.workunit.client.1.smithi026.stdout:1/323: chown d0/d17/d1f/d63/c69 13172652 1 2024-03-16T10:45:15.523 INFO:tasks.workunit.client.1.smithi026.stdout:9/371: symlink d0/da/d16/d17/d44/l7d 0 2024-03-16T10:45:15.524 INFO:tasks.workunit.client.1.smithi026.stdout:9/372: write d0/da/f75 [325704,4951] 0 2024-03-16T10:45:15.525 INFO:tasks.workunit.client.1.smithi026.stdout:2/332: rename d2/d29/d17/f49 to d2/d29/d17/d1a/d2b/d4e/f71 0 2024-03-16T10:45:15.531 INFO:tasks.workunit.client.1.smithi026.stdout:9/373: dwrite d0/f68 [0,4194304] 0 2024-03-16T10:45:15.533 INFO:tasks.workunit.client.1.smithi026.stdout:1/324: mkdir d0/d33/d6b 0 2024-03-16T10:45:15.536 INFO:tasks.workunit.client.1.smithi026.stdout:1/325: write d0/d17/d4d/f61 [41272,68799] 0 2024-03-16T10:45:15.537 INFO:tasks.workunit.client.1.smithi026.stdout:2/333: dwrite d2/d29/d17/d1a/d2b/d2e/f33 [0,4194304] 0 2024-03-16T10:45:15.540 INFO:tasks.workunit.client.1.smithi026.stdout:2/334: fsync d2/d29/d17/d1a/d2b/d2e/f31 0 2024-03-16T10:45:15.543 INFO:tasks.workunit.client.1.smithi026.stdout:9/374: write d0/da/d13/f26 [4729311,119832] 0 2024-03-16T10:45:15.547 INFO:tasks.workunit.client.1.smithi026.stdout:1/326: unlink d0/d17/d4d/f61 0 2024-03-16T10:45:15.547 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:15 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:15.547 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:15 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:15.551 INFO:tasks.workunit.client.1.smithi026.stdout:2/335: mkdir d2/d29/d51/d72 0 2024-03-16T10:45:15.553 INFO:tasks.workunit.client.1.smithi026.stdout:9/375: mknod d0/d63/c7e 0 2024-03-16T10:45:15.557 INFO:tasks.workunit.client.1.smithi026.stdout:1/327: creat d0/d33/f6c x:0 0 0 2024-03-16T10:45:15.561 INFO:tasks.workunit.client.1.smithi026.stdout:1/328: creat d0/d17/d4d/f6d x:0 0 0 2024-03-16T10:45:15.562 INFO:tasks.workunit.client.1.smithi026.stdout:1/329: dread - d0/d17/d4d/f6d zero size 2024-03-16T10:45:15.564 INFO:tasks.workunit.client.1.smithi026.stdout:1/330: rmdir d0/d17/d1f/d20 39 2024-03-16T10:45:15.570 INFO:tasks.workunit.client.1.smithi026.stdout:1/331: dwrite d0/d17/d1f/f3f [0,4194304] 0 2024-03-16T10:45:15.573 INFO:tasks.workunit.client.1.smithi026.stdout:1/332: creat d0/d17/d1f/d38/f6e x:0 0 0 2024-03-16T10:45:15.573 INFO:tasks.workunit.client.1.smithi026.stdout:1/333: chown d0/d33/d2e/d54 72553613 1 2024-03-16T10:45:15.578 INFO:tasks.workunit.client.1.smithi026.stdout:1/334: creat d0/d33/d3b/d64/f6f x:0 0 0 2024-03-16T10:45:15.580 INFO:tasks.workunit.client.1.smithi026.stdout:1/335: mkdir d0/d17/d4d/d70 0 2024-03-16T10:45:15.581 INFO:tasks.workunit.client.1.smithi026.stdout:1/336: write d0/d17/d4d/f6d [8996,138] 0 2024-03-16T10:45:15.582 INFO:tasks.workunit.client.1.smithi026.stdout:1/337: chown d0/d33/d2e/c3d 4 1 2024-03-16T10:45:15.587 INFO:tasks.workunit.client.1.smithi026.stdout:1/338: dread d0/d17/d1f/f35 [0,4194304] 0 2024-03-16T10:45:15.589 INFO:tasks.workunit.client.1.smithi026.stdout:1/339: dread d0/d17/d1f/d38/f4e [0,4194304] 0 2024-03-16T10:45:15.590 INFO:tasks.workunit.client.1.smithi026.stdout:1/340: write d0/d17/f1a [3903715,127374] 0 2024-03-16T10:45:15.591 INFO:tasks.workunit.client.1.smithi026.stdout:1/341: write d0/d17/f1a [226602,47530] 0 2024-03-16T10:45:15.597 INFO:tasks.workunit.client.1.smithi026.stdout:2/336: truncate d2/d29/d17/d1a/d2b/d4b/f5c 796868 0 2024-03-16T10:45:15.598 INFO:tasks.workunit.client.1.smithi026.stdout:1/342: fdatasync d0/d17/d1f/d20/f43 0 2024-03-16T10:45:15.600 INFO:tasks.workunit.client.1.smithi026.stdout:2/337: rmdir d2/d29/d17/d1a/d2b 39 2024-03-16T10:45:15.601 INFO:tasks.workunit.client.1.smithi026.stdout:9/376: write d0/da/f2a [759570,130696] 0 2024-03-16T10:45:15.610 INFO:tasks.workunit.client.1.smithi026.stdout:1/343: mknod d0/d33/d2e/d54/d62/c71 0 2024-03-16T10:45:15.616 INFO:tasks.workunit.client.1.smithi026.stdout:9/377: mkdir d0/d5f/d7f 0 2024-03-16T10:45:15.617 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:15 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:15.617 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:15 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:15.617 INFO:tasks.workunit.client.1.smithi026.stdout:2/338: truncate d2/d29/d17/f2d 3482066 0 2024-03-16T10:45:15.618 INFO:tasks.workunit.client.1.smithi026.stdout:1/344: write d0/fd [52462,86697] 0 2024-03-16T10:45:15.623 INFO:tasks.workunit.client.1.smithi026.stdout:9/378: creat d0/da/d16/d5d/d72/d62/f80 x:0 0 0 2024-03-16T10:45:15.626 INFO:tasks.workunit.client.1.smithi026.stdout:1/345: dwrite d0/d33/f6c [0,4194304] 0 2024-03-16T10:45:15.626 INFO:tasks.workunit.client.1.smithi026.stdout:1/346: stat d0/d33/d6b 0 2024-03-16T10:45:15.632 INFO:tasks.workunit.client.1.smithi026.stdout:1/347: rename d0/d33/l29 to d0/d33/d6b/l72 0 2024-03-16T10:45:15.633 INFO:tasks.workunit.client.1.smithi026.stdout:2/339: dwrite d2/d29/d17/d1a/d2b/d2e/f3f [0,4194304] 0 2024-03-16T10:45:15.640 INFO:tasks.workunit.client.1.smithi026.stdout:2/340: rmdir d2/d29/d17/d38 39 2024-03-16T10:45:15.641 INFO:tasks.workunit.client.1.smithi026.stdout:1/348: dwrite d0/d33/d3b/f27 [0,4194304] 0 2024-03-16T10:45:15.643 INFO:tasks.workunit.client.1.smithi026.stdout:1/349: write d0/d33/f6c [2215384,87625] 0 2024-03-16T10:45:15.722 INFO:tasks.workunit.client.1.smithi026.stdout:1/350: mknod d0/d33/d2e/c73 0 2024-03-16T10:45:15.724 INFO:tasks.workunit.client.1.smithi026.stdout:2/341: creat d2/d29/d17/d1a/d2b/f73 x:0 0 0 2024-03-16T10:45:15.726 INFO:tasks.workunit.client.1.smithi026.stdout:2/342: symlink d2/d29/d51/d55/l74 0 2024-03-16T10:45:15.730 INFO:tasks.workunit.client.1.smithi026.stdout:1/351: creat d0/f74 x:0 0 0 2024-03-16T10:45:15.731 INFO:tasks.workunit.client.1.smithi026.stdout:1/352: fdatasync d0/d33/f59 0 2024-03-16T10:45:15.732 INFO:tasks.workunit.client.1.smithi026.stdout:1/353: write d0/d33/d2e/f56 [716769,3232] 0 2024-03-16T10:45:15.732 INFO:tasks.workunit.client.1.smithi026.stdout:2/343: creat d2/d29/d51/d72/f75 x:0 0 0 2024-03-16T10:45:15.734 INFO:tasks.workunit.client.1.smithi026.stdout:1/354: fsync d0/d33/d2e/f56 0 2024-03-16T10:45:15.734 INFO:tasks.workunit.client.1.smithi026.stdout:1/355: fdatasync d0/f5a 0 2024-03-16T10:45:15.735 INFO:tasks.workunit.client.1.smithi026.stdout:2/344: creat d2/d29/d51/d72/f76 x:0 0 0 2024-03-16T10:45:15.735 INFO:tasks.workunit.client.1.smithi026.stdout:1/356: chown d0/l1d 2423 1 2024-03-16T10:45:15.739 INFO:tasks.workunit.client.1.smithi026.stdout:2/345: getdents d2 0 2024-03-16T10:45:15.742 INFO:tasks.workunit.client.1.smithi026.stdout:1/357: dwrite d0/f5a [0,4194304] 0 2024-03-16T10:45:15.748 INFO:tasks.workunit.client.1.smithi026.stdout:2/346: rename d2/d29/d17/d1a/d2b/d69/f6c to d2/d29/d17/d1a/d2b/d4e/d65/f77 0 2024-03-16T10:45:15.750 INFO:tasks.workunit.client.1.smithi026.stdout:1/358: mkdir d0/d17/d1f/d20/d75 0 2024-03-16T10:45:15.753 INFO:tasks.workunit.client.1.smithi026.stdout:1/359: symlink d0/d33/d2e/d54/l76 0 2024-03-16T10:45:15.754 INFO:tasks.workunit.client.1.smithi026.stdout:1/360: write d0/fd [5220850,66781] 0 2024-03-16T10:45:15.756 INFO:tasks.workunit.client.1.smithi026.stdout:2/347: link d2/d29/d3b/f66 d2/d29/d17/d1a/d2b/d4e/d67/f78 0 2024-03-16T10:45:15.759 INFO:tasks.workunit.client.1.smithi026.stdout:2/348: symlink d2/d29/d17/d1a/d2b/d4e/l79 0 2024-03-16T10:45:15.761 INFO:tasks.workunit.client.1.smithi026.stdout:2/349: mknod d2/d29/d51/d55/c7a 0 2024-03-16T10:45:15.766 INFO:tasks.workunit.client.1.smithi026.stdout:2/350: rmdir d2/d29/d17/d1a/d2b/d2e 39 2024-03-16T10:45:15.772 INFO:tasks.workunit.client.1.smithi026.stdout:2/351: dread d2/d29/d17/f2d [0,4194304] 0 2024-03-16T10:45:15.785 INFO:tasks.workunit.client.1.smithi026.stdout:2/352: dwrite d2/d29/d17/f2d [0,4194304] 0 2024-03-16T10:45:15.792 INFO:tasks.workunit.client.1.smithi026.stdout:2/353: dwrite d2/d29/d17/f2d [0,4194304] 0 2024-03-16T10:45:15.804 INFO:tasks.workunit.client.1.smithi026.stdout:2/354: mknod d2/d29/d17/d1a/d2b/c7b 0 2024-03-16T10:45:15.805 INFO:tasks.workunit.client.1.smithi026.stdout:2/355: stat d2/d29/l54 0 2024-03-16T10:45:15.805 INFO:tasks.workunit.client.1.smithi026.stdout:2/356: fdatasync d2/f35 0 2024-03-16T10:45:15.807 INFO:tasks.workunit.client.1.smithi026.stdout:2/357: fsync d2/d29/fe 0 2024-03-16T10:45:15.812 INFO:tasks.workunit.client.1.smithi026.stdout:2/358: write d2/d29/d17/d1a/d2b/d4e/f71 [1112594,61890] 0 2024-03-16T10:45:15.917 INFO:tasks.workunit.client.1.smithi026.stdout:7/335: sync 2024-03-16T10:45:15.917 INFO:tasks.workunit.client.1.smithi026.stdout:0/333: sync 2024-03-16T10:45:15.918 INFO:tasks.workunit.client.1.smithi026.stdout:7/336: chown d6/d1e/l2b 506 1 2024-03-16T10:45:15.918 INFO:tasks.workunit.client.1.smithi026.stdout:0/334: truncate d1/d12/f4b 735235 0 2024-03-16T10:45:15.934 INFO:tasks.workunit.client.1.smithi026.stdout:0/335: creat d1/d18/d1a/d24/f6d x:0 0 0 2024-03-16T10:45:15.939 INFO:tasks.workunit.client.1.smithi026.stdout:7/337: getdents d6/d35 0 2024-03-16T10:45:15.942 INFO:tasks.workunit.client.1.smithi026.stdout:7/338: mkdir d6/d35/d3a/d4e/d6c 0 2024-03-16T10:45:15.946 INFO:tasks.workunit.client.1.smithi026.stdout:0/336: getdents d1/d18/d1a/d2b/d4f 0 2024-03-16T10:45:15.947 INFO:tasks.workunit.client.1.smithi026.stdout:7/339: write d6/d35/d3a/d3c/f40 [1263732,13753] 0 2024-03-16T10:45:15.948 INFO:tasks.workunit.client.1.smithi026.stdout:7/340: readlink d6/d35/d3a/d59/l5b 0 2024-03-16T10:45:15.949 INFO:tasks.workunit.client.1.smithi026.stdout:7/341: fdatasync d6/d1e/f29 0 2024-03-16T10:45:15.955 INFO:tasks.workunit.client.1.smithi026.stdout:7/342: dwrite d6/d1e/f32 [0,4194304] 0 2024-03-16T10:45:15.961 INFO:tasks.workunit.client.1.smithi026.stdout:7/343: dread d6/de/d16/f49 [0,4194304] 0 2024-03-16T10:45:15.962 INFO:tasks.workunit.client.1.smithi026.stdout:7/344: truncate d6/d35/f66 692040 0 2024-03-16T10:45:15.967 INFO:tasks.workunit.client.1.smithi026.stdout:7/345: creat d6/d35/d3a/d4e/f6d x:0 0 0 2024-03-16T10:45:15.968 INFO:tasks.workunit.client.1.smithi026.stdout:7/346: chown d6/f15 7149 1 2024-03-16T10:45:15.969 INFO:tasks.workunit.client.1.smithi026.stdout:0/337: dread d1/d18/d1a/d24/d11/f54 [0,4194304] 0 2024-03-16T10:45:15.970 INFO:tasks.workunit.client.1.smithi026.stdout:7/347: truncate d6/de/f64 851005 0 2024-03-16T10:45:15.971 INFO:tasks.workunit.client.1.smithi026.stdout:0/338: dread d1/d18/d1a/f46 [0,4194304] 0 2024-03-16T10:45:15.972 INFO:tasks.workunit.client.1.smithi026.stdout:7/348: write d6/de/d16/f49 [44282,10951] 0 2024-03-16T10:45:15.976 INFO:tasks.workunit.client.1.smithi026.stdout:7/349: symlink d6/d35/d3a/l6e 0 2024-03-16T10:45:15.977 INFO:tasks.workunit.client.1.smithi026.stdout:7/350: chown d6/de/d16/f43 7221 1 2024-03-16T10:45:15.978 INFO:tasks.workunit.client.1.smithi026.stdout:7/351: fdatasync d6/d35/d3a/d4e/f6d 0 2024-03-16T10:45:15.982 INFO:tasks.workunit.client.1.smithi026.stdout:7/352: mkdir d6/d35/d3a/d3c/d3f/d65/d6f 0 2024-03-16T10:45:15.985 INFO:tasks.workunit.client.1.smithi026.stdout:7/353: mkdir d6/d35/d3a/d3c/d3f/d70 0 2024-03-16T10:45:15.986 INFO:tasks.workunit.client.1.smithi026.stdout:0/339: dread d1/d18/f36 [0,4194304] 0 2024-03-16T10:45:15.988 INFO:tasks.workunit.client.1.smithi026.stdout:7/354: truncate d6/d35/d3a/d3c/f51 506415 0 2024-03-16T10:45:15.989 INFO:tasks.workunit.client.1.smithi026.stdout:0/340: symlink d1/d18/d1a/d2b/d4f/l6e 0 2024-03-16T10:45:15.989 INFO:tasks.workunit.client.1.smithi026.stdout:0/341: read - d1/d18/d1a/d2b/f4e zero size 2024-03-16T10:45:15.994 INFO:tasks.workunit.client.1.smithi026.stdout:0/342: creat d1/d18/d1a/d24/f6f x:0 0 0 2024-03-16T10:45:15.997 INFO:tasks.workunit.client.1.smithi026.stdout:0/343: creat d1/d12/d2f/d48/d55/d60/f70 x:0 0 0 2024-03-16T10:45:15.999 INFO:tasks.workunit.client.1.smithi026.stdout:0/344: chown d1/d18/d1a/d24/l3f 15007921 1 2024-03-16T10:45:16.003 INFO:tasks.workunit.client.1.smithi026.stdout:0/345: creat d1/d12/d2f/d48/d55/f71 x:0 0 0 2024-03-16T10:45:16.018 INFO:tasks.workunit.client.1.smithi026.stdout:0/346: write d1/d18/d1a/d24/d51/f5e [262911,25562] 0 2024-03-16T10:45:16.021 INFO:tasks.workunit.client.1.smithi026.stdout:0/347: unlink d1/d18/f5c 0 2024-03-16T10:45:16.023 INFO:tasks.workunit.client.1.smithi026.stdout:0/348: creat d1/d18/d1a/d24/d51/f72 x:0 0 0 2024-03-16T10:45:16.047 INFO:tasks.workunit.client.1.smithi026.stdout:0/349: dwrite d1/d18/d1a/d24/f59 [0,4194304] 0 2024-03-16T10:45:16.049 INFO:tasks.workunit.client.1.smithi026.stdout:0/350: fsync d1/d18/d1a/f26 0 2024-03-16T10:45:16.053 INFO:tasks.workunit.client.1.smithi026.stdout:0/351: creat d1/d18/d1a/d24/d51/f73 x:0 0 0 2024-03-16T10:45:16.058 INFO:tasks.workunit.client.1.smithi026.stdout:0/352: unlink d1/d18/d1a/d24/d11/l3a 0 2024-03-16T10:45:16.060 INFO:tasks.workunit.client.1.smithi026.stdout:0/353: write d1/d18/d1a/d24/d51/f5d [740605,11630] 0 2024-03-16T10:45:16.063 INFO:tasks.workunit.client.1.smithi026.stdout:0/354: mknod d1/d18/d1a/d2b/c74 0 2024-03-16T10:45:16.075 INFO:tasks.workunit.client.1.smithi026.stdout:0/355: dread d1/d18/d1a/d2b/d4f/f53 [0,4194304] 0 2024-03-16T10:45:16.075 INFO:tasks.workunit.client.1.smithi026.stdout:0/356: chown d1/d18/d1a/d2b/c74 715 1 2024-03-16T10:45:16.078 INFO:tasks.workunit.client.1.smithi026.stdout:0/357: fsync d1/d18/d61/f6b 0 2024-03-16T10:45:16.079 INFO:tasks.workunit.client.1.smithi026.stdout:0/358: chown d1/d18/d61 308263 1 2024-03-16T10:45:16.134 INFO:tasks.workunit.client.1.smithi026.stdout:8/324: sync 2024-03-16T10:45:16.149 INFO:tasks.workunit.client.1.smithi026.stdout:0/359: rmdir d1/d18/d1a/d24/d51 39 2024-03-16T10:45:16.153 INFO:tasks.workunit.client.1.smithi026.stdout:8/325: truncate d5/dc/f1e 333814 0 2024-03-16T10:45:16.157 INFO:tasks.workunit.client.1.smithi026.stdout:8/326: creat d5/dc/d3f/f55 x:0 0 0 2024-03-16T10:45:16.159 INFO:tasks.workunit.client.1.smithi026.stdout:0/360: creat d1/d18/d1a/d24/d51/f75 x:0 0 0 2024-03-16T10:45:16.164 INFO:tasks.workunit.client.1.smithi026.stdout:8/327: dwrite d5/d7/d2c/f50 [0,4194304] 0 2024-03-16T10:45:16.170 INFO:tasks.workunit.client.1.smithi026.stdout:0/361: link d1/l3d d1/d12/d2f/d48/d55/l76 0 2024-03-16T10:45:16.173 INFO:tasks.workunit.client.1.smithi026.stdout:8/328: dwrite d5/dc/f4d [0,4194304] 0 2024-03-16T10:45:16.182 INFO:tasks.workunit.client.1.smithi026.stdout:8/329: dwrite d5/dc/f18 [0,4194304] 0 2024-03-16T10:45:16.187 INFO:tasks.workunit.client.1.smithi026.stdout:0/362: creat d1/d12/d2f/f77 x:0 0 0 2024-03-16T10:45:16.193 INFO:tasks.workunit.client.1.smithi026.stdout:8/330: link d5/dc/d3f/f55 d5/d7/d2c/f56 0 2024-03-16T10:45:16.196 INFO:tasks.workunit.client.1.smithi026.stdout:0/363: rename d1/d12/d2f/d48/d55/d60/lc to d1/d18/d1a/d24/d11/l78 0 2024-03-16T10:45:16.199 INFO:tasks.workunit.client.1.smithi026.stdout:0/364: symlink d1/d12/d2f/d48/d55/l79 0 2024-03-16T10:45:16.201 INFO:tasks.workunit.client.1.smithi026.stdout:0/365: write d1/d12/d2f/f77 [314239,110960] 0 2024-03-16T10:45:16.205 INFO:tasks.workunit.client.1.smithi026.stdout:0/366: dread d1/d18/d1a/d2b/d4f/f53 [0,4194304] 0 2024-03-16T10:45:16.210 INFO:tasks.workunit.client.1.smithi026.stdout:8/331: write d5/f3e [649215,108088] 0 2024-03-16T10:45:16.212 INFO:tasks.workunit.client.1.smithi026.stdout:0/367: creat d1/d18/d1a/f7a x:0 0 0 2024-03-16T10:45:16.214 INFO:tasks.workunit.client.1.smithi026.stdout:8/332: getdents d5/dc/d40/d53 0 2024-03-16T10:45:16.215 INFO:tasks.workunit.client.1.smithi026.stdout:8/333: readlink d5/dc/d40/l42 0 2024-03-16T10:45:16.219 INFO:tasks.workunit.client.1.smithi026.stdout:0/368: rename d1/d18/d1a/d24/f65 to d1/d12/d2f/f7b 0 2024-03-16T10:45:16.221 INFO:tasks.workunit.client.1.smithi026.stdout:8/334: dwrite d5/d7/f1d [0,4194304] 0 2024-03-16T10:45:16.223 INFO:tasks.workunit.client.1.smithi026.stdout:0/369: creat d1/d12/d2f/d48/d55/f7c x:0 0 0 2024-03-16T10:45:16.225 INFO:tasks.workunit.client.1.smithi026.stdout:8/335: creat d5/d7/f57 x:0 0 0 2024-03-16T10:45:16.225 INFO:tasks.workunit.client.1.smithi026.stdout:0/370: stat d1/d12/l2a 0 2024-03-16T10:45:16.228 INFO:tasks.workunit.client.1.smithi026.stdout:0/371: symlink d1/d12/d2f/d48/l7d 0 2024-03-16T10:45:16.229 INFO:tasks.workunit.client.1.smithi026.stdout:8/336: creat d5/dc/d40/d53/f58 x:0 0 0 2024-03-16T10:45:16.229 INFO:tasks.workunit.client.1.smithi026.stdout:0/372: stat d1/d18/d1a/d24/d11/f54 0 2024-03-16T10:45:16.234 INFO:tasks.workunit.client.1.smithi026.stdout:0/373: write d1/d18/d1a/d24/d57/f3e [3035862,42434] 0 2024-03-16T10:45:16.243 INFO:tasks.workunit.client.1.smithi026.stdout:0/374: dwrite d1/d18/d1a/d24/d51/f75 [0,4194304] 0 2024-03-16T10:45:16.248 INFO:tasks.workunit.client.1.smithi026.stdout:0/375: mknod d1/d18/d1a/d24/d51/d5f/c7e 0 2024-03-16T10:45:16.250 INFO:tasks.workunit.client.1.smithi026.stdout:8/337: rmdir d5/d7 39 2024-03-16T10:45:16.250 INFO:tasks.workunit.client.1.smithi026.stdout:0/376: truncate d1/d12/d2f/f7b 930017 0 2024-03-16T10:45:16.251 INFO:tasks.workunit.client.1.smithi026.stdout:0/377: stat d1/d12/f4a 0 2024-03-16T10:45:16.255 INFO:tasks.workunit.client.1.smithi026.stdout:6/324: sync 2024-03-16T10:45:16.259 INFO:tasks.workunit.client.1.smithi026.stdout:8/338: write d5/d7/f43 [1826005,89722] 0 2024-03-16T10:45:16.260 INFO:tasks.workunit.client.1.smithi026.stdout:8/339: chown d5/d23/f4b 434376 1 2024-03-16T10:45:16.261 INFO:tasks.workunit.client.1.smithi026.stdout:6/325: stat d0/d10/cb 0 2024-03-16T10:45:16.265 INFO:tasks.workunit.client.1.smithi026.stdout:0/378: symlink d1/d18/d1a/d24/l7f 0 2024-03-16T10:45:16.268 INFO:tasks.workunit.client.1.smithi026.stdout:6/326: dwrite d0/d28/d2d/f61 [0,4194304] 0 2024-03-16T10:45:16.275 INFO:tasks.workunit.client.1.smithi026.stdout:6/327: rename d0/d10/f36 to d0/d41/d4f/f77 0 2024-03-16T10:45:16.278 INFO:tasks.workunit.client.1.smithi026.stdout:0/379: unlink d1/d18/d1a/d24/f29 0 2024-03-16T10:45:16.280 INFO:tasks.workunit.client.1.smithi026.stdout:0/380: chown d1/d18/d1a/d24/d51 182230 1 2024-03-16T10:45:16.282 INFO:tasks.workunit.client.1.smithi026.stdout:6/328: creat d0/d28/d49/f78 x:0 0 0 2024-03-16T10:45:16.287 INFO:tasks.workunit.client.1.smithi026.stdout:8/340: dwrite d5/d7/f11 [4194304,4194304] 0 2024-03-16T10:45:16.290 INFO:tasks.workunit.client.1.smithi026.stdout:0/381: dread d1/d18/d1a/d24/d11/f54 [0,4194304] 0 2024-03-16T10:45:16.297 INFO:tasks.workunit.client.1.smithi026.stdout:8/341: mkdir d5/d7/d2c/d36/d59 0 2024-03-16T10:45:16.301 INFO:tasks.workunit.client.1.smithi026.stdout:6/329: dread d0/f2c [0,4194304] 0 2024-03-16T10:45:16.302 INFO:tasks.workunit.client.1.smithi026.stdout:0/382: dwrite d1/d12/d2f/f42 [0,4194304] 0 2024-03-16T10:45:16.304 INFO:tasks.workunit.client.1.smithi026.stdout:8/342: mknod d5/dc/c5a 0 2024-03-16T10:45:16.307 INFO:tasks.workunit.client.1.smithi026.stdout:0/383: creat d1/d18/d1a/d2b/d4f/f80 x:0 0 0 2024-03-16T10:45:16.310 INFO:tasks.workunit.client.1.smithi026.stdout:0/384: rename d1/d18/d1a to d1/d18/d1a/d24/d11/d81 22 2024-03-16T10:45:16.311 INFO:tasks.workunit.client.1.smithi026.stdout:8/343: dwrite d5/d28/f31 [0,4194304] 0 2024-03-16T10:45:16.312 INFO:tasks.workunit.client.1.smithi026.stdout:0/385: chown d1/d18/d1a/d24/d51/d5f/c7e 21296705 1 2024-03-16T10:45:16.314 INFO:tasks.workunit.client.1.smithi026.stdout:6/330: mkdir d0/d10/d79 0 2024-03-16T10:45:16.316 INFO:tasks.workunit.client.1.smithi026.stdout:0/386: chown d1/d18/d1a/d2b/l68 31 1 2024-03-16T10:45:16.317 INFO:tasks.workunit.client.1.smithi026.stdout:8/344: creat d5/d23/d4e/f5b x:0 0 0 2024-03-16T10:45:16.320 INFO:tasks.workunit.client.1.smithi026.stdout:0/387: unlink d1/d18/d1a/d24/d11/c2e 0 2024-03-16T10:45:16.323 INFO:tasks.workunit.client.1.smithi026.stdout:6/331: rename d0/d10/f31 to d0/d10/f7a 0 2024-03-16T10:45:16.325 INFO:tasks.workunit.client.1.smithi026.stdout:8/345: rename d5/d7/f1b to d5/dc/d40/d53/f5c 0 2024-03-16T10:45:16.327 INFO:tasks.workunit.client.1.smithi026.stdout:0/388: creat d1/d18/d1a/d24/f82 x:0 0 0 2024-03-16T10:45:16.328 INFO:tasks.workunit.client.1.smithi026.stdout:6/332: mkdir d0/d10/d79/d7b 0 2024-03-16T10:45:16.330 INFO:tasks.workunit.client.1.smithi026.stdout:4/354: sync 2024-03-16T10:45:16.330 INFO:tasks.workunit.client.1.smithi026.stdout:3/392: sync 2024-03-16T10:45:16.330 INFO:tasks.workunit.client.1.smithi026.stdout:9/379: sync 2024-03-16T10:45:16.330 INFO:tasks.workunit.client.1.smithi026.stdout:2/359: sync 2024-03-16T10:45:16.331 INFO:tasks.workunit.client.1.smithi026.stdout:1/361: sync 2024-03-16T10:45:16.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/355: sync 2024-03-16T10:45:16.331 INFO:tasks.workunit.client.1.smithi026.stdout:5/313: sync 2024-03-16T10:45:16.331 INFO:tasks.workunit.client.1.smithi026.stdout:3/393: stat d4/f41 0 2024-03-16T10:45:16.333 INFO:tasks.workunit.client.1.smithi026.stdout:4/355: chown d1/db/l39 107741 1 2024-03-16T10:45:16.333 INFO:tasks.workunit.client.1.smithi026.stdout:6/333: rmdir d0/d62 39 2024-03-16T10:45:16.335 INFO:tasks.workunit.client.1.smithi026.stdout:7/356: chown d6/d35/d3a/d4e/l58 5 1 2024-03-16T10:45:16.336 INFO:tasks.workunit.client.1.smithi026.stdout:0/389: rename d1/d18/d1a/d24/d51/f73 to d1/d18/d1a/d24/d57/f83 0 2024-03-16T10:45:16.339 INFO:tasks.workunit.client.1.smithi026.stdout:9/380: creat d0/d63/d6b/f81 x:0 0 0 2024-03-16T10:45:16.342 INFO:tasks.workunit.client.1.smithi026.stdout:0/390: dread - d1/d18/d1a/d24/f6d zero size 2024-03-16T10:45:16.343 INFO:tasks.workunit.client.1.smithi026.stdout:8/346: getdents d5/dc 0 2024-03-16T10:45:16.345 INFO:tasks.workunit.client.1.smithi026.stdout:5/314: dwrite d1/db/d27/f66 [0,4194304] 0 2024-03-16T10:45:16.345 INFO:tasks.workunit.client.1.smithi026.stdout:8/347: fsync d5/f3e 0 2024-03-16T10:45:16.349 INFO:tasks.workunit.client.1.smithi026.stdout:0/391: stat d1/d18/d1a/d24/d11/f54 0 2024-03-16T10:45:16.350 INFO:tasks.workunit.client.1.smithi026.stdout:4/356: dwrite d1/db/f26 [0,4194304] 0 2024-03-16T10:45:16.358 INFO:tasks.workunit.client.1.smithi026.stdout:2/360: write d2/d29/d17/d1a/d2b/d2e/f33 [5106261,111264] 0 2024-03-16T10:45:16.359 INFO:tasks.workunit.client.1.smithi026.stdout:2/361: read d2/f4d [269955,52853] 0 2024-03-16T10:45:16.360 INFO:tasks.workunit.client.1.smithi026.stdout:2/362: chown d2/d29/d17/d1a/d2b/c4a 2253061 1 2024-03-16T10:45:16.361 INFO:tasks.workunit.client.1.smithi026.stdout:2/363: chown d2/d29/d17/d1a/d2b/f15 559838666 1 2024-03-16T10:45:16.362 INFO:tasks.workunit.client.1.smithi026.stdout:2/364: write d2/d29/d17/d1a/d2b/f18 [7593814,74733] 0 2024-03-16T10:45:16.372 INFO:tasks.workunit.client.1.smithi026.stdout:5/315: mknod d1/db/d1f/c6a 0 2024-03-16T10:45:16.378 INFO:tasks.workunit.client.1.smithi026.stdout:7/357: rename d6/de/d16/f2e to d6/d35/d3a/d3c/d3f/d65/f71 0 2024-03-16T10:45:16.380 INFO:tasks.workunit.client.1.smithi026.stdout:0/392: dwrite d1/d18/d1a/d24/f67 [0,4194304] 0 2024-03-16T10:45:16.381 INFO:tasks.workunit.client.1.smithi026.stdout:6/334: fdatasync d0/d10/f7a 0 2024-03-16T10:45:16.382 INFO:tasks.workunit.client.1.smithi026.stdout:6/335: chown d0 43 1 2024-03-16T10:45:16.387 INFO:tasks.workunit.client.1.smithi026.stdout:7/358: dwrite d6/de/d16/f49 [0,4194304] 0 2024-03-16T10:45:16.402 INFO:tasks.workunit.client.1.smithi026.stdout:8/348: mkdir d5/d5d 0 2024-03-16T10:45:16.406 INFO:tasks.workunit.client.1.smithi026.stdout:3/394: creat d4/dc/dd/d60/d62/f85 x:0 0 0 2024-03-16T10:45:16.415 INFO:tasks.workunit.client.1.smithi026.stdout:3/395: dread d4/dc/f25 [0,4194304] 0 2024-03-16T10:45:16.415 INFO:tasks.workunit.client.1.smithi026.stdout:3/396: fdatasync d4/fe 0 2024-03-16T10:45:16.420 INFO:tasks.workunit.client.1.smithi026.stdout:5/316: dread d1/d1c/d2a/f57 [0,4194304] 0 2024-03-16T10:45:16.421 INFO:tasks.workunit.client.1.smithi026.stdout:5/317: write d1/fd [2195810,31547] 0 2024-03-16T10:45:16.422 INFO:tasks.workunit.client.1.smithi026.stdout:5/318: write d1/fd [2816089,2852] 0 2024-03-16T10:45:16.422 INFO:tasks.workunit.client.1.smithi026.stdout:0/393: mkdir d1/d12/d2f/d48/d84 0 2024-03-16T10:45:16.424 INFO:tasks.workunit.client.1.smithi026.stdout:0/394: truncate d1/d12/f45 241317 0 2024-03-16T10:45:16.425 INFO:tasks.workunit.client.1.smithi026.stdout:5/319: truncate d1/d1c/d2a/f3b 789424 0 2024-03-16T10:45:16.425 INFO:tasks.workunit.client.1.smithi026.stdout:0/395: dread - d1/d18/d1a/d24/d57/f83 zero size 2024-03-16T10:45:16.426 INFO:tasks.workunit.client.1.smithi026.stdout:5/320: chown d1/d1c/d2a/c62 119749479 1 2024-03-16T10:45:16.427 INFO:tasks.workunit.client.1.smithi026.stdout:6/336: unlink d0/d10/l58 0 2024-03-16T10:45:16.434 INFO:tasks.workunit.client.1.smithi026.stdout:5/321: dwrite d1/d1c/d2a/d31/f52 [0,4194304] 0 2024-03-16T10:45:16.436 INFO:tasks.workunit.client.1.smithi026.stdout:5/322: read d1/db/d27/f3e [3017572,74749] 0 2024-03-16T10:45:16.437 INFO:tasks.workunit.client.1.smithi026.stdout:1/362: getdents d0/d17/d4d 0 2024-03-16T10:45:16.438 INFO:tasks.workunit.client.1.smithi026.stdout:1/363: write d0/f1e [520775,82074] 0 2024-03-16T10:45:16.439 INFO:tasks.workunit.client.1.smithi026.stdout:1/364: read - d0/d33/f59 zero size 2024-03-16T10:45:16.441 INFO:tasks.workunit.client.1.smithi026.stdout:1/365: write d0/d33/f30 [3827812,21140] 0 2024-03-16T10:45:16.441 INFO:tasks.workunit.client.1.smithi026.stdout:1/366: stat d0 0 2024-03-16T10:45:16.442 INFO:tasks.workunit.client.1.smithi026.stdout:9/381: link d0/da/d13/f5a d0/f82 0 2024-03-16T10:45:16.443 INFO:tasks.workunit.client.1.smithi026.stdout:9/382: write d0/f4 [5914877,52056] 0 2024-03-16T10:45:16.445 INFO:tasks.workunit.client.1.smithi026.stdout:0/396: dread d1/d18/d1a/d24/d57/f3e [0,4194304] 0 2024-03-16T10:45:16.453 INFO:tasks.workunit.client.1.smithi026.stdout:4/357: creat d1/db/f74 x:0 0 0 2024-03-16T10:45:16.458 INFO:tasks.workunit.client.1.smithi026.stdout:9/383: dwrite d0/da/d16/d5d/d72/d62/f65 [0,4194304] 0 2024-03-16T10:45:16.461 INFO:tasks.workunit.client.1.smithi026.stdout:1/367: dread d0/d17/d1f/f35 [0,4194304] 0 2024-03-16T10:45:16.461 INFO:tasks.workunit.client.1.smithi026.stdout:3/397: write d4/dc/dd/d60/f64 [1174320,14026] 0 2024-03-16T10:45:16.463 INFO:tasks.workunit.client.1.smithi026.stdout:3/398: write d4/dc/dd/f24 [4789724,69506] 0 2024-03-16T10:45:16.489 INFO:tasks.workunit.client.1.smithi026.stdout:2/365: dwrite d2/d29/d3b/f47 [0,4194304] 0 2024-03-16T10:45:16.506 INFO:tasks.workunit.client.1.smithi026.stdout:4/358: readlink d1/db/de/l5e 0 2024-03-16T10:45:16.507 INFO:tasks.workunit.client.1.smithi026.stdout:4/359: write d1/db/de/f70 [56892,89285] 0 2024-03-16T10:45:16.513 INFO:tasks.workunit.client.1.smithi026.stdout:4/360: dwrite d1/db/f67 [4194304,4194304] 0 2024-03-16T10:45:16.520 INFO:tasks.workunit.client.1.smithi026.stdout:6/337: chown d0/d62/l73 29 1 2024-03-16T10:45:16.528 INFO:tasks.workunit.client.1.smithi026.stdout:2/366: truncate d2/d29/d17/f27 5184257 0 2024-03-16T10:45:16.533 INFO:tasks.workunit.client.1.smithi026.stdout:0/397: mkdir d1/d12/d2f/d48/d84/d85 0 2024-03-16T10:45:16.540 INFO:tasks.workunit.client.1.smithi026.stdout:9/384: link d0/f1d d0/d63/f83 0 2024-03-16T10:45:16.541 INFO:tasks.workunit.client.1.smithi026.stdout:9/385: chown d0/da/d4d/f5e 1 1 2024-03-16T10:45:16.543 INFO:tasks.workunit.client.1.smithi026.stdout:1/368: symlink d0/l77 0 2024-03-16T10:45:16.544 INFO:tasks.workunit.client.1.smithi026.stdout:5/323: write d1/db/d1f/f5c [591137,95837] 0 2024-03-16T10:45:16.544 INFO:tasks.workunit.client.1.smithi026.stdout:1/369: stat d0/d17/d1f/c15 0 2024-03-16T10:45:16.546 INFO:tasks.workunit.client.1.smithi026.stdout:4/361: rmdir d1/db/d29 39 2024-03-16T10:45:16.549 INFO:tasks.workunit.client.1.smithi026.stdout:1/370: dread d0/d33/f3e [0,4194304] 0 2024-03-16T10:45:16.555 INFO:tasks.workunit.client.1.smithi026.stdout:7/359: getdents d6 0 2024-03-16T10:45:16.556 INFO:tasks.workunit.client.1.smithi026.stdout:8/349: getdents d5/dc/d40/d53 0 2024-03-16T10:45:16.559 INFO:tasks.workunit.client.1.smithi026.stdout:5/324: symlink d1/d1c/d4c/d4d/l6b 0 2024-03-16T10:45:16.561 INFO:tasks.workunit.client.1.smithi026.stdout:4/362: read - d1/db/d29/d44/d48/f60 zero size 2024-03-16T10:45:16.563 INFO:tasks.workunit.client.1.smithi026.stdout:3/399: rename d4/dc/d31/d48/f58 to d4/dc/dd/d47/f86 0 2024-03-16T10:45:16.564 INFO:tasks.workunit.client.1.smithi026.stdout:3/400: stat d4/dc/dd/d3d/c81 0 2024-03-16T10:45:16.565 INFO:tasks.workunit.client.1.smithi026.stdout:1/371: creat d0/d17/d4d/f78 x:0 0 0 2024-03-16T10:45:16.568 INFO:tasks.workunit.client.1.smithi026.stdout:7/360: mknod d6/d35/c72 0 2024-03-16T10:45:16.572 INFO:tasks.workunit.client.1.smithi026.stdout:7/361: truncate d6/de/d16/f19 4892471 0 2024-03-16T10:45:16.573 INFO:tasks.workunit.client.1.smithi026.stdout:1/372: dwrite d0/d33/f6c [0,4194304] 0 2024-03-16T10:45:16.573 INFO:tasks.workunit.client.1.smithi026.stdout:7/362: fsync d6/f31 0 2024-03-16T10:45:16.575 INFO:tasks.workunit.client.1.smithi026.stdout:7/363: write d6/d1e/f20 [3020988,47463] 0 2024-03-16T10:45:16.576 INFO:tasks.workunit.client.1.smithi026.stdout:0/398: rename d1/d12/d2f/f42 to d1/d18/d1a/d2b/f86 0 2024-03-16T10:45:16.585 INFO:tasks.workunit.client.1.smithi026.stdout:8/350: symlink d5/d5d/l5e 0 2024-03-16T10:45:16.586 INFO:tasks.workunit.client.1.smithi026.stdout:8/351: chown d5/d23 41399 1 2024-03-16T10:45:16.587 INFO:tasks.workunit.client.1.smithi026.stdout:5/325: symlink d1/db/l6c 0 2024-03-16T10:45:16.587 INFO:tasks.workunit.client.1.smithi026.stdout:5/326: chown d1/d1c/d2a 20115 1 2024-03-16T10:45:16.595 INFO:tasks.workunit.client.1.smithi026.stdout:1/373: symlink d0/d33/d2e/l79 0 2024-03-16T10:45:16.597 INFO:tasks.workunit.client.1.smithi026.stdout:7/364: dread d6/de/d16/f43 [0,4194304] 0 2024-03-16T10:45:16.597 INFO:tasks.workunit.client.1.smithi026.stdout:0/399: symlink d1/d12/d2f/d48/d55/d60/l87 0 2024-03-16T10:45:16.599 INFO:tasks.workunit.client.1.smithi026.stdout:0/400: write d1/d18/d1a/f7a [493274,100570] 0 2024-03-16T10:45:16.600 INFO:tasks.workunit.client.1.smithi026.stdout:0/401: fdatasync d1/d12/f4b 0 2024-03-16T10:45:16.603 INFO:tasks.workunit.client.1.smithi026.stdout:3/401: creat d4/dc/d31/d48/d69/d78/f87 x:0 0 0 2024-03-16T10:45:16.604 INFO:tasks.workunit.client.1.smithi026.stdout:3/402: write d4/f41 [4393514,19141] 0 2024-03-16T10:45:16.607 INFO:tasks.workunit.client.1.smithi026.stdout:8/352: mknod d5/dc/d3f/c5f 0 2024-03-16T10:45:16.611 INFO:tasks.workunit.client.1.smithi026.stdout:1/374: creat d0/d33/d6b/f7a x:0 0 0 2024-03-16T10:45:16.614 INFO:tasks.workunit.client.1.smithi026.stdout:7/365: mkdir d6/d35/d3a/d3c/d73 0 2024-03-16T10:45:16.614 INFO:tasks.workunit.client.1.smithi026.stdout:3/403: dwrite d4/dc/dd/d3d/f6a [0,4194304] 0 2024-03-16T10:45:16.617 INFO:tasks.workunit.client.1.smithi026.stdout:6/338: dwrite d0/f1d [0,4194304] 0 2024-03-16T10:45:16.622 INFO:tasks.workunit.client.1.smithi026.stdout:6/339: write d0/f3 [1896870,104075] 0 2024-03-16T10:45:16.627 INFO:tasks.workunit.client.1.smithi026.stdout:5/327: truncate d1/db/d27/f60 1273517 0 2024-03-16T10:45:16.633 INFO:tasks.workunit.client.1.smithi026.stdout:1/375: write d0/d17/d1f/f35 [1865921,56449] 0 2024-03-16T10:45:16.638 INFO:tasks.workunit.client.1.smithi026.stdout:3/404: rename d4/d29/f30 to d4/d29/d7d/f88 0 2024-03-16T10:45:16.638 INFO:tasks.workunit.client.1.smithi026.stdout:5/328: dwrite d1/db/f39 [0,4194304] 0 2024-03-16T10:45:16.644 INFO:tasks.workunit.client.1.smithi026.stdout:6/340: mknod d0/d41/d4f/c7c 0 2024-03-16T10:45:16.645 INFO:tasks.workunit.client.1.smithi026.stdout:6/341: read d0/f2c [2819918,33661] 0 2024-03-16T10:45:16.648 INFO:tasks.workunit.client.1.smithi026.stdout:8/353: getdents d5/d7/d2c/d36/d59 0 2024-03-16T10:45:16.663 INFO:tasks.workunit.client.1.smithi026.stdout:1/376: symlink d0/d17/d4d/d70/l7b 0 2024-03-16T10:45:16.664 INFO:tasks.workunit.client.1.smithi026.stdout:1/377: dread - d0/d33/d3b/d64/f6f zero size 2024-03-16T10:45:16.666 INFO:tasks.workunit.client.1.smithi026.stdout:7/366: creat d6/de/f74 x:0 0 0 2024-03-16T10:45:16.668 INFO:tasks.workunit.client.1.smithi026.stdout:3/405: mkdir d4/d89 0 2024-03-16T10:45:16.671 INFO:tasks.workunit.client.1.smithi026.stdout:0/402: getdents d1/d18 0 2024-03-16T10:45:16.675 INFO:tasks.workunit.client.1.smithi026.stdout:3/406: dread d4/fa [0,4194304] 0 2024-03-16T10:45:16.677 INFO:tasks.workunit.client.1.smithi026.stdout:1/378: rename d0/d33 to d0/d17/d1f/d63/d6a/d7c 0 2024-03-16T10:45:16.679 INFO:tasks.workunit.client.1.smithi026.stdout:0/403: dwrite d1/d18/d1a/d24/f67 [0,4194304] 0 2024-03-16T10:45:16.691 INFO:tasks.workunit.client.1.smithi026.stdout:7/367: mknod d6/d35/d3a/d3c/d3f/d65/d68/c75 0 2024-03-16T10:45:16.691 INFO:tasks.workunit.client.1.smithi026.stdout:7/368: dread - d6/d1e/f6a zero size 2024-03-16T10:45:16.699 INFO:tasks.workunit.client.1.smithi026.stdout:8/354: getdents d5/dc/d40 0 2024-03-16T10:45:16.700 INFO:tasks.workunit.client.1.smithi026.stdout:7/369: mknod d6/de/d16/d44/c76 0 2024-03-16T10:45:16.701 INFO:tasks.workunit.client.1.smithi026.stdout:0/404: dread d1/d18/d1a/d24/d57/f3e [0,4194304] 0 2024-03-16T10:45:16.702 INFO:tasks.workunit.client.1.smithi026.stdout:8/355: write d5/dc/d40/d53/f58 [851933,12843] 0 2024-03-16T10:45:16.703 INFO:tasks.workunit.client.1.smithi026.stdout:7/370: truncate d6/f24 8656205 0 2024-03-16T10:45:16.704 INFO:tasks.workunit.client.1.smithi026.stdout:3/407: link d4/dc/d31/d48/c76 d4/d29/d7d/c8a 0 2024-03-16T10:45:16.706 INFO:tasks.workunit.client.1.smithi026.stdout:3/408: write d4/dc/f20 [206331,64200] 0 2024-03-16T10:45:16.707 INFO:tasks.workunit.client.1.smithi026.stdout:3/409: stat d4/d29/l2a 0 2024-03-16T10:45:16.708 INFO:tasks.workunit.client.1.smithi026.stdout:8/356: mkdir d5/d23/d4e/d60 0 2024-03-16T10:45:16.708 INFO:tasks.workunit.client.1.smithi026.stdout:3/410: chown d4/f9 1118667 1 2024-03-16T10:45:16.710 INFO:tasks.workunit.client.1.smithi026.stdout:1/379: getdents d0/d17/d1f/d38 0 2024-03-16T10:45:16.712 INFO:tasks.workunit.client.1.smithi026.stdout:1/380: truncate d0/d17/d1f/d63/d6a/d7c/f59 119940 0 2024-03-16T10:45:16.714 INFO:tasks.workunit.client.1.smithi026.stdout:7/371: write d6/d35/d3a/d3c/f5c [3128167,57880] 0 2024-03-16T10:45:16.718 INFO:tasks.workunit.client.1.smithi026.stdout:8/357: creat d5/d23/f61 x:0 0 0 2024-03-16T10:45:16.719 INFO:tasks.workunit.client.1.smithi026.stdout:3/411: truncate d4/fa 400140 0 2024-03-16T10:45:16.719 INFO:tasks.workunit.client.1.smithi026.stdout:8/358: chown d5/d7/f11 1408022 1 2024-03-16T10:45:16.719 INFO:tasks.workunit.client.1.smithi026.stdout:3/412: chown d4/dc/d31 4 1 2024-03-16T10:45:16.721 INFO:tasks.workunit.client.1.smithi026.stdout:1/381: unlink d0/d17/d1f/d20/l5b 0 2024-03-16T10:45:16.725 INFO:tasks.workunit.client.1.smithi026.stdout:7/372: symlink d6/de/d16/l77 0 2024-03-16T10:45:16.728 INFO:tasks.workunit.client.1.smithi026.stdout:7/373: fsync d6/fc 0 2024-03-16T10:45:16.730 INFO:tasks.workunit.client.1.smithi026.stdout:3/413: fsync d4/dc/d31/f3b 0 2024-03-16T10:45:16.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr fail", "who": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr fail", "who": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: osdmap e50: 6 total, 6 up, 6 in 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd='[{"prefix": "mgr fail", "who": "smithi026.omjkpo"}]': finished 2024-03-16T10:45:16.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: mgrmap e27: smithi002.zevrtm(active, starting, since 0.00653046s) 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:45:16.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:45:16.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:16 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:45:16.736 INFO:tasks.workunit.client.1.smithi026.stdout:1/382: dwrite d0/d17/d1f/d63/d6a/d7c/d3b/d64/f6f [0,4194304] 0 2024-03-16T10:45:16.736 INFO:tasks.workunit.client.1.smithi026.stdout:1/383: write d0/d17/d4d/f78 [120729,36563] 0 2024-03-16T10:45:16.736 INFO:tasks.workunit.client.1.smithi026.stdout:3/414: write d4/dc/dd/f75 [4437588,4743] 0 2024-03-16T10:45:16.743 INFO:tasks.workunit.client.1.smithi026.stdout:0/405: truncate d1/d18/d1a/d2b/d4f/f53 1415095 0 2024-03-16T10:45:16.746 INFO:tasks.workunit.client.1.smithi026.stdout:7/374: dread d6/d35/d3a/d3c/f40 [0,4194304] 0 2024-03-16T10:45:16.754 INFO:tasks.workunit.client.1.smithi026.stdout:8/359: dwrite d5/d7/f1c [0,4194304] 0 2024-03-16T10:45:16.759 INFO:tasks.workunit.client.1.smithi026.stdout:3/415: dread d4/dc/dd/f21 [0,4194304] 0 2024-03-16T10:45:16.762 INFO:tasks.workunit.client.1.smithi026.stdout:0/406: mkdir d1/d18/d1a/d24/d51/d88 0 2024-03-16T10:45:16.766 INFO:tasks.workunit.client.1.smithi026.stdout:0/407: dread d1/d18/d1a/f46 [0,4194304] 0 2024-03-16T10:45:16.767 INFO:tasks.workunit.client.1.smithi026.stdout:8/360: rmdir d5/d7/d2c 39 2024-03-16T10:45:16.767 INFO:tasks.workunit.client.1.smithi026.stdout:8/361: chown d5/dc/d3f/f20 115229 1 2024-03-16T10:45:16.767 INFO:tasks.workunit.client.1.smithi026.stdout:8/362: readlink d5/l41 0 2024-03-16T10:45:16.771 INFO:tasks.workunit.client.1.smithi026.stdout:7/375: link d6/de/f64 d6/d35/d3a/d5d/f78 0 2024-03-16T10:45:16.775 INFO:tasks.workunit.client.1.smithi026.stdout:3/416: mknod d4/dc/dd/d80/c8b 0 2024-03-16T10:45:16.778 INFO:tasks.workunit.client.1.smithi026.stdout:1/384: getdents d0/d17/d1f/d63/d6a/d7c/d6b 0 2024-03-16T10:45:16.778 INFO:tasks.workunit.client.1.smithi026.stdout:8/363: stat d5/d7/d2c/f2d 0 2024-03-16T10:45:16.779 INFO:tasks.workunit.client.1.smithi026.stdout:0/408: mkdir d1/d18/d1a/d24/d51/d88/d89 0 2024-03-16T10:45:16.779 INFO:tasks.workunit.client.1.smithi026.stdout:3/417: dread d4/dc/d31/f3b [0,4194304] 0 2024-03-16T10:45:16.786 INFO:tasks.workunit.client.1.smithi026.stdout:8/364: rmdir d5/dc/d40/d53 39 2024-03-16T10:45:16.788 INFO:tasks.workunit.client.1.smithi026.stdout:1/385: dwrite d0/d17/d1f/d20/f43 [0,4194304] 0 2024-03-16T10:45:16.789 INFO:tasks.workunit.client.1.smithi026.stdout:7/376: dread d6/de/d16/f1f [0,4194304] 0 2024-03-16T10:45:16.790 INFO:tasks.workunit.client.1.smithi026.stdout:3/418: write d4/fe [4980978,69536] 0 2024-03-16T10:45:16.790 INFO:tasks.workunit.client.1.smithi026.stdout:8/365: write f4 [3794305,30550] 0 2024-03-16T10:45:16.791 INFO:tasks.workunit.client.1.smithi026.stdout:3/419: readlink d4/d29/l2c 0 2024-03-16T10:45:16.792 INFO:tasks.workunit.client.1.smithi026.stdout:8/366: truncate d5/d23/d4e/f5b 433835 0 2024-03-16T10:45:16.798 INFO:tasks.workunit.client.1.smithi026.stdout:7/377: mkdir d6/d35/d3a/d5d/d79 0 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 172.21.15.26:0/4064026789' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr fail", "who": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "mgr fail", "who": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: osdmap e50: 6 total, 6 up, 6 in 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.24375 ' entity='mgr.smithi026.omjkpo' cmd='[{"prefix": "mgr fail", "who": "smithi026.omjkpo"}]': finished 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: mgrmap e27: smithi002.zevrtm(active, starting, since 0.00653046s) 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:45:16.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:16.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:16.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:16.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:45:16.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:16 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:45:16.801 INFO:tasks.workunit.client.1.smithi026.stdout:8/367: mkdir d5/dc/d3f/d62 0 2024-03-16T10:45:16.803 INFO:tasks.workunit.client.1.smithi026.stdout:1/386: rename d0/d17/c28 to d0/d17/d1f/d63/d6a/d7c/d3b/c7d 0 2024-03-16T10:45:16.803 INFO:tasks.workunit.client.1.smithi026.stdout:7/378: mknod d6/d35/d3a/d59/c7a 0 2024-03-16T10:45:16.804 INFO:tasks.workunit.client.1.smithi026.stdout:0/409: dwrite d1/f40 [0,4194304] 0 2024-03-16T10:45:16.814 INFO:tasks.workunit.client.1.smithi026.stdout:7/379: mkdir d6/d35/d3a/d5d/d7b 0 2024-03-16T10:45:16.815 INFO:tasks.workunit.client.1.smithi026.stdout:1/387: dwrite d0/f74 [0,4194304] 0 2024-03-16T10:45:16.816 INFO:tasks.workunit.client.1.smithi026.stdout:0/410: chown d1/d18/d1a/d24/l41 4386 1 2024-03-16T10:45:16.817 INFO:tasks.workunit.client.1.smithi026.stdout:8/368: creat d5/dc/d40/d53/f63 x:0 0 0 2024-03-16T10:45:16.819 INFO:tasks.workunit.client.1.smithi026.stdout:0/411: write d1/d18/d1a/f7a [761,85430] 0 2024-03-16T10:45:16.820 INFO:tasks.workunit.client.1.smithi026.stdout:3/420: dread d4/dc/d31/f34 [0,4194304] 0 2024-03-16T10:45:16.821 INFO:tasks.workunit.client.1.smithi026.stdout:7/380: creat d6/d35/d3a/d5d/f7c x:0 0 0 2024-03-16T10:45:16.825 INFO:tasks.workunit.client.1.smithi026.stdout:8/369: rename d5/dc/l3a to d5/d7/d2c/d36/d59/l64 0 2024-03-16T10:45:16.828 INFO:tasks.workunit.client.1.smithi026.stdout:3/421: rmdir d4/dc/d31 39 2024-03-16T10:45:16.829 INFO:tasks.workunit.client.1.smithi026.stdout:3/422: readlink d4/dc/l40 0 2024-03-16T10:45:16.833 INFO:tasks.workunit.client.1.smithi026.stdout:7/381: unlink d6/d35/d3a/d4e/l56 0 2024-03-16T10:45:16.834 INFO:tasks.workunit.client.1.smithi026.stdout:7/382: write d6/f24 [9356540,103220] 0 2024-03-16T10:45:16.840 INFO:tasks.workunit.client.1.smithi026.stdout:8/370: mknod d5/d28/c65 0 2024-03-16T10:45:16.841 INFO:tasks.workunit.client.1.smithi026.stdout:0/412: dread d1/d18/d1a/d24/f3 [0,4194304] 0 2024-03-16T10:45:16.842 INFO:tasks.workunit.client.1.smithi026.stdout:7/383: dwrite d6/f39 [0,4194304] 0 2024-03-16T10:45:16.845 INFO:tasks.workunit.client.1.smithi026.stdout:0/413: fsync d1/d12/f4b 0 2024-03-16T10:45:16.845 INFO:tasks.workunit.client.1.smithi026.stdout:7/384: write d6/fc [2198622,54091] 0 2024-03-16T10:45:16.849 INFO:tasks.workunit.client.1.smithi026.stdout:3/423: mknod d4/dc/d31/d48/d69/d78/c8c 0 2024-03-16T10:45:16.849 INFO:tasks.workunit.client.1.smithi026.stdout:3/424: fdatasync d4/f41 0 2024-03-16T10:45:16.851 INFO:tasks.workunit.client.1.smithi026.stdout:8/371: dwrite d5/d23/d4e/f5b [0,4194304] 0 2024-03-16T10:45:16.852 INFO:tasks.workunit.client.1.smithi026.stdout:7/385: mkdir d6/de/d7d 0 2024-03-16T10:45:16.854 INFO:tasks.workunit.client.1.smithi026.stdout:7/386: chown d6/de/d7d 3 1 2024-03-16T10:45:16.854 INFO:tasks.workunit.client.1.smithi026.stdout:8/372: fsync d5/d23/f61 0 2024-03-16T10:45:16.854 INFO:tasks.workunit.client.1.smithi026.stdout:8/373: chown d5/dc/f18 14974 1 2024-03-16T10:45:16.855 INFO:tasks.workunit.client.1.smithi026.stdout:7/387: read d6/d35/d3a/d5d/f78 [203675,72021] 0 2024-03-16T10:45:16.855 INFO:tasks.workunit.client.1.smithi026.stdout:7/388: readlink d6/de/d16/l6b 0 2024-03-16T10:45:16.856 INFO:tasks.workunit.client.1.smithi026.stdout:8/374: read d5/d7/f11 [2262046,52308] 0 2024-03-16T10:45:16.864 INFO:tasks.workunit.client.1.smithi026.stdout:7/389: dwrite d6/d1e/f6a [0,4194304] 0 2024-03-16T10:45:16.873 INFO:tasks.workunit.client.1.smithi026.stdout:7/390: rmdir d6/d1e 39 2024-03-16T10:45:16.887 INFO:tasks.workunit.client.1.smithi026.stdout:8/375: truncate d5/dc/d40/d53/f58 684804 0 2024-03-16T10:45:16.888 INFO:tasks.workunit.client.1.smithi026.stdout:7/391: dread d6/f22 [0,4194304] 0 2024-03-16T10:45:16.889 INFO:tasks.workunit.client.1.smithi026.stdout:7/392: dread - d6/f53 zero size 2024-03-16T10:45:16.896 INFO:tasks.workunit.client.1.smithi026.stdout:7/393: dread d6/d35/f66 [0,4194304] 0 2024-03-16T10:45:16.897 INFO:tasks.workunit.client.1.smithi026.stdout:7/394: write d6/d2f/f5f [761767,121651] 0 2024-03-16T10:45:16.898 INFO:tasks.workunit.client.1.smithi026.stdout:7/395: fsync d6/d35/d3a/d3c/f51 0 2024-03-16T10:45:16.918 INFO:tasks.workunit.client.1.smithi026.stdout:7/396: rename d6/de/f36 to d6/d35/f7e 0 2024-03-16T10:45:16.921 INFO:tasks.workunit.client.1.smithi026.stdout:8/376: getdents d5/d7/d2c/d36/d59 0 2024-03-16T10:45:16.927 INFO:tasks.workunit.client.1.smithi026.stdout:8/377: unlink d5/d7/f10 0 2024-03-16T10:45:16.931 INFO:tasks.workunit.client.1.smithi026.stdout:8/378: dread d5/dc/f18 [0,4194304] 0 2024-03-16T10:45:16.934 INFO:tasks.workunit.client.1.smithi026.stdout:8/379: creat d5/d23/d4e/f66 x:0 0 0 2024-03-16T10:45:16.938 INFO:tasks.workunit.client.1.smithi026.stdout:8/380: dread d5/dc/d3f/f46 [0,4194304] 0 2024-03-16T10:45:16.942 INFO:tasks.workunit.client.1.smithi026.stdout:7/397: dwrite d6/d35/d3a/d5d/f78 [0,4194304] 0 2024-03-16T10:45:16.944 INFO:tasks.workunit.client.1.smithi026.stdout:7/398: chown d6/d35/d3a/d5d/d7b 1820793 1 2024-03-16T10:45:16.944 INFO:tasks.workunit.client.1.smithi026.stdout:7/399: fsync d6/de/d16/f19 0 2024-03-16T10:45:16.950 INFO:tasks.workunit.client.1.smithi026.stdout:8/381: dread d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:16.952 INFO:tasks.workunit.client.1.smithi026.stdout:7/400: link d6/de/d16/c33 d6/d35/d3a/d4e/c7f 0 2024-03-16T10:45:16.953 INFO:tasks.workunit.client.1.smithi026.stdout:7/401: fsync d6/d35/d3a/d4e/f6d 0 2024-03-16T10:45:16.953 INFO:tasks.workunit.client.1.smithi026.stdout:8/382: creat d5/d23/f67 x:0 0 0 2024-03-16T10:45:16.954 INFO:tasks.workunit.client.1.smithi026.stdout:8/383: chown d5/dc/d3f/c5f 2997 1 2024-03-16T10:45:16.956 INFO:tasks.workunit.client.1.smithi026.stdout:7/402: symlink d6/d35/d3a/d3c/d3f/d65/l80 0 2024-03-16T10:45:16.958 INFO:tasks.workunit.client.1.smithi026.stdout:8/384: creat d5/d23/f68 x:0 0 0 2024-03-16T10:45:16.966 INFO:tasks.workunit.client.1.smithi026.stdout:8/385: getdents d5/d7/d2c 0 2024-03-16T10:45:16.967 INFO:tasks.workunit.client.1.smithi026.stdout:8/386: chown d5/dc/d3f/f55 44326 1 2024-03-16T10:45:16.969 INFO:tasks.workunit.client.1.smithi026.stdout:7/403: dread d6/de/d16/f1a [0,4194304] 0 2024-03-16T10:45:16.974 INFO:tasks.workunit.client.1.smithi026.stdout:7/404: mknod d6/de/d7d/c81 0 2024-03-16T10:45:16.979 INFO:tasks.workunit.client.1.smithi026.stdout:7/405: fsync d6/d1e/f20 0 2024-03-16T10:45:16.984 INFO:tasks.workunit.client.1.smithi026.stdout:8/387: getdents d5/d23 0 2024-03-16T10:45:16.987 INFO:tasks.workunit.client.1.smithi026.stdout:8/388: mkdir d5/d23/d4e/d69 0 2024-03-16T10:45:16.993 INFO:tasks.workunit.client.1.smithi026.stdout:8/389: dwrite d5/d7/d2c/f50 [0,4194304] 0 2024-03-16T10:45:16.995 INFO:tasks.workunit.client.1.smithi026.stdout:8/390: mknod d5/dc/d3f/c6a 0 2024-03-16T10:45:17.002 INFO:tasks.workunit.client.1.smithi026.stdout:8/391: write d5/d28/f39 [381873,79337] 0 2024-03-16T10:45:17.003 INFO:tasks.workunit.client.1.smithi026.stdout:8/392: readlink d5/d7/d2c/d36/l47 0 2024-03-16T10:45:17.011 INFO:tasks.workunit.client.1.smithi026.stdout:8/393: rename d5/d7/d2c/f2d to d5/d7/f6b 0 2024-03-16T10:45:17.012 INFO:tasks.workunit.client.1.smithi026.stdout:8/394: truncate d5/d7/d2c/d36/f4c 839774 0 2024-03-16T10:45:17.130 INFO:tasks.workunit.client.1.smithi026.stdout:2/367: sync 2024-03-16T10:45:17.149 INFO:tasks.workunit.client.1.smithi026.stdout:8/395: symlink d5/d23/l6c 0 2024-03-16T10:45:17.151 INFO:tasks.workunit.client.1.smithi026.stdout:2/368: symlink d2/d29/l7c 0 2024-03-16T10:45:17.159 INFO:tasks.workunit.client.1.smithi026.stdout:2/369: creat d2/d29/d17/d38/f7d x:0 0 0 2024-03-16T10:45:17.163 INFO:tasks.workunit.client.1.smithi026.stdout:2/370: rename d2/d29/d17/c3d to d2/d29/d17/d1a/d2b/d4e/d67/c7e 0 2024-03-16T10:45:17.168 INFO:tasks.workunit.client.1.smithi026.stdout:2/371: chown d2/d29/l62 13411437 1 2024-03-16T10:45:17.172 INFO:tasks.workunit.client.1.smithi026.stdout:2/372: dread d2/d29/d36/f58 [0,4194304] 0 2024-03-16T10:45:17.175 INFO:tasks.workunit.client.1.smithi026.stdout:2/373: mkdir d2/d29/d17/d1a/d2b/d2e/d60/d7f 0 2024-03-16T10:45:17.180 INFO:tasks.workunit.client.1.smithi026.stdout:8/396: dwrite d5/dc/d40/d53/f58 [0,4194304] 0 2024-03-16T10:45:17.185 INFO:tasks.workunit.client.1.smithi026.stdout:8/397: unlink d5/d7/f1d 0 2024-03-16T10:45:17.186 INFO:tasks.workunit.client.1.smithi026.stdout:8/398: write d5/d23/f68 [137034,25974] 0 2024-03-16T10:45:17.191 INFO:tasks.workunit.client.1.smithi026.stdout:2/374: creat d2/d29/d17/d1a/d2b/f80 x:0 0 0 2024-03-16T10:45:17.194 INFO:tasks.workunit.client.1.smithi026.stdout:2/375: creat d2/d29/d17/d38/f81 x:0 0 0 2024-03-16T10:45:17.201 INFO:tasks.workunit.client.1.smithi026.stdout:8/399: getdents d5/d7 0 2024-03-16T10:45:17.203 INFO:tasks.workunit.client.1.smithi026.stdout:2/376: creat d2/d29/d51/d55/d6d/f82 x:0 0 0 2024-03-16T10:45:17.204 INFO:tasks.workunit.client.1.smithi026.stdout:8/400: dread d5/dc/f4d [0,4194304] 0 2024-03-16T10:45:17.208 INFO:tasks.workunit.client.1.smithi026.stdout:8/401: dread d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:17.209 INFO:tasks.workunit.client.1.smithi026.stdout:8/402: fdatasync d5/dc/f38 0 2024-03-16T10:45:17.218 INFO:tasks.workunit.client.1.smithi026.stdout:8/403: truncate d5/d7/f1c 228285 0 2024-03-16T10:45:17.221 INFO:tasks.workunit.client.1.smithi026.stdout:8/404: write d5/d28/f39 [1965207,57742] 0 2024-03-16T10:45:17.221 INFO:tasks.workunit.client.1.smithi026.stdout:8/405: chown d5/dc/c5a 179 1 2024-03-16T10:45:17.222 INFO:tasks.workunit.client.1.smithi026.stdout:8/406: write d5/d23/d4e/f66 [408594,7992] 0 2024-03-16T10:45:17.222 INFO:tasks.workunit.client.1.smithi026.stdout:8/407: stat d5/d7/f49 0 2024-03-16T10:45:17.228 INFO:tasks.workunit.client.1.smithi026.stdout:8/408: symlink d5/d7/d2c/d36/d59/l6d 0 2024-03-16T10:45:17.231 INFO:tasks.workunit.client.1.smithi026.stdout:8/409: rename d5/d23/f2b to d5/dc/d3f/d62/f6e 0 2024-03-16T10:45:17.234 INFO:tasks.workunit.client.1.smithi026.stdout:9/386: sync 2024-03-16T10:45:17.235 INFO:tasks.workunit.client.1.smithi026.stdout:9/387: chown d0/da/d13/d6e 355 1 2024-03-16T10:45:17.241 INFO:tasks.workunit.client.1.smithi026.stdout:9/388: dwrite d0/f2f [0,4194304] 0 2024-03-16T10:45:17.247 INFO:tasks.workunit.client.1.smithi026.stdout:8/410: dread d5/dc/f38 [0,4194304] 0 2024-03-16T10:45:17.247 INFO:tasks.workunit.client.1.smithi026.stdout:9/389: unlink d0/f2f 0 2024-03-16T10:45:17.249 INFO:tasks.workunit.client.1.smithi026.stdout:8/411: write d5/dc/d40/d53/f58 [1550053,17058] 0 2024-03-16T10:45:17.250 INFO:tasks.workunit.client.1.smithi026.stdout:9/390: symlink d0/d63/l84 0 2024-03-16T10:45:17.251 INFO:tasks.workunit.client.1.smithi026.stdout:9/391: chown d0/da/d16/l18 3068343 1 2024-03-16T10:45:17.252 INFO:tasks.workunit.client.1.smithi026.stdout:9/392: fsync d0/da/d4d/f5e 0 2024-03-16T10:45:17.253 INFO:tasks.workunit.client.1.smithi026.stdout:9/393: dread - d0/da/d13/f78 zero size 2024-03-16T10:45:17.254 INFO:tasks.workunit.client.1.smithi026.stdout:9/394: chown d0/da/d13/f31 2688366 1 2024-03-16T10:45:17.256 INFO:tasks.workunit.client.1.smithi026.stdout:9/395: read d0/da/d13/f38 [4068157,41148] 0 2024-03-16T10:45:17.257 INFO:tasks.workunit.client.1.smithi026.stdout:8/412: link d5/d7/c13 d5/dc/d40/c6f 0 2024-03-16T10:45:17.261 INFO:tasks.workunit.client.1.smithi026.stdout:9/396: mknod d0/da/d16/d17/d44/c85 0 2024-03-16T10:45:17.265 INFO:tasks.workunit.client.1.smithi026.stdout:9/397: read d0/da/d13/f38 [3848146,98425] 0 2024-03-16T10:45:17.268 INFO:tasks.workunit.client.1.smithi026.stdout:9/398: write d0/d63/d6b/f81 [538764,105678] 0 2024-03-16T10:45:17.269 INFO:tasks.workunit.client.1.smithi026.stdout:9/399: write d0/da/d13/f26 [301642,69796] 0 2024-03-16T10:45:17.274 INFO:tasks.workunit.client.1.smithi026.stdout:9/400: unlink d0/da/d13/f78 0 2024-03-16T10:45:17.278 INFO:tasks.workunit.client.1.smithi026.stdout:8/413: dwrite d5/d28/f3c [0,4194304] 0 2024-03-16T10:45:17.280 INFO:tasks.workunit.client.1.smithi026.stdout:9/401: rename d0/da/d14/f3d to d0/d63/d6b/f86 0 2024-03-16T10:45:17.296 INFO:tasks.workunit.client.1.smithi026.stdout:8/414: truncate d5/d28/f31 4089768 0 2024-03-16T10:45:17.304 INFO:tasks.workunit.client.1.smithi026.stdout:9/402: dwrite d0/f9 [0,4194304] 0 2024-03-16T10:45:17.305 INFO:tasks.workunit.client.1.smithi026.stdout:8/415: write d5/dc/f4d [2908298,49795] 0 2024-03-16T10:45:17.314 INFO:tasks.workunit.client.1.smithi026.stdout:8/416: dwrite d5/d7/f57 [0,4194304] 0 2024-03-16T10:45:17.316 INFO:tasks.workunit.client.1.smithi026.stdout:9/403: write d0/da/d13/f69 [948181,62813] 0 2024-03-16T10:45:17.320 INFO:tasks.workunit.client.1.smithi026.stdout:9/404: symlink d0/d63/l87 0 2024-03-16T10:45:17.324 INFO:tasks.workunit.client.1.smithi026.stdout:8/417: link d5/d7/f43 d5/d7/d2c/f70 0 2024-03-16T10:45:17.325 INFO:tasks.workunit.client.1.smithi026.stdout:9/405: mknod d0/da/d4d/c88 0 2024-03-16T10:45:17.330 INFO:tasks.workunit.client.1.smithi026.stdout:9/406: unlink d0/d63/l84 0 2024-03-16T10:45:17.332 INFO:tasks.workunit.client.1.smithi026.stdout:8/418: symlink d5/dc/d40/l71 0 2024-03-16T10:45:17.336 INFO:tasks.workunit.client.1.smithi026.stdout:9/407: dwrite d0/da/f2a [0,4194304] 0 2024-03-16T10:45:17.400 INFO:tasks.workunit.client.1.smithi026.stdout:8/419: mkdir d5/dc/d3f/d62/d72 0 2024-03-16T10:45:17.401 INFO:tasks.workunit.client.1.smithi026.stdout:9/408: creat d0/d63/d6b/f89 x:0 0 0 2024-03-16T10:45:17.407 INFO:tasks.workunit.client.1.smithi026.stdout:9/409: creat d0/da/d16/d5d/d72/d62/f8a x:0 0 0 2024-03-16T10:45:17.409 INFO:tasks.workunit.client.1.smithi026.stdout:4/363: sync 2024-03-16T10:45:17.409 INFO:tasks.workunit.client.1.smithi026.stdout:5/329: sync 2024-03-16T10:45:17.414 INFO:tasks.workunit.client.1.smithi026.stdout:9/410: mknod d0/da/d16/d5d/c8b 0 2024-03-16T10:45:17.419 INFO:tasks.workunit.client.1.smithi026.stdout:5/330: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:17.419 INFO:tasks.workunit.client.1.smithi026.stdout:4/364: rename d1/f27 to d1/db/d29/d61/f75 0 2024-03-16T10:45:17.426 INFO:tasks.workunit.client.1.smithi026.stdout:4/365: dwrite d1/db/d29/d44/f5a [0,4194304] 0 2024-03-16T10:45:17.427 INFO:tasks.workunit.client.1.smithi026.stdout:4/366: dread - d1/db/d29/d44/d48/f73 zero size 2024-03-16T10:45:17.428 INFO:tasks.workunit.client.1.smithi026.stdout:5/331: creat d1/db/d1f/d61/f6d x:0 0 0 2024-03-16T10:45:17.428 INFO:tasks.workunit.client.1.smithi026.stdout:9/411: dread d0/da/d13/f53 [0,4194304] 0 2024-03-16T10:45:17.429 INFO:tasks.workunit.client.1.smithi026.stdout:5/332: stat d1/db/f38 0 2024-03-16T10:45:17.435 INFO:tasks.workunit.client.1.smithi026.stdout:8/420: dwrite d5/d7/f49 [0,4194304] 0 2024-03-16T10:45:17.441 INFO:tasks.workunit.client.1.smithi026.stdout:4/367: dwrite d1/f18 [0,4194304] 0 2024-03-16T10:45:17.444 INFO:tasks.workunit.client.1.smithi026.stdout:8/421: symlink d5/dc/l73 0 2024-03-16T10:45:17.449 INFO:tasks.workunit.client.1.smithi026.stdout:9/412: mkdir d0/d5f/d7f/d8c 0 2024-03-16T10:45:17.450 INFO:tasks.workunit.client.1.smithi026.stdout:9/413: chown d0/d63/d6b/f70 1856 1 2024-03-16T10:45:17.453 INFO:tasks.workunit.client.1.smithi026.stdout:4/368: dread - d1/db/de/f5b zero size 2024-03-16T10:45:17.456 INFO:tasks.workunit.client.1.smithi026.stdout:8/422: creat d5/d23/f74 x:0 0 0 2024-03-16T10:45:17.457 INFO:tasks.workunit.client.1.smithi026.stdout:8/423: stat d5/dc/d3f/f46 0 2024-03-16T10:45:17.460 INFO:tasks.workunit.client.1.smithi026.stdout:4/369: rmdir d1/db/de/d4c/d6f 39 2024-03-16T10:45:17.464 INFO:tasks.workunit.client.1.smithi026.stdout:9/414: link d0/da/d13/f31 d0/da/d13/d6e/f8d 0 2024-03-16T10:45:17.467 INFO:tasks.workunit.client.1.smithi026.stdout:4/370: rename d1/db/c30 to d1/db/de/d6e/c76 0 2024-03-16T10:45:17.469 INFO:tasks.workunit.client.1.smithi026.stdout:8/424: symlink d5/dc/d40/l75 0 2024-03-16T10:45:17.470 INFO:tasks.workunit.client.1.smithi026.stdout:9/415: symlink d0/da/d14/l8e 0 2024-03-16T10:45:17.476 INFO:tasks.workunit.client.1.smithi026.stdout:9/416: creat d0/da/d4d/f8f x:0 0 0 2024-03-16T10:45:17.476 INFO:tasks.workunit.client.1.smithi026.stdout:9/417: readlink d0/da/l29 0 2024-03-16T10:45:17.479 INFO:tasks.workunit.client.1.smithi026.stdout:8/425: mkdir d5/d23/d4e/d69/d76 0 2024-03-16T10:45:17.480 INFO:tasks.workunit.client.1.smithi026.stdout:8/426: readlink d5/d7/d2c/d36/d59/l64 0 2024-03-16T10:45:17.481 INFO:tasks.workunit.client.1.smithi026.stdout:9/418: dwrite d0/da/f75 [0,4194304] 0 2024-03-16T10:45:17.482 INFO:tasks.workunit.client.1.smithi026.stdout:9/419: write d0/da/f45 [640559,60874] 0 2024-03-16T10:45:17.483 INFO:tasks.workunit.client.1.smithi026.stdout:4/371: creat d1/f77 x:0 0 0 2024-03-16T10:45:17.488 INFO:tasks.workunit.client.1.smithi026.stdout:9/420: mknod d0/da/d4d/c90 0 2024-03-16T10:45:17.489 INFO:tasks.workunit.client.1.smithi026.stdout:9/421: readlink d0/da/d16/l36 0 2024-03-16T10:45:17.493 INFO:tasks.workunit.client.1.smithi026.stdout:4/372: mkdir d1/db/de/d4c/d78 0 2024-03-16T10:45:17.499 INFO:tasks.workunit.client.1.smithi026.stdout:4/373: fsync d1/db/de/f55 0 2024-03-16T10:45:17.507 INFO:tasks.workunit.client.1.smithi026.stdout:9/422: rmdir d0/da/d13/d6e 39 2024-03-16T10:45:17.517 INFO:tasks.workunit.client.1.smithi026.stdout:9/423: dread d0/da/d13/f38 [0,4194304] 0 2024-03-16T10:45:17.519 INFO:tasks.workunit.client.1.smithi026.stdout:9/424: truncate d0/da/d16/d5d/f7c 292644 0 2024-03-16T10:45:17.525 INFO:tasks.workunit.client.1.smithi026.stdout:9/425: link d0/da/d16/d17/d44/f2b d0/da/d14/f91 0 2024-03-16T10:45:17.530 INFO:tasks.workunit.client.1.smithi026.stdout:9/426: link d0/da/d13/d6e/f8d d0/da/d16/d5d/d72/d62/f92 0 2024-03-16T10:45:17.531 INFO:tasks.workunit.client.1.smithi026.stdout:9/427: symlink d0/da/d16/l93 0 2024-03-16T10:45:17.532 INFO:tasks.workunit.client.1.smithi026.stdout:8/427: truncate d5/d28/f3c 3994225 0 2024-03-16T10:45:17.533 INFO:tasks.workunit.client.1.smithi026.stdout:8/428: readlink d5/d7/d2c/d36/d59/l6d 0 2024-03-16T10:45:17.533 INFO:tasks.workunit.client.1.smithi026.stdout:9/428: symlink d0/da/d16/l94 0 2024-03-16T10:45:17.534 INFO:tasks.workunit.client.1.smithi026.stdout:9/429: read d0/da/f27 [3465698,36080] 0 2024-03-16T10:45:17.537 INFO:tasks.workunit.client.1.smithi026.stdout:9/430: symlink d0/da/d16/d17/d44/l95 0 2024-03-16T10:45:17.540 INFO:tasks.workunit.client.1.smithi026.stdout:1/388: sync 2024-03-16T10:45:17.540 INFO:tasks.workunit.client.1.smithi026.stdout:6/342: sync 2024-03-16T10:45:17.541 INFO:tasks.workunit.client.1.smithi026.stdout:8/429: creat d5/dc/d40/f77 x:0 0 0 2024-03-16T10:45:17.543 INFO:tasks.workunit.client.1.smithi026.stdout:8/430: fsync d5/d7/d2c/f50 0 2024-03-16T10:45:17.546 INFO:tasks.workunit.client.1.smithi026.stdout:6/343: mkdir d0/d28/d49/d7d 0 2024-03-16T10:45:17.547 INFO:tasks.workunit.client.1.smithi026.stdout:1/389: rmdir d0/d17/d1f/d63/d6a 39 2024-03-16T10:45:17.549 INFO:tasks.workunit.client.1.smithi026.stdout:4/374: dwrite d1/db/d29/f3e [0,4194304] 0 2024-03-16T10:45:17.550 INFO:tasks.workunit.client.1.smithi026.stdout:6/344: chown d0/d28/d49 7 1 2024-03-16T10:45:17.556 INFO:tasks.workunit.client.1.smithi026.stdout:8/431: dwrite d5/d23/d4e/f5b [4194304,4194304] 0 2024-03-16T10:45:17.556 INFO:tasks.workunit.client.1.smithi026.stdout:9/431: dread d0/da/f11 [0,4194304] 0 2024-03-16T10:45:17.566 INFO:tasks.workunit.client.1.smithi026.stdout:1/390: creat d0/d17/d1f/d63/d6a/d7c/d2e/f7e x:0 0 0 2024-03-16T10:45:17.568 INFO:tasks.workunit.client.1.smithi026.stdout:1/391: chown d0/d17/d1f/d63/d6a/d7c/d3b/f27 1 1 2024-03-16T10:45:17.568 INFO:tasks.workunit.client.1.smithi026.stdout:1/392: readlink d0/d17/d1f/l9 0 2024-03-16T10:45:17.570 INFO:tasks.workunit.client.1.smithi026.stdout:1/393: truncate d0/d17/d4d/f6d 914511 0 2024-03-16T10:45:17.571 INFO:tasks.workunit.client.1.smithi026.stdout:8/432: creat d5/d7/d2c/d36/d59/f78 x:0 0 0 2024-03-16T10:45:17.572 INFO:tasks.workunit.client.1.smithi026.stdout:1/394: dread d0/d17/d1f/d63/d6a/d7c/f59 [0,4194304] 0 2024-03-16T10:45:17.572 INFO:tasks.workunit.client.1.smithi026.stdout:9/432: mknod d0/d63/c96 0 2024-03-16T10:45:17.573 INFO:tasks.workunit.client.1.smithi026.stdout:8/433: truncate d5/d23/f61 438202 0 2024-03-16T10:45:17.574 INFO:tasks.workunit.client.1.smithi026.stdout:1/395: fdatasync d0/d17/d1f/d63/d6a/d7c/f59 0 2024-03-16T10:45:17.574 INFO:tasks.workunit.client.1.smithi026.stdout:8/434: write d5/f3e [264491,98513] 0 2024-03-16T10:45:17.582 INFO:tasks.workunit.client.1.smithi026.stdout:4/375: dread d1/db/de/f58 [0,4194304] 0 2024-03-16T10:45:17.583 INFO:tasks.workunit.client.1.smithi026.stdout:4/376: fdatasync d1/db/d29/d61/f6c 0 2024-03-16T10:45:17.586 INFO:tasks.workunit.client.1.smithi026.stdout:3/425: sync 2024-03-16T10:45:17.586 INFO:tasks.workunit.client.1.smithi026.stdout:7/406: sync 2024-03-16T10:45:17.586 INFO:tasks.workunit.client.1.smithi026.stdout:2/377: sync 2024-03-16T10:45:17.587 INFO:tasks.workunit.client.1.smithi026.stdout:5/333: sync 2024-03-16T10:45:17.587 INFO:tasks.workunit.client.1.smithi026.stdout:0/414: sync 2024-03-16T10:45:17.587 INFO:tasks.workunit.client.1.smithi026.stdout:1/396: unlink d0/d17/d1f/d63/d6a/d7c/d3b/f27 0 2024-03-16T10:45:17.588 INFO:tasks.workunit.client.1.smithi026.stdout:7/407: write d6/f31 [4575035,12274] 0 2024-03-16T10:45:17.588 INFO:tasks.workunit.client.1.smithi026.stdout:3/426: read - d4/d29/f6e zero size 2024-03-16T10:45:17.591 INFO:tasks.workunit.client.1.smithi026.stdout:5/334: dread - d1/d1c/d2a/d31/d69/d51/f5d zero size 2024-03-16T10:45:17.592 INFO:tasks.workunit.client.1.smithi026.stdout:2/378: write d2/d29/d17/d1a/d2b/d2e/f3c [2696555,26503] 0 2024-03-16T10:45:17.597 INFO:tasks.workunit.client.1.smithi026.stdout:1/397: dread d0/d17/d4d/f6d [0,4194304] 0 2024-03-16T10:45:17.597 INFO:tasks.workunit.client.1.smithi026.stdout:6/345: dwrite d0/d28/f37 [4194304,4194304] 0 2024-03-16T10:45:17.597 INFO:tasks.workunit.client.1.smithi026.stdout:2/379: chown d2/d29/d17/d1a/d2b/d4e/l79 51902 1 2024-03-16T10:45:17.605 INFO:tasks.workunit.client.1.smithi026.stdout:3/427: mknod d4/dc/d31/d48/c8d 0 2024-03-16T10:45:17.614 INFO:tasks.workunit.client.1.smithi026.stdout:8/435: dwrite d5/d7/f43 [4194304,4194304] 0 2024-03-16T10:45:17.616 INFO:tasks.workunit.client.1.smithi026.stdout:8/436: write d5/d23/f68 [1162349,98541] 0 2024-03-16T10:45:17.616 INFO:tasks.workunit.client.1.smithi026.stdout:1/398: dread d0/f1e [0,4194304] 0 2024-03-16T10:45:17.617 INFO:tasks.workunit.client.1.smithi026.stdout:8/437: chown d5/dc/f38 6892 1 2024-03-16T10:45:17.617 INFO:tasks.workunit.client.1.smithi026.stdout:2/380: creat d2/d29/d51/d55/f83 x:0 0 0 2024-03-16T10:45:17.625 INFO:tasks.workunit.client.1.smithi026.stdout:0/415: dwrite d1/d18/d1a/f46 [0,4194304] 0 2024-03-16T10:45:17.634 INFO:tasks.workunit.client.1.smithi026.stdout:7/408: truncate d6/fc 1381021 0 2024-03-16T10:45:17.638 INFO:tasks.workunit.client.1.smithi026.stdout:5/335: write d1/db/d27/f60 [1440754,44297] 0 2024-03-16T10:45:17.642 INFO:tasks.workunit.client.1.smithi026.stdout:8/438: rename d5/dc/d40/l75 to d5/d7/l79 0 2024-03-16T10:45:17.643 INFO:tasks.workunit.client.1.smithi026.stdout:2/381: rmdir d2/d29/d36 39 2024-03-16T10:45:17.653 INFO:tasks.workunit.client.1.smithi026.stdout:0/416: unlink d1/d12/f4a 0 2024-03-16T10:45:17.655 INFO:tasks.workunit.client.1.smithi026.stdout:4/377: truncate d1/db/de/f3c 3992460 0 2024-03-16T10:45:17.658 INFO:tasks.workunit.client.1.smithi026.stdout:5/336: mkdir d1/d6e 0 2024-03-16T10:45:17.659 INFO:tasks.workunit.client.1.smithi026.stdout:6/346: link d0/d10/f1a d0/d10/d79/d7b/f7e 0 2024-03-16T10:45:17.663 INFO:tasks.workunit.client.1.smithi026.stdout:3/428: creat d4/dc/dd/d60/d62/f8e x:0 0 0 2024-03-16T10:45:17.663 INFO:tasks.workunit.client.1.smithi026.stdout:8/439: dread d5/d28/f31 [0,4194304] 0 2024-03-16T10:45:17.666 INFO:tasks.workunit.client.1.smithi026.stdout:0/417: creat d1/d18/d1a/d24/d51/d5f/f8a x:0 0 0 2024-03-16T10:45:17.667 INFO:tasks.workunit.client.1.smithi026.stdout:7/409: getdents d6/d35/d3a/d4e/d6c 0 2024-03-16T10:45:17.670 INFO:tasks.workunit.client.1.smithi026.stdout:1/399: creat d0/f7f x:0 0 0 2024-03-16T10:45:17.673 INFO:tasks.workunit.client.1.smithi026.stdout:0/418: dwrite d1/d12/f45 [0,4194304] 0 2024-03-16T10:45:17.673 INFO:tasks.workunit.client.1.smithi026.stdout:5/337: unlink d1/fd 0 2024-03-16T10:45:17.675 INFO:tasks.workunit.client.1.smithi026.stdout:6/347: creat d0/d41/f7f x:0 0 0 2024-03-16T10:45:17.676 INFO:tasks.workunit.client.1.smithi026.stdout:6/348: readlink d0/d28/d49/l53 0 2024-03-16T10:45:17.680 INFO:tasks.workunit.client.1.smithi026.stdout:1/400: dread d0/d17/d1f/d38/f4e [0,4194304] 0 2024-03-16T10:45:17.687 INFO:tasks.workunit.client.1.smithi026.stdout:7/410: rename d6/d2f/c62 to d6/d1e/c82 0 2024-03-16T10:45:17.690 INFO:tasks.workunit.client.1.smithi026.stdout:0/419: fdatasync d1/d18/f36 0 2024-03-16T10:45:17.691 INFO:tasks.workunit.client.1.smithi026.stdout:5/338: truncate d1/db/d27/f23 2281377 0 2024-03-16T10:45:17.697 INFO:tasks.workunit.client.1.smithi026.stdout:1/401: dread d0/d17/d1f/f3f [0,4194304] 0 2024-03-16T10:45:17.697 INFO:tasks.workunit.client.1.smithi026.stdout:2/382: link d2/d29/d17/f2d d2/d29/d17/d1a/d2b/f84 0 2024-03-16T10:45:17.698 INFO:tasks.workunit.client.1.smithi026.stdout:4/378: dwrite d1/db/d29/d44/d48/f73 [0,4194304] 0 2024-03-16T10:45:17.699 INFO:tasks.workunit.client.1.smithi026.stdout:1/402: chown d0/f74 34200 1 2024-03-16T10:45:17.706 INFO:tasks.workunit.client.1.smithi026.stdout:8/440: fdatasync d5/d7/d2c/f56 0 2024-03-16T10:45:17.706 INFO:tasks.workunit.client.1.smithi026.stdout:7/411: dread d6/f31 [0,4194304] 0 2024-03-16T10:45:17.707 INFO:tasks.workunit.client.1.smithi026.stdout:4/379: dwrite d1/f3 [4194304,4194304] 0 2024-03-16T10:45:17.708 INFO:tasks.workunit.client.1.smithi026.stdout:7/412: readlink d6/de/l2c 0 2024-03-16T10:45:17.710 INFO:tasks.workunit.client.1.smithi026.stdout:4/380: readlink d1/db/d29/d44/d48/l71 0 2024-03-16T10:45:17.718 INFO:tasks.workunit.client.1.smithi026.stdout:8/441: dwrite d5/dc/d40/f77 [0,4194304] 0 2024-03-16T10:45:17.720 INFO:tasks.workunit.client.1.smithi026.stdout:6/349: unlink d0/d41/c6a 0 2024-03-16T10:45:17.722 INFO:tasks.workunit.client.1.smithi026.stdout:4/381: dwrite d1/db/d29/f43 [4194304,4194304] 0 2024-03-16T10:45:17.724 INFO:tasks.workunit.client.1.smithi026.stdout:5/339: symlink d1/d1c/d4c/d4d/l6f 0 2024-03-16T10:45:17.725 INFO:tasks.workunit.client.1.smithi026.stdout:5/340: dread - d1/d1c/d2a/d55/f58 zero size 2024-03-16T10:45:17.730 INFO:tasks.workunit.client.1.smithi026.stdout:0/420: symlink d1/d12/d2f/d48/d55/d60/l8b 0 2024-03-16T10:45:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:17 smithi026 ceph-mon[37041]: mgrmap e28: smithi002.zevrtm(active, since 1.01084s) 2024-03-16T10:45:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:17 smithi026 ceph-mon[37041]: [16/Mar/2024:10:45:16] ENGINE Bus STARTING 2024-03-16T10:45:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:17 smithi026 ceph-mon[37041]: [16/Mar/2024:10:45:16] ENGINE Serving on http://172.21.15.2:8765 2024-03-16T10:45:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:17 smithi026 ceph-mon[37041]: [16/Mar/2024:10:45:16] ENGINE Serving on https://172.21.15.2:7150 2024-03-16T10:45:17.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:17 smithi026 ceph-mon[37041]: [16/Mar/2024:10:45:16] ENGINE Bus STARTED 2024-03-16T10:45:17.734 INFO:tasks.workunit.client.1.smithi026.stdout:1/403: write d0/d17/f37 [216057,92651] 0 2024-03-16T10:45:17.738 INFO:tasks.workunit.client.1.smithi026.stdout:3/429: link d4/dc/d31/f3b d4/d89/f8f 0 2024-03-16T10:45:17.740 INFO:tasks.workunit.client.1.smithi026.stdout:6/350: unlink d0/d10/f7a 0 2024-03-16T10:45:17.741 INFO:tasks.workunit.client.1.smithi026.stdout:8/442: rename d5/dc/d3f/d62/d72 to d5/dc/d3f/d62/d7a 0 2024-03-16T10:45:17.742 INFO:tasks.workunit.client.1.smithi026.stdout:8/443: read d5/dc/d3f/f46 [461277,949] 0 2024-03-16T10:45:17.749 INFO:tasks.workunit.client.1.smithi026.stdout:0/421: symlink d1/d18/d61/l8c 0 2024-03-16T10:45:17.749 INFO:tasks.workunit.client.1.smithi026.stdout:4/382: dwrite d1/db/de/f2b [0,4194304] 0 2024-03-16T10:45:17.756 INFO:tasks.workunit.client.1.smithi026.stdout:4/383: dwrite d1/db/d29/f43 [4194304,4194304] 0 2024-03-16T10:45:17.757 INFO:tasks.workunit.client.1.smithi026.stdout:2/383: rename d2/d29/d17/d1a/d2b/d4e/d65 to d2/d29/d85 0 2024-03-16T10:45:17.763 INFO:tasks.workunit.client.1.smithi026.stdout:0/422: symlink d1/d12/d2f/d48/d84/l8d 0 2024-03-16T10:45:17.765 INFO:tasks.workunit.client.1.smithi026.stdout:1/404: mknod d0/d17/d1f/d63/d6a/d7c/c80 0 2024-03-16T10:45:17.767 INFO:tasks.workunit.client.1.smithi026.stdout:7/413: creat d6/f83 x:0 0 0 2024-03-16T10:45:17.767 INFO:tasks.workunit.client.1.smithi026.stdout:7/414: chown d6/f38 799582524 1 2024-03-16T10:45:17.768 INFO:tasks.workunit.client.1.smithi026.stdout:3/430: creat d4/d89/f90 x:0 0 0 2024-03-16T10:45:17.770 INFO:tasks.workunit.client.1.smithi026.stdout:3/431: write d4/dc/d31/d48/d69/d78/f87 [945915,58223] 0 2024-03-16T10:45:17.771 INFO:tasks.workunit.client.1.smithi026.stdout:8/444: mkdir d5/d7/d7b 0 2024-03-16T10:45:17.774 INFO:tasks.workunit.client.1.smithi026.stdout:2/384: mknod d2/d29/d17/d1a/d2b/d2e/c86 0 2024-03-16T10:45:17.775 INFO:tasks.workunit.client.1.smithi026.stdout:9/433: sync 2024-03-16T10:45:17.778 INFO:tasks.workunit.client.1.smithi026.stdout:8/445: dwrite d5/d23/d4e/f66 [0,4194304] 0 2024-03-16T10:45:17.779 INFO:tasks.workunit.client.1.smithi026.stdout:1/405: read d0/d17/d4d/f6d [660443,27363] 0 2024-03-16T10:45:17.787 INFO:tasks.workunit.client.1.smithi026.stdout:5/341: getdents d1/db 0 2024-03-16T10:45:17.789 INFO:tasks.workunit.client.1.smithi026.stdout:4/384: mkdir d1/db/de/d4c/d79 0 2024-03-16T10:45:17.790 INFO:tasks.workunit.client.1.smithi026.stdout:7/415: dread d6/d1e/f29 [0,4194304] 0 2024-03-16T10:45:17.792 INFO:tasks.workunit.client.1.smithi026.stdout:2/385: unlink d2/d29/d17/d1a/c22 0 2024-03-16T10:45:17.795 INFO:tasks.workunit.client.1.smithi026.stdout:5/342: dwrite d1/d1c/d2a/d31/d69/d51/f54 [0,4194304] 0 2024-03-16T10:45:17.795 INFO:tasks.workunit.client.1.smithi026.stdout:2/386: stat d2/d29/d3b/f66 0 2024-03-16T10:45:17.798 INFO:tasks.workunit.client.1.smithi026.stdout:4/385: dread d1/db/d29/f5f [0,4194304] 0 2024-03-16T10:45:17.798 INFO:tasks.workunit.client.1.smithi026.stdout:5/343: chown d1/d1c/d2a/d31/d69/f67 1 1 2024-03-16T10:45:17.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:17 smithi002 ceph-mon[29248]: mgrmap e28: smithi002.zevrtm(active, since 1.01084s) 2024-03-16T10:45:17.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:17 smithi002 ceph-mon[29248]: [16/Mar/2024:10:45:16] ENGINE Bus STARTING 2024-03-16T10:45:17.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:17 smithi002 ceph-mon[29248]: [16/Mar/2024:10:45:16] ENGINE Serving on http://172.21.15.2:8765 2024-03-16T10:45:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:17 smithi002 ceph-mon[29248]: [16/Mar/2024:10:45:16] ENGINE Serving on https://172.21.15.2:7150 2024-03-16T10:45:17.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:17 smithi002 ceph-mon[29248]: [16/Mar/2024:10:45:16] ENGINE Bus STARTED 2024-03-16T10:45:17.799 INFO:tasks.workunit.client.1.smithi026.stdout:8/446: dread - d5/d7/d2c/f56 zero size 2024-03-16T10:45:17.799 INFO:tasks.workunit.client.1.smithi026.stdout:1/406: creat d0/d17/f81 x:0 0 0 2024-03-16T10:45:17.799 INFO:tasks.workunit.client.1.smithi026.stdout:3/432: symlink d4/dc/l91 0 2024-03-16T10:45:17.805 INFO:tasks.workunit.client.1.smithi026.stdout:6/351: rename d0/f32 to d0/d10/f80 0 2024-03-16T10:45:17.809 INFO:tasks.workunit.client.1.smithi026.stdout:9/434: symlink d0/d5f/d7f/d8c/l97 0 2024-03-16T10:45:17.816 INFO:tasks.workunit.client.1.smithi026.stdout:5/344: write d1/f5 [2217250,58028] 0 2024-03-16T10:45:17.817 INFO:tasks.workunit.client.1.smithi026.stdout:5/345: dread - d1/db/d27/f41 zero size 2024-03-16T10:45:17.818 INFO:tasks.workunit.client.1.smithi026.stdout:4/386: unlink d1/db/c5c 0 2024-03-16T10:45:17.821 INFO:tasks.workunit.client.1.smithi026.stdout:8/447: creat d5/d23/d4e/f7c x:0 0 0 2024-03-16T10:45:17.822 INFO:tasks.workunit.client.1.smithi026.stdout:3/433: mknod d4/dc/dd/d60/d62/d4f/c92 0 2024-03-16T10:45:17.835 INFO:tasks.workunit.client.1.smithi026.stdout:0/423: link d1/d18/d1a/f25 d1/d18/d1a/d24/d11/f8e 0 2024-03-16T10:45:17.842 INFO:tasks.workunit.client.1.smithi026.stdout:3/434: truncate d4/dc/d73/d83/d7c/f5b 593116 0 2024-03-16T10:45:17.846 INFO:tasks.workunit.client.1.smithi026.stdout:0/424: dwrite d1/d18/d61/f62 [0,4194304] 0 2024-03-16T10:45:17.847 INFO:tasks.workunit.client.1.smithi026.stdout:7/416: link d6/de/d16/d44/c4d d6/d35/d3a/d3c/d3f/d65/d6f/c84 0 2024-03-16T10:45:17.850 INFO:tasks.workunit.client.1.smithi026.stdout:5/346: dread d1/db/d27/f23 [0,4194304] 0 2024-03-16T10:45:17.851 INFO:tasks.workunit.client.1.smithi026.stdout:2/387: link d2/d29/d17/d1a/d2b/c28 d2/d29/d17/d1a/d2b/d69/c87 0 2024-03-16T10:45:17.851 INFO:tasks.workunit.client.1.smithi026.stdout:6/352: dwrite d0/f33 [0,4194304] 0 2024-03-16T10:45:17.854 INFO:tasks.workunit.client.1.smithi026.stdout:4/387: mkdir d1/db/de/d4c/d6b/d7a 0 2024-03-16T10:45:17.857 INFO:tasks.workunit.client.1.smithi026.stdout:8/448: dwrite d5/d7/f34 [0,4194304] 0 2024-03-16T10:45:17.857 INFO:tasks.workunit.client.1.smithi026.stdout:1/407: rename d0/d17/d1f/fc to d0/d17/d1f/d63/d6a/d7c/f82 0 2024-03-16T10:45:17.861 INFO:tasks.workunit.client.1.smithi026.stdout:9/435: link d0/da/d16/d17/d44/l56 d0/d5f/d7f/d8c/l98 0 2024-03-16T10:45:17.867 INFO:tasks.workunit.client.1.smithi026.stdout:1/408: write d0/d17/d1f/d63/d6a/d7c/d3b/f2f [945067,109793] 0 2024-03-16T10:45:17.869 INFO:tasks.workunit.client.1.smithi026.stdout:8/449: truncate d5/d7/d2c/d36/f4c 1445775 0 2024-03-16T10:45:17.870 INFO:tasks.workunit.client.1.smithi026.stdout:5/347: dwrite d1/db/d27/f60 [0,4194304] 0 2024-03-16T10:45:17.873 INFO:tasks.workunit.client.1.smithi026.stdout:1/409: read d0/d17/d1f/d38/f4e [467738,74987] 0 2024-03-16T10:45:17.875 INFO:tasks.workunit.client.1.smithi026.stdout:6/353: unlink d0/d5b/f26 0 2024-03-16T10:45:17.877 INFO:tasks.workunit.client.1.smithi026.stdout:0/425: dwrite d1/d12/d2f/f7b [0,4194304] 0 2024-03-16T10:45:17.879 INFO:tasks.workunit.client.1.smithi026.stdout:1/410: read d0/d17/f1a [3161870,90889] 0 2024-03-16T10:45:17.879 INFO:tasks.workunit.client.1.smithi026.stdout:4/388: symlink d1/db/d29/d44/d48/l7b 0 2024-03-16T10:45:17.880 INFO:tasks.workunit.client.1.smithi026.stdout:3/435: symlink d4/dc/d73/d83/d57/l93 0 2024-03-16T10:45:17.880 INFO:tasks.workunit.client.1.smithi026.stdout:5/348: dread d1/f21 [0,4194304] 0 2024-03-16T10:45:17.881 INFO:tasks.workunit.client.1.smithi026.stdout:1/411: chown d0/d17/d1f/d63/d6a/d7c/f6c 12033 1 2024-03-16T10:45:17.883 INFO:tasks.workunit.client.1.smithi026.stdout:9/436: mknod d0/d63/c99 0 2024-03-16T10:45:17.884 INFO:tasks.workunit.client.1.smithi026.stdout:7/417: mkdir d6/d35/d3a/d4e/d6c/d85 0 2024-03-16T10:45:17.886 INFO:tasks.workunit.client.1.smithi026.stdout:8/450: creat d5/d23/d4e/f7d x:0 0 0 2024-03-16T10:45:17.891 INFO:tasks.workunit.client.1.smithi026.stdout:6/354: write d0/d10/f27 [1806035,117522] 0 2024-03-16T10:45:17.893 INFO:tasks.workunit.client.1.smithi026.stdout:0/426: fdatasync d1/d18/d1a/d24/d11/f8e 0 2024-03-16T10:45:17.899 INFO:tasks.workunit.client.1.smithi026.stdout:4/389: creat d1/db/d29/d61/f7c x:0 0 0 2024-03-16T10:45:17.902 INFO:tasks.workunit.client.1.smithi026.stdout:9/437: symlink d0/d63/d6b/l9a 0 2024-03-16T10:45:17.903 INFO:tasks.workunit.client.1.smithi026.stdout:6/355: dwrite d0/d28/d49/f78 [0,4194304] 0 2024-03-16T10:45:17.905 INFO:tasks.workunit.client.1.smithi026.stdout:6/356: chown d0/d10/d79 256158063 1 2024-03-16T10:45:17.908 INFO:tasks.workunit.client.1.smithi026.stdout:8/451: unlink d5/dc/d40/f52 0 2024-03-16T10:45:17.911 INFO:tasks.workunit.client.1.smithi026.stdout:9/438: dwrite d0/da/d14/f39 [4194304,4194304] 0 2024-03-16T10:45:17.916 INFO:tasks.workunit.client.1.smithi026.stdout:5/349: creat d1/d6e/f70 x:0 0 0 2024-03-16T10:45:17.921 INFO:tasks.workunit.client.1.smithi026.stdout:3/436: write d4/dc/d31/f3b [1062409,124171] 0 2024-03-16T10:45:17.930 INFO:tasks.workunit.client.1.smithi026.stdout:4/390: readlink d1/l5 0 2024-03-16T10:45:17.937 INFO:tasks.workunit.client.1.smithi026.stdout:7/418: getdents d6/d35/d3a/d4e/d6c/d85 0 2024-03-16T10:45:17.939 INFO:tasks.workunit.client.1.smithi026.stdout:8/452: mknod d5/d5d/c7e 0 2024-03-16T10:45:17.941 INFO:tasks.workunit.client.1.smithi026.stdout:9/439: creat d0/da/d13/d6e/f9b x:0 0 0 2024-03-16T10:45:17.942 INFO:tasks.workunit.client.1.smithi026.stdout:5/350: creat d1/f71 x:0 0 0 2024-03-16T10:45:17.944 INFO:tasks.workunit.client.1.smithi026.stdout:7/419: dwrite d6/d35/d3a/d3c/f51 [0,4194304] 0 2024-03-16T10:45:17.946 INFO:tasks.workunit.client.1.smithi026.stdout:3/437: mkdir d4/dc/dd/d3d/d94 0 2024-03-16T10:45:17.947 INFO:tasks.workunit.client.1.smithi026.stdout:4/391: rmdir d1/db/de/d4c/d6b 39 2024-03-16T10:45:17.949 INFO:tasks.workunit.client.1.smithi026.stdout:1/412: creat d0/d17/d1f/d63/f83 x:0 0 0 2024-03-16T10:45:17.954 INFO:tasks.workunit.client.1.smithi026.stdout:9/440: creat d0/d5f/d7f/f9c x:0 0 0 2024-03-16T10:45:17.955 INFO:tasks.workunit.client.1.smithi026.stdout:5/351: fdatasync d1/f7 0 2024-03-16T10:45:17.958 INFO:tasks.workunit.client.1.smithi026.stdout:7/420: creat d6/de/d7d/f86 x:0 0 0 2024-03-16T10:45:17.959 INFO:tasks.workunit.client.1.smithi026.stdout:7/421: write d6/de/f64 [1113593,93141] 0 2024-03-16T10:45:17.960 INFO:tasks.workunit.client.1.smithi026.stdout:7/422: chown d6/d35/d3a/d59/l5b 1785 1 2024-03-16T10:45:17.961 INFO:tasks.workunit.client.1.smithi026.stdout:6/357: link d0/c1 d0/d62/c81 0 2024-03-16T10:45:17.962 INFO:tasks.workunit.client.1.smithi026.stdout:9/441: creat d0/d5f/d7f/f9d x:0 0 0 2024-03-16T10:45:17.964 INFO:tasks.workunit.client.1.smithi026.stdout:9/442: truncate d0/da/d16/d5d/f7c 761862 0 2024-03-16T10:45:17.964 INFO:tasks.workunit.client.1.smithi026.stdout:0/427: write d1/d18/d1a/d24/f3 [5072984,77123] 0 2024-03-16T10:45:17.965 INFO:tasks.workunit.client.1.smithi026.stdout:5/352: creat d1/d1c/d2a/d31/d69/f72 x:0 0 0 2024-03-16T10:45:17.966 INFO:tasks.workunit.client.1.smithi026.stdout:5/353: fdatasync d1/d1c/f3c 0 2024-03-16T10:45:17.968 INFO:tasks.workunit.client.1.smithi026.stdout:4/392: mknod d1/db/c7d 0 2024-03-16T10:45:17.978 INFO:tasks.workunit.client.1.smithi026.stdout:4/393: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:18.188 INFO:tasks.workunit.client.1.smithi026.stdout:9/443: symlink d0/d5f/d7f/l9e 0 2024-03-16T10:45:18.190 INFO:tasks.workunit.client.1.smithi026.stdout:0/428: chown d1/d18/d1a/d24/d51/f5d 36 1 2024-03-16T10:45:18.192 INFO:tasks.workunit.client.1.smithi026.stdout:5/354: creat d1/db/d1f/d61/f73 x:0 0 0 2024-03-16T10:45:18.202 INFO:tasks.workunit.client.1.smithi026.stdout:0/429: creat d1/d18/d1a/d24/d51/d88/f8f x:0 0 0 2024-03-16T10:45:18.214 INFO:tasks.workunit.client.1.smithi026.stdout:0/430: symlink d1/d18/d1a/d2b/d4f/l90 0 2024-03-16T10:45:18.216 INFO:tasks.workunit.client.1.smithi026.stdout:5/355: creat d1/d1c/d68/f74 x:0 0 0 2024-03-16T10:45:18.217 INFO:tasks.workunit.client.1.smithi026.stdout:4/394: creat d1/db/f7e x:0 0 0 2024-03-16T10:45:18.223 INFO:tasks.workunit.client.1.smithi026.stdout:0/431: chown d1/d18/d1a/l49 6 1 2024-03-16T10:45:18.228 INFO:tasks.workunit.client.1.smithi026.stdout:2/388: sync 2024-03-16T10:45:18.232 INFO:tasks.workunit.client.1.smithi026.stdout:4/395: mknod d1/db/de/d4c/d6f/c7f 0 2024-03-16T10:45:18.234 INFO:tasks.workunit.client.1.smithi026.stdout:0/432: write d1/d18/d1a/d24/d57/f83 [32877,18757] 0 2024-03-16T10:45:18.235 INFO:tasks.workunit.client.1.smithi026.stdout:0/433: fsync d1/d18/d1a/d24/d51/f5e 0 2024-03-16T10:45:18.235 INFO:tasks.workunit.client.1.smithi026.stdout:2/389: symlink d2/d29/d85/l88 0 2024-03-16T10:45:18.237 INFO:tasks.workunit.client.1.smithi026.stdout:0/434: read d1/d18/d1a/f25 [3839916,72398] 0 2024-03-16T10:45:18.242 INFO:tasks.workunit.client.1.smithi026.stdout:4/396: dwrite d1/db/de/f3d [4194304,4194304] 0 2024-03-16T10:45:18.244 INFO:tasks.workunit.client.1.smithi026.stdout:0/435: creat d1/d18/d1a/d2b/f91 x:0 0 0 2024-03-16T10:45:18.245 INFO:tasks.workunit.client.1.smithi026.stdout:2/390: fsync d2/d29/d36/f58 0 2024-03-16T10:45:18.247 INFO:tasks.workunit.client.1.smithi026.stdout:4/397: creat d1/db/de/d4c/d59/f80 x:0 0 0 2024-03-16T10:45:18.251 INFO:tasks.workunit.client.1.smithi026.stdout:2/391: symlink d2/d29/d36/l89 0 2024-03-16T10:45:18.252 INFO:tasks.workunit.client.1.smithi026.stdout:2/392: readlink d2/d29/d17/d1a/d2b/l5e 0 2024-03-16T10:45:18.253 INFO:tasks.workunit.client.1.smithi026.stdout:4/398: dread - d1/db/de/f65 zero size 2024-03-16T10:45:18.255 INFO:tasks.workunit.client.1.smithi026.stdout:2/393: write d2/d29/d17/d1a/d2b/d4b/f5d [845374,85674] 0 2024-03-16T10:45:18.257 INFO:tasks.workunit.client.1.smithi026.stdout:2/394: write d2/d29/d17/d1a/d2b/d2e/f3f [3142790,97570] 0 2024-03-16T10:45:18.265 INFO:tasks.workunit.client.1.smithi026.stdout:2/395: mknod d2/d29/d17/d38/c8a 0 2024-03-16T10:45:18.318 INFO:tasks.workunit.client.1.smithi026.stdout:3/438: sync 2024-03-16T10:45:18.318 INFO:tasks.workunit.client.1.smithi026.stdout:8/453: sync 2024-03-16T10:45:18.318 INFO:tasks.workunit.client.1.smithi026.stdout:7/423: sync 2024-03-16T10:45:18.319 INFO:tasks.workunit.client.1.smithi026.stdout:3/439: chown d4/dc/dd/d80/c8b 408 1 2024-03-16T10:45:18.322 INFO:tasks.workunit.client.1.smithi026.stdout:7/424: write d6/d35/d3a/d3c/f51 [4188691,102609] 0 2024-03-16T10:45:18.322 INFO:tasks.workunit.client.1.smithi026.stdout:6/358: dwrite d0/d5b/f30 [0,4194304] 0 2024-03-16T10:45:18.327 INFO:tasks.workunit.client.1.smithi026.stdout:3/440: dread d4/d29/d7d/f88 [4194304,4194304] 0 2024-03-16T10:45:18.329 INFO:tasks.workunit.client.1.smithi026.stdout:3/441: read d4/d89/f8f [31757,61437] 0 2024-03-16T10:45:18.330 INFO:tasks.workunit.client.1.smithi026.stdout:7/425: dread - d6/d2f/f4a zero size 2024-03-16T10:45:18.335 INFO:tasks.workunit.client.1.smithi026.stdout:9/444: dwrite d0/da/f1b [0,4194304] 0 2024-03-16T10:45:18.337 INFO:tasks.workunit.client.1.smithi026.stdout:3/442: symlink d4/dc/dd/d3d/l95 0 2024-03-16T10:45:18.340 INFO:tasks.workunit.client.1.smithi026.stdout:7/426: rename d6/d35/d3a/d3c/f51 to d6/d35/d3a/d3c/d3f/f87 0 2024-03-16T10:45:18.350 INFO:tasks.workunit.client.1.smithi026.stdout:3/443: creat d4/dc/f96 x:0 0 0 2024-03-16T10:45:18.351 INFO:tasks.workunit.client.1.smithi026.stdout:3/444: truncate d4/dc/d31/f51 1704567 0 2024-03-16T10:45:18.352 INFO:tasks.workunit.client.1.smithi026.stdout:3/445: chown d4/dc/dd/d3d/l68 97 1 2024-03-16T10:45:18.355 INFO:tasks.workunit.client.1.smithi026.stdout:3/446: rename d4/d29 to d4/d89/d97 0 2024-03-16T10:45:18.357 INFO:tasks.workunit.client.1.smithi026.stdout:3/447: truncate d4/dc/dd/f75 5247524 0 2024-03-16T10:45:18.364 INFO:tasks.workunit.client.1.smithi026.stdout:3/448: dwrite d4/dc/d31/d48/d69/f6f [0,4194304] 0 2024-03-16T10:45:18.366 INFO:tasks.workunit.client.1.smithi026.stdout:3/449: chown d4/dc/dd/l2b 110 1 2024-03-16T10:45:18.374 INFO:tasks.workunit.client.1.smithi026.stdout:3/450: symlink d4/dc/dd/d3d/d94/l98 0 2024-03-16T10:45:18.377 INFO:tasks.workunit.client.1.smithi026.stdout:4/399: dwrite d1/db/de/f3c [0,4194304] 0 2024-03-16T10:45:18.378 INFO:tasks.workunit.client.1.smithi026.stdout:2/396: write d2/d29/d17/d1a/d2b/f1f [875369,118826] 0 2024-03-16T10:45:18.381 INFO:tasks.workunit.client.1.smithi026.stdout:3/451: creat d4/d89/d97/f99 x:0 0 0 2024-03-16T10:45:18.385 INFO:tasks.workunit.client.1.smithi026.stdout:1/413: mkdir d0/d17/d1f/d63/d6a/d7c/d84 0 2024-03-16T10:45:18.388 INFO:tasks.workunit.client.1.smithi026.stdout:2/397: symlink d2/d29/d51/d55/l8b 0 2024-03-16T10:45:18.393 INFO:tasks.workunit.client.1.smithi026.stdout:4/400: dread d1/db/f67 [0,4194304] 0 2024-03-16T10:45:18.406 INFO:tasks.workunit.client.1.smithi026.stdout:2/398: dread d2/d29/d17/d1a/d2b/d2e/f3f [0,4194304] 0 2024-03-16T10:45:18.407 INFO:tasks.workunit.client.1.smithi026.stdout:2/399: fsync d2/d29/d17/d1a/d2b/d4e/f71 0 2024-03-16T10:45:18.408 INFO:tasks.workunit.client.1.smithi026.stdout:2/400: chown d2/d29/d17/d1a/d2b/d4b/f5a 498501801 1 2024-03-16T10:45:18.410 INFO:tasks.workunit.client.1.smithi026.stdout:2/401: write d2/d29/d17/d1a/d2b/f73 [392224,104160] 0 2024-03-16T10:45:18.411 INFO:tasks.workunit.client.1.smithi026.stdout:0/436: creat d1/f92 x:0 0 0 2024-03-16T10:45:18.411 INFO:tasks.workunit.client.1.smithi026.stdout:0/437: chown d1 3749874 1 2024-03-16T10:45:18.414 INFO:tasks.workunit.client.1.smithi026.stdout:1/414: dread d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:18.415 INFO:tasks.workunit.client.1.smithi026.stdout:1/415: fsync d0/d17/d1f/f35 0 2024-03-16T10:45:18.416 INFO:tasks.workunit.client.1.smithi026.stdout:4/401: mkdir d1/db/de/d4c/d81 0 2024-03-16T10:45:18.425 INFO:tasks.workunit.client.1.smithi026.stdout:1/416: rmdir d0/d17/d1f/d20 39 2024-03-16T10:45:18.428 INFO:tasks.workunit.client.1.smithi026.stdout:8/454: truncate d5/f3e 502990 0 2024-03-16T10:45:18.429 INFO:tasks.workunit.client.1.smithi026.stdout:8/455: dread - d5/dc/d3f/f55 zero size 2024-03-16T10:45:18.431 INFO:tasks.workunit.client.1.smithi026.stdout:2/402: link d2/d29/d85/f77 d2/d29/d17/d1a/d2b/d2e/d60/f8c 0 2024-03-16T10:45:18.432 INFO:tasks.workunit.client.1.smithi026.stdout:2/403: dread d2/d29/f56 [0,4194304] 0 2024-03-16T10:45:18.433 INFO:tasks.workunit.client.1.smithi026.stdout:5/356: link d1/d1c/d2a/d31/d69/d51/f54 d1/d1c/f75 0 2024-03-16T10:45:18.434 INFO:tasks.workunit.client.1.smithi026.stdout:5/357: chown d1/db/d27/f66 1791250 1 2024-03-16T10:45:18.436 INFO:tasks.workunit.client.1.smithi026.stdout:0/438: rename d1/d18/d1a/d24/d11/f2d to d1/d12/d2f/d48/d55/f93 0 2024-03-16T10:45:18.439 INFO:tasks.workunit.client.1.smithi026.stdout:1/417: unlink d0/d17/d1f/d38/l4b 0 2024-03-16T10:45:18.442 INFO:tasks.workunit.client.1.smithi026.stdout:0/439: dwrite d1/d18/f36 [0,4194304] 0 2024-03-16T10:45:18.442 INFO:tasks.workunit.client.1.smithi026.stdout:4/402: creat d1/db/de/d4c/d59/d6a/f82 x:0 0 0 2024-03-16T10:45:18.442 INFO:tasks.workunit.client.1.smithi026.stdout:9/445: write d0/da/d13/f53 [748869,19874] 0 2024-03-16T10:45:18.444 INFO:tasks.workunit.client.1.smithi026.stdout:4/403: fdatasync d1/db/de/f55 0 2024-03-16T10:45:18.445 INFO:tasks.workunit.client.1.smithi026.stdout:9/446: chown d0/da/f1b 287319413 1 2024-03-16T10:45:18.445 INFO:tasks.workunit.client.1.smithi026.stdout:8/456: rmdir d5/dc/d3f 39 2024-03-16T10:45:18.453 INFO:tasks.workunit.client.1.smithi026.stdout:5/358: truncate d1/db/f44 2424074 0 2024-03-16T10:45:18.457 INFO:tasks.workunit.client.1.smithi026.stdout:2/404: dwrite d2/d29/f56 [0,4194304] 0 2024-03-16T10:45:18.459 INFO:tasks.workunit.client.1.smithi026.stdout:0/440: chown d1/l3d 713834 1 2024-03-16T10:45:18.461 INFO:tasks.workunit.client.1.smithi026.stdout:2/405: dread d2/d29/fe [0,4194304] 0 2024-03-16T10:45:18.475 INFO:tasks.workunit.client.1.smithi026.stdout:1/418: creat d0/d17/d1f/d20/f85 x:0 0 0 2024-03-16T10:45:18.488 INFO:tasks.workunit.client.1.smithi026.stdout:3/452: dwrite d4/dc/d31/f34 [0,4194304] 0 2024-03-16T10:45:18.490 INFO:tasks.workunit.client.1.smithi026.stdout:9/447: link d0/da/f7a d0/da/d16/d5d/d72/d62/f9f 0 2024-03-16T10:45:18.495 INFO:tasks.workunit.client.1.smithi026.stdout:5/359: link d1/db/d1f/d61/f6d d1/d1c/d2a/d31/f76 0 2024-03-16T10:45:18.496 INFO:tasks.workunit.client.1.smithi026.stdout:9/448: dwrite d0/f68 [0,4194304] 0 2024-03-16T10:45:18.499 INFO:tasks.workunit.client.1.smithi026.stdout:9/449: dread - d0/da/d16/d5d/d72/d62/f9f zero size 2024-03-16T10:45:18.507 INFO:tasks.workunit.client.1.smithi026.stdout:2/406: creat d2/d29/d17/d1a/d2b/d2e/d60/d7f/f8d x:0 0 0 2024-03-16T10:45:18.507 INFO:tasks.workunit.client.1.smithi026.stdout:5/360: dwrite d1/f71 [0,4194304] 0 2024-03-16T10:45:18.510 INFO:tasks.workunit.client.1.smithi026.stdout:1/419: dwrite d0/d17/d1f/f3f [0,4194304] 0 2024-03-16T10:45:18.511 INFO:tasks.workunit.client.1.smithi026.stdout:8/457: creat d5/f7f x:0 0 0 2024-03-16T10:45:18.519 INFO:tasks.workunit.client.1.smithi026.stdout:2/407: dread d2/d29/d17/d1a/d2b/d4b/f5d [0,4194304] 0 2024-03-16T10:45:18.520 INFO:tasks.workunit.client.1.smithi026.stdout:2/408: write d2/d29/d17/d1a/d2b/d4b/f5d [121951,83996] 0 2024-03-16T10:45:18.522 INFO:tasks.workunit.client.1.smithi026.stdout:2/409: dread - d2/d29/d17/d1a/d2b/f80 zero size 2024-03-16T10:45:18.529 INFO:tasks.workunit.client.1.smithi026.stdout:0/441: creat d1/d18/d1a/f94 x:0 0 0 2024-03-16T10:45:18.530 INFO:tasks.workunit.client.1.smithi026.stdout:0/442: chown d1/d18/d61/l8c 0 1 2024-03-16T10:45:18.535 INFO:tasks.workunit.client.1.smithi026.stdout:9/450: dread d0/da/d13/d6e/f8d [0,4194304] 0 2024-03-16T10:45:18.535 INFO:tasks.workunit.client.1.smithi026.stdout:5/361: symlink d1/db/l77 0 2024-03-16T10:45:18.537 INFO:tasks.workunit.client.1.smithi026.stdout:9/451: fsync d0/f66 0 2024-03-16T10:45:18.539 INFO:tasks.workunit.client.1.smithi026.stdout:0/443: dwrite d1/d12/f4b [0,4194304] 0 2024-03-16T10:45:18.540 INFO:tasks.workunit.client.1.smithi026.stdout:9/452: truncate d0/da/d14/f39 8707056 0 2024-03-16T10:45:18.547 INFO:tasks.workunit.client.1.smithi026.stdout:1/420: mknod d0/d17/d4d/d70/c86 0 2024-03-16T10:45:18.548 INFO:tasks.workunit.client.1.smithi026.stdout:1/421: chown d0/d17/d1f/d63/d6a/d7c/f6c 36589 1 2024-03-16T10:45:18.550 INFO:tasks.workunit.client.1.smithi026.stdout:1/422: chown d0/d17/d1f/d63/d6a/d7c/d84 998817 1 2024-03-16T10:45:18.553 INFO:tasks.workunit.client.1.smithi026.stdout:1/423: write d0/d17/d1f/d38/f6e [45301,58491] 0 2024-03-16T10:45:18.559 INFO:tasks.workunit.client.1.smithi026.stdout:8/458: dread d5/d7/f43 [0,4194304] 0 2024-03-16T10:45:18.567 INFO:tasks.workunit.client.1.smithi026.stdout:6/359: sync 2024-03-16T10:45:18.576 INFO:tasks.workunit.client.1.smithi026.stdout:0/444: mknod d1/d12/d2f/d48/d84/c95 0 2024-03-16T10:45:18.592 INFO:tasks.workunit.client.1.smithi026.stdout:1/424: creat d0/d17/d1f/d63/d6a/d7c/d6b/f87 x:0 0 0 2024-03-16T10:45:18.593 INFO:tasks.workunit.client.1.smithi026.stdout:1/425: fsync d0/d17/d4d/f78 0 2024-03-16T10:45:18.598 INFO:tasks.workunit.client.1.smithi026.stdout:6/360: unlink d0/d28/d2d/f57 0 2024-03-16T10:45:18.598 INFO:tasks.workunit.client.1.smithi026.stdout:6/361: dread - d0/d62/f6d zero size 2024-03-16T10:45:18.602 INFO:tasks.workunit.client.1.smithi026.stdout:4/404: write d1/f56 [1403394,76386] 0 2024-03-16T10:45:18.606 INFO:tasks.workunit.client.1.smithi026.stdout:2/410: rename d2/d29/f63 to d2/d29/d17/d1a/f8e 0 2024-03-16T10:45:18.607 INFO:tasks.workunit.client.1.smithi026.stdout:2/411: chown d2/d29/d51/d55/l8b 1067599 1 2024-03-16T10:45:18.624 INFO:tasks.workunit.client.1.smithi026.stdout:5/362: rename d1/d1c/f3f to d1/d6e/f78 0 2024-03-16T10:45:18.625 INFO:tasks.workunit.client.1.smithi026.stdout:5/363: chown d1/d1c/d2a/c62 27 1 2024-03-16T10:45:18.626 INFO:tasks.workunit.client.1.smithi026.stdout:2/412: unlink d2/d29/d51/d55/f83 0 2024-03-16T10:45:18.628 INFO:tasks.workunit.client.1.smithi026.stdout:2/413: truncate d2/d29/d17/d1a/d2b/d4b/f5b 89611 0 2024-03-16T10:45:18.629 INFO:tasks.workunit.client.1.smithi026.stdout:6/362: creat d0/d10/d79/d7b/f82 x:0 0 0 2024-03-16T10:45:18.630 INFO:tasks.workunit.client.1.smithi026.stdout:6/363: chown d0/d28/f42 0 1 2024-03-16T10:45:18.635 INFO:tasks.workunit.client.1.smithi026.stdout:8/459: getdents d5/dc/d40 0 2024-03-16T10:45:18.636 INFO:tasks.workunit.client.1.smithi026.stdout:2/414: chown d2/d29/d17/d38/d64/l6a 30146 1 2024-03-16T10:45:18.637 INFO:tasks.workunit.client.1.smithi026.stdout:2/415: fsync d2/d29/d17/d1a/d2b/f1f 0 2024-03-16T10:45:18.638 INFO:tasks.workunit.client.1.smithi026.stdout:2/416: chown d2/d29/d17/d38/d64 0 1 2024-03-16T10:45:18.639 INFO:tasks.workunit.client.1.smithi026.stdout:6/364: unlink d0/f33 0 2024-03-16T10:45:18.640 INFO:tasks.workunit.client.1.smithi026.stdout:5/364: mknod d1/d1c/c79 0 2024-03-16T10:45:18.641 INFO:tasks.workunit.client.1.smithi026.stdout:8/460: mknod d5/dc/d40/c80 0 2024-03-16T10:45:18.643 INFO:tasks.workunit.client.1.smithi026.stdout:2/417: creat d2/f8f x:0 0 0 2024-03-16T10:45:18.647 INFO:tasks.workunit.client.1.smithi026.stdout:6/365: link d0/d28/d49/f78 d0/d41/d4f/f83 0 2024-03-16T10:45:18.648 INFO:tasks.workunit.client.1.smithi026.stdout:8/461: mknod d5/dc/d3f/d62/c81 0 2024-03-16T10:45:18.650 INFO:tasks.workunit.client.1.smithi026.stdout:8/462: creat d5/d23/f82 x:0 0 0 2024-03-16T10:45:18.651 INFO:tasks.workunit.client.1.smithi026.stdout:6/366: link d0/d28/f6f d0/d62/f84 0 2024-03-16T10:45:18.653 INFO:tasks.workunit.client.1.smithi026.stdout:6/367: creat d0/d62/f85 x:0 0 0 2024-03-16T10:45:18.655 INFO:tasks.workunit.client.1.smithi026.stdout:8/463: symlink d5/l83 0 2024-03-16T10:45:18.657 INFO:tasks.workunit.client.1.smithi026.stdout:6/368: mknod d0/d10/d79/d7b/c86 0 2024-03-16T10:45:18.728 INFO:tasks.workunit.client.1.smithi026.stdout:7/427: sync 2024-03-16T10:45:18.730 INFO:tasks.workunit.client.1.smithi026.stdout:7/428: stat d6/d35/d3a/c4f 0 2024-03-16T10:45:18.730 INFO:tasks.workunit.client.1.smithi026.stdout:3/453: truncate d4/dc/d31/f34 450902 0 2024-03-16T10:45:18.731 INFO:tasks.workunit.client.1.smithi026.stdout:0/445: rename d1/d12/d2f/d48/d84 to d1/d18/d1a/d2b/d4f/d96 0 2024-03-16T10:45:18.735 INFO:tasks.workunit.client.1.smithi026.stdout:8/464: dwrite d5/d23/d4e/f5b [0,4194304] 0 2024-03-16T10:45:18.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:18 smithi002 ceph-mon[29248]: pgmap v4: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail 2024-03-16T10:45:18.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:18 smithi002 ceph-mon[29248]: mgrmap e29: smithi002.zevrtm(active, since 2s) 2024-03-16T10:45:18.820 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:18 smithi026 ceph-mon[37041]: pgmap v4: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail 2024-03-16T10:45:18.820 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:18 smithi026 ceph-mon[37041]: mgrmap e29: smithi002.zevrtm(active, since 2s) 2024-03-16T10:45:18.856 INFO:tasks.workunit.client.1.smithi026.stdout:9/453: dwrite d0/da/d13/d6e/f8d [0,4194304] 0 2024-03-16T10:45:18.856 INFO:tasks.workunit.client.1.smithi026.stdout:6/369: creat d0/d41/d4f/f87 x:0 0 0 2024-03-16T10:45:18.857 INFO:tasks.workunit.client.1.smithi026.stdout:7/429: mkdir d6/de/d16/d88 0 2024-03-16T10:45:18.859 INFO:tasks.workunit.client.1.smithi026.stdout:3/454: creat d4/dc/d73/d83/d57/f9a x:0 0 0 2024-03-16T10:45:18.859 INFO:tasks.workunit.client.1.smithi026.stdout:4/405: dwrite d1/f22 [0,4194304] 0 2024-03-16T10:45:18.859 INFO:tasks.workunit.client.1.smithi026.stdout:1/426: dwrite d0/d17/d4d/f6d [0,4194304] 0 2024-03-16T10:45:18.860 INFO:tasks.workunit.client.1.smithi026.stdout:0/446: creat d1/d18/f97 x:0 0 0 2024-03-16T10:45:18.863 INFO:tasks.workunit.client.1.smithi026.stdout:7/430: chown d6/d35/d3a/d5d/d7b 120468025 1 2024-03-16T10:45:18.871 INFO:tasks.workunit.client.1.smithi026.stdout:3/455: dwrite d4/d89/d97/f3a [0,4194304] 0 2024-03-16T10:45:18.875 INFO:tasks.workunit.client.1.smithi026.stdout:9/454: symlink d0/da/d16/d5d/d72/d62/la0 0 2024-03-16T10:45:18.875 INFO:tasks.workunit.client.1.smithi026.stdout:2/418: creat d2/d29/d17/d38/d64/f90 x:0 0 0 2024-03-16T10:45:18.881 INFO:tasks.workunit.client.1.smithi026.stdout:3/456: dwrite d4/dc/dd/f24 [0,4194304] 0 2024-03-16T10:45:18.888 INFO:tasks.workunit.client.1.smithi026.stdout:9/455: mknod d0/d63/d6b/ca1 0 2024-03-16T10:45:18.889 INFO:tasks.workunit.client.1.smithi026.stdout:4/406: mknod d1/db/de/d4c/d6b/c83 0 2024-03-16T10:45:18.889 INFO:tasks.workunit.client.1.smithi026.stdout:3/457: dwrite d4/dc/dd/d60/f64 [0,4194304] 0 2024-03-16T10:45:18.890 INFO:tasks.workunit.client.1.smithi026.stdout:9/456: chown d0/da/d14 414511 1 2024-03-16T10:45:18.892 INFO:tasks.workunit.client.1.smithi026.stdout:6/370: rmdir d0/d28/d2d/d54 0 2024-03-16T10:45:18.892 INFO:tasks.workunit.client.1.smithi026.stdout:6/371: chown d0/d28/d2d/f55 64840 1 2024-03-16T10:45:18.896 INFO:tasks.workunit.client.1.smithi026.stdout:7/431: link d6/de/d16/f1a d6/d35/d3a/d3c/d3f/d65/f89 0 2024-03-16T10:45:18.897 INFO:tasks.workunit.client.1.smithi026.stdout:0/447: truncate d1/d18/d1a/d24/f82 548242 0 2024-03-16T10:45:18.902 INFO:tasks.workunit.client.1.smithi026.stdout:0/448: dread - d1/d18/d1a/d2b/f4e zero size 2024-03-16T10:45:18.902 INFO:tasks.workunit.client.1.smithi026.stdout:9/457: rmdir d0/da/d13/d6e 39 2024-03-16T10:45:18.905 INFO:tasks.workunit.client.1.smithi026.stdout:4/407: rename d1/db/d29/d44/l46 to d1/db/de/d4c/d59/d6a/l84 0 2024-03-16T10:45:18.905 INFO:tasks.workunit.client.1.smithi026.stdout:0/449: readlink d1/d12/d2f/d48/d55/l76 0 2024-03-16T10:45:18.911 INFO:tasks.workunit.client.1.smithi026.stdout:7/432: dwrite d6/d35/d3a/d3c/f5c [0,4194304] 0 2024-03-16T10:45:18.915 INFO:tasks.workunit.client.1.smithi026.stdout:3/458: write d4/d89/f8f [1032723,116732] 0 2024-03-16T10:45:18.916 INFO:tasks.workunit.client.1.smithi026.stdout:9/458: dwrite d0/d5f/d7f/f9c [0,4194304] 0 2024-03-16T10:45:18.918 INFO:tasks.workunit.client.1.smithi026.stdout:3/459: fsync d4/dc/f11 0 2024-03-16T10:45:18.929 INFO:tasks.workunit.client.1.smithi026.stdout:4/408: chown d1/db/de/c1e 446057294 1 2024-03-16T10:45:18.930 INFO:tasks.workunit.client.1.smithi026.stdout:0/450: dread - d1/d18/d1a/d24/f6f zero size 2024-03-16T10:45:18.934 INFO:tasks.workunit.client.1.smithi026.stdout:7/433: write d6/d35/d3a/d3c/f40 [2886646,104763] 0 2024-03-16T10:45:18.937 INFO:tasks.workunit.client.1.smithi026.stdout:3/460: fsync d4/dc/dd/d3d/f49 0 2024-03-16T10:45:18.938 INFO:tasks.workunit.client.1.smithi026.stdout:3/461: dread - d4/f77 zero size 2024-03-16T10:45:18.939 INFO:tasks.workunit.client.1.smithi026.stdout:3/462: write d4/dc/d31/f3b [70886,127875] 0 2024-03-16T10:45:18.941 INFO:tasks.workunit.client.1.smithi026.stdout:4/409: dread d1/db/de/d4c/f4d [0,4194304] 0 2024-03-16T10:45:18.944 INFO:tasks.workunit.client.1.smithi026.stdout:3/463: dread d4/dc/d31/d48/d69/f6f [0,4194304] 0 2024-03-16T10:45:18.945 INFO:tasks.workunit.client.1.smithi026.stdout:0/451: mknod d1/d12/d2f/d48/d55/c98 0 2024-03-16T10:45:18.946 INFO:tasks.workunit.client.1.smithi026.stdout:0/452: fsync d1/d12/d2f/d48/d55/f71 0 2024-03-16T10:45:18.951 INFO:tasks.workunit.client.1.smithi026.stdout:7/434: dread d6/f31 [0,4194304] 0 2024-03-16T10:45:18.952 INFO:tasks.workunit.client.1.smithi026.stdout:3/464: dwrite d4/d89/f8f [0,4194304] 0 2024-03-16T10:45:18.953 INFO:tasks.workunit.client.1.smithi026.stdout:7/435: stat d6/de/d16/f1a 0 2024-03-16T10:45:18.954 INFO:tasks.workunit.client.1.smithi026.stdout:7/436: read - d6/de/f57 zero size 2024-03-16T10:45:18.958 INFO:tasks.workunit.client.1.smithi026.stdout:3/465: dread d4/d89/d97/f3a [0,4194304] 0 2024-03-16T10:45:18.962 INFO:tasks.workunit.client.1.smithi026.stdout:6/372: write d0/d10/fd [3037997,33835] 0 2024-03-16T10:45:18.963 INFO:tasks.workunit.client.1.smithi026.stdout:6/373: chown d0/d28/d2d/f68 1541276793 1 2024-03-16T10:45:18.968 INFO:tasks.workunit.client.1.smithi026.stdout:9/459: truncate d0/f4 454313 0 2024-03-16T10:45:18.968 INFO:tasks.workunit.client.1.smithi026.stdout:9/460: stat d0/da/d14/f34 0 2024-03-16T10:45:18.970 INFO:tasks.workunit.client.1.smithi026.stdout:9/461: write d0/da/d16/f3f [721367,35879] 0 2024-03-16T10:45:18.971 INFO:tasks.workunit.client.1.smithi026.stdout:9/462: truncate d0/da/d4d/f8f 161731 0 2024-03-16T10:45:18.974 INFO:tasks.workunit.client.1.smithi026.stdout:3/466: dread d4/dc/f4e [0,4194304] 0 2024-03-16T10:45:18.979 INFO:tasks.workunit.client.1.smithi026.stdout:9/463: dwrite d0/da/d16/d17/f40 [0,4194304] 0 2024-03-16T10:45:18.981 INFO:tasks.workunit.client.1.smithi026.stdout:7/437: write d6/d35/d3a/d3c/d3f/f4b [866093,75168] 0 2024-03-16T10:45:18.983 INFO:tasks.workunit.client.1.smithi026.stdout:7/438: truncate d6/d35/d3a/d3c/d3f/f4b 1911118 0 2024-03-16T10:45:18.993 INFO:tasks.workunit.client.1.smithi026.stdout:4/410: rename d1/db/de/d4c/d78 to d1/d85 0 2024-03-16T10:45:18.995 INFO:tasks.workunit.client.1.smithi026.stdout:9/464: dwrite d0/da/d16/d5d/f7c [0,4194304] 0 2024-03-16T10:45:18.999 INFO:tasks.workunit.client.1.smithi026.stdout:7/439: creat d6/de/d7d/f8a x:0 0 0 2024-03-16T10:45:19.004 INFO:tasks.workunit.client.1.smithi026.stdout:6/374: rename d0/d41/d4f/f65 to d0/d28/d49/d7d/f88 0 2024-03-16T10:45:19.006 INFO:tasks.workunit.client.1.smithi026.stdout:0/453: link d1/d18/c47 d1/d12/d2f/d48/d55/d60/c99 0 2024-03-16T10:45:19.015 INFO:tasks.workunit.client.1.smithi026.stdout:7/440: unlink d6/d35/d3a/l6e 0 2024-03-16T10:45:19.019 INFO:tasks.workunit.client.1.smithi026.stdout:3/467: rename d4/f18 to d4/dc/d73/d83/d57/f9b 0 2024-03-16T10:45:19.020 INFO:tasks.workunit.client.1.smithi026.stdout:6/375: rmdir d0/d10 39 2024-03-16T10:45:19.024 INFO:tasks.workunit.client.1.smithi026.stdout:9/465: write d0/da/d16/d17/d44/f2b [878638,120228] 0 2024-03-16T10:45:19.025 INFO:tasks.workunit.client.1.smithi026.stdout:9/466: write d0/da/f45 [1131957,79168] 0 2024-03-16T10:45:19.027 INFO:tasks.workunit.client.1.smithi026.stdout:7/441: rename d6/d35/d3a/d4e/f5a to d6/d35/d3a/d3c/d3f/d54/f8b 0 2024-03-16T10:45:19.030 INFO:tasks.workunit.client.1.smithi026.stdout:4/411: link d1/db/de/d4c/d6f/c7f d1/db/de/d4c/d6b/d7a/c86 0 2024-03-16T10:45:19.035 INFO:tasks.workunit.client.1.smithi026.stdout:4/412: dwrite d1/f6 [0,4194304] 0 2024-03-16T10:45:19.038 INFO:tasks.workunit.client.1.smithi026.stdout:9/467: chown d0/da/d14/l52 16200918 1 2024-03-16T10:45:19.044 INFO:tasks.workunit.client.1.smithi026.stdout:4/413: mknod d1/db/de/d6e/c87 0 2024-03-16T10:45:19.044 INFO:tasks.workunit.client.1.smithi026.stdout:0/454: dwrite d1/d18/d1a/d24/f59 [0,4194304] 0 2024-03-16T10:45:19.047 INFO:tasks.workunit.client.1.smithi026.stdout:9/468: mkdir d0/d5f/da2 0 2024-03-16T10:45:19.049 INFO:tasks.workunit.client.1.smithi026.stdout:7/442: symlink d6/d35/d3a/d4e/d6c/d85/l8c 0 2024-03-16T10:45:19.053 INFO:tasks.workunit.client.1.smithi026.stdout:6/376: fsync d0/d10/f4a 0 2024-03-16T10:45:19.054 INFO:tasks.workunit.client.1.smithi026.stdout:4/414: mknod d1/db/de/d4c/d6b/c88 0 2024-03-16T10:45:19.057 INFO:tasks.workunit.client.1.smithi026.stdout:3/468: getdents d4/dc/dd/d47 0 2024-03-16T10:45:19.057 INFO:tasks.workunit.client.1.smithi026.stdout:0/455: dwrite d1/d18/d1a/d24/f6d [0,4194304] 0 2024-03-16T10:45:19.058 INFO:tasks.workunit.client.1.smithi026.stdout:9/469: getdents d0/d5f/da2 0 2024-03-16T10:45:19.060 INFO:tasks.workunit.client.1.smithi026.stdout:6/377: dread - d0/d28/f4d zero size 2024-03-16T10:45:19.061 INFO:tasks.workunit.client.1.smithi026.stdout:4/415: rmdir d1/db/d29 39 2024-03-16T10:45:19.063 INFO:tasks.workunit.client.1.smithi026.stdout:9/470: dread d0/da/d14/f91 [0,4194304] 0 2024-03-16T10:45:19.063 INFO:tasks.workunit.client.1.smithi026.stdout:0/456: unlink d1/d18/d1a/d24/d51/f63 0 2024-03-16T10:45:19.071 INFO:tasks.workunit.client.1.smithi026.stdout:9/471: dwrite d0/d5f/d7f/f9c [0,4194304] 0 2024-03-16T10:45:19.078 INFO:tasks.workunit.client.1.smithi026.stdout:7/443: write d6/de/f11 [2580101,59556] 0 2024-03-16T10:45:19.079 INFO:tasks.workunit.client.1.smithi026.stdout:5/365: sync 2024-03-16T10:45:19.083 INFO:tasks.workunit.client.1.smithi026.stdout:3/469: truncate d4/f41 60232 0 2024-03-16T10:45:19.085 INFO:tasks.workunit.client.1.smithi026.stdout:0/457: dread d1/d18/d1a/d24/d51/f75 [0,4194304] 0 2024-03-16T10:45:19.097 INFO:tasks.workunit.client.1.smithi026.stdout:3/470: dread d4/dc/dd/d60/d62/d4f/f7e [0,4194304] 0 2024-03-16T10:45:19.097 INFO:tasks.workunit.client.1.smithi026.stdout:7/444: creat d6/d35/d3a/d3c/d3f/d54/f8d x:0 0 0 2024-03-16T10:45:19.098 INFO:tasks.workunit.client.1.smithi026.stdout:5/366: mknod d1/d1c/d4c/d4d/c7a 0 2024-03-16T10:45:19.100 INFO:tasks.workunit.client.1.smithi026.stdout:5/367: write d1/d1c/d2a/d31/d69/f67 [848739,17818] 0 2024-03-16T10:45:19.104 INFO:tasks.workunit.client.1.smithi026.stdout:6/378: rename d0/d28/d2d/f61 to d0/f89 0 2024-03-16T10:45:19.105 INFO:tasks.workunit.client.1.smithi026.stdout:6/379: dread - d0/d41/f7f zero size 2024-03-16T10:45:19.107 INFO:tasks.workunit.client.1.smithi026.stdout:9/472: chown d0/da/d13/d6e/c67 300713 1 2024-03-16T10:45:19.111 INFO:tasks.workunit.client.1.smithi026.stdout:3/471: mknod d4/dc/d31/c9c 0 2024-03-16T10:45:19.114 INFO:tasks.workunit.client.1.smithi026.stdout:3/472: write d4/dc/d31/d48/d69/d78/f87 [361396,113302] 0 2024-03-16T10:45:19.115 INFO:tasks.workunit.client.1.smithi026.stdout:5/368: dread d1/d1c/d2a/d31/f52 [0,4194304] 0 2024-03-16T10:45:19.117 INFO:tasks.workunit.client.1.smithi026.stdout:0/458: dwrite d1/d12/d2f/d48/d55/d60/f28 [4194304,4194304] 0 2024-03-16T10:45:19.119 INFO:tasks.workunit.client.1.smithi026.stdout:5/369: dread d1/f9 [0,4194304] 0 2024-03-16T10:45:19.119 INFO:tasks.workunit.client.1.smithi026.stdout:7/445: rename d6/de/d16/f19 to d6/d35/d3a/f8e 0 2024-03-16T10:45:19.123 INFO:tasks.workunit.client.1.smithi026.stdout:9/473: truncate d0/da/d14/f91 83683 0 2024-03-16T10:45:19.127 INFO:tasks.workunit.client.1.smithi026.stdout:5/370: dwrite d1/d1c/d68/f74 [0,4194304] 0 2024-03-16T10:45:19.130 INFO:tasks.workunit.client.1.smithi026.stdout:0/459: creat d1/d18/d1a/d24/d57/f9a x:0 0 0 2024-03-16T10:45:19.131 INFO:tasks.workunit.client.1.smithi026.stdout:6/380: truncate d0/d10/f1a 51404 0 2024-03-16T10:45:19.133 INFO:tasks.workunit.client.1.smithi026.stdout:4/416: getdents d1/db/d29 0 2024-03-16T10:45:19.137 INFO:tasks.workunit.client.1.smithi026.stdout:9/474: unlink d0/da/d4d/l76 0 2024-03-16T10:45:19.138 INFO:tasks.workunit.client.1.smithi026.stdout:9/475: dread - d0/da/d16/d5d/d72/d62/f9f zero size 2024-03-16T10:45:19.140 INFO:tasks.workunit.client.1.smithi026.stdout:9/476: write d0/d63/d6b/f81 [1539128,54733] 0 2024-03-16T10:45:19.148 INFO:tasks.workunit.client.1.smithi026.stdout:4/417: mkdir d1/db/de/d4c/d6b/d89 0 2024-03-16T10:45:19.149 INFO:tasks.workunit.client.1.smithi026.stdout:0/460: rename d1/d18/d1a/d24 to d1/d12/d2f/d48/d55/d60/d9b 0 2024-03-16T10:45:19.152 INFO:tasks.workunit.client.1.smithi026.stdout:3/473: creat d4/f9d x:0 0 0 2024-03-16T10:45:19.157 INFO:tasks.workunit.client.1.smithi026.stdout:9/477: creat d0/da/d16/d17/d44/fa3 x:0 0 0 2024-03-16T10:45:19.158 INFO:tasks.workunit.client.1.smithi026.stdout:7/446: dwrite d6/f31 [0,4194304] 0 2024-03-16T10:45:19.159 INFO:tasks.workunit.client.1.smithi026.stdout:9/478: write d0/da/f2a [1556138,84688] 0 2024-03-16T10:45:19.162 INFO:tasks.workunit.client.1.smithi026.stdout:6/381: write d0/f44 [829971,3040] 0 2024-03-16T10:45:19.164 INFO:tasks.workunit.client.1.smithi026.stdout:4/418: dread d1/db/d29/d44/f5a [0,4194304] 0 2024-03-16T10:45:19.166 INFO:tasks.workunit.client.1.smithi026.stdout:0/461: creat d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f9c x:0 0 0 2024-03-16T10:45:19.169 INFO:tasks.workunit.client.1.smithi026.stdout:7/447: dwrite d6/de/d16/f1f [0,4194304] 0 2024-03-16T10:45:19.172 INFO:tasks.workunit.client.1.smithi026.stdout:7/448: dread d6/f31 [4194304,4194304] 0 2024-03-16T10:45:19.173 INFO:tasks.workunit.client.1.smithi026.stdout:7/449: chown d6/d35/d3a 111242513 1 2024-03-16T10:45:19.173 INFO:tasks.workunit.client.1.smithi026.stdout:9/479: symlink d0/da/d16/d17/la4 0 2024-03-16T10:45:19.175 INFO:tasks.workunit.client.1.smithi026.stdout:6/382: dread - d0/d62/f63 zero size 2024-03-16T10:45:19.178 INFO:tasks.workunit.client.1.smithi026.stdout:3/474: unlink d4/fa 0 2024-03-16T10:45:19.185 INFO:tasks.workunit.client.1.smithi026.stdout:9/480: write d0/da/f11 [4726050,125493] 0 2024-03-16T10:45:19.189 INFO:tasks.workunit.client.1.smithi026.stdout:3/475: fdatasync d4/dc/f25 0 2024-03-16T10:45:19.190 INFO:tasks.workunit.client.1.smithi026.stdout:3/476: readlink d4/dc/dd/d60/d62/d4f/l5d 0 2024-03-16T10:45:19.191 INFO:tasks.workunit.client.1.smithi026.stdout:7/450: creat d6/d35/d3a/d3c/d3f/d65/d6f/f8f x:0 0 0 2024-03-16T10:45:19.192 INFO:tasks.workunit.client.1.smithi026.stdout:7/451: fdatasync d6/f39 0 2024-03-16T10:45:19.192 INFO:tasks.workunit.client.1.smithi026.stdout:9/481: creat d0/d63/d6b/fa5 x:0 0 0 2024-03-16T10:45:19.195 INFO:tasks.workunit.client.1.smithi026.stdout:6/383: mkdir d0/d8a 0 2024-03-16T10:45:19.196 INFO:tasks.workunit.client.1.smithi026.stdout:0/462: dread d1/d12/d2f/d48/d55/d60/f19 [0,4194304] 0 2024-03-16T10:45:19.196 INFO:tasks.workunit.client.1.smithi026.stdout:4/419: creat d1/db/de/d4c/f8a x:0 0 0 2024-03-16T10:45:19.197 INFO:tasks.workunit.client.1.smithi026.stdout:0/463: dread - d1/d12/d2f/d48/d55/d60/d9b/d51/d5f/f8a zero size 2024-03-16T10:45:19.198 INFO:tasks.workunit.client.1.smithi026.stdout:4/420: write d1/db/de/f5b [865726,126835] 0 2024-03-16T10:45:19.199 INFO:tasks.workunit.client.1.smithi026.stdout:4/421: chown d1/db/d29/d44 139610096 1 2024-03-16T10:45:19.203 INFO:tasks.workunit.client.1.smithi026.stdout:7/452: creat d6/d1e/f90 x:0 0 0 2024-03-16T10:45:19.205 INFO:tasks.workunit.client.1.smithi026.stdout:9/482: rename d0/da/f45 to d0/d5f/da2/fa6 0 2024-03-16T10:45:19.206 INFO:tasks.workunit.client.1.smithi026.stdout:0/464: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f8f [0,4194304] 0 2024-03-16T10:45:19.208 INFO:tasks.workunit.client.1.smithi026.stdout:0/465: truncate d1/d18/d1a/d2b/f91 555666 0 2024-03-16T10:45:19.209 INFO:tasks.workunit.client.1.smithi026.stdout:0/466: stat d1/d12/d2f/f7b 0 2024-03-16T10:45:19.212 INFO:tasks.workunit.client.1.smithi026.stdout:4/422: fsync d1/f2d 0 2024-03-16T10:45:19.224 INFO:tasks.workunit.client.1.smithi026.stdout:4/423: unlink d1/db/d29/d44/d48/f60 0 2024-03-16T10:45:19.225 INFO:tasks.workunit.client.1.smithi026.stdout:4/424: write d1/db/de/f65 [544848,21871] 0 2024-03-16T10:45:19.230 INFO:tasks.workunit.client.1.smithi026.stdout:9/483: truncate d0/da/f11 3309068 0 2024-03-16T10:45:19.233 INFO:tasks.workunit.client.1.smithi026.stdout:7/453: creat d6/f91 x:0 0 0 2024-03-16T10:45:19.234 INFO:tasks.workunit.client.1.smithi026.stdout:7/454: write d6/de/d7d/f86 [484542,1846] 0 2024-03-16T10:45:19.240 INFO:tasks.workunit.client.1.smithi026.stdout:0/467: rename d1/d18/l34 to d1/d18/d61/l9d 0 2024-03-16T10:45:19.241 INFO:tasks.workunit.client.1.smithi026.stdout:0/468: read d1/d12/d2f/d48/d55/d60/d9b/d51/f75 [628668,96050] 0 2024-03-16T10:45:19.242 INFO:tasks.workunit.client.1.smithi026.stdout:9/484: mkdir d0/da/d14/da7 0 2024-03-16T10:45:19.244 INFO:tasks.workunit.client.1.smithi026.stdout:4/425: mknod d1/db/de/c8b 0 2024-03-16T10:45:19.246 INFO:tasks.workunit.client.1.smithi026.stdout:6/384: getdents d0/d10/d79/d7b 0 2024-03-16T10:45:19.256 INFO:tasks.workunit.client.1.smithi026.stdout:7/455: link d6/f83 d6/d35/d3a/d59/f92 0 2024-03-16T10:45:19.262 INFO:tasks.workunit.client.1.smithi026.stdout:9/485: rmdir d0/da/d16/d17/d6c 0 2024-03-16T10:45:19.263 INFO:tasks.workunit.client.1.smithi026.stdout:9/486: chown d0/da/d14 65 1 2024-03-16T10:45:19.265 INFO:tasks.workunit.client.1.smithi026.stdout:7/456: creat d6/de/d16/d88/f93 x:0 0 0 2024-03-16T10:45:19.267 INFO:tasks.workunit.client.1.smithi026.stdout:0/469: creat d1/d18/f9e x:0 0 0 2024-03-16T10:45:19.268 INFO:tasks.workunit.client.1.smithi026.stdout:6/385: getdents d0/d5b 0 2024-03-16T10:45:19.268 INFO:tasks.workunit.client.1.smithi026.stdout:6/386: truncate d0/f89 4911520 0 2024-03-16T10:45:19.270 INFO:tasks.workunit.client.1.smithi026.stdout:9/487: rename d0/d5f/d7f/l9e to d0/da/d14/da7/la8 0 2024-03-16T10:45:19.271 INFO:tasks.workunit.client.1.smithi026.stdout:7/457: chown d6/de/d16/d44/c4d 3486794 1 2024-03-16T10:45:19.271 INFO:tasks.workunit.client.1.smithi026.stdout:7/458: stat d6/de/d16/d44/c76 0 2024-03-16T10:45:19.272 INFO:tasks.workunit.client.1.smithi026.stdout:7/459: chown d6/d1e/f90 1 1 2024-03-16T10:45:19.277 INFO:tasks.workunit.client.1.smithi026.stdout:9/488: creat d0/da/d16/d17/fa9 x:0 0 0 2024-03-16T10:45:19.277 INFO:tasks.workunit.client.1.smithi026.stdout:0/470: mknod d1/d18/d1a/d2b/c9f 0 2024-03-16T10:45:19.279 INFO:tasks.workunit.client.1.smithi026.stdout:7/460: chown d6/d35/d3a/d3c/d3f/d54/f8b 32884065 1 2024-03-16T10:45:19.280 INFO:tasks.workunit.client.1.smithi026.stdout:6/387: symlink d0/d8a/l8b 0 2024-03-16T10:45:19.280 INFO:tasks.workunit.client.1.smithi026.stdout:7/461: read d6/d1e/f32 [4079149,36624] 0 2024-03-16T10:45:19.281 INFO:tasks.workunit.client.1.smithi026.stdout:7/462: truncate d6/f91 214718 0 2024-03-16T10:45:19.283 INFO:tasks.workunit.client.1.smithi026.stdout:0/471: fdatasync d1/d18/d1a/d2b/f32 0 2024-03-16T10:45:19.286 INFO:tasks.workunit.client.1.smithi026.stdout:6/388: chown d0/d10/l29 7 1 2024-03-16T10:45:19.291 INFO:tasks.workunit.client.1.smithi026.stdout:1/427: sync 2024-03-16T10:45:19.291 INFO:tasks.workunit.client.1.smithi026.stdout:8/465: sync 2024-03-16T10:45:19.291 INFO:tasks.workunit.client.1.smithi026.stdout:2/419: sync 2024-03-16T10:45:19.293 INFO:tasks.workunit.client.1.smithi026.stdout:1/428: write d0/d17/d1f/d63/d6a/d7c/f30 [408292,11014] 0 2024-03-16T10:45:19.299 INFO:tasks.workunit.client.1.smithi026.stdout:0/472: fdatasync d1/d18/d1a/f25 0 2024-03-16T10:45:19.304 INFO:tasks.workunit.client.1.smithi026.stdout:6/389: truncate d0/d28/d2d/f60 481632 0 2024-03-16T10:45:19.309 INFO:tasks.workunit.client.1.smithi026.stdout:0/473: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/f72 [0,4194304] 0 2024-03-16T10:45:19.312 INFO:tasks.workunit.client.1.smithi026.stdout:2/420: fdatasync d2/d29/d85/f77 0 2024-03-16T10:45:19.314 INFO:tasks.workunit.client.1.smithi026.stdout:9/489: dwrite d0/da/d16/d17/d44/f2b [0,4194304] 0 2024-03-16T10:45:19.316 INFO:tasks.workunit.client.1.smithi026.stdout:2/421: read - d2/d29/d17/d38/d64/f90 zero size 2024-03-16T10:45:19.318 INFO:tasks.workunit.client.1.smithi026.stdout:0/474: dread d1/d12/d2f/d48/d55/d60/d9b/d57/f83 [0,4194304] 0 2024-03-16T10:45:19.327 INFO:tasks.workunit.client.1.smithi026.stdout:7/463: creat d6/de/f94 x:0 0 0 2024-03-16T10:45:19.327 INFO:tasks.workunit.client.1.smithi026.stdout:2/422: dwrite d2/d29/d17/d38/d64/f90 [0,4194304] 0 2024-03-16T10:45:19.330 INFO:tasks.workunit.client.1.smithi026.stdout:7/464: dread - d6/d35/d3a/d59/f92 zero size 2024-03-16T10:45:19.335 INFO:tasks.workunit.client.1.smithi026.stdout:8/466: creat d5/d23/d4e/d60/f84 x:0 0 0 2024-03-16T10:45:19.342 INFO:tasks.workunit.client.1.smithi026.stdout:1/429: creat d0/d17/d1f/d20/d65/f88 x:0 0 0 2024-03-16T10:45:19.352 INFO:tasks.workunit.client.1.smithi026.stdout:2/423: dwrite d2/d29/d17/d1a/d2b/d2e/f3f [0,4194304] 0 2024-03-16T10:45:19.354 INFO:tasks.workunit.client.1.smithi026.stdout:8/467: mknod d5/dc/d3f/d62/c85 0 2024-03-16T10:45:19.360 INFO:tasks.workunit.client.1.smithi026.stdout:1/430: chown d0/d17/d1f/d20/c45 15 1 2024-03-16T10:45:19.361 INFO:tasks.workunit.client.1.smithi026.stdout:1/431: chown d0/d17/d1f/f35 37 1 2024-03-16T10:45:19.362 INFO:tasks.workunit.client.1.smithi026.stdout:1/432: fdatasync d0/d17/d1f/f35 0 2024-03-16T10:45:19.364 INFO:tasks.workunit.client.1.smithi026.stdout:6/390: truncate d0/d28/d2d/f60 677887 0 2024-03-16T10:45:19.371 INFO:tasks.workunit.client.1.smithi026.stdout:9/490: truncate d0/da/d16/f3f 120555 0 2024-03-16T10:45:19.372 INFO:tasks.workunit.client.1.smithi026.stdout:0/475: link d1/d12/d2f/d48/d55/d60/d9b/d11/f8e d1/d12/d2f/d48/d55/d60/fa0 0 2024-03-16T10:45:19.376 INFO:tasks.workunit.client.1.smithi026.stdout:0/476: dread d1/d12/d2f/f77 [0,4194304] 0 2024-03-16T10:45:19.380 INFO:tasks.workunit.client.1.smithi026.stdout:1/433: creat d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/f89 x:0 0 0 2024-03-16T10:45:19.382 INFO:tasks.workunit.client.1.smithi026.stdout:8/468: dwrite d5/dc/d3f/f46 [0,4194304] 0 2024-03-16T10:45:19.389 INFO:tasks.workunit.client.1.smithi026.stdout:9/491: mknod d0/d63/d6b/caa 0 2024-03-16T10:45:19.391 INFO:tasks.workunit.client.1.smithi026.stdout:7/465: getdents d6/d2f 0 2024-03-16T10:45:19.394 INFO:tasks.workunit.client.1.smithi026.stdout:0/477: mknod d1/d12/d2f/d48/d55/ca1 0 2024-03-16T10:45:19.396 INFO:tasks.workunit.client.1.smithi026.stdout:2/424: creat d2/d29/d17/d1a/d2b/f91 x:0 0 0 2024-03-16T10:45:19.398 INFO:tasks.workunit.client.1.smithi026.stdout:1/434: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a 0 2024-03-16T10:45:19.399 INFO:tasks.workunit.client.1.smithi026.stdout:8/469: mknod d5/d23/d4e/d69/c86 0 2024-03-16T10:45:19.406 INFO:tasks.workunit.client.1.smithi026.stdout:1/435: dwrite d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/f89 [0,4194304] 0 2024-03-16T10:45:19.408 INFO:tasks.workunit.client.1.smithi026.stdout:1/436: truncate d0/d17/f81 477427 0 2024-03-16T10:45:19.415 INFO:tasks.workunit.client.1.smithi026.stdout:8/470: dread d5/dc/d3f/f20 [0,4194304] 0 2024-03-16T10:45:19.417 INFO:tasks.workunit.client.1.smithi026.stdout:7/466: fdatasync d6/f83 0 2024-03-16T10:45:19.424 INFO:tasks.workunit.client.1.smithi026.stdout:6/391: link d0/d5b/f18 d0/d28/d49/d7d/f8c 0 2024-03-16T10:45:19.432 INFO:tasks.workunit.client.1.smithi026.stdout:0/478: mknod d1/d12/d2f/d48/d55/d60/d9b/ca2 0 2024-03-16T10:45:19.434 INFO:tasks.workunit.client.1.smithi026.stdout:2/425: rename d2/d29/d17/d1a/d2b/c28 to d2/d29/d17/d38/c92 0 2024-03-16T10:45:19.436 INFO:tasks.workunit.client.1.smithi026.stdout:6/392: truncate d0/d28/d49/f5a 1885956 0 2024-03-16T10:45:19.437 INFO:tasks.workunit.client.1.smithi026.stdout:9/492: creat d0/da/d13/fab x:0 0 0 2024-03-16T10:45:19.439 INFO:tasks.workunit.client.1.smithi026.stdout:1/437: mkdir d0/d8b 0 2024-03-16T10:45:19.439 INFO:tasks.workunit.client.1.smithi026.stdout:1/438: stat d0/d17/d1f/d20/d75 0 2024-03-16T10:45:19.441 INFO:tasks.workunit.client.1.smithi026.stdout:8/471: creat d5/d23/d4e/d69/d76/f87 x:0 0 0 2024-03-16T10:45:19.443 INFO:tasks.workunit.client.1.smithi026.stdout:0/479: mknod d1/d18/d1a/d2b/ca3 0 2024-03-16T10:45:19.445 INFO:tasks.workunit.client.1.smithi026.stdout:2/426: symlink d2/d29/d17/d38/l93 0 2024-03-16T10:45:19.449 INFO:tasks.workunit.client.1.smithi026.stdout:9/493: unlink d0/da/d16/d17/f40 0 2024-03-16T10:45:19.451 INFO:tasks.workunit.client.1.smithi026.stdout:1/439: creat d0/d17/d1f/d20/d65/f8c x:0 0 0 2024-03-16T10:45:19.453 INFO:tasks.workunit.client.1.smithi026.stdout:1/440: readlink d0/d17/d1f/d63/d6a/d7c/d2e/l39 0 2024-03-16T10:45:19.454 INFO:tasks.workunit.client.1.smithi026.stdout:8/472: rmdir d5/d7/d2c/d36/d59 39 2024-03-16T10:45:19.455 INFO:tasks.workunit.client.1.smithi026.stdout:9/494: dwrite d0/d5f/d7f/f9c [4194304,4194304] 0 2024-03-16T10:45:19.470 INFO:tasks.workunit.client.1.smithi026.stdout:5/371: sync 2024-03-16T10:45:19.473 INFO:tasks.workunit.client.1.smithi026.stdout:0/480: symlink d1/d18/la4 0 2024-03-16T10:45:19.478 INFO:tasks.workunit.client.1.smithi026.stdout:0/481: dread d1/d12/d2f/d48/d55/d60/d9b/d51/f72 [0,4194304] 0 2024-03-16T10:45:19.481 INFO:tasks.workunit.client.1.smithi026.stdout:8/473: symlink d5/dc/l88 0 2024-03-16T10:45:19.485 INFO:tasks.workunit.client.1.smithi026.stdout:1/441: mkdir d0/d17/d1f/d63/d8d 0 2024-03-16T10:45:19.487 INFO:tasks.workunit.client.1.smithi026.stdout:1/442: chown d0/d17/d1f/d20/d65/f8c 152233 1 2024-03-16T10:45:19.492 INFO:tasks.workunit.client.1.smithi026.stdout:2/427: link d2/d29/d17/d1a/l30 d2/d29/d17/d1a/d2b/d4b/l94 0 2024-03-16T10:45:19.493 INFO:tasks.workunit.client.1.smithi026.stdout:0/482: mkdir d1/d12/d2f/d48/da5 0 2024-03-16T10:45:19.498 INFO:tasks.workunit.client.1.smithi026.stdout:2/428: dwrite d2/d29/d36/f58 [0,4194304] 0 2024-03-16T10:45:19.500 INFO:tasks.workunit.client.1.smithi026.stdout:5/372: write d1/db/f12 [513228,112042] 0 2024-03-16T10:45:19.500 INFO:tasks.workunit.client.1.smithi026.stdout:9/495: link d0/da/d16/d5d/d72/d62/la0 d0/da/d16/d5d/d72/lac 0 2024-03-16T10:45:19.505 INFO:tasks.workunit.client.1.smithi026.stdout:0/483: symlink d1/d12/d2f/d48/d55/d60/d9b/d51/d88/la6 0 2024-03-16T10:45:19.507 INFO:tasks.workunit.client.1.smithi026.stdout:8/474: readlink d5/dc/d40/l42 0 2024-03-16T10:45:19.507 INFO:tasks.workunit.client.1.smithi026.stdout:0/484: write d1/d12/d2f/d48/d55/d60/d9b/d51/f5d [764896,127781] 0 2024-03-16T10:45:19.508 INFO:tasks.workunit.client.1.smithi026.stdout:0/485: stat d1/d12/d2f/d48 0 2024-03-16T10:45:19.511 INFO:tasks.workunit.client.1.smithi026.stdout:2/429: mknod d2/d29/d17/d38/d64/c95 0 2024-03-16T10:45:19.516 INFO:tasks.workunit.client.1.smithi026.stdout:5/373: mknod d1/d1c/d2a/d31/d69/d51/c7b 0 2024-03-16T10:45:19.522 INFO:tasks.workunit.client.1.smithi026.stdout:8/475: dwrite d5/d23/d4e/f66 [0,4194304] 0 2024-03-16T10:45:19.523 INFO:tasks.workunit.client.1.smithi026.stdout:0/486: dwrite d1/d18/f97 [0,4194304] 0 2024-03-16T10:45:19.524 INFO:tasks.workunit.client.1.smithi026.stdout:6/393: dwrite d0/d5b/f1f [4194304,4194304] 0 2024-03-16T10:45:19.525 INFO:tasks.workunit.client.1.smithi026.stdout:2/430: truncate d2/d29/d17/d1a/d2b/d4e/d6f/f70 685893 0 2024-03-16T10:45:19.528 INFO:tasks.workunit.client.1.smithi026.stdout:5/374: stat d1/db/d27/l42 0 2024-03-16T10:45:19.533 INFO:tasks.workunit.client.1.smithi026.stdout:2/431: mkdir d2/d29/d51/d55/d96 0 2024-03-16T10:45:19.540 INFO:tasks.workunit.client.1.smithi026.stdout:6/394: unlink d0/d5b/f1f 0 2024-03-16T10:45:19.541 INFO:tasks.workunit.client.1.smithi026.stdout:0/487: stat d1/d12/d2f/d48/d55/d60/c99 0 2024-03-16T10:45:19.544 INFO:tasks.workunit.client.1.smithi026.stdout:5/375: chown d1/db/f44 122 1 2024-03-16T10:45:19.547 INFO:tasks.workunit.client.1.smithi026.stdout:5/376: dread d1/d1c/d2a/f57 [0,4194304] 0 2024-03-16T10:45:19.548 INFO:tasks.workunit.client.1.smithi026.stdout:6/395: fsync d0/f5f 0 2024-03-16T10:45:19.550 INFO:tasks.workunit.client.1.smithi026.stdout:6/396: chown d0/f51 183615 1 2024-03-16T10:45:19.553 INFO:tasks.workunit.client.1.smithi026.stdout:2/432: dread d2/d29/f3a [0,4194304] 0 2024-03-16T10:45:19.554 INFO:tasks.workunit.client.1.smithi026.stdout:8/476: rename d5/dc/d3f/d62 to d5/dc/d89 0 2024-03-16T10:45:19.567 INFO:tasks.workunit.client.1.smithi026.stdout:8/477: write d5/dc/d40/d53/f58 [4881430,4349] 0 2024-03-16T10:45:19.570 INFO:tasks.workunit.client.1.smithi026.stdout:6/397: rename d0/d28/d49/d7d/f88 to d0/d10/d79/f8d 0 2024-03-16T10:45:19.570 INFO:tasks.workunit.client.1.smithi026.stdout:9/496: write d0/d63/d6b/f77 [596470,53431] 0 2024-03-16T10:45:19.572 INFO:tasks.workunit.client.1.smithi026.stdout:9/497: write d0/da/d13/f69 [576564,126685] 0 2024-03-16T10:45:19.572 INFO:tasks.workunit.client.1.smithi026.stdout:9/498: dread - d0/f66 zero size 2024-03-16T10:45:19.574 INFO:tasks.workunit.client.1.smithi026.stdout:5/377: creat d1/db/d1f/d61/d65/f7c x:0 0 0 2024-03-16T10:45:19.575 INFO:tasks.workunit.client.1.smithi026.stdout:8/478: truncate d5/dc/f18 1856685 0 2024-03-16T10:45:19.577 INFO:tasks.workunit.client.1.smithi026.stdout:6/398: mkdir d0/d10/d79/d8e 0 2024-03-16T10:45:19.580 INFO:tasks.workunit.client.1.smithi026.stdout:9/499: creat d0/da/d16/fad x:0 0 0 2024-03-16T10:45:19.586 INFO:tasks.workunit.client.1.smithi026.stdout:0/488: write d1/d12/d2f/d48/d55/d60/d9b/d57/f83 [820189,74023] 0 2024-03-16T10:45:19.588 INFO:tasks.workunit.client.1.smithi026.stdout:8/479: write d5/d7/d2c/d36/d59/f78 [657885,46041] 0 2024-03-16T10:45:19.590 INFO:tasks.workunit.client.1.smithi026.stdout:6/399: symlink d0/d10/d79/d7b/l8f 0 2024-03-16T10:45:19.591 INFO:tasks.workunit.client.1.smithi026.stdout:2/433: write d2/d29/d17/d1a/f39 [137583,123251] 0 2024-03-16T10:45:19.592 INFO:tasks.workunit.client.1.smithi026.stdout:2/434: read d2/d29/d17/d1a/d2b/d4b/f5b [80531,77248] 0 2024-03-16T10:45:19.593 INFO:tasks.workunit.client.1.smithi026.stdout:0/489: mkdir d1/d18/d1a/d2b/d4f/da7 0 2024-03-16T10:45:19.594 INFO:tasks.workunit.client.1.smithi026.stdout:2/435: readlink d2/d29/d17/d1a/d2b/l5e 0 2024-03-16T10:45:19.596 INFO:tasks.workunit.client.1.smithi026.stdout:0/490: truncate d1/d12/d2f/d48/d55/d60/d9b/f6f 740078 0 2024-03-16T10:45:19.600 INFO:tasks.workunit.client.1.smithi026.stdout:5/378: creat d1/db/f7d x:0 0 0 2024-03-16T10:45:19.605 INFO:tasks.workunit.client.1.smithi026.stdout:2/436: rmdir d2/d29/d17/d1a/d2b/d4e 39 2024-03-16T10:45:19.607 INFO:tasks.workunit.client.1.smithi026.stdout:0/491: creat d1/d18/d1a/d2b/d4f/fa8 x:0 0 0 2024-03-16T10:45:19.611 INFO:tasks.workunit.client.1.smithi026.stdout:5/379: symlink d1/d1c/d2a/d31/l7e 0 2024-03-16T10:45:19.612 INFO:tasks.workunit.client.1.smithi026.stdout:2/437: dwrite d2/d29/d17/d38/f59 [0,4194304] 0 2024-03-16T10:45:19.613 INFO:tasks.workunit.client.1.smithi026.stdout:5/380: chown d1/d1c/d2a/f57 6 1 2024-03-16T10:45:19.614 INFO:tasks.workunit.client.1.smithi026.stdout:2/438: fsync d2/d29/d17/d1a/d2b/d2e/f31 0 2024-03-16T10:45:19.619 INFO:tasks.workunit.client.1.smithi026.stdout:0/492: creat d1/d12/fa9 x:0 0 0 2024-03-16T10:45:19.620 INFO:tasks.workunit.client.1.smithi026.stdout:6/400: link d0/d5b/c1e d0/d8a/c90 0 2024-03-16T10:45:19.620 INFO:tasks.workunit.client.1.smithi026.stdout:8/480: dwrite d5/dc/f18 [0,4194304] 0 2024-03-16T10:45:19.622 INFO:tasks.workunit.client.1.smithi026.stdout:6/401: write d0/d10/d79/d7b/f82 [625025,84551] 0 2024-03-16T10:45:19.622 INFO:tasks.workunit.client.1.smithi026.stdout:8/481: write d5/d7/d2c/d36/d59/f78 [1095447,22962] 0 2024-03-16T10:45:19.627 INFO:tasks.workunit.client.1.smithi026.stdout:2/439: rmdir d2/d29 39 2024-03-16T10:45:19.629 INFO:tasks.workunit.client.1.smithi026.stdout:5/381: creat d1/d1c/d2a/d31/d69/d51/f7f x:0 0 0 2024-03-16T10:45:19.635 INFO:tasks.workunit.client.1.smithi026.stdout:6/402: mkdir d0/d10/d79/d91 0 2024-03-16T10:45:19.635 INFO:tasks.workunit.client.1.smithi026.stdout:5/382: dread d1/db/d1f/f5c [0,4194304] 0 2024-03-16T10:45:19.636 INFO:tasks.workunit.client.1.smithi026.stdout:6/403: chown d0/f44 617 1 2024-03-16T10:45:19.637 INFO:tasks.workunit.client.1.smithi026.stdout:6/404: stat d0/d28/f40 0 2024-03-16T10:45:19.637 INFO:tasks.workunit.client.1.smithi026.stdout:8/482: rename d5/dc/d40/d53/f5c to d5/dc/d89/f8a 0 2024-03-16T10:45:19.638 INFO:tasks.workunit.client.1.smithi026.stdout:8/483: chown d5/d7/d2c/d36/d59/l64 105 1 2024-03-16T10:45:19.642 INFO:tasks.workunit.client.1.smithi026.stdout:0/493: mkdir d1/d12/d2f/daa 0 2024-03-16T10:45:19.643 INFO:tasks.workunit.client.1.smithi026.stdout:5/383: mkdir d1/d1c/d2a/d31/d69/d51/d80 0 2024-03-16T10:45:19.645 INFO:tasks.workunit.client.1.smithi026.stdout:6/405: mknod d0/d28/d49/d7d/c92 0 2024-03-16T10:45:19.646 INFO:tasks.workunit.client.1.smithi026.stdout:8/484: write d5/dc/d40/f77 [424010,107836] 0 2024-03-16T10:45:19.649 INFO:tasks.workunit.client.1.smithi026.stdout:5/384: creat d1/d1c/d2a/d31/f81 x:0 0 0 2024-03-16T10:45:19.651 INFO:tasks.workunit.client.1.smithi026.stdout:2/440: chown d2/d29/d17/d38/c92 902752 1 2024-03-16T10:45:19.653 INFO:tasks.workunit.client.1.smithi026.stdout:0/494: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/cab 0 2024-03-16T10:45:19.659 INFO:tasks.workunit.client.1.smithi026.stdout:2/441: creat d2/d29/d17/d38/f97 x:0 0 0 2024-03-16T10:45:19.663 INFO:tasks.workunit.client.1.smithi026.stdout:5/385: dwrite d1/db/f38 [0,4194304] 0 2024-03-16T10:45:19.664 INFO:tasks.workunit.client.1.smithi026.stdout:5/386: chown d1/d1c/d2a/d55 715160711 1 2024-03-16T10:45:19.669 INFO:tasks.workunit.client.1.smithi026.stdout:6/406: write d0/d5b/f18 [1804559,64640] 0 2024-03-16T10:45:19.670 INFO:tasks.workunit.client.1.smithi026.stdout:2/442: fsync d2/d29/d17/d1a/d2b/d2e/d60/f8c 0 2024-03-16T10:45:19.672 INFO:tasks.workunit.client.1.smithi026.stdout:8/485: truncate d5/d23/d4e/f5b 987246 0 2024-03-16T10:45:19.672 INFO:tasks.workunit.client.1.smithi026.stdout:0/495: getdents d1/d18/d1a/d2b/d4f/da7 0 2024-03-16T10:45:19.676 INFO:tasks.workunit.client.1.smithi026.stdout:2/443: mkdir d2/d29/d36/d98 0 2024-03-16T10:45:19.682 INFO:tasks.workunit.client.1.smithi026.stdout:8/486: rmdir d5 39 2024-03-16T10:45:19.687 INFO:tasks.workunit.client.1.smithi026.stdout:0/496: mkdir d1/d12/d2f/daa/dac 0 2024-03-16T10:45:19.690 INFO:tasks.workunit.client.1.smithi026.stdout:2/444: symlink d2/d29/d51/d55/d96/l99 0 2024-03-16T10:45:19.694 INFO:tasks.workunit.client.1.smithi026.stdout:5/387: getdents d1/d1c/d68 0 2024-03-16T10:45:19.694 INFO:tasks.workunit.client.1.smithi026.stdout:0/497: dwrite d1/d12/d2f/d48/d55/d60/f28 [4194304,4194304] 0 2024-03-16T10:45:19.699 INFO:tasks.workunit.client.1.smithi026.stdout:5/388: mkdir d1/db/d1f/d61/d65/d82 0 2024-03-16T10:45:19.701 INFO:tasks.workunit.client.1.smithi026.stdout:5/389: read d1/d1c/d2a/d31/f49 [69383,8546] 0 2024-03-16T10:45:19.701 INFO:tasks.workunit.client.1.smithi026.stdout:2/445: creat d2/d29/d17/d1a/d2b/f9a x:0 0 0 2024-03-16T10:45:19.702 INFO:tasks.workunit.client.1.smithi026.stdout:0/498: symlink d1/d18/lad 0 2024-03-16T10:45:19.707 INFO:tasks.workunit.client.1.smithi026.stdout:2/446: rmdir d2/d29/d17/d38 39 2024-03-16T10:45:19.714 INFO:tasks.workunit.client.1.smithi026.stdout:0/499: dwrite d1/d18/d1a/d2b/f86 [0,4194304] 0 2024-03-16T10:45:19.719 INFO:tasks.workunit.client.1.smithi026.stdout:5/390: dwrite d1/f18 [0,4194304] 0 2024-03-16T10:45:19.725 INFO:tasks.workunit.client.1.smithi026.stdout:5/391: rename d1/db/f12 to d1/db/d1f/d61/d65/f83 0 2024-03-16T10:45:19.734 INFO:tasks.workunit.client.1.smithi026.stdout:2/447: truncate d2/d29/d17/d1a/d2b/d2e/f3c 4200604 0 2024-03-16T10:45:19.734 INFO:tasks.workunit.client.1.smithi026.stdout:0/500: dread d1/f40 [0,4194304] 0 2024-03-16T10:45:19.736 INFO:tasks.workunit.client.1.smithi026.stdout:0/501: dread - d1/f92 zero size 2024-03-16T10:45:19.738 INFO:tasks.workunit.client.1.smithi026.stdout:0/502: write d1/d12/d2f/d48/d55/d60/d9b/d51/f5d [572033,116296] 0 2024-03-16T10:45:19.738 INFO:tasks.workunit.client.1.smithi026.stdout:5/392: fdatasync d1/db/d1f/d61/d65/f83 0 2024-03-16T10:45:19.742 INFO:tasks.workunit.client.1.smithi026.stdout:0/503: truncate d1/d12/d2f/d48/d55/d60/d9b/f82 701575 0 2024-03-16T10:45:19.748 INFO:tasks.workunit.client.1.smithi026.stdout:2/448: rename d2/d29/d17/d1a/d2b/d4e/d6f/f70 to d2/d29/d3b/f9b 0 2024-03-16T10:45:19.751 INFO:tasks.workunit.client.1.smithi026.stdout:0/504: truncate d1/d12/d2f/d48/d55/d60/d9b/d51/f72 5019950 0 2024-03-16T10:45:19.752 INFO:tasks.workunit.client.1.smithi026.stdout:2/449: dread d2/d29/d36/f3e [0,4194304] 0 2024-03-16T10:45:19.753 INFO:tasks.workunit.client.1.smithi026.stdout:5/393: symlink d1/db/l84 0 2024-03-16T10:45:19.757 INFO:tasks.workunit.client.1.smithi026.stdout:3/477: sync 2024-03-16T10:45:19.757 INFO:tasks.workunit.client.1.smithi026.stdout:4/426: sync 2024-03-16T10:45:19.760 INFO:tasks.workunit.client.1.smithi026.stdout:0/505: dwrite d1/d18/d1a/d2b/d4f/f80 [0,4194304] 0 2024-03-16T10:45:19.761 INFO:tasks.workunit.client.1.smithi026.stdout:0/506: chown d1/d18/d1a/d2b/d4f/d96 3 1 2024-03-16T10:45:19.765 INFO:tasks.workunit.client.1.smithi026.stdout:2/450: dwrite d2/f35 [0,4194304] 0 2024-03-16T10:45:19.766 INFO:tasks.workunit.client.1.smithi026.stdout:5/394: truncate d1/db/d1f/d61/d65/f83 1348936 0 2024-03-16T10:45:19.776 INFO:tasks.workunit.client.1.smithi026.stdout:5/395: dwrite d1/d1c/f3c [0,4194304] 0 2024-03-16T10:45:19.776 INFO:tasks.workunit.client.1.smithi026.stdout:2/451: readlink d2/l1c 0 2024-03-16T10:45:19.779 INFO:tasks.workunit.client.1.smithi026.stdout:3/478: dread d4/dc/d73/d83/d7c/f5b [0,4194304] 0 2024-03-16T10:45:19.786 INFO:tasks.workunit.client.1.smithi026.stdout:3/479: dread d4/dc/d31/f3b [0,4194304] 0 2024-03-16T10:45:19.789 INFO:tasks.workunit.client.1.smithi026.stdout:4/427: symlink d1/l8c 0 2024-03-16T10:45:19.790 INFO:tasks.workunit.client.1.smithi026.stdout:5/396: creat d1/db/d1f/f85 x:0 0 0 2024-03-16T10:45:19.791 INFO:tasks.workunit.client.1.smithi026.stdout:4/428: stat d1/db/de/d4c/d6b/d89 0 2024-03-16T10:45:19.792 INFO:tasks.workunit.client.1.smithi026.stdout:3/480: creat d4/d89/d97/f9e x:0 0 0 2024-03-16T10:45:19.794 INFO:tasks.workunit.client.1.smithi026.stdout:0/507: dread d1/d12/d2f/d48/d55/d60/d9b/f3 [0,4194304] 0 2024-03-16T10:45:19.802 INFO:tasks.workunit.client.1.smithi026.stdout:5/397: creat d1/d1c/d2a/d31/d69/d51/d80/f86 x:0 0 0 2024-03-16T10:45:19.803 INFO:tasks.workunit.client.1.smithi026.stdout:3/481: rmdir d4/d89 39 2024-03-16T10:45:19.806 INFO:tasks.workunit.client.1.smithi026.stdout:5/398: write d1/db/d1f/f5b [772144,37449] 0 2024-03-16T10:45:19.807 INFO:tasks.workunit.client.1.smithi026.stdout:2/452: dwrite d2/d29/d85/f77 [0,4194304] 0 2024-03-16T10:45:19.810 INFO:tasks.workunit.client.1.smithi026.stdout:4/429: link d1/l21 d1/db/de/d4c/d6b/d89/l8d 0 2024-03-16T10:45:19.810 INFO:tasks.workunit.client.1.smithi026.stdout:2/453: write d2/d29/d17/d1a/d2b/d2e/d60/d7f/f8d [685984,102642] 0 2024-03-16T10:45:19.814 INFO:tasks.workunit.client.1.smithi026.stdout:4/430: readlink d1/db/de/d4c/l69 0 2024-03-16T10:45:19.814 INFO:tasks.workunit.client.1.smithi026.stdout:3/482: creat d4/dc/d31/d48/f9f x:0 0 0 2024-03-16T10:45:19.814 INFO:tasks.workunit.client.1.smithi026.stdout:0/508: dread d1/d18/d61/f6b [0,4194304] 0 2024-03-16T10:45:19.814 INFO:tasks.workunit.client.1.smithi026.stdout:5/399: truncate d1/d1c/d2a/d31/d69/d51/f7f 815676 0 2024-03-16T10:45:19.816 INFO:tasks.workunit.client.1.smithi026.stdout:4/431: chown d1/db/de/f70 85 1 2024-03-16T10:45:19.818 INFO:tasks.workunit.client.1.smithi026.stdout:2/454: creat d2/d29/d17/d1a/d2b/d4e/d6f/f9c x:0 0 0 2024-03-16T10:45:19.822 INFO:tasks.workunit.client.1.smithi026.stdout:3/483: mknod d4/dc/d31/d48/d69/d78/ca0 0 2024-03-16T10:45:19.825 INFO:tasks.workunit.client.1.smithi026.stdout:4/432: read d1/db/de/f65 [483756,113391] 0 2024-03-16T10:45:19.826 INFO:tasks.workunit.client.1.smithi026.stdout:3/484: mknod d4/dc/d73/d83/ca1 0 2024-03-16T10:45:19.829 INFO:tasks.workunit.client.1.smithi026.stdout:0/509: getdents d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89 0 2024-03-16T10:45:19.830 INFO:tasks.workunit.client.1.smithi026.stdout:5/400: getdents d1/db/d1f/d61 0 2024-03-16T10:45:19.830 INFO:tasks.workunit.client.1.smithi026.stdout:2/455: dwrite d2/d29/d17/d1a/f39 [0,4194304] 0 2024-03-16T10:45:19.834 INFO:tasks.workunit.client.1.smithi026.stdout:4/433: mkdir d1/db/d29/d44/d48/d8e 0 2024-03-16T10:45:19.834 INFO:tasks.workunit.client.1.smithi026.stdout:2/456: write d2/d29/d85/f77 [3354302,124685] 0 2024-03-16T10:45:19.842 INFO:tasks.workunit.client.1.smithi026.stdout:3/485: dwrite d4/dc/dd/d3d/f79 [0,4194304] 0 2024-03-16T10:45:19.846 INFO:tasks.workunit.client.1.smithi026.stdout:4/434: write d1/db/de/f58 [143063,102218] 0 2024-03-16T10:45:19.849 INFO:tasks.workunit.client.1.smithi026.stdout:0/510: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/d88/cae 0 2024-03-16T10:45:19.852 INFO:tasks.workunit.client.1.smithi026.stdout:2/457: creat d2/d29/d17/d1a/d2b/d69/f9d x:0 0 0 2024-03-16T10:45:19.857 INFO:tasks.workunit.client.1.smithi026.stdout:4/435: mknod d1/db/de/d4c/d79/c8f 0 2024-03-16T10:45:19.863 INFO:tasks.workunit.client.1.smithi026.stdout:0/511: rename d1/d12/d2f/d48/d55/f93 to d1/d12/d2f/d48/faf 0 2024-03-16T10:45:19.863 INFO:tasks.workunit.client.1.smithi026.stdout:0/512: fdatasync d1/f92 0 2024-03-16T10:45:19.869 INFO:tasks.workunit.client.1.smithi026.stdout:4/436: mknod d1/db/de/c90 0 2024-03-16T10:45:19.870 INFO:tasks.workunit.client.1.smithi026.stdout:3/486: truncate d4/dc/dd/d47/f86 1312350 0 2024-03-16T10:45:19.871 INFO:tasks.workunit.client.1.smithi026.stdout:3/487: dread - d4/dc/dd/d60/d62/f85 zero size 2024-03-16T10:45:19.874 INFO:tasks.workunit.client.1.smithi026.stdout:4/437: mknod d1/db/d29/d44/d48/c91 0 2024-03-16T10:45:19.877 INFO:tasks.workunit.client.1.smithi026.stdout:0/513: dread d1/d12/d2f/d48/d55/d60/d9b/d51/f72 [0,4194304] 0 2024-03-16T10:45:19.877 INFO:tasks.workunit.client.1.smithi026.stdout:3/488: fsync d4/dc/d73/d83/d7c/f5b 0 2024-03-16T10:45:19.880 INFO:tasks.workunit.client.1.smithi026.stdout:4/438: symlink d1/db/de/d6e/l92 0 2024-03-16T10:45:19.884 INFO:tasks.workunit.client.1.smithi026.stdout:0/514: dwrite d1/d12/f4b [0,4194304] 0 2024-03-16T10:45:19.889 INFO:tasks.workunit.client.1.smithi026.stdout:3/489: creat d4/dc/d31/d48/d69/d78/fa2 x:0 0 0 2024-03-16T10:45:19.894 INFO:tasks.workunit.client.1.smithi026.stdout:3/490: mknod d4/dc/d31/d48/d7f/ca3 0 2024-03-16T10:45:19.900 INFO:tasks.workunit.client.1.smithi026.stdout:0/515: dread d1/d18/d1a/f26 [4194304,4194304] 0 2024-03-16T10:45:19.957 INFO:tasks.workunit.client.1.smithi026.stdout:7/467: sync 2024-03-16T10:45:19.961 INFO:tasks.workunit.client.1.smithi026.stdout:4/439: creat d1/db/d29/d44/d48/d8e/f93 x:0 0 0 2024-03-16T10:45:19.982 INFO:tasks.workunit.client.1.smithi026.stdout:7/468: creat d6/d35/d3a/d3c/d3f/d65/d6f/f95 x:0 0 0 2024-03-16T10:45:19.985 INFO:tasks.workunit.client.1.smithi026.stdout:0/516: symlink d1/d18/d1a/lb0 0 2024-03-16T10:45:19.988 INFO:tasks.workunit.client.1.smithi026.stdout:4/440: creat d1/db/de/d4c/d81/f94 x:0 0 0 2024-03-16T10:45:19.988 INFO:tasks.workunit.client.1.smithi026.stdout:7/469: dwrite d6/de/f11 [0,4194304] 0 2024-03-16T10:45:19.988 INFO:tasks.workunit.client.1.smithi026.stdout:0/517: write d1/d12/d2f/d48/d55/d60/d9b/d57/f83 [1826503,51571] 0 2024-03-16T10:45:19.990 INFO:tasks.workunit.client.1.smithi026.stdout:4/441: stat d1/db/de/d4c/l69 0 2024-03-16T10:45:19.991 INFO:tasks.workunit.client.1.smithi026.stdout:7/470: read d6/de/d16/f49 [3437616,53929] 0 2024-03-16T10:45:19.994 INFO:tasks.workunit.client.1.smithi026.stdout:4/442: fsync d1/f8 0 2024-03-16T10:45:19.995 INFO:tasks.workunit.client.1.smithi026.stdout:0/518: fsync d1/d12/d2f/d48/d55/d60/d9b/d11/f54 0 2024-03-16T10:45:19.998 INFO:tasks.workunit.client.1.smithi026.stdout:7/471: rename d6/de/d16/f1a to d6/d35/d3a/d59/f96 0 2024-03-16T10:45:20.002 INFO:tasks.workunit.client.1.smithi026.stdout:0/519: unlink d1/d12/d2f/d48/d55/d60/f31 0 2024-03-16T10:45:20.004 INFO:tasks.workunit.client.1.smithi026.stdout:4/443: rename d1/db/f3a to d1/db/d29/f95 0 2024-03-16T10:45:20.005 INFO:tasks.workunit.client.1.smithi026.stdout:4/444: chown d1/db/d29/f3e 8 1 2024-03-16T10:45:20.010 INFO:tasks.workunit.client.1.smithi026.stdout:0/520: symlink d1/d18/d1a/d2b/d4f/d96/lb1 0 2024-03-16T10:45:20.012 INFO:tasks.workunit.client.1.smithi026.stdout:3/491: truncate d4/dc/d31/f51 1611447 0 2024-03-16T10:45:20.013 INFO:tasks.workunit.client.1.smithi026.stdout:7/472: creat d6/d35/d3a/d5d/d79/f97 x:0 0 0 2024-03-16T10:45:20.019 INFO:tasks.workunit.client.1.smithi026.stdout:3/492: creat d4/dc/dd/d3d/d94/fa4 x:0 0 0 2024-03-16T10:45:20.019 INFO:tasks.workunit.client.1.smithi026.stdout:4/445: dread d1/db/de/f3d [4194304,4194304] 0 2024-03-16T10:45:20.020 INFO:tasks.workunit.client.1.smithi026.stdout:4/446: chown d1/db/f5d 36507249 1 2024-03-16T10:45:20.022 INFO:tasks.workunit.client.1.smithi026.stdout:4/447: write d1/db/d29/d44/d48/d8e/f93 [918570,55089] 0 2024-03-16T10:45:20.026 INFO:tasks.workunit.client.1.smithi026.stdout:3/493: symlink d4/dc/dd/d60/la5 0 2024-03-16T10:45:20.030 INFO:tasks.workunit.client.1.smithi026.stdout:0/521: getdents d1/d12/d2f/d48/d55/d60 0 2024-03-16T10:45:20.039 INFO:tasks.workunit.client.1.smithi026.stdout:3/494: dread d4/dc/f11 [4194304,4194304] 0 2024-03-16T10:45:20.042 INFO:tasks.workunit.client.1.smithi026.stdout:7/473: link d6/c10 d6/c98 0 2024-03-16T10:45:20.044 INFO:tasks.workunit.client.1.smithi026.stdout:0/522: creat d1/d18/d1a/d2b/d4f/da7/fb2 x:0 0 0 2024-03-16T10:45:20.046 INFO:tasks.workunit.client.1.smithi026.stdout:3/495: mkdir d4/dc/dd/d60/d62/d4f/d5a/da6 0 2024-03-16T10:45:20.048 INFO:tasks.workunit.client.1.smithi026.stdout:0/523: dread d1/f40 [0,4194304] 0 2024-03-16T10:45:20.049 INFO:tasks.workunit.client.1.smithi026.stdout:4/448: truncate d1/db/fc 7297 0 2024-03-16T10:45:20.052 INFO:tasks.workunit.client.1.smithi026.stdout:3/496: rename d4/dc/d31/c9c to d4/dc/dd/d60/d62/d4f/d5a/da6/ca7 0 2024-03-16T10:45:20.060 INFO:tasks.workunit.client.1.smithi026.stdout:4/449: dwrite d1/f6 [0,4194304] 0 2024-03-16T10:45:20.068 INFO:tasks.workunit.client.1.smithi026.stdout:3/497: link d4/dc/l32 d4/dc/dd/d47/d53/la8 0 2024-03-16T10:45:20.068 INFO:tasks.workunit.client.1.smithi026.stdout:3/498: readlink d4/l6d 0 2024-03-16T10:45:20.069 INFO:tasks.workunit.client.1.smithi026.stdout:7/474: truncate d6/d1e/f6a 1020910 0 2024-03-16T10:45:20.069 INFO:tasks.workunit.client.1.smithi026.stdout:3/499: readlink d4/dc/d31/l37 0 2024-03-16T10:45:20.070 INFO:tasks.workunit.client.1.smithi026.stdout:3/500: read - d4/dc/f96 zero size 2024-03-16T10:45:20.071 INFO:tasks.workunit.client.1.smithi026.stdout:7/475: write d6/d35/d3a/d5d/d79/f97 [91527,53781] 0 2024-03-16T10:45:20.071 INFO:tasks.workunit.client.1.smithi026.stdout:3/501: chown d4/dc/f16 102431 1 2024-03-16T10:45:20.074 INFO:tasks.workunit.client.1.smithi026.stdout:4/450: symlink d1/l96 0 2024-03-16T10:45:20.077 INFO:tasks.workunit.client.1.smithi026.stdout:0/524: write d1/d18/d1a/f25 [4360717,104653] 0 2024-03-16T10:45:20.079 INFO:tasks.workunit.client.1.smithi026.stdout:7/476: chown d6/d35/d3a/d4e/c7f 51 1 2024-03-16T10:45:20.082 INFO:tasks.workunit.client.1.smithi026.stdout:3/502: dread d4/dc/d31/f34 [0,4194304] 0 2024-03-16T10:45:20.086 INFO:tasks.workunit.client.1.smithi026.stdout:7/477: dwrite d6/d1e/f20 [0,4194304] 0 2024-03-16T10:45:20.090 INFO:tasks.workunit.client.1.smithi026.stdout:4/451: creat d1/f97 x:0 0 0 2024-03-16T10:45:20.092 INFO:tasks.workunit.client.1.smithi026.stdout:7/478: fsync d6/fc 0 2024-03-16T10:45:20.095 INFO:tasks.workunit.client.1.smithi026.stdout:4/452: write d1/db/d29/d44/f5a [3734804,23421] 0 2024-03-16T10:45:20.097 INFO:tasks.workunit.client.1.smithi026.stdout:4/453: write d1/db/de/d4c/d59/f80 [58152,52476] 0 2024-03-16T10:45:20.100 INFO:tasks.workunit.client.1.smithi026.stdout:0/525: link d1/d12/d2f/d48/d55/d60/d9b/d57/f3e d1/d18/fb3 0 2024-03-16T10:45:20.101 INFO:tasks.workunit.client.1.smithi026.stdout:3/503: getdents d4/dc/dd/d60/d62/d4f 0 2024-03-16T10:45:20.102 INFO:tasks.workunit.client.1.smithi026.stdout:3/504: stat d4/dc/d73/d83/d7c 0 2024-03-16T10:45:20.103 INFO:tasks.workunit.client.1.smithi026.stdout:4/454: creat d1/db/de/d4c/d6b/f98 x:0 0 0 2024-03-16T10:45:20.105 INFO:tasks.workunit.client.1.smithi026.stdout:4/455: write d1/db/f74 [289536,91514] 0 2024-03-16T10:45:20.105 INFO:tasks.workunit.client.1.smithi026.stdout:7/479: write d6/f83 [714806,66305] 0 2024-03-16T10:45:20.107 INFO:tasks.workunit.client.1.smithi026.stdout:0/526: mkdir d1/d18/d1a/d2b/db4 0 2024-03-16T10:45:20.110 INFO:tasks.workunit.client.1.smithi026.stdout:7/480: mkdir d6/d35/d3a/d3c/d3f/d99 0 2024-03-16T10:45:20.112 INFO:tasks.workunit.client.1.smithi026.stdout:3/505: unlink d4/dc/dd/d47/d53/la8 0 2024-03-16T10:45:20.112 INFO:tasks.workunit.client.1.smithi026.stdout:4/456: dwrite d1/db/f42 [0,4194304] 0 2024-03-16T10:45:20.116 INFO:tasks.workunit.client.1.smithi026.stdout:7/481: truncate d6/f31 317172 0 2024-03-16T10:45:20.117 INFO:tasks.workunit.client.1.smithi026.stdout:0/527: creat d1/d12/d2f/daa/dac/fb5 x:0 0 0 2024-03-16T10:45:20.118 INFO:tasks.workunit.client.1.smithi026.stdout:3/506: chown d4/dc/dd/c56 48 1 2024-03-16T10:45:20.121 INFO:tasks.workunit.client.1.smithi026.stdout:4/457: chown d1/db/d29/d44/d48/l4a 10334 1 2024-03-16T10:45:20.132 INFO:tasks.workunit.client.1.smithi026.stdout:0/528: rename d1/d18/d1a/d2b/l64 to d1/d18/d1a/lb6 0 2024-03-16T10:45:20.134 INFO:tasks.workunit.client.1.smithi026.stdout:7/482: dread d6/de/d16/f43 [0,4194304] 0 2024-03-16T10:45:20.134 INFO:tasks.workunit.client.1.smithi026.stdout:3/507: getdents d4/dc/dd/d60 0 2024-03-16T10:45:20.139 INFO:tasks.workunit.client.1.smithi026.stdout:7/483: unlink d6/de/d16/f43 0 2024-03-16T10:45:20.141 INFO:tasks.workunit.client.1.smithi026.stdout:7/484: write d6/d35/d3a/d5d/d79/f97 [854211,101135] 0 2024-03-16T10:45:20.141 INFO:tasks.workunit.client.1.smithi026.stdout:3/508: mkdir d4/dc/dd/d47/da9 0 2024-03-16T10:45:20.142 INFO:tasks.workunit.client.1.smithi026.stdout:4/458: link d1/db/l39 d1/db/d29/d61/l99 0 2024-03-16T10:45:20.149 INFO:tasks.workunit.client.1.smithi026.stdout:4/459: dwrite d1/f6 [0,4194304] 0 2024-03-16T10:45:20.152 INFO:tasks.workunit.client.1.smithi026.stdout:4/460: fsync d1/f52 0 2024-03-16T10:45:20.161 INFO:tasks.workunit.client.1.smithi026.stdout:7/485: write d6/f22 [1084916,45130] 0 2024-03-16T10:45:20.166 INFO:tasks.workunit.client.1.smithi026.stdout:3/509: dread d4/dc/dd/d3d/f3e [0,4194304] 0 2024-03-16T10:45:20.175 INFO:tasks.workunit.client.1.smithi026.stdout:4/461: dwrite d1/f52 [0,4194304] 0 2024-03-16T10:45:20.177 INFO:tasks.workunit.client.1.smithi026.stdout:3/510: rmdir d4/dc/dd/d47/da9 0 2024-03-16T10:45:20.177 INFO:tasks.workunit.client.1.smithi026.stdout:7/486: dread d6/d35/d3a/f8e [0,4194304] 0 2024-03-16T10:45:20.179 INFO:tasks.workunit.client.1.smithi026.stdout:4/462: fdatasync d1/db/de/d4c/f4d 0 2024-03-16T10:45:20.191 INFO:tasks.workunit.client.1.smithi026.stdout:3/511: creat d4/faa x:0 0 0 2024-03-16T10:45:20.195 INFO:tasks.workunit.client.1.smithi026.stdout:7/487: dread d6/d35/d3a/d3c/d3f/d65/f89 [0,4194304] 0 2024-03-16T10:45:20.198 INFO:tasks.workunit.client.1.smithi026.stdout:7/488: symlink d6/d35/d3a/d5d/d7b/l9a 0 2024-03-16T10:45:20.200 INFO:tasks.workunit.client.1.smithi026.stdout:4/463: dwrite d1/db/d29/f95 [0,4194304] 0 2024-03-16T10:45:20.203 INFO:tasks.workunit.client.1.smithi026.stdout:7/489: dread d6/f83 [0,4194304] 0 2024-03-16T10:45:20.210 INFO:tasks.workunit.client.1.smithi026.stdout:4/464: write d1/f2d [4002710,23225] 0 2024-03-16T10:45:20.212 INFO:tasks.workunit.client.1.smithi026.stdout:4/465: dread - d1/db/d29/d61/f7c zero size 2024-03-16T10:45:20.215 INFO:tasks.workunit.client.1.smithi026.stdout:7/490: write d6/d1e/f6a [852514,60733] 0 2024-03-16T10:45:20.217 INFO:tasks.workunit.client.1.smithi026.stdout:4/466: creat d1/db/de/d4c/d59/d6a/f9a x:0 0 0 2024-03-16T10:45:20.219 INFO:tasks.workunit.client.1.smithi026.stdout:7/491: creat d6/d35/d3a/d3c/d3f/d54/f9b x:0 0 0 2024-03-16T10:45:20.220 INFO:tasks.workunit.client.1.smithi026.stdout:4/467: creat d1/db/de/d4c/d59/d6a/f9b x:0 0 0 2024-03-16T10:45:20.220 INFO:tasks.workunit.client.1.smithi026.stdout:7/492: truncate d6/de/f57 798334 0 2024-03-16T10:45:20.224 INFO:tasks.workunit.client.1.smithi026.stdout:4/468: mkdir d1/db/de/d4c/d6b/d89/d9c 0 2024-03-16T10:45:20.225 INFO:tasks.workunit.client.1.smithi026.stdout:7/493: dread d6/d1e/f6a [0,4194304] 0 2024-03-16T10:45:20.226 INFO:tasks.workunit.client.1.smithi026.stdout:7/494: readlink d6/d35/d3a/d59/l5b 0 2024-03-16T10:45:20.226 INFO:tasks.workunit.client.1.smithi026.stdout:7/495: chown d6/d35/d3a/d4e 1 1 2024-03-16T10:45:20.227 INFO:tasks.workunit.client.1.smithi026.stdout:4/469: mknod d1/db/de/d4c/d59/c9d 0 2024-03-16T10:45:20.227 INFO:tasks.workunit.client.1.smithi026.stdout:7/496: dread - d6/d35/d3a/d3c/d3f/d65/d6f/f95 zero size 2024-03-16T10:45:20.228 INFO:tasks.workunit.client.1.smithi026.stdout:4/470: fdatasync d1/db/de/f70 0 2024-03-16T10:45:20.235 INFO:tasks.workunit.client.1.smithi026.stdout:7/497: link d6/d1e/l2b d6/de/d16/d88/l9c 0 2024-03-16T10:45:20.239 INFO:tasks.workunit.client.1.smithi026.stdout:4/471: getdents d1/db 0 2024-03-16T10:45:20.239 INFO:tasks.workunit.client.1.smithi026.stdout:4/472: write d1/f3 [4115121,89035] 0 2024-03-16T10:45:20.242 INFO:tasks.workunit.client.1.smithi026.stdout:4/473: write d1/db/d29/d44/d48/d8e/f93 [1046295,90728] 0 2024-03-16T10:45:20.242 INFO:tasks.workunit.client.1.smithi026.stdout:7/498: rename d6/d35/d3a/d5d/d79 to d6/d35/d3a/d3c/d3f/d9d 0 2024-03-16T10:45:20.244 INFO:tasks.workunit.client.1.smithi026.stdout:7/499: chown d6/d35/d3a/d3c/d3f/d65/d6f/c84 22572 1 2024-03-16T10:45:20.250 INFO:tasks.workunit.client.1.smithi026.stdout:7/500: mkdir d6/d35/d3a/d59/d9e 0 2024-03-16T10:45:20.254 INFO:tasks.workunit.client.1.smithi026.stdout:7/501: dread d6/d1e/f20 [0,4194304] 0 2024-03-16T10:45:20.257 INFO:tasks.workunit.client.1.smithi026.stdout:4/474: rename d1/db/de/c8b to d1/c9e 0 2024-03-16T10:45:20.265 INFO:tasks.workunit.client.1.smithi026.stdout:1/443: sync 2024-03-16T10:45:20.265 INFO:tasks.workunit.client.1.smithi026.stdout:7/502: creat d6/d35/d3a/d3c/d3f/d70/f9f x:0 0 0 2024-03-16T10:45:20.266 INFO:tasks.workunit.client.1.smithi026.stdout:4/475: mknod d1/db/de/c9f 0 2024-03-16T10:45:20.270 INFO:tasks.workunit.client.1.smithi026.stdout:7/503: write d6/d1e/f6a [1366673,36325] 0 2024-03-16T10:45:20.273 INFO:tasks.workunit.client.1.smithi026.stdout:4/476: dwrite d1/f52 [0,4194304] 0 2024-03-16T10:45:20.275 INFO:tasks.workunit.client.1.smithi026.stdout:4/477: truncate d1/db/f7e 458139 0 2024-03-16T10:45:20.276 INFO:tasks.workunit.client.1.smithi026.stdout:4/478: write d1/f18 [847137,95362] 0 2024-03-16T10:45:20.276 INFO:tasks.workunit.client.1.smithi026.stdout:4/479: dread - d1/f97 zero size 2024-03-16T10:45:20.277 INFO:tasks.workunit.client.1.smithi026.stdout:4/480: stat d1/f8 0 2024-03-16T10:45:20.278 INFO:tasks.workunit.client.1.smithi026.stdout:1/444: truncate d0/d17/d1f/f35 2194017 0 2024-03-16T10:45:20.280 INFO:tasks.workunit.client.1.smithi026.stdout:1/445: chown d0/d17/d1f/d63/d6a/d7c/d6b/f87 26293898 1 2024-03-16T10:45:20.281 INFO:tasks.workunit.client.1.smithi026.stdout:7/504: write d6/d35/d3a/d3c/d3f/d65/f89 [1109409,9235] 0 2024-03-16T10:45:20.286 INFO:tasks.workunit.client.1.smithi026.stdout:4/481: mknod d1/db/de/d4c/d59/d6a/ca0 0 2024-03-16T10:45:20.288 INFO:tasks.workunit.client.1.smithi026.stdout:4/482: fsync d1/db/de/f70 0 2024-03-16T10:45:20.290 INFO:tasks.workunit.client.1.smithi026.stdout:7/505: dwrite d6/d35/d3a/d3c/d3f/d9d/f97 [0,4194304] 0 2024-03-16T10:45:20.296 INFO:tasks.workunit.client.1.smithi026.stdout:1/446: dread d0/fd [0,4194304] 0 2024-03-16T10:45:20.300 INFO:tasks.workunit.client.1.smithi026.stdout:7/506: link d6/d35/d3a/d3c/d3f/d65/f89 d6/d35/d3a/d3c/d3f/fa0 0 2024-03-16T10:45:20.300 INFO:tasks.workunit.client.1.smithi026.stdout:4/483: creat d1/db/de/d4c/fa1 x:0 0 0 2024-03-16T10:45:20.302 INFO:tasks.workunit.client.1.smithi026.stdout:1/447: creat d0/d17/d1f/d20/d65/f8e x:0 0 0 2024-03-16T10:45:20.303 INFO:tasks.workunit.client.1.smithi026.stdout:7/507: mkdir d6/d35/d3a/d4e/da1 0 2024-03-16T10:45:20.304 INFO:tasks.workunit.client.1.smithi026.stdout:1/448: chown d0/d17/d1f/c15 3 1 2024-03-16T10:45:20.305 INFO:tasks.workunit.client.1.smithi026.stdout:4/484: getdents d1/db/de/d4c/d6b/d89/d9c 0 2024-03-16T10:45:20.306 INFO:tasks.workunit.client.1.smithi026.stdout:1/449: write d0/f1e [1253445,6695] 0 2024-03-16T10:45:20.309 INFO:tasks.workunit.client.1.smithi026.stdout:7/508: mknod d6/d35/d3a/d3c/d73/ca2 0 2024-03-16T10:45:20.312 INFO:tasks.workunit.client.1.smithi026.stdout:7/509: fdatasync d6/d2f/f34 0 2024-03-16T10:45:20.315 INFO:tasks.workunit.client.1.smithi026.stdout:7/510: truncate d6/d35/d3a/d3c/d3f/f4b 2190214 0 2024-03-16T10:45:20.316 INFO:tasks.workunit.client.1.smithi026.stdout:7/511: stat d6/d35/d3a/d3c/d3f/d9d/f97 0 2024-03-16T10:45:20.316 INFO:tasks.workunit.client.1.smithi026.stdout:7/512: fsync d6/de/d7d/f8a 0 2024-03-16T10:45:20.320 INFO:tasks.workunit.client.1.smithi026.stdout:4/485: getdents d1/db/de 0 2024-03-16T10:45:20.322 INFO:tasks.workunit.client.1.smithi026.stdout:4/486: mkdir d1/d85/da2 0 2024-03-16T10:45:20.326 INFO:tasks.workunit.client.1.smithi026.stdout:4/487: dread d1/db/f42 [0,4194304] 0 2024-03-16T10:45:20.329 INFO:tasks.workunit.client.1.smithi026.stdout:7/513: getdents d6/d35/d3a/d3c 0 2024-03-16T10:45:20.332 INFO:tasks.workunit.client.1.smithi026.stdout:7/514: rmdir d6/d35/d3a/d3c/d3f/d9d 39 2024-03-16T10:45:20.334 INFO:tasks.workunit.client.1.smithi026.stdout:9/500: sync 2024-03-16T10:45:20.334 INFO:tasks.workunit.client.1.smithi026.stdout:6/407: sync 2024-03-16T10:45:20.336 INFO:tasks.workunit.client.1.smithi026.stdout:7/515: creat d6/d35/d3a/d3c/d3f/d70/fa3 x:0 0 0 2024-03-16T10:45:20.339 INFO:tasks.workunit.client.1.smithi026.stdout:4/488: dread d1/db/f26 [0,4194304] 0 2024-03-16T10:45:20.340 INFO:tasks.workunit.client.1.smithi026.stdout:4/489: write d1/f6 [3023085,112180] 0 2024-03-16T10:45:20.340 INFO:tasks.workunit.client.1.smithi026.stdout:7/516: rename d6/d2f/f34 to d6/d35/d3a/d4e/d6c/fa4 0 2024-03-16T10:45:20.345 INFO:tasks.workunit.client.1.smithi026.stdout:4/490: symlink d1/db/de/d4c/d79/la3 0 2024-03-16T10:45:20.346 INFO:tasks.workunit.client.1.smithi026.stdout:4/491: truncate d1/db/f74 930915 0 2024-03-16T10:45:20.348 INFO:tasks.workunit.client.1.smithi026.stdout:9/501: rename d0/f9 to d0/fae 0 2024-03-16T10:45:20.350 INFO:tasks.workunit.client.1.smithi026.stdout:7/517: rename d6/d35/d3a/d5d to d6/d35/d3a/d4e/d6c/d85/da5 0 2024-03-16T10:45:20.353 INFO:tasks.workunit.client.1.smithi026.stdout:6/408: dwrite d0/d10/f1a [0,4194304] 0 2024-03-16T10:45:20.367 INFO:tasks.workunit.client.1.smithi026.stdout:4/492: rename d1/l21 to d1/la4 0 2024-03-16T10:45:20.368 INFO:tasks.workunit.client.1.smithi026.stdout:4/493: truncate d1/db/de/d4c/f8a 355642 0 2024-03-16T10:45:20.369 INFO:tasks.workunit.client.1.smithi026.stdout:4/494: read - d1/db/d29/d61/f7c zero size 2024-03-16T10:45:20.375 INFO:tasks.workunit.client.1.smithi026.stdout:4/495: dwrite d1/db/de/d4c/d81/f94 [0,4194304] 0 2024-03-16T10:45:20.376 INFO:tasks.workunit.client.1.smithi026.stdout:4/496: dread - d1/f97 zero size 2024-03-16T10:45:20.384 INFO:tasks.workunit.client.1.smithi026.stdout:4/497: dwrite d1/db/de/f58 [4194304,4194304] 0 2024-03-16T10:45:20.385 INFO:tasks.workunit.client.1.smithi026.stdout:4/498: chown d1/db/f74 1404 1 2024-03-16T10:45:20.393 INFO:tasks.workunit.client.1.smithi026.stdout:4/499: dwrite d1/db/f64 [0,4194304] 0 2024-03-16T10:45:20.395 INFO:tasks.workunit.client.1.smithi026.stdout:4/500: fdatasync d1/db/de/d4c/d59/d6a/f82 0 2024-03-16T10:45:20.396 INFO:tasks.workunit.client.1.smithi026.stdout:4/501: fsync d1/db/de/d4c/d59/d6a/f82 0 2024-03-16T10:45:20.398 INFO:tasks.workunit.client.1.smithi026.stdout:4/502: mkdir d1/db/de/d4c/d6b/d89/da5 0 2024-03-16T10:45:20.400 INFO:tasks.workunit.client.1.smithi026.stdout:4/503: fsync d1/db/d29/d61/f75 0 2024-03-16T10:45:20.402 INFO:tasks.workunit.client.1.smithi026.stdout:4/504: creat d1/db/de/d4c/d6b/d89/d9c/fa6 x:0 0 0 2024-03-16T10:45:20.406 INFO:tasks.workunit.client.1.smithi026.stdout:4/505: dread d1/f22 [0,4194304] 0 2024-03-16T10:45:20.414 INFO:tasks.workunit.client.1.smithi026.stdout:4/506: fdatasync d1/db/f64 0 2024-03-16T10:45:20.420 INFO:tasks.workunit.client.1.smithi026.stdout:4/507: dwrite d1/db/d29/f95 [0,4194304] 0 2024-03-16T10:45:20.421 INFO:tasks.workunit.client.1.smithi026.stdout:4/508: write d1/db/de/f2b [3375753,7475] 0 2024-03-16T10:45:20.437 INFO:tasks.workunit.client.1.smithi026.stdout:4/509: dwrite d1/f8 [0,4194304] 0 2024-03-16T10:45:20.439 INFO:tasks.workunit.client.1.smithi026.stdout:4/510: fsync d1/db/de/d4c/d59/f80 0 2024-03-16T10:45:20.445 INFO:tasks.workunit.client.1.smithi026.stdout:4/511: creat d1/db/d29/d44/d48/fa7 x:0 0 0 2024-03-16T10:45:20.447 INFO:tasks.workunit.client.1.smithi026.stdout:4/512: write d1/db/de/d4c/d59/f80 [876799,129261] 0 2024-03-16T10:45:20.452 INFO:tasks.workunit.client.1.smithi026.stdout:4/513: mkdir d1/d85/da2/da8 0 2024-03-16T10:45:20.453 INFO:tasks.workunit.client.1.smithi026.stdout:4/514: stat d1/db/de/d4c/d59/d6a/f9a 0 2024-03-16T10:45:20.459 INFO:tasks.workunit.client.1.smithi026.stdout:4/515: chown d1/la4 230927351 1 2024-03-16T10:45:20.461 INFO:tasks.workunit.client.1.smithi026.stdout:4/516: dread d1/db/f7e [0,4194304] 0 2024-03-16T10:45:20.466 INFO:tasks.workunit.client.1.smithi026.stdout:4/517: unlink d1/db/d29/f3e 0 2024-03-16T10:45:20.472 INFO:tasks.workunit.client.1.smithi026.stdout:4/518: mknod d1/db/de/d4c/d59/d6a/ca9 0 2024-03-16T10:45:20.546 INFO:tasks.workunit.client.1.smithi026.stdout:4/519: rename d1/db/d29 to d1/db/de/d4c/d79/daa 0 2024-03-16T10:45:20.587 INFO:tasks.workunit.client.1.smithi026.stdout:8/487: sync 2024-03-16T10:45:20.589 INFO:tasks.workunit.client.1.smithi026.stdout:4/520: fdatasync d1/db/de/f65 0 2024-03-16T10:45:20.599 INFO:tasks.workunit.client.1.smithi026.stdout:8/488: write d5/d7/d2c/f32 [2243647,129796] 0 2024-03-16T10:45:20.603 INFO:tasks.workunit.client.1.smithi026.stdout:4/521: dread d1/db/de/d4c/d79/daa/f5f [0,4194304] 0 2024-03-16T10:45:20.666 INFO:tasks.workunit.client.1.smithi026.stdout:8/489: write d5/d7/f11 [1162983,7325] 0 2024-03-16T10:45:20.669 INFO:tasks.workunit.client.1.smithi026.stdout:4/522: dwrite d1/db/de/d4c/d79/daa/f95 [0,4194304] 0 2024-03-16T10:45:20.671 INFO:tasks.workunit.client.1.smithi026.stdout:4/523: write d1/db/de/d4c/d79/daa/f43 [3780529,88796] 0 2024-03-16T10:45:20.677 INFO:tasks.workunit.client.1.smithi026.stdout:4/524: mknod d1/cab 0 2024-03-16T10:45:20.677 INFO:tasks.workunit.client.1.smithi026.stdout:8/490: link d5/c45 d5/d7/d2c/d36/c8b 0 2024-03-16T10:45:20.702 INFO:tasks.workunit.client.1.smithi026.stdout:2/458: sync 2024-03-16T10:45:20.702 INFO:tasks.workunit.client.1.smithi026.stdout:5/401: sync 2024-03-16T10:45:20.706 INFO:tasks.workunit.client.1.smithi026.stdout:8/491: mkdir d5/d7/d2c/d36/d8c 0 2024-03-16T10:45:20.708 INFO:tasks.workunit.client.1.smithi026.stdout:2/459: write d2/d29/d17/d1a/d2b/d2e/f3f [4852013,25743] 0 2024-03-16T10:45:20.708 INFO:tasks.workunit.client.1.smithi026.stdout:8/492: truncate d5/d23/f82 418400 0 2024-03-16T10:45:20.710 INFO:tasks.workunit.client.1.smithi026.stdout:5/402: rmdir d1/d1c/d2a/d31/d69/d51/d80 39 2024-03-16T10:45:20.710 INFO:tasks.workunit.client.1.smithi026.stdout:4/525: dwrite d1/db/de/d4c/d59/d6a/f9b [0,4194304] 0 2024-03-16T10:45:20.712 INFO:tasks.workunit.client.1.smithi026.stdout:5/403: chown d1/d6e/f78 107649942 1 2024-03-16T10:45:20.713 INFO:tasks.workunit.client.1.smithi026.stdout:4/526: chown d1/db/c33 239617509 1 2024-03-16T10:45:20.716 INFO:tasks.workunit.client.1.smithi026.stdout:2/460: dread d2/d29/d17/d1a/d2b/d4b/f5b [0,4194304] 0 2024-03-16T10:45:20.718 INFO:tasks.workunit.client.1.smithi026.stdout:8/493: dwrite d5/dc/f38 [4194304,4194304] 0 2024-03-16T10:45:20.720 INFO:tasks.workunit.client.1.smithi026.stdout:5/404: rmdir d1/d1c/d2a 39 2024-03-16T10:45:20.733 INFO:tasks.workunit.client.1.smithi026.stdout:2/461: rename d2/d29/d17/d1a/d2b/d2e/f33 to d2/d29/d17/d1a/f9e 0 2024-03-16T10:45:20.734 INFO:tasks.workunit.client.1.smithi026.stdout:2/462: fsync d2/d29/d51/f5f 0 2024-03-16T10:45:20.735 INFO:tasks.workunit.client.1.smithi026.stdout:2/463: dread - d2/d29/d17/d1a/d2b/f9a zero size 2024-03-16T10:45:20.736 INFO:tasks.workunit.client.1.smithi026.stdout:8/494: link d5/d23/f74 d5/d7/f8d 0 2024-03-16T10:45:20.736 INFO:tasks.workunit.client.1.smithi026.stdout:5/405: write d1/db/d27/f3e [2314033,1859] 0 2024-03-16T10:45:20.738 INFO:tasks.workunit.client.1.smithi026.stdout:4/527: getdents d1/db/de/d4c 0 2024-03-16T10:45:20.739 INFO:tasks.workunit.client.1.smithi026.stdout:4/528: chown d1/db/de/d4c/d79/daa/d44 10681473 1 2024-03-16T10:45:20.740 INFO:tasks.workunit.client.1.smithi026.stdout:4/529: fdatasync d1/db/de/f3c 0 2024-03-16T10:45:20.743 INFO:tasks.workunit.client.1.smithi026.stdout:2/464: dread d2/d29/d17/d1a/d2b/d4b/f5d [0,4194304] 0 2024-03-16T10:45:20.744 INFO:tasks.workunit.client.1.smithi026.stdout:8/495: unlink d5/dc/d40/c51 0 2024-03-16T10:45:20.748 INFO:tasks.workunit.client.1.smithi026.stdout:5/406: write d1/d1c/d2a/d31/d69/d51/d80/f86 [652634,118986] 0 2024-03-16T10:45:20.750 INFO:tasks.workunit.client.1.smithi026.stdout:5/407: dread - d1/d1c/d2a/d31/d69/d51/f5e zero size 2024-03-16T10:45:20.751 INFO:tasks.workunit.client.1.smithi026.stdout:5/408: write d1/db/f7d [849133,118938] 0 2024-03-16T10:45:20.755 INFO:tasks.workunit.client.1.smithi026.stdout:4/530: creat d1/fac x:0 0 0 2024-03-16T10:45:20.758 INFO:tasks.workunit.client.1.smithi026.stdout:2/465: dwrite d2/d29/d17/d1a/f8e [0,4194304] 0 2024-03-16T10:45:20.758 INFO:tasks.workunit.client.1.smithi026.stdout:4/531: readlink d1/db/de/d4c/d79/daa/d44/d48/l7b 0 2024-03-16T10:45:20.759 INFO:tasks.workunit.client.1.smithi026.stdout:5/409: symlink d1/d1c/l87 0 2024-03-16T10:45:20.765 INFO:tasks.workunit.client.1.smithi026.stdout:2/466: symlink d2/d29/d51/d72/l9f 0 2024-03-16T10:45:20.769 INFO:tasks.workunit.client.1.smithi026.stdout:4/532: creat d1/db/de/d4c/d6b/fad x:0 0 0 2024-03-16T10:45:20.770 INFO:tasks.workunit.client.1.smithi026.stdout:5/410: creat d1/d1c/d4c/f88 x:0 0 0 2024-03-16T10:45:20.773 INFO:tasks.workunit.client.1.smithi026.stdout:8/496: rename d5/d7/l17 to d5/d7/d2c/d36/l8e 0 2024-03-16T10:45:20.786 INFO:tasks.workunit.client.1.smithi026.stdout:2/467: write d2/d29/d17/d1a/d2b/d4e/d67/f78 [748247,130027] 0 2024-03-16T10:45:20.789 INFO:tasks.workunit.client.1.smithi026.stdout:4/533: getdents d1/db/de/d4c/d79/daa 0 2024-03-16T10:45:20.791 INFO:tasks.workunit.client.1.smithi026.stdout:4/534: read d1/db/de/f2b [1600954,188] 0 2024-03-16T10:45:20.792 INFO:tasks.workunit.client.1.smithi026.stdout:8/497: dread d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:20.794 INFO:tasks.workunit.client.1.smithi026.stdout:5/411: dwrite d1/db/f28 [0,4194304] 0 2024-03-16T10:45:20.795 INFO:tasks.workunit.client.1.smithi026.stdout:5/412: fsync d1/db/f38 0 2024-03-16T10:45:20.796 INFO:tasks.workunit.client.1.smithi026.stdout:0/529: sync 2024-03-16T10:45:20.796 INFO:tasks.workunit.client.1.smithi026.stdout:3/512: sync 2024-03-16T10:45:20.798 INFO:tasks.workunit.client.1.smithi026.stdout:2/468: dwrite d2/d29/d17/f23 [0,4194304] 0 2024-03-16T10:45:20.799 INFO:tasks.workunit.client.1.smithi026.stdout:5/413: write d1/db/d1f/d61/d65/f7c [274055,122072] 0 2024-03-16T10:45:20.800 INFO:tasks.workunit.client.1.smithi026.stdout:0/530: fdatasync d1/d18/d1a/d2b/d4f/da7/fb2 0 2024-03-16T10:45:20.804 INFO:tasks.workunit.client.1.smithi026.stdout:0/531: truncate d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f9c 609529 0 2024-03-16T10:45:20.806 INFO:tasks.workunit.client.1.smithi026.stdout:0/532: write d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f9c [1166891,39101] 0 2024-03-16T10:45:20.812 INFO:tasks.workunit.client.1.smithi026.stdout:0/533: dwrite d1/d12/d2f/d48/d55/d60/f70 [0,4194304] 0 2024-03-16T10:45:20.851 INFO:tasks.workunit.client.1.smithi026.stdout:2/469: mknod d2/d29/d36/ca0 0 2024-03-16T10:45:20.861 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:20 smithi026 ceph-mon[37041]: pgmap v5: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail 2024-03-16T10:45:20.861 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:20 smithi026 ceph-mon[37041]: mgrmap e30: smithi002.zevrtm(active, since 4s) 2024-03-16T10:45:20.863 INFO:tasks.workunit.client.1.smithi026.stdout:5/414: creat d1/d1c/d2a/f89 x:0 0 0 2024-03-16T10:45:20.868 INFO:tasks.workunit.client.1.smithi026.stdout:2/470: mknod d2/d29/d68/ca1 0 2024-03-16T10:45:20.872 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:20 smithi002 ceph-mon[29248]: pgmap v5: 65 pgs: 65 active+clean; 2.9 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail 2024-03-16T10:45:20.872 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:20 smithi002 ceph-mon[29248]: mgrmap e30: smithi002.zevrtm(active, since 4s) 2024-03-16T10:45:20.872 INFO:tasks.workunit.client.1.smithi026.stdout:5/415: chown d1/d1c/d2a/d31/l5f 446763 1 2024-03-16T10:45:20.873 INFO:tasks.workunit.client.1.smithi026.stdout:0/534: getdents d1/d12/d2f/d48/d55 0 2024-03-16T10:45:20.874 INFO:tasks.workunit.client.1.smithi026.stdout:3/513: dwrite d4/dc/dd/d47/d53/f5c [0,4194304] 0 2024-03-16T10:45:20.875 INFO:tasks.workunit.client.1.smithi026.stdout:5/416: fdatasync d1/d1c/d2a/d55/f58 0 2024-03-16T10:45:20.877 INFO:tasks.workunit.client.1.smithi026.stdout:2/471: creat d2/d29/d17/d38/d64/fa2 x:0 0 0 2024-03-16T10:45:20.880 INFO:tasks.workunit.client.1.smithi026.stdout:0/535: creat d1/d12/d2f/d48/d55/d60/d9b/d11/fb7 x:0 0 0 2024-03-16T10:45:20.882 INFO:tasks.workunit.client.1.smithi026.stdout:3/514: truncate d4/dc/d31/f34 1216024 0 2024-03-16T10:45:20.884 INFO:tasks.workunit.client.1.smithi026.stdout:5/417: mkdir d1/db/d8a 0 2024-03-16T10:45:20.890 INFO:tasks.workunit.client.1.smithi026.stdout:0/536: fdatasync d1/d18/fb3 0 2024-03-16T10:45:20.893 INFO:tasks.workunit.client.1.smithi026.stdout:5/418: dread d1/db/d27/f23 [0,4194304] 0 2024-03-16T10:45:20.894 INFO:tasks.workunit.client.1.smithi026.stdout:5/419: fsync d1/d6e/f70 0 2024-03-16T10:45:20.894 INFO:tasks.workunit.client.1.smithi026.stdout:0/537: getdents d1/d18/d1a/d2b/d4f/d96/d85 0 2024-03-16T10:45:20.898 INFO:tasks.workunit.client.1.smithi026.stdout:5/420: mknod d1/db/d8a/c8b 0 2024-03-16T10:45:20.901 INFO:tasks.workunit.client.1.smithi026.stdout:2/472: truncate d2/f35 1839322 0 2024-03-16T10:45:20.902 INFO:tasks.workunit.client.1.smithi026.stdout:5/421: creat d1/d1c/d2a/d31/d69/f8c x:0 0 0 2024-03-16T10:45:20.904 INFO:tasks.workunit.client.1.smithi026.stdout:3/515: write d4/dc/d31/f34 [1978739,91604] 0 2024-03-16T10:45:20.910 INFO:tasks.workunit.client.1.smithi026.stdout:5/422: dwrite d1/d1c/f50 [0,4194304] 0 2024-03-16T10:45:20.910 INFO:tasks.workunit.client.1.smithi026.stdout:0/538: getdents d1 0 2024-03-16T10:45:20.911 INFO:tasks.workunit.client.1.smithi026.stdout:5/423: readlink l0 0 2024-03-16T10:45:20.912 INFO:tasks.workunit.client.1.smithi026.stdout:2/473: creat d2/d29/d36/d98/fa3 x:0 0 0 2024-03-16T10:45:20.913 INFO:tasks.workunit.client.1.smithi026.stdout:5/424: read - d1/d1c/d2a/d31/d69/d51/f5d zero size 2024-03-16T10:45:20.914 INFO:tasks.workunit.client.1.smithi026.stdout:2/474: chown d2/d29/d17/d1a/d2b/d4e/d67 46974233 1 2024-03-16T10:45:20.919 INFO:tasks.workunit.client.1.smithi026.stdout:3/516: truncate d4/f41 525123 0 2024-03-16T10:45:20.923 INFO:tasks.workunit.client.1.smithi026.stdout:0/539: creat d1/d12/d2f/d48/da5/fb8 x:0 0 0 2024-03-16T10:45:20.925 INFO:tasks.workunit.client.1.smithi026.stdout:2/475: mkdir d2/d29/da4 0 2024-03-16T10:45:20.928 INFO:tasks.workunit.client.1.smithi026.stdout:3/517: creat d4/dc/d73/d83/d57/fab x:0 0 0 2024-03-16T10:45:20.929 INFO:tasks.workunit.client.1.smithi026.stdout:0/540: read d1/d18/d1a/f2c [842876,67476] 0 2024-03-16T10:45:20.930 INFO:tasks.workunit.client.1.smithi026.stdout:2/476: creat d2/d29/d3b/fa5 x:0 0 0 2024-03-16T10:45:20.931 INFO:tasks.workunit.client.1.smithi026.stdout:3/518: chown d4/c33 10979 1 2024-03-16T10:45:20.932 INFO:tasks.workunit.client.1.smithi026.stdout:2/477: write d2/d29/d17/d38/f7d [357845,36255] 0 2024-03-16T10:45:20.933 INFO:tasks.workunit.client.1.smithi026.stdout:3/519: write d4/dc/dd/f24 [379591,97388] 0 2024-03-16T10:45:20.937 INFO:tasks.workunit.client.1.smithi026.stdout:2/478: symlink d2/d29/d3b/la6 0 2024-03-16T10:45:20.939 INFO:tasks.workunit.client.1.smithi026.stdout:0/541: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f9c [0,4194304] 0 2024-03-16T10:45:20.939 INFO:tasks.workunit.client.1.smithi026.stdout:3/520: mknod d4/dc/d73/d83/cac 0 2024-03-16T10:45:20.943 INFO:tasks.workunit.client.1.smithi026.stdout:0/542: symlink d1/d12/d2f/d48/d55/d60/lb9 0 2024-03-16T10:45:20.944 INFO:tasks.workunit.client.1.smithi026.stdout:3/521: mknod d4/dc/dd/d3d/d94/cad 0 2024-03-16T10:45:20.945 INFO:tasks.workunit.client.1.smithi026.stdout:2/479: fdatasync d2/d29/d17/d1a/d2b/d4b/f5c 0 2024-03-16T10:45:20.947 INFO:tasks.workunit.client.1.smithi026.stdout:2/480: write d2/d29/d17/d38/f81 [400119,47746] 0 2024-03-16T10:45:20.951 INFO:tasks.workunit.client.1.smithi026.stdout:0/543: mkdir d1/d12/d2f/dba 0 2024-03-16T10:45:20.952 INFO:tasks.workunit.client.1.smithi026.stdout:0/544: dread - d1/d18/d1a/d2b/d4f/da7/fb2 zero size 2024-03-16T10:45:20.953 INFO:tasks.workunit.client.1.smithi026.stdout:3/522: mkdir d4/d89/d97/d7d/dae 0 2024-03-16T10:45:20.957 INFO:tasks.workunit.client.1.smithi026.stdout:0/545: creat d1/d12/d2f/d48/d55/d60/d9b/d57/fbb x:0 0 0 2024-03-16T10:45:20.958 INFO:tasks.workunit.client.1.smithi026.stdout:3/523: chown d4/dc/dd/d60/d62/d4f/d5a/da6/ca7 1189373 1 2024-03-16T10:45:21.005 INFO:tasks.workunit.client.1.smithi026.stdout:1/450: sync 2024-03-16T10:45:21.005 INFO:tasks.workunit.client.1.smithi026.stdout:9/502: sync 2024-03-16T10:45:21.005 INFO:tasks.workunit.client.1.smithi026.stdout:6/409: sync 2024-03-16T10:45:21.006 INFO:tasks.workunit.client.1.smithi026.stdout:7/518: sync 2024-03-16T10:45:21.010 INFO:tasks.workunit.client.1.smithi026.stdout:0/546: truncate d1/d18/d1a/d2b/d4f/da7/fb2 759324 0 2024-03-16T10:45:21.011 INFO:tasks.workunit.client.1.smithi026.stdout:3/524: fdatasync d4/dc/dd/d60/d62/f52 0 2024-03-16T10:45:21.013 INFO:tasks.workunit.client.1.smithi026.stdout:9/503: truncate d0/d5f/d7f/f9d 664677 0 2024-03-16T10:45:21.019 INFO:tasks.workunit.client.1.smithi026.stdout:9/504: dwrite d0/da/f1b [4194304,4194304] 0 2024-03-16T10:45:21.039 INFO:tasks.workunit.client.1.smithi026.stdout:1/451: fsync d0/d17/d1f/d20/f43 0 2024-03-16T10:45:21.041 INFO:tasks.workunit.client.1.smithi026.stdout:0/547: creat d1/d12/d2f/d48/d55/d60/d9b/d51/d88/fbc x:0 0 0 2024-03-16T10:45:21.043 INFO:tasks.workunit.client.1.smithi026.stdout:6/410: dread d0/d10/ff [0,4194304] 0 2024-03-16T10:45:21.044 INFO:tasks.workunit.client.1.smithi026.stdout:7/519: rename d6/d35/d3a/d3c/d3f/d65/d6f to d6/d1e/da6 0 2024-03-16T10:45:21.044 INFO:tasks.workunit.client.1.smithi026.stdout:0/548: write d1/d12/d2f/d48/d55/d60/d9b/d51/d88/fbc [828993,108370] 0 2024-03-16T10:45:21.046 INFO:tasks.workunit.client.1.smithi026.stdout:3/525: unlink d4/dc/dd/d60/d62/f8e 0 2024-03-16T10:45:21.048 INFO:tasks.workunit.client.1.smithi026.stdout:0/549: write d1/d12/d2f/d48/d55/d60/fa0 [1598624,110010] 0 2024-03-16T10:45:21.051 INFO:tasks.workunit.client.1.smithi026.stdout:0/550: write d1/d18/d1a/d2b/d4f/f80 [4080691,105509] 0 2024-03-16T10:45:21.051 INFO:tasks.workunit.client.1.smithi026.stdout:7/520: write d6/de/f74 [2664,37075] 0 2024-03-16T10:45:21.054 INFO:tasks.workunit.client.1.smithi026.stdout:0/551: write d1/d18/d1a/d2b/f4e [693629,75719] 0 2024-03-16T10:45:21.056 INFO:tasks.workunit.client.1.smithi026.stdout:6/411: rename d0/d28/l3f to d0/d41/d71/l93 0 2024-03-16T10:45:21.059 INFO:tasks.workunit.client.1.smithi026.stdout:3/526: dwrite d4/dc/d31/d48/d69/d78/fa2 [0,4194304] 0 2024-03-16T10:45:21.065 INFO:tasks.workunit.client.1.smithi026.stdout:7/521: dread d6/d35/d3a/d3c/d3f/f4b [0,4194304] 0 2024-03-16T10:45:21.067 INFO:tasks.workunit.client.1.smithi026.stdout:1/452: rename d0/f5a to d0/f8f 0 2024-03-16T10:45:21.070 INFO:tasks.workunit.client.1.smithi026.stdout:3/527: symlink d4/dc/d73/d83/d57/laf 0 2024-03-16T10:45:21.072 INFO:tasks.workunit.client.1.smithi026.stdout:1/453: dread d0/d17/d1f/d38/f4e [0,4194304] 0 2024-03-16T10:45:21.073 INFO:tasks.workunit.client.1.smithi026.stdout:0/552: rename d1/d12/d2f/d48/d55/d60/d9b/ca2 to d1/d18/d1a/d2b/d4f/d96/cbd 0 2024-03-16T10:45:21.076 INFO:tasks.workunit.client.1.smithi026.stdout:3/528: mknod d4/dc/d73/d83/cb0 0 2024-03-16T10:45:21.080 INFO:tasks.workunit.client.1.smithi026.stdout:7/522: creat d6/d35/d3a/d3c/d3f/d9d/fa7 x:0 0 0 2024-03-16T10:45:21.085 INFO:tasks.workunit.client.1.smithi026.stdout:1/454: dread d0/d17/d1f/f3f [0,4194304] 0 2024-03-16T10:45:21.086 INFO:tasks.workunit.client.1.smithi026.stdout:1/455: fsync d0/d17/d1f/d38/f4e 0 2024-03-16T10:45:21.087 INFO:tasks.workunit.client.1.smithi026.stdout:1/456: chown d0/d17/d1f/d63/d6a/d7c/f3e 1112 1 2024-03-16T10:45:21.088 INFO:tasks.workunit.client.1.smithi026.stdout:1/457: fdatasync d0/f74 0 2024-03-16T10:45:21.090 INFO:tasks.workunit.client.1.smithi026.stdout:1/458: write d0/d17/d1f/d20/f43 [2765024,73737] 0 2024-03-16T10:45:21.095 INFO:tasks.workunit.client.1.smithi026.stdout:9/505: dwrite d0/da/d13/d6e/f74 [0,4194304] 0 2024-03-16T10:45:21.101 INFO:tasks.workunit.client.1.smithi026.stdout:1/459: rename d0/d17/d1f/d63/d6a/d7c/f3e to d0/d17/d1f/f90 0 2024-03-16T10:45:21.105 INFO:tasks.workunit.client.1.smithi026.stdout:3/529: rmdir d4 39 2024-03-16T10:45:21.107 INFO:tasks.workunit.client.1.smithi026.stdout:6/412: dwrite d0/d10/ff [0,4194304] 0 2024-03-16T10:45:21.107 INFO:tasks.workunit.client.1.smithi026.stdout:1/460: symlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/l91 0 2024-03-16T10:45:21.109 INFO:tasks.workunit.client.1.smithi026.stdout:1/461: stat d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62 0 2024-03-16T10:45:21.111 INFO:tasks.workunit.client.1.smithi026.stdout:1/462: chown d0/d17/d1f/d63/d6a/d7c/c5d 50189732 1 2024-03-16T10:45:21.112 INFO:tasks.workunit.client.1.smithi026.stdout:0/553: truncate d1/d18/d1a/d2b/f86 959341 0 2024-03-16T10:45:21.114 INFO:tasks.workunit.client.1.smithi026.stdout:6/413: getdents d0/d10/d79/d91 0 2024-03-16T10:45:21.116 INFO:tasks.workunit.client.1.smithi026.stdout:1/463: truncate d0/d17/d1f/d63/d6a/d7c/d6b/f87 878719 0 2024-03-16T10:45:21.116 INFO:tasks.workunit.client.1.smithi026.stdout:3/530: truncate d4/dc/d73/d83/d57/f9b 7528824 0 2024-03-16T10:45:21.117 INFO:tasks.workunit.client.1.smithi026.stdout:9/506: write d0/da/d16/d5d/d72/d62/f80 [445678,129039] 0 2024-03-16T10:45:21.119 INFO:tasks.workunit.client.1.smithi026.stdout:1/464: rename d0 to d0/d17/d1f/d63/d6a/d7c/d3b/d64/d92 22 2024-03-16T10:45:21.121 INFO:tasks.workunit.client.1.smithi026.stdout:0/554: getdents d1/d18/d1a/d2b/d4f/d96/d85 0 2024-03-16T10:45:21.125 INFO:tasks.workunit.client.1.smithi026.stdout:3/531: truncate d4/dc/d73/d83/d7c/f5b 318762 0 2024-03-16T10:45:21.126 INFO:tasks.workunit.client.1.smithi026.stdout:0/555: dwrite d1/d12/d2f/d48/d55/d60/fa0 [0,4194304] 0 2024-03-16T10:45:21.127 INFO:tasks.workunit.client.1.smithi026.stdout:1/465: mknod d0/d17/d1f/d63/d6a/c93 0 2024-03-16T10:45:21.139 INFO:tasks.workunit.client.1.smithi026.stdout:6/414: write d0/d10/f38 [761218,64401] 0 2024-03-16T10:45:21.142 INFO:tasks.workunit.client.1.smithi026.stdout:6/415: read d0/d28/f37 [5926477,63449] 0 2024-03-16T10:45:21.143 INFO:tasks.workunit.client.1.smithi026.stdout:6/416: write d0/d28/f37 [2021051,52843] 0 2024-03-16T10:45:21.146 INFO:tasks.workunit.client.1.smithi026.stdout:9/507: getdents d0/da/d16 0 2024-03-16T10:45:21.148 INFO:tasks.workunit.client.1.smithi026.stdout:3/532: chown d4/dc/d73/d83/d7c/f5b 0 1 2024-03-16T10:45:21.150 INFO:tasks.workunit.client.1.smithi026.stdout:6/417: rmdir d0/d5b 39 2024-03-16T10:45:21.155 INFO:tasks.workunit.client.1.smithi026.stdout:0/556: getdents d1 0 2024-03-16T10:45:21.156 INFO:tasks.workunit.client.1.smithi026.stdout:3/533: truncate d4/dc/dd/d3d/f4a 3725696 0 2024-03-16T10:45:21.157 INFO:tasks.workunit.client.1.smithi026.stdout:0/557: dread - d1/d12/d2f/d48/d55/d60/d9b/d11/fb7 zero size 2024-03-16T10:45:21.159 INFO:tasks.workunit.client.1.smithi026.stdout:6/418: rename d0/d28/f4d to d0/d10/d79/d91/f94 0 2024-03-16T10:45:21.161 INFO:tasks.workunit.client.1.smithi026.stdout:3/534: mkdir d4/dc/dd/d47/d53/db1 0 2024-03-16T10:45:21.163 INFO:tasks.workunit.client.1.smithi026.stdout:4/535: sync 2024-03-16T10:45:21.169 INFO:tasks.workunit.client.1.smithi026.stdout:3/535: rmdir d4/dc/d31/d48/d7f 39 2024-03-16T10:45:21.171 INFO:tasks.workunit.client.1.smithi026.stdout:4/536: dwrite d1/f97 [0,4194304] 0 2024-03-16T10:45:21.171 INFO:tasks.workunit.client.1.smithi026.stdout:3/536: dread - d4/d89/d97/f99 zero size 2024-03-16T10:45:21.171 INFO:tasks.workunit.client.1.smithi026.stdout:6/419: creat d0/d5b/f95 x:0 0 0 2024-03-16T10:45:21.172 INFO:tasks.workunit.client.1.smithi026.stdout:4/537: fdatasync d1/db/de/f3c 0 2024-03-16T10:45:21.173 INFO:tasks.workunit.client.1.smithi026.stdout:6/420: truncate d0/d62/f85 738305 0 2024-03-16T10:45:21.175 INFO:tasks.workunit.client.1.smithi026.stdout:9/508: getdents d0/d63/d6b 0 2024-03-16T10:45:21.176 INFO:tasks.workunit.client.1.smithi026.stdout:9/509: chown d0/da/d16/d17/d44/c30 950 1 2024-03-16T10:45:21.181 INFO:tasks.workunit.client.1.smithi026.stdout:4/538: write d1/db/de/d4c/d79/daa/d44/d48/f73 [2490280,7597] 0 2024-03-16T10:45:21.182 INFO:tasks.workunit.client.1.smithi026.stdout:6/421: creat d0/d41/d4f/f96 x:0 0 0 2024-03-16T10:45:21.185 INFO:tasks.workunit.client.1.smithi026.stdout:0/558: rename d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f8f to d1/d12/d2f/d48/d55/d60/d9b/fbe 0 2024-03-16T10:45:21.186 INFO:tasks.workunit.client.1.smithi026.stdout:9/510: symlink d0/da/d16/d5d/d72/laf 0 2024-03-16T10:45:21.187 INFO:tasks.workunit.client.1.smithi026.stdout:9/511: readlink d0/da/d14/l37 0 2024-03-16T10:45:21.188 INFO:tasks.workunit.client.1.smithi026.stdout:9/512: chown d0/d63/d6b/f77 131625 1 2024-03-16T10:45:21.189 INFO:tasks.workunit.client.1.smithi026.stdout:4/539: creat d1/db/de/d4c/d6b/fae x:0 0 0 2024-03-16T10:45:21.194 INFO:tasks.workunit.client.1.smithi026.stdout:9/513: unlink d0/da/d16/d5d/d72/d62/f9f 0 2024-03-16T10:45:21.194 INFO:tasks.workunit.client.1.smithi026.stdout:3/537: link d4/dc/dd/d47/d53/c55 d4/d89/cb2 0 2024-03-16T10:45:21.198 INFO:tasks.workunit.client.1.smithi026.stdout:4/540: mkdir d1/db/de/d4c/d79/daf 0 2024-03-16T10:45:21.198 INFO:tasks.workunit.client.1.smithi026.stdout:4/541: chown d1/db/de/d4c/d81/f94 3203503 1 2024-03-16T10:45:21.202 INFO:tasks.workunit.client.1.smithi026.stdout:0/559: mknod d1/d18/d1a/d2b/d4f/d96/d85/cbf 0 2024-03-16T10:45:21.203 INFO:tasks.workunit.client.1.smithi026.stdout:0/560: chown d1/d18/d1a/d2b/f4e 41 1 2024-03-16T10:45:21.205 INFO:tasks.workunit.client.1.smithi026.stdout:0/561: dread d1/d12/d2f/d48/d55/d60/d9b/f6f [0,4194304] 0 2024-03-16T10:45:21.206 INFO:tasks.workunit.client.1.smithi026.stdout:3/538: mkdir d4/dc/d31/d48/d69/d78/db3 0 2024-03-16T10:45:21.209 INFO:tasks.workunit.client.1.smithi026.stdout:9/514: mknod d0/cb0 0 2024-03-16T10:45:21.211 INFO:tasks.workunit.client.1.smithi026.stdout:0/562: creat d1/d12/d2f/d48/d55/d60/d9b/d11/fc0 x:0 0 0 2024-03-16T10:45:21.212 INFO:tasks.workunit.client.1.smithi026.stdout:9/515: write d0/da/d16/d17/fa9 [36981,81215] 0 2024-03-16T10:45:21.212 INFO:tasks.workunit.client.1.smithi026.stdout:4/542: link d1/fac d1/d85/da2/fb0 0 2024-03-16T10:45:21.213 INFO:tasks.workunit.client.1.smithi026.stdout:6/422: dwrite d0/d10/d79/d91/f94 [0,4194304] 0 2024-03-16T10:45:21.217 INFO:tasks.workunit.client.1.smithi026.stdout:9/516: write d0/d63/d6b/f77 [1481527,58565] 0 2024-03-16T10:45:21.219 INFO:tasks.workunit.client.1.smithi026.stdout:9/517: write d0/d5f/d7f/f9c [5950555,87935] 0 2024-03-16T10:45:21.222 INFO:tasks.workunit.client.1.smithi026.stdout:0/563: dwrite d1/d12/d2f/d48/d55/d60/f70 [0,4194304] 0 2024-03-16T10:45:21.223 INFO:tasks.workunit.client.1.smithi026.stdout:6/423: chown d0/d10/f67 1644554107 1 2024-03-16T10:45:21.229 INFO:tasks.workunit.client.1.smithi026.stdout:3/539: rename d4/dc/dd/d3d/f79 to d4/dc/dd/fb4 0 2024-03-16T10:45:21.234 INFO:tasks.workunit.client.1.smithi026.stdout:9/518: rmdir d0/da/d16 39 2024-03-16T10:45:21.239 INFO:tasks.workunit.client.1.smithi026.stdout:4/543: mkdir d1/db/de/d4c/d6b/d7a/db1 0 2024-03-16T10:45:21.240 INFO:tasks.workunit.client.1.smithi026.stdout:4/544: stat d1/db/de/d4c/d6b/fae 0 2024-03-16T10:45:21.244 INFO:tasks.workunit.client.1.smithi026.stdout:0/564: write d1/d18/d61/f62 [5227706,79653] 0 2024-03-16T10:45:21.244 INFO:tasks.workunit.client.1.smithi026.stdout:3/540: creat d4/dc/d31/d48/d69/d78/db3/fb5 x:0 0 0 2024-03-16T10:45:21.253 INFO:tasks.workunit.client.1.smithi026.stdout:9/519: dread d0/da/d16/d17/d44/f2b [0,4194304] 0 2024-03-16T10:45:21.254 INFO:tasks.workunit.client.1.smithi026.stdout:9/520: fsync d0/da/f1b 0 2024-03-16T10:45:21.254 INFO:tasks.workunit.client.1.smithi026.stdout:0/565: rename d1/d18/d1a/l49 to d1/d12/d2f/d48/lc1 0 2024-03-16T10:45:21.256 INFO:tasks.workunit.client.1.smithi026.stdout:0/566: read d1/d18/d61/f62 [2439465,52801] 0 2024-03-16T10:45:21.258 INFO:tasks.workunit.client.1.smithi026.stdout:0/567: write d1/d12/d2f/d48/d55/d60/d9b/d51/d88/fbc [1620033,76052] 0 2024-03-16T10:45:21.259 INFO:tasks.workunit.client.1.smithi026.stdout:3/541: write d4/dc/f13 [5125812,86399] 0 2024-03-16T10:45:21.262 INFO:tasks.workunit.client.1.smithi026.stdout:9/521: chown d0/da/f7a 3583 1 2024-03-16T10:45:21.268 INFO:tasks.workunit.client.1.smithi026.stdout:3/542: mknod d4/d89/d97/d7d/cb6 0 2024-03-16T10:45:21.269 INFO:tasks.workunit.client.1.smithi026.stdout:9/522: mkdir d0/da/d16/d17/db1 0 2024-03-16T10:45:21.275 INFO:tasks.workunit.client.1.smithi026.stdout:0/568: creat d1/d18/fc2 x:0 0 0 2024-03-16T10:45:21.277 INFO:tasks.workunit.client.1.smithi026.stdout:9/523: creat d0/da/fb2 x:0 0 0 2024-03-16T10:45:21.278 INFO:tasks.workunit.client.1.smithi026.stdout:0/569: creat d1/d18/d1a/d2b/d4f/da7/fc3 x:0 0 0 2024-03-16T10:45:21.279 INFO:tasks.workunit.client.1.smithi026.stdout:0/570: fsync d1/d12/d2f/d48/d55/d60/fa0 0 2024-03-16T10:45:21.289 INFO:tasks.workunit.client.1.smithi026.stdout:9/524: write d0/da/d13/f55 [821170,12299] 0 2024-03-16T10:45:21.290 INFO:tasks.workunit.client.1.smithi026.stdout:3/543: dwrite d4/d89/d97/d7d/f88 [0,4194304] 0 2024-03-16T10:45:21.294 INFO:tasks.workunit.client.1.smithi026.stdout:9/525: mknod d0/da/d13/d6e/cb3 0 2024-03-16T10:45:21.295 INFO:tasks.workunit.client.1.smithi026.stdout:3/544: creat d4/d89/d97/fb7 x:0 0 0 2024-03-16T10:45:21.296 INFO:tasks.workunit.client.1.smithi026.stdout:0/571: dwrite d1/d18/d61/f6b [0,4194304] 0 2024-03-16T10:45:21.296 INFO:tasks.workunit.client.1.smithi026.stdout:9/526: dread - d0/da/d14/f6a zero size 2024-03-16T10:45:21.302 INFO:tasks.workunit.client.1.smithi026.stdout:0/572: mknod d1/d18/d1a/d2b/d4f/da7/cc4 0 2024-03-16T10:45:21.303 INFO:tasks.workunit.client.1.smithi026.stdout:3/545: symlink d4/dc/d31/d48/d69/d78/db3/lb8 0 2024-03-16T10:45:21.308 INFO:tasks.workunit.client.1.smithi026.stdout:0/573: mknod d1/d12/d2f/d48/d55/d60/d9b/cc5 0 2024-03-16T10:45:21.315 INFO:tasks.workunit.client.1.smithi026.stdout:3/546: dread d4/dc/dd/d3d/f39 [0,4194304] 0 2024-03-16T10:45:21.316 INFO:tasks.workunit.client.1.smithi026.stdout:9/527: dread d0/da/d13/f31 [0,4194304] 0 2024-03-16T10:45:21.317 INFO:tasks.workunit.client.1.smithi026.stdout:0/574: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/f5d [0,4194304] 0 2024-03-16T10:45:21.318 INFO:tasks.workunit.client.1.smithi026.stdout:9/528: chown d0/da/d16/d5d/c8b 7754390 1 2024-03-16T10:45:21.319 INFO:tasks.workunit.client.1.smithi026.stdout:3/547: write d4/dc/d31/d48/d69/d78/db3/fb5 [957833,8146] 0 2024-03-16T10:45:21.322 INFO:tasks.workunit.client.1.smithi026.stdout:0/575: symlink d1/d12/d2f/d48/d55/d60/d9b/d51/lc6 0 2024-03-16T10:45:21.322 INFO:tasks.workunit.client.1.smithi026.stdout:3/548: write d4/dc/dd/d60/f64 [1660021,55821] 0 2024-03-16T10:45:21.333 INFO:tasks.workunit.client.1.smithi026.stdout:0/576: creat d1/d12/d2f/dba/fc7 x:0 0 0 2024-03-16T10:45:21.334 INFO:tasks.workunit.client.1.smithi026.stdout:0/577: dread - d1/d18/d1a/f94 zero size 2024-03-16T10:45:21.340 INFO:tasks.workunit.client.1.smithi026.stdout:0/578: fsync d1/d18/d61/f62 0 2024-03-16T10:45:21.342 INFO:tasks.workunit.client.1.smithi026.stdout:9/529: dwrite d0/da/d16/f3f [0,4194304] 0 2024-03-16T10:45:21.343 INFO:tasks.workunit.client.1.smithi026.stdout:0/579: readlink d1/d18/d1a/d2b/d4f/l6e 0 2024-03-16T10:45:21.344 INFO:tasks.workunit.client.1.smithi026.stdout:3/549: dwrite d4/dc/dd/d3d/f39 [0,4194304] 0 2024-03-16T10:45:21.346 INFO:tasks.workunit.client.1.smithi026.stdout:3/550: fsync d4/dc/dd/f24 0 2024-03-16T10:45:21.347 INFO:tasks.workunit.client.1.smithi026.stdout:9/530: truncate d0/da/d16/d17/fa9 543802 0 2024-03-16T10:45:21.348 INFO:tasks.workunit.client.1.smithi026.stdout:0/580: truncate d1/d18/d1a/f94 506765 0 2024-03-16T10:45:21.352 INFO:tasks.workunit.client.1.smithi026.stdout:3/551: mkdir d4/dc/dd/d60/d62/d4f/d5a/da6/db9 0 2024-03-16T10:45:21.353 INFO:tasks.workunit.client.1.smithi026.stdout:9/531: mknod d0/da/d16/d17/db1/cb4 0 2024-03-16T10:45:21.355 INFO:tasks.workunit.client.1.smithi026.stdout:0/581: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/cc8 0 2024-03-16T10:45:21.356 INFO:tasks.workunit.client.1.smithi026.stdout:3/552: symlink d4/d89/d97/lba 0 2024-03-16T10:45:21.357 INFO:tasks.workunit.client.1.smithi026.stdout:3/553: fdatasync d4/dc/d73/d83/d57/fab 0 2024-03-16T10:45:21.362 INFO:tasks.workunit.client.1.smithi026.stdout:3/554: symlink d4/d89/d97/d7d/dae/lbb 0 2024-03-16T10:45:21.363 INFO:tasks.workunit.client.1.smithi026.stdout:3/555: readlink d4/dc/d31/l7a 0 2024-03-16T10:45:21.364 INFO:tasks.workunit.client.1.smithi026.stdout:0/582: link d1/d12/d2f/c4d d1/d12/d2f/d48/d55/d60/d9b/d11/cc9 0 2024-03-16T10:45:21.366 INFO:tasks.workunit.client.1.smithi026.stdout:9/532: truncate d0/da/d14/f91 379191 0 2024-03-16T10:45:21.368 INFO:tasks.workunit.client.1.smithi026.stdout:3/556: mkdir d4/dc/d73/d83/dbc 0 2024-03-16T10:45:21.368 INFO:tasks.workunit.client.1.smithi026.stdout:0/583: dread d1/d18/d61/f6b [0,4194304] 0 2024-03-16T10:45:21.370 INFO:tasks.workunit.client.1.smithi026.stdout:9/533: symlink d0/da/d16/d17/lb5 0 2024-03-16T10:45:21.373 INFO:tasks.workunit.client.1.smithi026.stdout:3/557: mknod d4/dc/d31/cbd 0 2024-03-16T10:45:21.377 INFO:tasks.workunit.client.1.smithi026.stdout:9/534: dwrite d0/da/d13/d6e/f74 [0,4194304] 0 2024-03-16T10:45:21.381 INFO:tasks.workunit.client.1.smithi026.stdout:3/558: mkdir d4/dc/d31/dbe 0 2024-03-16T10:45:21.383 INFO:tasks.workunit.client.1.smithi026.stdout:8/498: sync 2024-03-16T10:45:21.384 INFO:tasks.workunit.client.1.smithi026.stdout:9/535: rename d0/da/d16/d5d/d72/d62/f92 to d0/d5f/fb6 0 2024-03-16T10:45:21.386 INFO:tasks.workunit.client.1.smithi026.stdout:9/536: write d0/d5f/d7f/f9c [7146231,68624] 0 2024-03-16T10:45:21.388 INFO:tasks.workunit.client.1.smithi026.stdout:8/499: dread d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:21.394 INFO:tasks.workunit.client.1.smithi026.stdout:8/500: dwrite d5/d23/f61 [0,4194304] 0 2024-03-16T10:45:21.526 INFO:tasks.workunit.client.1.smithi026.stdout:0/584: creat d1/d18/fca x:0 0 0 2024-03-16T10:45:21.530 INFO:tasks.workunit.client.1.smithi026.stdout:0/585: dwrite d1/d18/fc2 [0,4194304] 0 2024-03-16T10:45:21.560 INFO:tasks.workunit.client.1.smithi026.stdout:5/425: sync 2024-03-16T10:45:21.566 INFO:tasks.workunit.client.1.smithi026.stdout:5/426: dwrite d1/d1c/d4c/f88 [0,4194304] 0 2024-03-16T10:45:21.587 INFO:tasks.workunit.client.1.smithi026.stdout:3/559: dwrite d4/dc/f5f [0,4194304] 0 2024-03-16T10:45:21.589 INFO:tasks.workunit.client.1.smithi026.stdout:8/501: rename d5/d23/f74 to d5/d23/d4e/d60/f8f 0 2024-03-16T10:45:21.595 INFO:tasks.workunit.client.1.smithi026.stdout:9/537: dwrite d0/da/d16/d17/d44/f1e [0,4194304] 0 2024-03-16T10:45:21.600 INFO:tasks.workunit.client.1.smithi026.stdout:3/560: rmdir d4/dc/dd/d60/d62/d4f/d5a/da6 39 2024-03-16T10:45:21.600 INFO:tasks.workunit.client.1.smithi026.stdout:3/561: chown l2 4671 1 2024-03-16T10:45:21.601 INFO:tasks.workunit.client.1.smithi026.stdout:3/562: fsync d4/d89/d97/fb7 0 2024-03-16T10:45:21.602 INFO:tasks.workunit.client.1.smithi026.stdout:8/502: chown d5/d7/d2c/d36/c3b 44 1 2024-03-16T10:45:21.608 INFO:tasks.workunit.client.1.smithi026.stdout:0/586: dread d1/d12/d2f/d48/d55/d60/d9b/f67 [0,4194304] 0 2024-03-16T10:45:21.612 INFO:tasks.workunit.client.1.smithi026.stdout:5/427: link d1/l6 d1/d1c/d2a/d55/l8d 0 2024-03-16T10:45:21.614 INFO:tasks.workunit.client.1.smithi026.stdout:8/503: dread d5/d7/f4f [0,4194304] 0 2024-03-16T10:45:21.616 INFO:tasks.workunit.client.1.smithi026.stdout:8/504: write d5/d23/d4e/f7c [420464,48703] 0 2024-03-16T10:45:21.619 INFO:tasks.workunit.client.1.smithi026.stdout:5/428: mkdir d1/d1c/d2a/d8e 0 2024-03-16T10:45:21.620 INFO:tasks.workunit.client.1.smithi026.stdout:5/429: chown d1/d1c/d2a/d31/f76 242294715 1 2024-03-16T10:45:21.622 INFO:tasks.workunit.client.1.smithi026.stdout:9/538: link d0/da/d16/d5d/d72/lac d0/d5f/lb7 0 2024-03-16T10:45:21.623 INFO:tasks.workunit.client.1.smithi026.stdout:5/430: dread d1/db/d1f/f26 [0,4194304] 0 2024-03-16T10:45:21.626 INFO:tasks.workunit.client.1.smithi026.stdout:0/587: dread d1/d12/d2f/d48/d55/d60/d9b/d57/f3e [0,4194304] 0 2024-03-16T10:45:21.627 INFO:tasks.workunit.client.1.smithi026.stdout:5/431: mknod d1/d1c/d4c/c8f 0 2024-03-16T10:45:21.632 INFO:tasks.workunit.client.1.smithi026.stdout:8/505: getdents d5/d7/d2c/d36 0 2024-03-16T10:45:21.634 INFO:tasks.workunit.client.1.smithi026.stdout:0/588: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/d88/fbc [0,4194304] 0 2024-03-16T10:45:21.638 INFO:tasks.workunit.client.1.smithi026.stdout:8/506: rename d5/dc/d89/d7a to d5/d28/d90 0 2024-03-16T10:45:21.639 INFO:tasks.workunit.client.1.smithi026.stdout:8/507: fdatasync d5/d23/f68 0 2024-03-16T10:45:21.642 INFO:tasks.workunit.client.1.smithi026.stdout:0/589: creat d1/d12/d2f/d48/d55/d60/d9b/d51/fcb x:0 0 0 2024-03-16T10:45:21.644 INFO:tasks.workunit.client.1.smithi026.stdout:8/508: chown d5/d7/d2c/d36/c8b 177 1 2024-03-16T10:45:21.645 INFO:tasks.workunit.client.1.smithi026.stdout:5/432: link d1/db/c35 d1/d1c/d2a/d8e/c90 0 2024-03-16T10:45:21.646 INFO:tasks.workunit.client.1.smithi026.stdout:8/509: read d5/dc/d3f/f46 [1416389,82911] 0 2024-03-16T10:45:21.648 INFO:tasks.workunit.client.1.smithi026.stdout:8/510: write d5/d7/d2c/d36/d59/f78 [538914,2887] 0 2024-03-16T10:45:21.650 INFO:tasks.workunit.client.1.smithi026.stdout:0/590: unlink d1/d12/d2f/d48/d55/d60/d9b/d51/f75 0 2024-03-16T10:45:21.653 INFO:tasks.workunit.client.1.smithi026.stdout:5/433: mkdir d1/d1c/d2a/d55/d91 0 2024-03-16T10:45:21.661 INFO:tasks.workunit.client.1.smithi026.stdout:0/591: unlink d1/d18/f9e 0 2024-03-16T10:45:21.668 INFO:tasks.workunit.client.1.smithi026.stdout:5/434: creat d1/d6e/f92 x:0 0 0 2024-03-16T10:45:21.669 INFO:tasks.workunit.client.1.smithi026.stdout:5/435: dread - d1/d1c/d2a/d31/f81 zero size 2024-03-16T10:45:21.675 INFO:tasks.workunit.client.1.smithi026.stdout:9/539: write d0/da/d13/f38 [4490835,75014] 0 2024-03-16T10:45:21.679 INFO:tasks.workunit.client.1.smithi026.stdout:5/436: mknod d1/d6e/c93 0 2024-03-16T10:45:21.685 INFO:tasks.workunit.client.1.smithi026.stdout:9/540: rename d0/f68 to d0/da/d13/d6e/fb8 0 2024-03-16T10:45:21.687 INFO:tasks.workunit.client.1.smithi026.stdout:5/437: unlink d1/db/d27/l30 0 2024-03-16T10:45:21.690 INFO:tasks.workunit.client.1.smithi026.stdout:0/592: link d1/d18/fc2 d1/d12/d2f/d48/d55/d60/d9b/fcc 0 2024-03-16T10:45:21.691 INFO:tasks.workunit.client.1.smithi026.stdout:0/593: readlink d1/d18/d1a/d2b/l30 0 2024-03-16T10:45:21.694 INFO:tasks.workunit.client.1.smithi026.stdout:0/594: truncate d1/d12/d2f/d48/d55/d60/d9b/d11/fc0 587064 0 2024-03-16T10:45:21.695 INFO:tasks.workunit.client.1.smithi026.stdout:5/438: mknod d1/db/d1f/d61/d65/d82/c94 0 2024-03-16T10:45:21.697 INFO:tasks.workunit.client.1.smithi026.stdout:5/439: creat d1/db/d8a/f95 x:0 0 0 2024-03-16T10:45:21.727 INFO:tasks.workunit.client.1.smithi026.stdout:5/440: dwrite d1/db/d1f/d61/d65/f83 [0,4194304] 0 2024-03-16T10:45:21.732 INFO:tasks.workunit.client.1.smithi026.stdout:5/441: creat d1/db/d8a/f96 x:0 0 0 2024-03-16T10:45:21.735 INFO:tasks.workunit.client.1.smithi026.stdout:5/442: rename d1/db/d1f/d61/f73 to d1/d1c/d2a/d31/f97 0 2024-03-16T10:45:21.736 INFO:tasks.workunit.client.1.smithi026.stdout:5/443: fsync d1/db/d27/f41 0 2024-03-16T10:45:21.739 INFO:tasks.workunit.client.1.smithi026.stdout:2/481: sync 2024-03-16T10:45:21.739 INFO:tasks.workunit.client.1.smithi026.stdout:5/444: mknod d1/d1c/d4c/d4d/c98 0 2024-03-16T10:45:21.739 INFO:tasks.workunit.client.1.smithi026.stdout:5/445: chown d1/f9 12803367 1 2024-03-16T10:45:21.746 INFO:tasks.workunit.client.1.smithi026.stdout:5/446: rmdir d1/d1c/d2a 39 2024-03-16T10:45:21.751 INFO:tasks.workunit.client.1.smithi026.stdout:2/482: dread d2/d29/d17/d1a/d2b/d4b/f5a [0,4194304] 0 2024-03-16T10:45:21.752 INFO:tasks.workunit.client.1.smithi026.stdout:2/483: write d2/d29/d17/d1a/d2b/d69/f9d [684654,54591] 0 2024-03-16T10:45:21.753 INFO:tasks.workunit.client.1.smithi026.stdout:5/447: rename d1/db/d8a to d1/d1c/d2a/d55/d99 0 2024-03-16T10:45:21.754 INFO:tasks.workunit.client.1.smithi026.stdout:5/448: dread - d1/d1c/d2a/d31/d69/f72 zero size 2024-03-16T10:45:21.758 INFO:tasks.workunit.client.1.smithi026.stdout:2/484: rename d2/d29/d68 to d2/d29/d36/da7 0 2024-03-16T10:45:21.762 INFO:tasks.workunit.client.1.smithi026.stdout:2/485: unlink d2/d29/d17/d1a/d2b/f18 0 2024-03-16T10:45:21.767 INFO:tasks.workunit.client.1.smithi026.stdout:5/449: link d1/d1c/d4c/d4d/c7a d1/d1c/d2a/d55/d99/c9a 0 2024-03-16T10:45:21.777 INFO:tasks.workunit.client.1.smithi026.stdout:5/450: creat d1/d1c/d2a/d31/d69/d51/f9b x:0 0 0 2024-03-16T10:45:21.778 INFO:tasks.workunit.client.1.smithi026.stdout:2/486: write d2/d29/d17/f40 [818632,76591] 0 2024-03-16T10:45:21.780 INFO:tasks.workunit.client.1.smithi026.stdout:5/451: chown d1/d1c/d2a/d55/l8d 6960546 1 2024-03-16T10:45:21.784 INFO:tasks.workunit.client.1.smithi026.stdout:5/452: mknod d1/d1c/d2a/d55/d99/c9c 0 2024-03-16T10:45:21.784 INFO:tasks.workunit.client.1.smithi026.stdout:5/453: truncate d1/d1c/d2a/d31/f97 917585 0 2024-03-16T10:45:21.787 INFO:tasks.workunit.client.1.smithi026.stdout:5/454: creat d1/db/d1f/d61/d65/f9d x:0 0 0 2024-03-16T10:45:21.789 INFO:tasks.workunit.client.1.smithi026.stdout:2/487: link d2/d29/d17/d1a/l6b d2/d29/d17/d1a/d2b/d4b/la8 0 2024-03-16T10:45:21.791 INFO:tasks.workunit.client.1.smithi026.stdout:5/455: symlink d1/db/d1f/d61/d65/l9e 0 2024-03-16T10:45:21.793 INFO:tasks.workunit.client.1.smithi026.stdout:2/488: symlink d2/d29/d51/d55/d96/la9 0 2024-03-16T10:45:21.794 INFO:tasks.workunit.client.1.smithi026.stdout:5/456: mkdir d1/d6e/d9f 0 2024-03-16T10:45:21.798 INFO:tasks.workunit.client.1.smithi026.stdout:5/457: dread d1/db/d1f/f5c [0,4194304] 0 2024-03-16T10:45:21.806 INFO:tasks.workunit.client.1.smithi026.stdout:2/489: getdents d2/d29/d17/d1a/d2b/d4e/d67 0 2024-03-16T10:45:21.807 INFO:tasks.workunit.client.1.smithi026.stdout:2/490: readlink d2/d29/d51/d55/d96/l99 0 2024-03-16T10:45:21.813 INFO:tasks.workunit.client.1.smithi026.stdout:2/491: dwrite d2/d29/d3b/fa5 [0,4194304] 0 2024-03-16T10:45:21.816 INFO:tasks.workunit.client.1.smithi026.stdout:5/458: write d1/d1c/d2a/f47 [36152,66314] 0 2024-03-16T10:45:21.818 INFO:tasks.workunit.client.1.smithi026.stdout:5/459: write d1/db/d27/f66 [4047795,112669] 0 2024-03-16T10:45:21.820 INFO:tasks.workunit.client.1.smithi026.stdout:2/492: symlink d2/d29/d17/laa 0 2024-03-16T10:45:21.822 INFO:tasks.workunit.client.1.smithi026.stdout:2/493: write d2/d29/d17/d1a/d2b/d4e/d6f/f9c [44340,119431] 0 2024-03-16T10:45:21.825 INFO:tasks.workunit.client.1.smithi026.stdout:5/460: symlink d1/d1c/d4c/d4d/la0 0 2024-03-16T10:45:21.827 INFO:tasks.workunit.client.1.smithi026.stdout:5/461: chown d1/d1c/d4c/f88 892 1 2024-03-16T10:45:21.829 INFO:tasks.workunit.client.1.smithi026.stdout:5/462: fsync d1/d1c/d2a/f89 0 2024-03-16T10:45:21.831 INFO:tasks.workunit.client.1.smithi026.stdout:2/494: dwrite d2/d29/d3b/fa5 [4194304,4194304] 0 2024-03-16T10:45:21.836 INFO:tasks.workunit.client.1.smithi026.stdout:5/463: write d1/d1c/d2a/d31/f76 [541699,116067] 0 2024-03-16T10:45:21.843 INFO:tasks.workunit.client.1.smithi026.stdout:5/464: mknod d1/d1c/d2a/d31/d69/d51/ca1 0 2024-03-16T10:45:21.845 INFO:tasks.workunit.client.1.smithi026.stdout:2/495: dread d2/d29/d17/d1a/d2b/f1f [4194304,4194304] 0 2024-03-16T10:45:21.846 INFO:tasks.workunit.client.1.smithi026.stdout:2/496: fdatasync d2/f8f 0 2024-03-16T10:45:21.848 INFO:tasks.workunit.client.1.smithi026.stdout:2/497: write d2/d29/d17/d1a/d2b/f1f [5990320,23338] 0 2024-03-16T10:45:21.849 INFO:tasks.workunit.client.1.smithi026.stdout:5/465: creat d1/d1c/d2a/d31/d69/d51/fa2 x:0 0 0 2024-03-16T10:45:21.851 INFO:tasks.workunit.client.1.smithi026.stdout:5/466: stat d1/db/d1f/d61/d65/d82 0 2024-03-16T10:45:21.852 INFO:tasks.workunit.client.1.smithi026.stdout:5/467: truncate d1/d1c/d2a/d31/d69/d51/f5e 526564 0 2024-03-16T10:45:21.863 INFO:tasks.workunit.client.1.smithi026.stdout:5/468: rmdir d1/d1c/d2a/d31/d69/d51 39 2024-03-16T10:45:21.865 INFO:tasks.workunit.client.1.smithi026.stdout:2/498: truncate d2/d29/d17/d38/f59 85828 0 2024-03-16T10:45:21.866 INFO:tasks.workunit.client.1.smithi026.stdout:5/469: rename d1/d6e/f78 to d1/d1c/d68/fa3 0 2024-03-16T10:45:21.875 INFO:tasks.workunit.client.1.smithi026.stdout:7/523: sync 2024-03-16T10:45:21.876 INFO:tasks.workunit.client.1.smithi026.stdout:2/499: mknod d2/d29/d17/d1a/d2b/d2e/d60/cab 0 2024-03-16T10:45:21.877 INFO:tasks.workunit.client.1.smithi026.stdout:2/500: fdatasync d2/d29/d17/d1a/d2b/d4e/f71 0 2024-03-16T10:45:21.878 INFO:tasks.workunit.client.1.smithi026.stdout:5/470: symlink d1/d1c/d2a/d55/la4 0 2024-03-16T10:45:21.878 INFO:tasks.workunit.client.1.smithi026.stdout:5/471: readlink d1/db/l84 0 2024-03-16T10:45:21.881 INFO:tasks.workunit.client.1.smithi026.stdout:7/524: creat d6/d35/d3a/d4e/da1/fa8 x:0 0 0 2024-03-16T10:45:21.882 INFO:tasks.workunit.client.1.smithi026.stdout:2/501: fsync d2/d29/d17/d1a/d2b/d2e/f45 0 2024-03-16T10:45:21.885 INFO:tasks.workunit.client.1.smithi026.stdout:5/472: fdatasync d1/f9 0 2024-03-16T10:45:21.888 INFO:tasks.workunit.client.1.smithi026.stdout:7/525: creat d6/d1e/fa9 x:0 0 0 2024-03-16T10:45:21.898 INFO:tasks.workunit.client.1.smithi026.stdout:5/473: creat d1/d1c/d2a/d55/d99/fa5 x:0 0 0 2024-03-16T10:45:21.903 INFO:tasks.workunit.client.1.smithi026.stdout:2/502: write d2/d29/d36/f3e [1775533,622] 0 2024-03-16T10:45:21.907 INFO:tasks.workunit.client.1.smithi026.stdout:2/503: mknod d2/d29/d17/d38/cac 0 2024-03-16T10:45:21.908 INFO:tasks.workunit.client.1.smithi026.stdout:5/474: dwrite d1/d1c/d68/fa3 [0,4194304] 0 2024-03-16T10:45:21.910 INFO:tasks.workunit.client.1.smithi026.stdout:7/526: write d6/d35/d3a/d3c/d3f/d54/f8b [344636,59702] 0 2024-03-16T10:45:21.910 INFO:tasks.workunit.client.1.smithi026.stdout:5/475: chown d1/db/d1f/d61/d65/l9e 0 1 2024-03-16T10:45:21.914 INFO:tasks.workunit.client.1.smithi026.stdout:7/527: truncate d6/de/f64 4474220 0 2024-03-16T10:45:21.914 INFO:tasks.workunit.client.1.smithi026.stdout:2/504: dread d2/d29/d3b/fa5 [4194304,4194304] 0 2024-03-16T10:45:21.917 INFO:tasks.workunit.client.1.smithi026.stdout:2/505: unlink l0 0 2024-03-16T10:45:21.922 INFO:tasks.workunit.client.1.smithi026.stdout:5/476: link d1/f18 d1/d1c/d2a/d31/d69/d51/d80/fa6 0 2024-03-16T10:45:21.925 INFO:tasks.workunit.client.1.smithi026.stdout:7/528: creat d6/de/faa x:0 0 0 2024-03-16T10:45:21.925 INFO:tasks.workunit.client.1.smithi026.stdout:7/529: dread - d6/f53 zero size 2024-03-16T10:45:21.926 INFO:tasks.workunit.client.1.smithi026.stdout:5/477: creat d1/db/d1f/d61/fa7 x:0 0 0 2024-03-16T10:45:21.932 INFO:tasks.workunit.client.1.smithi026.stdout:7/530: dwrite d6/d35/d3a/d4e/da1/fa8 [0,4194304] 0 2024-03-16T10:45:21.934 INFO:tasks.workunit.client.1.smithi026.stdout:2/506: creat d2/d29/d17/d1a/fad x:0 0 0 2024-03-16T10:45:21.937 INFO:tasks.workunit.client.1.smithi026.stdout:7/531: mkdir d6/d35/d3a/dab 0 2024-03-16T10:45:21.940 INFO:tasks.workunit.client.1.smithi026.stdout:2/507: unlink d2/d29/d17/d1a/d2b/d2e/d60/f8c 0 2024-03-16T10:45:21.945 INFO:tasks.workunit.client.1.smithi026.stdout:5/478: rmdir d1 39 2024-03-16T10:45:21.947 INFO:tasks.workunit.client.1.smithi026.stdout:2/508: dwrite d2/d29/d17/d1a/d2b/d2e/f3f [0,4194304] 0 2024-03-16T10:45:21.951 INFO:tasks.workunit.client.1.smithi026.stdout:2/509: symlink d2/d29/d17/d1a/d2b/d4b/lae 0 2024-03-16T10:45:21.954 INFO:tasks.workunit.client.1.smithi026.stdout:7/532: getdents d6/d35/d3a/d4e/d6c/d85/da5/d7b 0 2024-03-16T10:45:21.958 INFO:tasks.workunit.client.1.smithi026.stdout:7/533: link d6/d1e/f90 d6/d35/d3a/d4e/fac 0 2024-03-16T10:45:21.961 INFO:tasks.workunit.client.1.smithi026.stdout:2/510: getdents d2/d29/d51/d55 0 2024-03-16T10:45:21.963 INFO:tasks.workunit.client.1.smithi026.stdout:2/511: mknod d2/d29/d51/d55/d96/caf 0 2024-03-16T10:45:21.966 INFO:tasks.workunit.client.1.smithi026.stdout:5/479: truncate d1/d1c/d68/f74 2101452 0 2024-03-16T10:45:21.969 INFO:tasks.workunit.client.1.smithi026.stdout:2/512: rename d2/d29/d3b/f66 to d2/d29/d17/d1a/d2b/d4b/fb0 0 2024-03-16T10:45:21.970 INFO:tasks.workunit.client.1.smithi026.stdout:7/534: dread d6/d35/d3a/f8e [0,4194304] 0 2024-03-16T10:45:21.970 INFO:tasks.workunit.client.1.smithi026.stdout:2/513: chown d2/d29/d17/d1a/l2f 4866 1 2024-03-16T10:45:21.971 INFO:tasks.workunit.client.1.smithi026.stdout:7/535: dread - d6/d1e/fa9 zero size 2024-03-16T10:45:21.971 INFO:tasks.workunit.client.1.smithi026.stdout:2/514: readlink d2/d29/d3b/l52 0 2024-03-16T10:45:21.973 INFO:tasks.workunit.client.1.smithi026.stdout:5/480: mkdir d1/d1c/d2a/d31/d69/d51/d80/da8 0 2024-03-16T10:45:21.974 INFO:tasks.workunit.client.1.smithi026.stdout:5/481: write d1/d1c/d68/fa3 [178311,113276] 0 2024-03-16T10:45:21.979 INFO:tasks.workunit.client.1.smithi026.stdout:5/482: rename d1/db/d1f/f5b to d1/d1c/d2a/d31/fa9 0 2024-03-16T10:45:21.981 INFO:tasks.workunit.client.1.smithi026.stdout:5/483: readlink d1/d1c/d2a/d31/d69/l63 0 2024-03-16T10:45:21.983 INFO:tasks.workunit.client.1.smithi026.stdout:7/536: dread d6/f22 [0,4194304] 0 2024-03-16T10:45:21.996 INFO:tasks.workunit.client.1.smithi026.stdout:5/484: truncate d1/d1c/f50 1649166 0 2024-03-16T10:45:21.999 INFO:tasks.workunit.client.1.smithi026.stdout:7/537: write d6/fc [1297815,78743] 0 2024-03-16T10:45:22.002 INFO:tasks.workunit.client.1.smithi026.stdout:7/538: rmdir d6/d35/d3a/d4e/d6c/d85/da5 39 2024-03-16T10:45:22.009 INFO:tasks.workunit.client.1.smithi026.stdout:5/485: dwrite d1/f9 [0,4194304] 0 2024-03-16T10:45:22.014 INFO:tasks.workunit.client.1.smithi026.stdout:5/486: chown d1/db/f28 481999856 1 2024-03-16T10:45:22.030 INFO:tasks.workunit.client.1.smithi026.stdout:5/487: dread d1/db/d1f/f33 [4194304,4194304] 0 2024-03-16T10:45:22.031 INFO:tasks.workunit.client.1.smithi026.stdout:7/539: getdents d6/d35/d3a/d3c/d3f/d54 0 2024-03-16T10:45:22.033 INFO:tasks.workunit.client.1.smithi026.stdout:5/488: symlink d1/d1c/d2a/d31/laa 0 2024-03-16T10:45:22.035 INFO:tasks.workunit.client.1.smithi026.stdout:5/489: rmdir d1/d1c/d2a/d31/d69/d51/d80 39 2024-03-16T10:45:22.037 INFO:tasks.workunit.client.1.smithi026.stdout:7/540: dread d6/d35/f52 [0,4194304] 0 2024-03-16T10:45:22.041 INFO:tasks.workunit.client.1.smithi026.stdout:5/490: dwrite d1/f5 [0,4194304] 0 2024-03-16T10:45:22.045 INFO:tasks.workunit.client.1.smithi026.stdout:7/541: dwrite d6/f22 [0,4194304] 0 2024-03-16T10:45:22.050 INFO:tasks.workunit.client.1.smithi026.stdout:7/542: dread d6/de/d7d/f86 [0,4194304] 0 2024-03-16T10:45:22.055 INFO:tasks.workunit.client.1.smithi026.stdout:7/543: rename d6/d35/d3a/d4e/d6c/d85/da5 to d6/d2f/dad 0 2024-03-16T10:45:22.055 INFO:tasks.workunit.client.1.smithi026.stdout:7/544: chown d6/de/d7d 12733 1 2024-03-16T10:45:22.070 INFO:tasks.workunit.client.1.smithi026.stdout:5/491: dwrite d1/f18 [0,4194304] 0 2024-03-16T10:45:22.074 INFO:tasks.workunit.client.1.smithi026.stdout:7/545: truncate d6/d35/d3a/d3c/d3f/f87 4208331 0 2024-03-16T10:45:22.076 INFO:tasks.workunit.client.1.smithi026.stdout:5/492: mkdir d1/d1c/d2a/dab 0 2024-03-16T10:45:22.079 INFO:tasks.workunit.client.1.smithi026.stdout:7/546: truncate d6/d35/f7e 857804 0 2024-03-16T10:45:22.081 INFO:tasks.workunit.client.1.smithi026.stdout:5/493: rename d1/f21 to d1/d1c/d2a/d31/d69/d51/d80/fac 0 2024-03-16T10:45:22.083 INFO:tasks.workunit.client.1.smithi026.stdout:7/547: creat d6/de/d7d/fae x:0 0 0 2024-03-16T10:45:22.083 INFO:tasks.workunit.client.1.smithi026.stdout:5/494: read - d1/d1c/d2a/d55/f58 zero size 2024-03-16T10:45:22.085 INFO:tasks.workunit.client.1.smithi026.stdout:5/495: mknod d1/db/d1f/d61/cad 0 2024-03-16T10:45:22.094 INFO:tasks.workunit.client.1.smithi026.stdout:7/548: rename d6/d35/d3a/d3c to d6/d2f/daf 0 2024-03-16T10:45:22.097 INFO:tasks.workunit.client.1.smithi026.stdout:5/496: rename d1/d6e/c93 to d1/d1c/d2a/d31/d69/d51/d80/cae 0 2024-03-16T10:45:22.101 INFO:tasks.workunit.client.1.smithi026.stdout:5/497: symlink d1/d1c/d2a/d31/d69/d51/d80/da8/laf 0 2024-03-16T10:45:22.103 INFO:tasks.workunit.client.1.smithi026.stdout:5/498: write d1/d6e/f92 [387371,73173] 0 2024-03-16T10:45:22.108 INFO:tasks.workunit.client.1.smithi026.stdout:7/549: dread d6/d2f/daf/d3f/fa0 [0,4194304] 0 2024-03-16T10:45:22.108 INFO:tasks.workunit.client.1.smithi026.stdout:5/499: mkdir d1/d1c/d2a/db0 0 2024-03-16T10:45:22.117 INFO:tasks.workunit.client.1.smithi026.stdout:5/500: write d1/d1c/f50 [2520156,58608] 0 2024-03-16T10:45:22.120 INFO:tasks.workunit.client.1.smithi026.stdout:5/501: truncate d1/d1c/d2a/d55/f58 769551 0 2024-03-16T10:45:22.120 INFO:tasks.workunit.client.1.smithi026.stdout:7/550: dwrite d6/d2f/f4a [0,4194304] 0 2024-03-16T10:45:22.122 INFO:tasks.workunit.client.1.smithi026.stdout:7/551: write d6/d1e/f6a [282594,71480] 0 2024-03-16T10:45:22.127 INFO:tasks.workunit.client.1.smithi026.stdout:5/502: truncate d1/d1c/d2a/d31/f49 534765 0 2024-03-16T10:45:22.128 INFO:tasks.workunit.client.1.smithi026.stdout:5/503: dread - d1/d1c/d2a/d31/d69/d51/fa2 zero size 2024-03-16T10:45:22.133 INFO:tasks.workunit.client.1.smithi026.stdout:5/504: rename d1/d1c/d2a/d31/d69/d51/d80/da8 to d1/d1c/d2a/d55/db1 0 2024-03-16T10:45:22.134 INFO:tasks.workunit.client.1.smithi026.stdout:7/552: getdents d6/d35/d3a/d4e/d6c/d85 0 2024-03-16T10:45:22.134 INFO:tasks.workunit.client.1.smithi026.stdout:7/553: chown d6/d2f/dad 2028 1 2024-03-16T10:45:22.137 INFO:tasks.workunit.client.1.smithi026.stdout:5/505: creat d1/d1c/d2a/d55/d91/fb2 x:0 0 0 2024-03-16T10:45:22.138 INFO:tasks.workunit.client.1.smithi026.stdout:7/554: truncate d6/de/d16/f5e 102645 0 2024-03-16T10:45:22.139 INFO:tasks.workunit.client.1.smithi026.stdout:5/506: write d1/db/d1f/d61/d65/f83 [2144810,125349] 0 2024-03-16T10:45:22.140 INFO:tasks.workunit.client.1.smithi026.stdout:7/555: mkdir d6/d35/d3a/d4e/db0 0 2024-03-16T10:45:22.144 INFO:tasks.workunit.client.1.smithi026.stdout:7/556: creat d6/d2f/daf/d3f/d65/fb1 x:0 0 0 2024-03-16T10:45:22.145 INFO:tasks.workunit.client.1.smithi026.stdout:7/557: chown d6/d35/d3a/d4e/l58 24936187 1 2024-03-16T10:45:22.151 INFO:tasks.workunit.client.1.smithi026.stdout:7/558: dwrite d6/d2f/daf/d3f/d9d/fa7 [0,4194304] 0 2024-03-16T10:45:22.156 INFO:tasks.workunit.client.1.smithi026.stdout:7/559: mknod d6/d2f/dad/cb2 0 2024-03-16T10:45:22.159 INFO:tasks.workunit.client.1.smithi026.stdout:5/507: write d1/d1c/d2a/f3b [586273,108453] 0 2024-03-16T10:45:22.161 INFO:tasks.workunit.client.1.smithi026.stdout:5/508: write d1/d1c/d2a/d55/d99/f95 [477489,63546] 0 2024-03-16T10:45:22.162 INFO:tasks.workunit.client.1.smithi026.stdout:7/560: dread d6/d35/d3a/d59/f96 [0,4194304] 0 2024-03-16T10:45:22.165 INFO:tasks.workunit.client.1.smithi026.stdout:7/561: unlink d6/l1d 0 2024-03-16T10:45:22.167 INFO:tasks.workunit.client.1.smithi026.stdout:7/562: dread - d6/d2f/daf/d3f/d70/f9f zero size 2024-03-16T10:45:22.167 INFO:tasks.workunit.client.1.smithi026.stdout:5/509: dwrite d1/f9 [4194304,4194304] 0 2024-03-16T10:45:22.169 INFO:tasks.workunit.client.1.smithi026.stdout:7/563: truncate d6/d1e/f29 2053848 0 2024-03-16T10:45:22.172 INFO:tasks.workunit.client.1.smithi026.stdout:5/510: rmdir d1/d1c/d2a/d31 39 2024-03-16T10:45:22.175 INFO:tasks.workunit.client.1.smithi026.stdout:7/564: creat d6/d35/d3a/dab/fb3 x:0 0 0 2024-03-16T10:45:22.179 INFO:tasks.workunit.client.1.smithi026.stdout:5/511: dwrite d1/db/d1f/d61/fa7 [0,4194304] 0 2024-03-16T10:45:22.181 INFO:tasks.workunit.client.1.smithi026.stdout:7/565: chown d6/d1e/l2b 5 1 2024-03-16T10:45:22.186 INFO:tasks.workunit.client.1.smithi026.stdout:7/566: dread - d6/d2f/daf/d3f/d65/fb1 zero size 2024-03-16T10:45:22.187 INFO:tasks.workunit.client.1.smithi026.stdout:7/567: write d6/f39 [914409,63472] 0 2024-03-16T10:45:22.188 INFO:tasks.workunit.client.1.smithi026.stdout:5/512: dwrite d1/db/d27/f3e [4194304,4194304] 0 2024-03-16T10:45:22.189 INFO:tasks.workunit.client.1.smithi026.stdout:5/513: chown d1/c29 4520 1 2024-03-16T10:45:22.190 INFO:tasks.workunit.client.1.smithi026.stdout:7/568: read d6/d1e/f20 [1561406,73281] 0 2024-03-16T10:45:22.192 INFO:tasks.workunit.client.1.smithi026.stdout:5/514: truncate d1/d1c/d2a/d55/d99/f96 391063 0 2024-03-16T10:45:22.194 INFO:tasks.workunit.client.1.smithi026.stdout:5/515: truncate d1/db/d27/f3e 9348988 0 2024-03-16T10:45:22.197 INFO:tasks.workunit.client.1.smithi026.stdout:7/569: link d6/d2f/daf/d3f/d65/d68/c75 d6/d2f/dad/d7b/cb4 0 2024-03-16T10:45:22.198 INFO:tasks.workunit.client.1.smithi026.stdout:7/570: write d6/d2f/daf/f40 [4237230,76679] 0 2024-03-16T10:45:22.199 INFO:tasks.workunit.client.1.smithi026.stdout:5/516: rename d1/d1c/d2a/d31/d69/c4b to d1/d1c/d2a/d8e/cb3 0 2024-03-16T10:45:22.212 INFO:tasks.workunit.client.1.smithi026.stdout:7/571: write d6/d1e/f90 [535659,12653] 0 2024-03-16T10:45:22.213 INFO:tasks.workunit.client.1.smithi026.stdout:7/572: chown d6/de/d16/f49 7110 1 2024-03-16T10:45:22.214 INFO:tasks.workunit.client.1.smithi026.stdout:5/517: write d1/db/f59 [698688,93680] 0 2024-03-16T10:45:22.215 INFO:tasks.workunit.client.1.smithi026.stdout:7/573: stat d6/de/d16/l77 0 2024-03-16T10:45:22.215 INFO:tasks.workunit.client.1.smithi026.stdout:7/574: fdatasync d6/d1e/da6/f8f 0 2024-03-16T10:45:22.216 INFO:tasks.workunit.client.1.smithi026.stdout:5/518: fdatasync d1/d1c/d2a/d31/f76 0 2024-03-16T10:45:22.217 INFO:tasks.workunit.client.1.smithi026.stdout:7/575: chown d6/de/d7d/fae 1 1 2024-03-16T10:45:22.217 INFO:tasks.workunit.client.1.smithi026.stdout:7/576: stat d6/c25 0 2024-03-16T10:45:22.220 INFO:tasks.workunit.client.1.smithi026.stdout:5/519: creat d1/d1c/d2a/d55/d99/fb4 x:0 0 0 2024-03-16T10:45:22.222 INFO:tasks.workunit.client.1.smithi026.stdout:5/520: chown d1/d1c/d2a/d31/d69/d51/d80/cae 22 1 2024-03-16T10:45:22.225 INFO:tasks.workunit.client.1.smithi026.stdout:5/521: symlink d1/d1c/d2a/dab/lb5 0 2024-03-16T10:45:22.227 INFO:tasks.workunit.client.1.smithi026.stdout:5/522: mkdir d1/d1c/d2a/d31/d69/d51/d80/db6 0 2024-03-16T10:45:22.228 INFO:tasks.workunit.client.1.smithi026.stdout:5/523: truncate d1/d1c/d2a/d31/f81 253702 0 2024-03-16T10:45:22.229 INFO:tasks.workunit.client.1.smithi026.stdout:5/524: fsync d1/db/f28 0 2024-03-16T10:45:22.231 INFO:tasks.workunit.client.1.smithi026.stdout:5/525: rename d1/f7 to d1/d6e/fb7 0 2024-03-16T10:45:22.234 INFO:tasks.workunit.client.1.smithi026.stdout:1/466: sync 2024-03-16T10:45:22.236 INFO:tasks.workunit.client.1.smithi026.stdout:5/526: getdents d1/db 0 2024-03-16T10:45:22.240 INFO:tasks.workunit.client.1.smithi026.stdout:1/467: creat d0/d17/d1f/d63/d6a/f94 x:0 0 0 2024-03-16T10:45:22.242 INFO:tasks.workunit.client.1.smithi026.stdout:1/468: stat d0/l16 0 2024-03-16T10:45:22.248 INFO:tasks.workunit.client.1.smithi026.stdout:5/527: link d1/db/f38 d1/d1c/fb8 0 2024-03-16T10:45:22.252 INFO:tasks.workunit.client.1.smithi026.stdout:5/528: creat d1/db/d1f/d61/fb9 x:0 0 0 2024-03-16T10:45:22.254 INFO:tasks.workunit.client.1.smithi026.stdout:1/469: dread d0/f40 [0,4194304] 0 2024-03-16T10:45:22.255 INFO:tasks.workunit.client.1.smithi026.stdout:1/470: dread - d0/d17/d1f/d63/d6a/f94 zero size 2024-03-16T10:45:22.257 INFO:tasks.workunit.client.1.smithi026.stdout:5/529: unlink d1/d1c/d2a/d31/d69/d51/d80/f86 0 2024-03-16T10:45:22.257 INFO:tasks.workunit.client.1.smithi026.stdout:5/530: chown d1/f71 13119 1 2024-03-16T10:45:22.264 INFO:tasks.workunit.client.1.smithi026.stdout:5/531: creat d1/d1c/d2a/d31/fba x:0 0 0 2024-03-16T10:45:22.265 INFO:tasks.workunit.client.1.smithi026.stdout:1/471: creat d0/d17/d1f/d63/d8d/f95 x:0 0 0 2024-03-16T10:45:22.267 INFO:tasks.workunit.client.1.smithi026.stdout:5/532: dread d1/d6e/f92 [0,4194304] 0 2024-03-16T10:45:22.270 INFO:tasks.workunit.client.1.smithi026.stdout:5/533: truncate d1/d1c/d2a/d31/d69/f8c 1039004 0 2024-03-16T10:45:22.273 INFO:tasks.workunit.client.1.smithi026.stdout:1/472: dwrite d0/d17/d1f/f3f [4194304,4194304] 0 2024-03-16T10:45:22.275 INFO:tasks.workunit.client.1.smithi026.stdout:5/534: mkdir d1/db/d1f/d61/d65/d82/dbb 0 2024-03-16T10:45:22.279 INFO:tasks.workunit.client.1.smithi026.stdout:1/473: mknod d0/d17/d4d/d70/c96 0 2024-03-16T10:45:22.280 INFO:tasks.workunit.client.1.smithi026.stdout:1/474: write d0/f1e [656715,118765] 0 2024-03-16T10:45:22.285 INFO:tasks.workunit.client.1.smithi026.stdout:5/535: link d1/d1c/d2a/d55/d99/fa5 d1/d1c/d2a/d55/db1/fbc 0 2024-03-16T10:45:22.288 INFO:tasks.workunit.client.1.smithi026.stdout:1/475: symlink d0/d17/d1f/l97 0 2024-03-16T10:45:22.289 INFO:tasks.workunit.client.1.smithi026.stdout:1/476: truncate d0/d17/d1f/d63/d8d/f95 90592 0 2024-03-16T10:45:22.290 INFO:tasks.workunit.client.1.smithi026.stdout:1/477: read - d0/d17/d1f/d20/d65/f88 zero size 2024-03-16T10:45:22.293 INFO:tasks.workunit.client.1.smithi026.stdout:1/478: getdents d0/d17/d1f/d63/d58 0 2024-03-16T10:45:22.294 INFO:tasks.workunit.client.1.smithi026.stdout:1/479: write d0/d17/f81 [845023,14681] 0 2024-03-16T10:45:22.297 INFO:tasks.workunit.client.1.smithi026.stdout:1/480: unlink d0/d17/d1f/d63/d8d/f95 0 2024-03-16T10:45:22.299 INFO:tasks.workunit.client.1.smithi026.stdout:1/481: creat d0/d17/d4d/f98 x:0 0 0 2024-03-16T10:45:22.300 INFO:tasks.workunit.client.1.smithi026.stdout:1/482: read d0/d17/d1f/f3f [367543,119459] 0 2024-03-16T10:45:22.303 INFO:tasks.workunit.client.1.smithi026.stdout:1/483: unlink d0/d17/d1f/d63/d6a/d7c/d2e/f56 0 2024-03-16T10:45:22.304 INFO:tasks.workunit.client.1.smithi026.stdout:1/484: fsync d0/f1e 0 2024-03-16T10:45:22.306 INFO:tasks.workunit.client.1.smithi026.stdout:1/485: symlink d0/d17/d1f/d63/d6a/d7c/d2e/l99 0 2024-03-16T10:45:22.318 INFO:tasks.workunit.client.1.smithi026.stdout:1/486: dwrite d0/f8f [0,4194304] 0 2024-03-16T10:45:22.323 INFO:tasks.workunit.client.1.smithi026.stdout:1/487: read - d0/d17/d1f/d63/d6a/d7c/d2e/d54/f5e zero size 2024-03-16T10:45:22.324 INFO:tasks.workunit.client.1.smithi026.stdout:1/488: write d0/d17/d1f/d38/f4e [1105668,52834] 0 2024-03-16T10:45:22.331 INFO:tasks.workunit.client.1.smithi026.stdout:1/489: dwrite d0/d17/d1f/d38/f6e [0,4194304] 0 2024-03-16T10:45:22.332 INFO:tasks.workunit.client.1.smithi026.stdout:1/490: chown d0/d17/d1f/d20/f85 419687413 1 2024-03-16T10:45:22.341 INFO:tasks.workunit.client.1.smithi026.stdout:1/491: creat d0/d17/d1f/d63/d58/f9a x:0 0 0 2024-03-16T10:45:22.343 INFO:tasks.workunit.client.1.smithi026.stdout:1/492: creat d0/d17/d4d/d70/f9b x:0 0 0 2024-03-16T10:45:22.353 INFO:tasks.workunit.client.1.smithi026.stdout:1/493: dwrite d0/f40 [0,4194304] 0 2024-03-16T10:45:22.360 INFO:tasks.workunit.client.1.smithi026.stdout:1/494: dwrite d0/fd [4194304,4194304] 0 2024-03-16T10:45:22.362 INFO:tasks.workunit.client.1.smithi026.stdout:1/495: write d0/d17/d1f/d63/d6a/d7c/f30 [645548,87914] 0 2024-03-16T10:45:22.364 INFO:tasks.workunit.client.1.smithi026.stdout:1/496: chown d0/d17/d1f/d63/d6a/d7c/d2e/f7e 1739990474 1 2024-03-16T10:45:22.367 INFO:tasks.workunit.client.1.smithi026.stdout:1/497: rmdir d0/d17/d4d/d70 39 2024-03-16T10:45:22.372 INFO:tasks.workunit.client.1.smithi026.stdout:1/498: dread d0/d17/d4d/f78 [0,4194304] 0 2024-03-16T10:45:22.375 INFO:tasks.workunit.client.1.smithi026.stdout:1/499: readlink d0/d17/d4d/d70/l7b 0 2024-03-16T10:45:22.377 INFO:tasks.workunit.client.1.smithi026.stdout:1/500: mknod d0/c9c 0 2024-03-16T10:45:22.378 INFO:tasks.workunit.client.1.smithi026.stdout:1/501: write d0/f40 [322062,47298] 0 2024-03-16T10:45:22.379 INFO:tasks.workunit.client.1.smithi026.stdout:1/502: dread - d0/d17/d4d/f98 zero size 2024-03-16T10:45:22.380 INFO:tasks.workunit.client.1.smithi026.stdout:1/503: write d0/fd [4030137,60691] 0 2024-03-16T10:45:22.381 INFO:tasks.workunit.client.1.smithi026.stdout:1/504: dread - d0/d17/d1f/d63/f83 zero size 2024-03-16T10:45:22.382 INFO:tasks.workunit.client.1.smithi026.stdout:1/505: chown d0/d17/d1f/d63/f83 146080 1 2024-03-16T10:45:22.386 INFO:tasks.workunit.client.1.smithi026.stdout:1/506: read d0/d17/d1f/d63/d6a/d7c/f6c [3280756,111952] 0 2024-03-16T10:45:22.390 INFO:tasks.workunit.client.1.smithi026.stdout:1/507: rename d0/d17/d1f/d38 to d0/d17/d1f/d63/d6a/d7c/d84/d9d 0 2024-03-16T10:45:22.392 INFO:tasks.workunit.client.1.smithi026.stdout:1/508: mknod d0/d17/d1f/d63/d58/c9e 0 2024-03-16T10:45:22.394 INFO:tasks.workunit.client.1.smithi026.stdout:1/509: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/c9f 0 2024-03-16T10:45:22.398 INFO:tasks.workunit.client.1.smithi026.stdout:1/510: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0 0 2024-03-16T10:45:22.400 INFO:tasks.workunit.client.1.smithi026.stdout:1/511: symlink d0/d17/d1f/la1 0 2024-03-16T10:45:22.409 INFO:tasks.workunit.client.1.smithi026.stdout:1/512: link d0/d17/d1f/d63/d6a/d7c/d3b/f2f d0/d17/d1f/d63/d6a/fa2 0 2024-03-16T10:45:22.411 INFO:tasks.workunit.client.1.smithi026.stdout:1/513: creat d0/d17/d1f/d63/d6a/fa3 x:0 0 0 2024-03-16T10:45:22.413 INFO:tasks.workunit.client.1.smithi026.stdout:1/514: chown d0/d17/d1f/d63/d6a/d7c/d2e/c57 22466 1 2024-03-16T10:45:22.423 INFO:tasks.workunit.client.1.smithi026.stdout:1/515: dwrite d0/d17/f48 [0,4194304] 0 2024-03-16T10:45:22.423 INFO:tasks.workunit.client.1.smithi026.stdout:1/516: stat d0/d8b 0 2024-03-16T10:45:22.430 INFO:tasks.workunit.client.1.smithi026.stdout:8/511: sync 2024-03-16T10:45:22.430 INFO:tasks.workunit.client.1.smithi026.stdout:9/541: sync 2024-03-16T10:45:22.430 INFO:tasks.workunit.client.1.smithi026.stdout:4/545: sync 2024-03-16T10:45:22.430 INFO:tasks.workunit.client.1.smithi026.stdout:0/595: sync 2024-03-16T10:45:22.430 INFO:tasks.workunit.client.1.smithi026.stdout:6/424: sync 2024-03-16T10:45:22.430 INFO:tasks.workunit.client.1.smithi026.stdout:3/563: sync 2024-03-16T10:45:22.430 INFO:tasks.workunit.client.1.smithi026.stdout:2/515: sync 2024-03-16T10:45:22.435 INFO:tasks.workunit.client.1.smithi026.stdout:8/512: creat d5/d28/d90/f91 x:0 0 0 2024-03-16T10:45:22.435 INFO:tasks.workunit.client.1.smithi026.stdout:4/546: chown d1/db/de/d4c/d59/d6a/l84 3 1 2024-03-16T10:45:22.436 INFO:tasks.workunit.client.1.smithi026.stdout:0/596: fdatasync d1/d12/d2f/d48/d55/d60/d9b/f6d 0 2024-03-16T10:45:22.438 INFO:tasks.workunit.client.1.smithi026.stdout:2/516: creat d2/d29/d3b/fb1 x:0 0 0 2024-03-16T10:45:22.440 INFO:tasks.workunit.client.1.smithi026.stdout:4/547: chown d1/d85/da2/da8 71832 1 2024-03-16T10:45:22.441 INFO:tasks.workunit.client.1.smithi026.stdout:4/548: fdatasync d1/f18 0 2024-03-16T10:45:22.442 INFO:tasks.workunit.client.1.smithi026.stdout:9/542: rename d0/da/d14/c1c to d0/d5f/d7f/d8c/cb9 0 2024-03-16T10:45:22.443 INFO:tasks.workunit.client.1.smithi026.stdout:2/517: write d2/d29/d36/f58 [2842373,3198] 0 2024-03-16T10:45:22.445 INFO:tasks.workunit.client.1.smithi026.stdout:8/513: creat d5/d5d/f92 x:0 0 0 2024-03-16T10:45:22.445 INFO:tasks.workunit.client.1.smithi026.stdout:6/425: dwrite d0/d10/d79/d7b/f7e [0,4194304] 0 2024-03-16T10:45:22.449 INFO:tasks.workunit.client.1.smithi026.stdout:8/514: chown d5/d5d 10736 1 2024-03-16T10:45:22.451 INFO:tasks.workunit.client.1.smithi026.stdout:1/517: dread d0/d17/d1f/d63/d6a/d7c/f82 [0,4194304] 0 2024-03-16T10:45:22.451 INFO:tasks.workunit.client.1.smithi026.stdout:3/564: dread d4/dc/f11 [4194304,4194304] 0 2024-03-16T10:45:22.454 INFO:tasks.workunit.client.1.smithi026.stdout:9/543: mknod d0/da/d14/da7/cba 0 2024-03-16T10:45:22.459 INFO:tasks.workunit.client.1.smithi026.stdout:1/518: write d0/d17/d1f/d63/d6a/f94 [696374,98361] 0 2024-03-16T10:45:22.460 INFO:tasks.workunit.client.1.smithi026.stdout:6/426: mknod d0/d10/c97 0 2024-03-16T10:45:22.461 INFO:tasks.workunit.client.1.smithi026.stdout:2/518: write d2/d29/f3a [2426131,116424] 0 2024-03-16T10:45:22.463 INFO:tasks.workunit.client.1.smithi026.stdout:8/515: dwrite d5/f7f [0,4194304] 0 2024-03-16T10:45:22.467 INFO:tasks.workunit.client.1.smithi026.stdout:8/516: readlink d5/d23/l54 0 2024-03-16T10:45:22.469 INFO:tasks.workunit.client.1.smithi026.stdout:4/549: write d1/db/de/f1d [5758055,7046] 0 2024-03-16T10:45:22.473 INFO:tasks.workunit.client.1.smithi026.stdout:1/519: dwrite d0/f1e [0,4194304] 0 2024-03-16T10:45:22.475 INFO:tasks.workunit.client.1.smithi026.stdout:4/550: write d1/db/de/d4c/fa1 [221937,122588] 0 2024-03-16T10:45:22.476 INFO:tasks.workunit.client.1.smithi026.stdout:3/565: dwrite d4/dc/dd/fb4 [0,4194304] 0 2024-03-16T10:45:22.480 INFO:tasks.workunit.client.1.smithi026.stdout:8/517: mkdir d5/d7/d2c/d36/d59/d93 0 2024-03-16T10:45:22.481 INFO:tasks.workunit.client.1.smithi026.stdout:3/566: readlink d4/dc/dd/d60/d62/d4f/l5d 0 2024-03-16T10:45:22.482 INFO:tasks.workunit.client.1.smithi026.stdout:2/519: truncate d2/d29/d85/f77 4526575 0 2024-03-16T10:45:22.485 INFO:tasks.workunit.client.1.smithi026.stdout:1/520: unlink d0/d17/d4d/d70/l7b 0 2024-03-16T10:45:22.489 INFO:tasks.workunit.client.1.smithi026.stdout:1/521: dread - d0/d17/d1f/d20/d65/f8c zero size 2024-03-16T10:45:22.493 INFO:tasks.workunit.client.1.smithi026.stdout:8/518: dwrite d5/d28/f31 [0,4194304] 0 2024-03-16T10:45:22.494 INFO:tasks.workunit.client.1.smithi026.stdout:4/551: dwrite d1/db/de/d4c/d79/daa/d44/d48/d8e/f93 [0,4194304] 0 2024-03-16T10:45:22.498 INFO:tasks.workunit.client.1.smithi026.stdout:4/552: fdatasync d1/db/de/d4c/d81/f94 0 2024-03-16T10:45:22.502 INFO:tasks.workunit.client.1.smithi026.stdout:3/567: dread d4/f41 [0,4194304] 0 2024-03-16T10:45:22.502 INFO:tasks.workunit.client.1.smithi026.stdout:4/553: stat d1/l35 0 2024-03-16T10:45:22.502 INFO:tasks.workunit.client.1.smithi026.stdout:9/544: link d0/da/d16/c35 d0/cbb 0 2024-03-16T10:45:22.505 INFO:tasks.workunit.client.1.smithi026.stdout:6/427: rename d0/d28/d2d/c59 to d0/c98 0 2024-03-16T10:45:22.512 INFO:tasks.workunit.client.1.smithi026.stdout:1/522: mkdir d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4 0 2024-03-16T10:45:22.513 INFO:tasks.workunit.client.1.smithi026.stdout:1/523: fdatasync d0/d17/d1f/d63/d6a/d7c/d6b/f87 0 2024-03-16T10:45:22.516 INFO:tasks.workunit.client.1.smithi026.stdout:8/519: write d5/dc/d3f/f46 [2975004,69163] 0 2024-03-16T10:45:22.516 INFO:tasks.workunit.client.1.smithi026.stdout:3/568: chown d4/dc/d31/d48/d7f/l84 252406 1 2024-03-16T10:45:22.518 INFO:tasks.workunit.client.1.smithi026.stdout:9/545: creat d0/da/d16/d17/fbc x:0 0 0 2024-03-16T10:45:22.520 INFO:tasks.workunit.client.1.smithi026.stdout:4/554: unlink d1/db/de/f65 0 2024-03-16T10:45:22.521 INFO:tasks.workunit.client.1.smithi026.stdout:4/555: fsync d1/f18 0 2024-03-16T10:45:22.523 INFO:tasks.workunit.client.1.smithi026.stdout:2/520: creat d2/fb2 x:0 0 0 2024-03-16T10:45:22.526 INFO:tasks.workunit.client.1.smithi026.stdout:6/428: write d0/d28/f37 [4545873,15893] 0 2024-03-16T10:45:22.530 INFO:tasks.workunit.client.1.smithi026.stdout:8/520: unlink d5/f4a 0 2024-03-16T10:45:22.531 INFO:tasks.workunit.client.1.smithi026.stdout:8/521: fdatasync d5/dc/f38 0 2024-03-16T10:45:22.533 INFO:tasks.workunit.client.1.smithi026.stdout:4/556: truncate d1/db/de/d4c/f4d 1251563 0 2024-03-16T10:45:22.536 INFO:tasks.workunit.client.1.smithi026.stdout:1/524: unlink d0/d17/d1f/d20/l46 0 2024-03-16T10:45:22.539 INFO:tasks.workunit.client.1.smithi026.stdout:8/522: read d5/d7/f6b [1043973,76733] 0 2024-03-16T10:45:22.540 INFO:tasks.workunit.client.1.smithi026.stdout:9/546: link d0/d63/d6b/f81 d0/d5f/d7f/d8c/fbd 0 2024-03-16T10:45:22.597 INFO:tasks.workunit.client.1.smithi026.stdout:5/536: sync 2024-03-16T10:45:22.597 INFO:tasks.workunit.client.1.smithi026.stdout:7/577: sync 2024-03-16T10:45:22.612 INFO:tasks.workunit.client.1.smithi026.stdout:4/557: rmdir d1/db/de/d4c/d79/daa 39 2024-03-16T10:45:22.613 INFO:tasks.workunit.client.1.smithi026.stdout:1/525: creat d0/d17/d1f/d63/d6a/d7c/d84/fa5 x:0 0 0 2024-03-16T10:45:22.614 INFO:tasks.workunit.client.1.smithi026.stdout:5/537: write d1/db/d1f/d61/d65/f83 [4474190,98079] 0 2024-03-16T10:45:22.616 INFO:tasks.workunit.client.1.smithi026.stdout:3/569: rename d4/dc/f25 to d4/dc/dd/d60/d62/d4f/d5a/fbf 0 2024-03-16T10:45:22.618 INFO:tasks.workunit.client.1.smithi026.stdout:4/558: read d1/db/f42 [2526017,37155] 0 2024-03-16T10:45:22.690 INFO:tasks.workunit.client.1.smithi026.stdout:9/547: truncate d0/da/d16/d5d/d72/d62/f65 5011767 0 2024-03-16T10:45:22.690 INFO:tasks.workunit.client.1.smithi026.stdout:7/578: mknod d6/d35/d3a/d4e/cb5 0 2024-03-16T10:45:22.692 INFO:tasks.workunit.client.1.smithi026.stdout:1/526: dwrite d0/d17/f37 [0,4194304] 0 2024-03-16T10:45:22.693 INFO:tasks.workunit.client.1.smithi026.stdout:5/538: symlink d1/d1c/d4c/d4d/lbd 0 2024-03-16T10:45:22.693 INFO:tasks.workunit.client.1.smithi026.stdout:4/559: fsync d1/db/de/d4c/d81/f94 0 2024-03-16T10:45:22.695 INFO:tasks.workunit.client.1.smithi026.stdout:9/548: chown d0/da/d13/d6e/f74 2080866041 1 2024-03-16T10:45:22.696 INFO:tasks.workunit.client.1.smithi026.stdout:5/539: dread - d1/db/d1f/f85 zero size 2024-03-16T10:45:22.698 INFO:tasks.workunit.client.1.smithi026.stdout:7/579: symlink d6/d35/d3a/d4e/da1/lb6 0 2024-03-16T10:45:22.698 INFO:tasks.workunit.client.1.smithi026.stdout:2/521: dwrite d2/d29/d17/f27 [0,4194304] 0 2024-03-16T10:45:22.701 INFO:tasks.workunit.client.1.smithi026.stdout:7/580: fsync d6/d2f/f4a 0 2024-03-16T10:45:22.701 INFO:tasks.workunit.client.1.smithi026.stdout:4/560: mkdir d1/db/de/d4c/d59/d6a/db2 0 2024-03-16T10:45:22.707 INFO:tasks.workunit.client.1.smithi026.stdout:7/581: chown d6/de/cf 2329711 1 2024-03-16T10:45:22.709 INFO:tasks.workunit.client.1.smithi026.stdout:9/549: dwrite d0/f66 [0,4194304] 0 2024-03-16T10:45:22.711 INFO:tasks.workunit.client.1.smithi026.stdout:1/527: creat d0/fa6 x:0 0 0 2024-03-16T10:45:22.716 INFO:tasks.workunit.client.1.smithi026.stdout:9/550: write d0/da/d16/d17/d44/f1e [2685246,91679] 0 2024-03-16T10:45:22.716 INFO:tasks.workunit.client.1.smithi026.stdout:5/540: dwrite d1/d1c/d2a/f57 [4194304,4194304] 0 2024-03-16T10:45:22.718 INFO:tasks.workunit.client.1.smithi026.stdout:9/551: chown d0/da/d14/f6a 171123 1 2024-03-16T10:45:22.719 INFO:tasks.workunit.client.1.smithi026.stdout:5/541: read d1/d1c/d2a/d31/f97 [821287,119256] 0 2024-03-16T10:45:22.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:22 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:22.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:22 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:22.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:22 smithi026 ceph-mon[37041]: pgmap v6: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 151 op/s 2024-03-16T10:45:22.735 INFO:tasks.workunit.client.1.smithi026.stdout:9/552: mknod d0/d63/d6b/cbe 0 2024-03-16T10:45:22.738 INFO:tasks.workunit.client.1.smithi026.stdout:4/561: dread d1/db/de/d4c/d79/daa/d61/f75 [4194304,4194304] 0 2024-03-16T10:45:22.739 INFO:tasks.workunit.client.1.smithi026.stdout:7/582: dread d6/d2f/daf/d3f/d9d/f97 [0,4194304] 0 2024-03-16T10:45:22.742 INFO:tasks.workunit.client.1.smithi026.stdout:7/583: dread d6/f83 [0,4194304] 0 2024-03-16T10:45:22.744 INFO:tasks.workunit.client.1.smithi026.stdout:7/584: write d6/de/f64 [669097,9445] 0 2024-03-16T10:45:22.744 INFO:tasks.workunit.client.1.smithi026.stdout:2/522: getdents d2/d29/d51/d55 0 2024-03-16T10:45:22.748 INFO:tasks.workunit.client.1.smithi026.stdout:1/528: unlink d0/d17/d1f/f35 0 2024-03-16T10:45:22.749 INFO:tasks.workunit.client.1.smithi026.stdout:9/553: rmdir d0/da/d16/d5d/d72 39 2024-03-16T10:45:22.751 INFO:tasks.workunit.client.1.smithi026.stdout:4/562: unlink d1/db/de/d4c/d79/daa/f54 0 2024-03-16T10:45:22.752 INFO:tasks.workunit.client.1.smithi026.stdout:5/542: mkdir d1/d1c/d2a/dbe 0 2024-03-16T10:45:22.759 INFO:tasks.workunit.client.1.smithi026.stdout:3/570: write d4/dc/dd/d60/d62/d4f/d5a/fbf [1942561,58324] 0 2024-03-16T10:45:22.762 INFO:tasks.workunit.client.1.smithi026.stdout:5/543: symlink d1/db/d1f/lbf 0 2024-03-16T10:45:22.765 INFO:tasks.workunit.client.1.smithi026.stdout:2/523: unlink d2/d29/d85/f77 0 2024-03-16T10:45:22.766 INFO:tasks.workunit.client.1.smithi026.stdout:3/571: dwrite d4/dc/dd/d60/d62/d4f/d5a/fbf [0,4194304] 0 2024-03-16T10:45:22.768 INFO:tasks.workunit.client.1.smithi026.stdout:4/563: mkdir d1/db/de/d4c/d59/d6a/db2/db3 0 2024-03-16T10:45:22.769 INFO:tasks.workunit.client.1.smithi026.stdout:3/572: write d4/dc/d31/d48/d69/d78/db3/fb5 [95689,70596] 0 2024-03-16T10:45:22.769 INFO:tasks.workunit.client.1.smithi026.stdout:5/544: mknod d1/d1c/d4c/d4d/cc0 0 2024-03-16T10:45:22.772 INFO:tasks.workunit.client.1.smithi026.stdout:5/545: read d1/db/f38 [1869103,111835] 0 2024-03-16T10:45:22.775 INFO:tasks.workunit.client.1.smithi026.stdout:0/597: sync 2024-03-16T10:45:22.775 INFO:tasks.workunit.client.1.smithi026.stdout:6/429: sync 2024-03-16T10:45:22.778 INFO:tasks.workunit.client.1.smithi026.stdout:3/573: fdatasync d4/dc/dd/f21 0 2024-03-16T10:45:22.781 INFO:tasks.workunit.client.1.smithi026.stdout:2/524: rename d2/d29/d17/d1a/d2b/d4e/c4f to d2/d29/d36/d98/cb3 0 2024-03-16T10:45:22.784 INFO:tasks.workunit.client.1.smithi026.stdout:7/585: write d6/d35/f66 [166543,83391] 0 2024-03-16T10:45:22.785 INFO:tasks.workunit.client.1.smithi026.stdout:1/529: write d0/d17/d1f/d63/d6a/d7c/d6b/f7a [793232,47826] 0 2024-03-16T10:45:22.787 INFO:tasks.workunit.client.1.smithi026.stdout:9/554: write d0/da/d16/d17/d44/f42 [143566,5348] 0 2024-03-16T10:45:22.790 INFO:tasks.workunit.client.1.smithi026.stdout:6/430: creat d0/d41/d4f/f99 x:0 0 0 2024-03-16T10:45:22.790 INFO:tasks.workunit.client.1.smithi026.stdout:6/431: chown d0/d8a 1974 1 2024-03-16T10:45:22.792 INFO:tasks.workunit.client.1.smithi026.stdout:6/432: chown d0/d28/d49/f78 3461922 1 2024-03-16T10:45:22.792 INFO:tasks.workunit.client.1.smithi026.stdout:0/598: dwrite d1/f40 [0,4194304] 0 2024-03-16T10:45:22.794 INFO:tasks.workunit.client.1.smithi026.stdout:3/574: read d4/dc/d73/d83/d57/f9b [4104829,81956] 0 2024-03-16T10:45:22.795 INFO:tasks.workunit.client.1.smithi026.stdout:3/575: chown d4/dc 63734 1 2024-03-16T10:45:22.796 INFO:tasks.workunit.client.1.smithi026.stdout:4/564: rename d1/db/de/f70 to d1/db/de/d4c/d59/d6a/db2/fb4 0 2024-03-16T10:45:22.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:22 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:22.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:22 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:22.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:22 smithi002 ceph-mon[29248]: pgmap v6: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 151 op/s 2024-03-16T10:45:22.798 INFO:tasks.workunit.client.1.smithi026.stdout:4/565: dread - d1/db/de/d4c/d6b/fae zero size 2024-03-16T10:45:22.800 INFO:tasks.workunit.client.1.smithi026.stdout:5/546: symlink d1/d1c/lc1 0 2024-03-16T10:45:22.802 INFO:tasks.workunit.client.1.smithi026.stdout:1/530: creat d0/d17/d1f/d63/d6a/d7c/d84/fa7 x:0 0 0 2024-03-16T10:45:22.804 INFO:tasks.workunit.client.1.smithi026.stdout:0/599: mkdir d1/d12/dcd 0 2024-03-16T10:45:22.806 INFO:tasks.workunit.client.1.smithi026.stdout:3/576: rmdir d4/dc/d73/d83 39 2024-03-16T10:45:22.811 INFO:tasks.workunit.client.1.smithi026.stdout:5/547: creat d1/d1c/d2a/d8e/fc2 x:0 0 0 2024-03-16T10:45:22.813 INFO:tasks.workunit.client.1.smithi026.stdout:0/600: mknod d1/d12/d2f/d48/cce 0 2024-03-16T10:45:22.816 INFO:tasks.workunit.client.1.smithi026.stdout:9/555: dread d0/f3c [0,4194304] 0 2024-03-16T10:45:22.817 INFO:tasks.workunit.client.1.smithi026.stdout:7/586: creat d6/fb7 x:0 0 0 2024-03-16T10:45:22.818 INFO:tasks.workunit.client.1.smithi026.stdout:9/556: fsync d0/da/f3b 0 2024-03-16T10:45:22.820 INFO:tasks.workunit.client.1.smithi026.stdout:7/587: write d6/d1e/fa9 [740740,15842] 0 2024-03-16T10:45:22.820 INFO:tasks.workunit.client.1.smithi026.stdout:5/548: dwrite d1/db/d1f/f5c [4194304,4194304] 0 2024-03-16T10:45:22.823 INFO:tasks.workunit.client.1.smithi026.stdout:6/433: rename d0/d28/f42 to d0/f9a 0 2024-03-16T10:45:22.824 INFO:tasks.workunit.client.1.smithi026.stdout:2/525: getdents d2/d29/d51/d72 0 2024-03-16T10:45:22.829 INFO:tasks.workunit.client.1.smithi026.stdout:4/566: link d1/db/de/d4c/d79/daa/d44/d48/c91 d1/db/de/d4c/d6f/cb5 0 2024-03-16T10:45:22.829 INFO:tasks.workunit.client.1.smithi026.stdout:4/567: readlink d1/l8c 0 2024-03-16T10:45:22.831 INFO:tasks.workunit.client.1.smithi026.stdout:5/549: symlink d1/d6e/lc3 0 2024-03-16T10:45:22.832 INFO:tasks.workunit.client.1.smithi026.stdout:7/588: symlink d6/d35/d3a/lb8 0 2024-03-16T10:45:22.833 INFO:tasks.workunit.client.1.smithi026.stdout:5/550: dread d1/d1c/d2a/f47 [0,4194304] 0 2024-03-16T10:45:22.837 INFO:tasks.workunit.client.1.smithi026.stdout:1/531: rename d0/d17/d1f/la1 to d0/d17/d1f/d63/d6a/d7c/d84/d9d/la8 0 2024-03-16T10:45:22.840 INFO:tasks.workunit.client.1.smithi026.stdout:0/601: mkdir d1/d12/d2f/d48/d55/d60/d9b/dcf 0 2024-03-16T10:45:22.840 INFO:tasks.workunit.client.1.smithi026.stdout:0/602: fsync d1/d18/d1a/d2b/d4f/f80 0 2024-03-16T10:45:22.847 INFO:tasks.workunit.client.1.smithi026.stdout:9/557: dwrite d0/f82 [0,4194304] 0 2024-03-16T10:45:22.851 INFO:tasks.workunit.client.1.smithi026.stdout:5/551: unlink d1/d1c/d2a/d8e/fc2 0 2024-03-16T10:45:22.853 INFO:tasks.workunit.client.1.smithi026.stdout:6/434: dwrite d0/d10/f4a [0,4194304] 0 2024-03-16T10:45:22.853 INFO:tasks.workunit.client.1.smithi026.stdout:1/532: mknod d0/d17/d1f/d63/d6a/d7c/d2e/ca9 0 2024-03-16T10:45:22.858 INFO:tasks.workunit.client.1.smithi026.stdout:0/603: unlink d1/d12/d2f/d48/d55/d60/l8b 0 2024-03-16T10:45:22.867 INFO:tasks.workunit.client.1.smithi026.stdout:7/589: fsync d6/de/d16/f5e 0 2024-03-16T10:45:22.869 INFO:tasks.workunit.client.1.smithi026.stdout:2/526: truncate d2/d29/d36/f3e 418076 0 2024-03-16T10:45:22.874 INFO:tasks.workunit.client.1.smithi026.stdout:8/523: sync 2024-03-16T10:45:22.874 INFO:tasks.workunit.client.1.smithi026.stdout:5/552: mknod d1/d1c/d2a/d55/db1/cc4 0 2024-03-16T10:45:22.879 INFO:tasks.workunit.client.1.smithi026.stdout:4/568: link d1/d85/da2/fb0 d1/db/de/d6e/fb6 0 2024-03-16T10:45:22.879 INFO:tasks.workunit.client.1.smithi026.stdout:9/558: dread d0/da/f27 [0,4194304] 0 2024-03-16T10:45:22.881 INFO:tasks.workunit.client.1.smithi026.stdout:1/533: write d0/d17/d1f/d63/d6a/d7c/d2e/d54/f5e [138914,34871] 0 2024-03-16T10:45:22.883 INFO:tasks.workunit.client.1.smithi026.stdout:1/534: write d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/f89 [462697,69797] 0 2024-03-16T10:45:22.885 INFO:tasks.workunit.client.1.smithi026.stdout:2/527: unlink d2/d29/d51/d55/d6d/f82 0 2024-03-16T10:45:22.887 INFO:tasks.workunit.client.1.smithi026.stdout:1/535: truncate d0/d17/d1f/d63/d6a/d7c/d84/fa5 1021328 0 2024-03-16T10:45:22.887 INFO:tasks.workunit.client.1.smithi026.stdout:5/553: rmdir d1/db/d1f/d61/d65/d82 39 2024-03-16T10:45:22.888 INFO:tasks.workunit.client.1.smithi026.stdout:1/536: read - d0/f7f zero size 2024-03-16T10:45:22.889 INFO:tasks.workunit.client.1.smithi026.stdout:2/528: dread d2/d29/d17/d38/f7d [0,4194304] 0 2024-03-16T10:45:22.890 INFO:tasks.workunit.client.1.smithi026.stdout:5/554: write d1/d1c/d2a/d31/d69/d51/fa2 [1030242,31510] 0 2024-03-16T10:45:22.890 INFO:tasks.workunit.client.1.smithi026.stdout:1/537: fsync d0/d17/d1f/d63/d6a/fa3 0 2024-03-16T10:45:22.891 INFO:tasks.workunit.client.1.smithi026.stdout:0/604: unlink d1/d12/d2f/d48/d55/d60/d9b/d11/cc9 0 2024-03-16T10:45:22.893 INFO:tasks.workunit.client.1.smithi026.stdout:0/605: chown d1/d12/d2f/dba/fc7 15 1 2024-03-16T10:45:22.894 INFO:tasks.workunit.client.1.smithi026.stdout:9/559: unlink d0/d63/d6b/cbe 0 2024-03-16T10:45:22.895 INFO:tasks.workunit.client.1.smithi026.stdout:0/606: write d1/d12/d2f/d48/da5/fb8 [998467,84353] 0 2024-03-16T10:45:22.896 INFO:tasks.workunit.client.1.smithi026.stdout:4/569: symlink d1/db/de/d4c/d59/d6a/lb7 0 2024-03-16T10:45:22.897 INFO:tasks.workunit.client.1.smithi026.stdout:1/538: dread d0/d17/f37 [0,4194304] 0 2024-03-16T10:45:22.901 INFO:tasks.workunit.client.1.smithi026.stdout:6/435: truncate d0/d28/d2d/f68 339571 0 2024-03-16T10:45:22.901 INFO:tasks.workunit.client.1.smithi026.stdout:6/436: stat d0/d5b/c3b 0 2024-03-16T10:45:22.902 INFO:tasks.workunit.client.1.smithi026.stdout:6/437: dread - d0/d62/f84 zero size 2024-03-16T10:45:22.908 INFO:tasks.workunit.client.1.smithi026.stdout:0/607: dread d1/d12/f4b [0,4194304] 0 2024-03-16T10:45:22.909 INFO:tasks.workunit.client.1.smithi026.stdout:3/577: rename d4/l28 to d4/dc/dd/lc0 0 2024-03-16T10:45:22.911 INFO:tasks.workunit.client.1.smithi026.stdout:0/608: write d1/d12/d2f/d48/d55/d60/d9b/d51/f5e [263242,4809] 0 2024-03-16T10:45:22.914 INFO:tasks.workunit.client.1.smithi026.stdout:7/590: truncate d6/d35/d3a/d59/f96 1493990 0 2024-03-16T10:45:22.918 INFO:tasks.workunit.client.1.smithi026.stdout:3/578: dwrite d4/d89/d97/d7d/f88 [4194304,4194304] 0 2024-03-16T10:45:22.920 INFO:tasks.workunit.client.1.smithi026.stdout:3/579: chown d4/dc/dd/d60/f64 28537 1 2024-03-16T10:45:22.920 INFO:tasks.workunit.client.1.smithi026.stdout:9/560: rmdir d0/d5f/da2 39 2024-03-16T10:45:22.922 INFO:tasks.workunit.client.1.smithi026.stdout:3/580: write d4/dc/f13 [1235347,52585] 0 2024-03-16T10:45:22.929 INFO:tasks.workunit.client.1.smithi026.stdout:1/539: unlink d0/d17/d1f/d63/d6a/d7c/c80 0 2024-03-16T10:45:22.930 INFO:tasks.workunit.client.1.smithi026.stdout:0/609: dread d1/d12/d2f/d48/d55/d60/fa0 [0,4194304] 0 2024-03-16T10:45:22.935 INFO:tasks.workunit.client.1.smithi026.stdout:8/524: rename d5/d28/f3c to d5/d7/d7b/f94 0 2024-03-16T10:45:22.942 INFO:tasks.workunit.client.1.smithi026.stdout:3/581: fsync d4/dc/f20 0 2024-03-16T10:45:22.945 INFO:tasks.workunit.client.1.smithi026.stdout:4/570: symlink d1/lb8 0 2024-03-16T10:45:22.946 INFO:tasks.workunit.client.1.smithi026.stdout:4/571: dread d1/f22 [0,4194304] 0 2024-03-16T10:45:22.953 INFO:tasks.workunit.client.1.smithi026.stdout:2/529: rename d2/d29/d17/d1a/d2b/d2e/f3f to d2/d29/d17/d1a/d2b/d2e/d60/d7f/fb4 0 2024-03-16T10:45:22.954 INFO:tasks.workunit.client.1.smithi026.stdout:8/525: unlink d5/c16 0 2024-03-16T10:45:22.955 INFO:tasks.workunit.client.1.smithi026.stdout:9/561: write d0/da/d14/f6a [472938,38930] 0 2024-03-16T10:45:22.956 INFO:tasks.workunit.client.1.smithi026.stdout:9/562: write d0/d63/d6b/fa5 [904869,112260] 0 2024-03-16T10:45:22.958 INFO:tasks.workunit.client.1.smithi026.stdout:8/526: dread d5/d7/d2c/d36/f4c [0,4194304] 0 2024-03-16T10:45:22.958 INFO:tasks.workunit.client.1.smithi026.stdout:7/591: link d6/de/f64 d6/d35/d3a/d59/fb9 0 2024-03-16T10:45:22.960 INFO:tasks.workunit.client.1.smithi026.stdout:9/563: dread d0/da/d14/f6a [0,4194304] 0 2024-03-16T10:45:22.962 INFO:tasks.workunit.client.1.smithi026.stdout:1/540: mkdir d0/d17/d4d/daa 0 2024-03-16T10:45:22.964 INFO:tasks.workunit.client.1.smithi026.stdout:0/610: symlink d1/d18/ld0 0 2024-03-16T10:45:22.968 INFO:tasks.workunit.client.1.smithi026.stdout:3/582: rename d4/dc/dd/d60/d62/d4f/c92 to d4/dc/dd/d47/d53/cc1 0 2024-03-16T10:45:22.969 INFO:tasks.workunit.client.1.smithi026.stdout:2/530: symlink d2/d29/d17/d1a/d2b/d4e/d6f/lb5 0 2024-03-16T10:45:22.971 INFO:tasks.workunit.client.1.smithi026.stdout:2/531: truncate d2/d29/d17/d38/f81 1372865 0 2024-03-16T10:45:22.972 INFO:tasks.workunit.client.1.smithi026.stdout:9/564: dread d0/da/d16/d17/d44/f1e [0,4194304] 0 2024-03-16T10:45:22.975 INFO:tasks.workunit.client.1.smithi026.stdout:5/555: getdents d1/d1c/d2a 0 2024-03-16T10:45:22.976 INFO:tasks.workunit.client.1.smithi026.stdout:8/527: symlink d5/dc/d3f/l95 0 2024-03-16T10:45:22.978 INFO:tasks.workunit.client.1.smithi026.stdout:5/556: dread d1/db/f59 [0,4194304] 0 2024-03-16T10:45:22.982 INFO:tasks.workunit.client.1.smithi026.stdout:4/572: creat d1/db/de/d4c/d79/daf/fb9 x:0 0 0 2024-03-16T10:45:22.984 INFO:tasks.workunit.client.1.smithi026.stdout:1/541: readlink d0/l2d 0 2024-03-16T10:45:22.991 INFO:tasks.workunit.client.1.smithi026.stdout:7/592: dread d6/f24 [4194304,4194304] 0 2024-03-16T10:45:22.991 INFO:tasks.workunit.client.1.smithi026.stdout:3/583: mkdir d4/dc/dd/d80/dc2 0 2024-03-16T10:45:23.002 INFO:tasks.workunit.client.1.smithi026.stdout:5/557: rmdir d1/db/d27 39 2024-03-16T10:45:23.002 INFO:tasks.workunit.client.1.smithi026.stdout:5/558: fsync d1/d1c/f3c 0 2024-03-16T10:45:23.007 INFO:tasks.workunit.client.1.smithi026.stdout:0/611: rename d1/d18 to d1/d12/d2f/daa/dd1 0 2024-03-16T10:45:23.007 INFO:tasks.workunit.client.1.smithi026.stdout:5/559: dread d1/f18 [0,4194304] 0 2024-03-16T10:45:23.008 INFO:tasks.workunit.client.1.smithi026.stdout:0/612: stat d1/d12/d2f/d48 0 2024-03-16T10:45:23.009 INFO:tasks.workunit.client.1.smithi026.stdout:7/593: creat d6/d2f/daf/d3f/d9d/fba x:0 0 0 2024-03-16T10:45:23.012 INFO:tasks.workunit.client.1.smithi026.stdout:8/528: dread d5/d7/d2c/f70 [0,4194304] 0 2024-03-16T10:45:23.018 INFO:tasks.workunit.client.1.smithi026.stdout:3/584: mknod d4/dc/d31/d48/d69/cc3 0 2024-03-16T10:45:23.021 INFO:tasks.workunit.client.1.smithi026.stdout:4/573: dwrite d1/db/f67 [8388608,4194304] 0 2024-03-16T10:45:23.029 INFO:tasks.workunit.client.1.smithi026.stdout:3/585: dwrite d4/dc/f96 [0,4194304] 0 2024-03-16T10:45:23.030 INFO:tasks.workunit.client.1.smithi026.stdout:2/532: rename d2/d29/d17/d1a/d2b/d4b/f5d to d2/d29/d17/d1a/d2b/d69/fb6 0 2024-03-16T10:45:23.035 INFO:tasks.workunit.client.1.smithi026.stdout:2/533: dread d2/d29/d17/d1a/d2b/d4b/fb0 [0,4194304] 0 2024-03-16T10:45:23.036 INFO:tasks.workunit.client.1.smithi026.stdout:7/594: creat d6/d35/d3a/d4e/fbb x:0 0 0 2024-03-16T10:45:23.053 INFO:tasks.workunit.client.1.smithi026.stdout:0/613: dwrite d1/d12/d2f/daa/dd1/fb3 [0,4194304] 0 2024-03-16T10:45:23.054 INFO:tasks.workunit.client.1.smithi026.stdout:0/614: write d1/d12/d2f/d48/d55/d60/d9b/d57/fbb [839731,102754] 0 2024-03-16T10:45:23.060 INFO:tasks.workunit.client.1.smithi026.stdout:0/615: dwrite d1/d12/d2f/daa/dd1/d1a/d2b/d4f/f56 [0,4194304] 0 2024-03-16T10:45:23.061 INFO:tasks.workunit.client.1.smithi026.stdout:0/616: dread - d1/d12/d2f/d48/d55/d60/d9b/d11/fb7 zero size 2024-03-16T10:45:23.226 INFO:tasks.workunit.client.1.smithi026.stdout:6/438: sync 2024-03-16T10:45:23.227 INFO:tasks.workunit.client.1.smithi026.stdout:6/439: chown d0/d10/l29 92270106 1 2024-03-16T10:45:23.265 INFO:tasks.workunit.client.1.smithi026.stdout:8/529: truncate d5/d7/d2c/f50 4312958 0 2024-03-16T10:45:23.266 INFO:tasks.workunit.client.1.smithi026.stdout:5/560: fsync d1/d6e/fb7 0 2024-03-16T10:45:23.270 INFO:tasks.workunit.client.1.smithi026.stdout:3/586: truncate d4/dc/d73/d83/d57/fab 678448 0 2024-03-16T10:45:23.272 INFO:tasks.workunit.client.1.smithi026.stdout:9/565: rename d0/da/d16/d17/d44/c3a to d0/da/d13/d6e/cbf 0 2024-03-16T10:45:23.284 INFO:tasks.workunit.client.1.smithi026.stdout:1/542: sync 2024-03-16T10:45:23.286 INFO:tasks.workunit.client.1.smithi026.stdout:6/440: dwrite d0/d41/d4f/f77 [4194304,4194304] 0 2024-03-16T10:45:23.297 INFO:tasks.workunit.client.1.smithi026.stdout:8/530: unlink d5/d7/l2a 0 2024-03-16T10:45:23.298 INFO:tasks.workunit.client.1.smithi026.stdout:3/587: mknod d4/dc/dd/d80/dc2/cc4 0 2024-03-16T10:45:23.303 INFO:tasks.workunit.client.1.smithi026.stdout:6/441: dread d0/d28/d2d/f60 [0,4194304] 0 2024-03-16T10:45:23.303 INFO:tasks.workunit.client.1.smithi026.stdout:6/442: fdatasync d0/d41/d4f/f96 0 2024-03-16T10:45:23.307 INFO:tasks.workunit.client.1.smithi026.stdout:0/617: rename d1/d12/d2f/d48/d55/d60/d9b/l7f to d1/d12/d2f/d48/d55/d60/d9b/dcf/ld2 0 2024-03-16T10:45:23.307 INFO:tasks.workunit.client.1.smithi026.stdout:4/574: getdents d1/db/de/d4c/d79 0 2024-03-16T10:45:23.309 INFO:tasks.workunit.client.1.smithi026.stdout:1/543: dread d0/d17/d1f/d63/d6a/d7c/f30 [0,4194304] 0 2024-03-16T10:45:23.322 INFO:tasks.workunit.client.1.smithi026.stdout:0/618: creat d1/d12/d2f/d48/d55/d60/d9b/dcf/fd3 x:0 0 0 2024-03-16T10:45:23.327 INFO:tasks.workunit.client.1.smithi026.stdout:0/619: creat d1/fd4 x:0 0 0 2024-03-16T10:45:23.332 INFO:tasks.workunit.client.1.smithi026.stdout:0/620: creat d1/d12/d2f/daa/dd1/d1a/d2b/fd5 x:0 0 0 2024-03-16T10:45:23.347 INFO:tasks.workunit.client.1.smithi026.stdout:0/621: dread d1/d12/d2f/d48/d55/d60/d9b/fbe [0,4194304] 0 2024-03-16T10:45:23.350 INFO:tasks.workunit.client.1.smithi026.stdout:0/622: creat d1/d12/d2f/daa/dac/fd6 x:0 0 0 2024-03-16T10:45:23.379 INFO:tasks.workunit.client.1.smithi026.stdout:9/566: symlink d0/da/d4d/lc0 0 2024-03-16T10:45:23.389 INFO:tasks.workunit.client.1.smithi026.stdout:3/588: write d4/dc/f96 [4829995,78862] 0 2024-03-16T10:45:23.391 INFO:tasks.workunit.client.1.smithi026.stdout:3/589: write d4/f9d [215714,74091] 0 2024-03-16T10:45:23.393 INFO:tasks.workunit.client.1.smithi026.stdout:3/590: write d4/dc/dd/f24 [626547,86372] 0 2024-03-16T10:45:23.398 INFO:tasks.workunit.client.1.smithi026.stdout:3/591: mknod d4/dc/dd/d47/cc5 0 2024-03-16T10:45:23.401 INFO:tasks.workunit.client.1.smithi026.stdout:3/592: mknod d4/dc/d31/d48/d69/cc6 0 2024-03-16T10:45:23.402 INFO:tasks.workunit.client.1.smithi026.stdout:3/593: chown d4/dc/d31/f34 5616094 1 2024-03-16T10:45:23.408 INFO:tasks.workunit.client.1.smithi026.stdout:3/594: dread d4/d89/d97/d7d/f88 [4194304,4194304] 0 2024-03-16T10:45:23.412 INFO:tasks.workunit.client.1.smithi026.stdout:8/531: write d5/d23/d4e/f7d [48969,96185] 0 2024-03-16T10:45:23.414 INFO:tasks.workunit.client.1.smithi026.stdout:6/443: write d0/d62/f63 [721508,89605] 0 2024-03-16T10:45:23.415 INFO:tasks.workunit.client.1.smithi026.stdout:4/575: write d1/f77 [53147,26346] 0 2024-03-16T10:45:23.418 INFO:tasks.workunit.client.1.smithi026.stdout:6/444: rmdir d0/d62 39 2024-03-16T10:45:23.419 INFO:tasks.workunit.client.1.smithi026.stdout:4/576: creat d1/db/de/d4c/d6b/d7a/fba x:0 0 0 2024-03-16T10:45:23.423 INFO:tasks.workunit.client.1.smithi026.stdout:2/534: unlink d2/d29/fe 0 2024-03-16T10:45:23.430 INFO:tasks.workunit.client.1.smithi026.stdout:5/561: rename d1/d1c/d2a/d31 to d1/d1c/d2a/dab/dc5 0 2024-03-16T10:45:23.432 INFO:tasks.workunit.client.1.smithi026.stdout:0/623: write d1/d12/d2f/d48/d55/d60/d9b/fcc [3959847,9946] 0 2024-03-16T10:45:23.433 INFO:tasks.workunit.client.1.smithi026.stdout:0/624: stat d1/d12/d2f/d48/d55/d60/fa0 0 2024-03-16T10:45:23.434 INFO:tasks.workunit.client.1.smithi026.stdout:0/625: write d1/d12/d2f/d48/d55/d60/d9b/d57/f83 [278368,23424] 0 2024-03-16T10:45:23.435 INFO:tasks.workunit.client.1.smithi026.stdout:9/567: rename d0/da/d4d/c90 to d0/d5f/cc1 0 2024-03-16T10:45:23.436 INFO:tasks.workunit.client.1.smithi026.stdout:9/568: chown d0/da/d14/l8e 85423372 1 2024-03-16T10:45:23.437 INFO:tasks.workunit.client.1.smithi026.stdout:8/532: unlink f4 0 2024-03-16T10:45:23.439 INFO:tasks.workunit.client.1.smithi026.stdout:3/595: rename d4/dc/d73/d83/ca1 to d4/dc/d31/d48/d7f/cc7 0 2024-03-16T10:45:23.441 INFO:tasks.workunit.client.1.smithi026.stdout:1/544: creat d0/fab x:0 0 0 2024-03-16T10:45:23.442 INFO:tasks.workunit.client.1.smithi026.stdout:0/626: mkdir d1/d12/d2f/d48/d55/d60/d9b/d57/dd7 0 2024-03-16T10:45:23.444 INFO:tasks.workunit.client.1.smithi026.stdout:8/533: mkdir d5/d7/d7b/d96 0 2024-03-16T10:45:23.446 INFO:tasks.workunit.client.1.smithi026.stdout:9/569: dread d0/d5f/da2/fa6 [0,4194304] 0 2024-03-16T10:45:23.446 INFO:tasks.workunit.client.1.smithi026.stdout:7/595: unlink d6/c98 0 2024-03-16T10:45:23.449 INFO:tasks.workunit.client.1.smithi026.stdout:8/534: creat d5/d5d/f97 x:0 0 0 2024-03-16T10:45:23.450 INFO:tasks.workunit.client.1.smithi026.stdout:3/596: dwrite d4/dc/d31/d48/d69/d78/db3/fb5 [0,4194304] 0 2024-03-16T10:45:23.451 INFO:tasks.workunit.client.1.smithi026.stdout:0/627: truncate d1/d12/d2f/d48/d55/d60/d9b/fcc 4913778 0 2024-03-16T10:45:23.453 INFO:tasks.workunit.client.1.smithi026.stdout:6/445: rename d0/c3d to d0/d10/c9b 0 2024-03-16T10:45:23.454 INFO:tasks.workunit.client.1.smithi026.stdout:9/570: creat d0/d63/d6b/fc2 x:0 0 0 2024-03-16T10:45:23.454 INFO:tasks.workunit.client.1.smithi026.stdout:8/535: truncate d5/d28/f39 1075241 0 2024-03-16T10:45:23.456 INFO:tasks.workunit.client.1.smithi026.stdout:3/597: mknod d4/dc/dd/d47/d53/cc8 0 2024-03-16T10:45:23.460 INFO:tasks.workunit.client.1.smithi026.stdout:6/446: creat d0/d28/f9c x:0 0 0 2024-03-16T10:45:23.461 INFO:tasks.workunit.client.1.smithi026.stdout:0/628: write d1/d12/d2f/daa/dd1/d1a/d2b/d4f/f80 [3693517,68675] 0 2024-03-16T10:45:23.461 INFO:tasks.workunit.client.1.smithi026.stdout:7/596: dwrite d6/d2f/daf/d3f/d70/fa3 [0,4194304] 0 2024-03-16T10:45:23.465 INFO:tasks.workunit.client.1.smithi026.stdout:6/447: truncate d0/d28/d2d/f4c 5111098 0 2024-03-16T10:45:23.467 INFO:tasks.workunit.client.1.smithi026.stdout:8/536: symlink d5/d7/d2c/d36/l98 0 2024-03-16T10:45:23.467 INFO:tasks.workunit.client.1.smithi026.stdout:8/537: fsync d5/dc/d3f/f46 0 2024-03-16T10:45:23.469 INFO:tasks.workunit.client.1.smithi026.stdout:1/545: rename d0/d17/d1f/d63/d6a/d7c/d84/d9d/la8 to d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/lac 0 2024-03-16T10:45:23.475 INFO:tasks.workunit.client.1.smithi026.stdout:7/597: dwrite d6/d35/d3a/f8e [0,4194304] 0 2024-03-16T10:45:23.479 INFO:tasks.workunit.client.1.smithi026.stdout:4/577: write d1/fac [769180,119794] 0 2024-03-16T10:45:23.481 INFO:tasks.workunit.client.1.smithi026.stdout:1/546: dread d0/d17/d1f/d63/d6a/d7c/f59 [0,4194304] 0 2024-03-16T10:45:23.481 INFO:tasks.workunit.client.1.smithi026.stdout:4/578: stat d1/db/de/d6e/fb6 0 2024-03-16T10:45:23.483 INFO:tasks.workunit.client.1.smithi026.stdout:4/579: read d1/f97 [1870652,92255] 0 2024-03-16T10:45:23.484 INFO:tasks.workunit.client.1.smithi026.stdout:2/535: write d2/f4d [1346433,52105] 0 2024-03-16T10:45:23.488 INFO:tasks.workunit.client.1.smithi026.stdout:0/629: link d1/d12/d2f/dba/fc7 d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/fd8 0 2024-03-16T10:45:23.496 INFO:tasks.workunit.client.1.smithi026.stdout:3/598: getdents d4/dc/d73/d83 0 2024-03-16T10:45:23.500 INFO:tasks.workunit.client.1.smithi026.stdout:8/538: rmdir d5/d5d 39 2024-03-16T10:45:23.505 INFO:tasks.workunit.client.1.smithi026.stdout:0/630: mknod d1/d12/d2f/daa/dd1/d1a/d2b/cd9 0 2024-03-16T10:45:23.506 INFO:tasks.workunit.client.1.smithi026.stdout:0/631: write d1/d12/d2f/daa/dd1/fc2 [5455176,13291] 0 2024-03-16T10:45:23.509 INFO:tasks.workunit.client.1.smithi026.stdout:9/571: write d0/d63/d6b/f81 [2557913,18157] 0 2024-03-16T10:45:23.510 INFO:tasks.workunit.client.1.smithi026.stdout:9/572: fsync d0/da/d16/d17/fa9 0 2024-03-16T10:45:23.512 INFO:tasks.workunit.client.1.smithi026.stdout:1/547: truncate d0/d17/d1f/d63/d6a/fa2 556238 0 2024-03-16T10:45:23.519 INFO:tasks.workunit.client.1.smithi026.stdout:9/573: dwrite d0/d5f/d7f/d8c/fbd [0,4194304] 0 2024-03-16T10:45:23.521 INFO:tasks.workunit.client.1.smithi026.stdout:4/580: truncate d1/db/de/f3d 1843001 0 2024-03-16T10:45:23.525 INFO:tasks.workunit.client.1.smithi026.stdout:2/536: symlink d2/d29/d17/d1a/d2b/lb7 0 2024-03-16T10:45:23.525 INFO:tasks.workunit.client.1.smithi026.stdout:9/574: write d0/da/d13/f38 [419388,26459] 0 2024-03-16T10:45:23.526 INFO:tasks.workunit.client.1.smithi026.stdout:6/448: dwrite d0/d5b/f25 [4194304,4194304] 0 2024-03-16T10:45:23.527 INFO:tasks.workunit.client.1.smithi026.stdout:4/581: fsync d1/db/de/d4c/d6b/fae 0 2024-03-16T10:45:23.529 INFO:tasks.workunit.client.1.smithi026.stdout:7/598: write d6/f31 [104721,129868] 0 2024-03-16T10:45:23.538 INFO:tasks.workunit.client.1.smithi026.stdout:4/582: dwrite d1/db/de/d4c/d59/d6a/f9a [0,4194304] 0 2024-03-16T10:45:23.539 INFO:tasks.workunit.client.1.smithi026.stdout:3/599: mknod d4/dc/cc9 0 2024-03-16T10:45:23.549 INFO:tasks.workunit.client.1.smithi026.stdout:8/539: symlink d5/d5d/l99 0 2024-03-16T10:45:23.555 INFO:tasks.workunit.client.1.smithi026.stdout:8/540: dwrite d5/dc/d40/f77 [0,4194304] 0 2024-03-16T10:45:23.556 INFO:tasks.workunit.client.1.smithi026.stdout:2/537: unlink d2/d29/d17/d1a/d2b/l5e 0 2024-03-16T10:45:23.562 INFO:tasks.workunit.client.1.smithi026.stdout:0/632: rename d1/d12/d2f/daa/dd1/d1a/d2b/d4f/f53 to d1/d12/d2f/d48/d55/d60/d9b/d57/fda 0 2024-03-16T10:45:23.563 INFO:tasks.workunit.client.1.smithi026.stdout:0/633: fdatasync d1/d12/d2f/daa/dac/fb5 0 2024-03-16T10:45:23.567 INFO:tasks.workunit.client.1.smithi026.stdout:6/449: write d0/d28/f6f [792393,82009] 0 2024-03-16T10:45:23.567 INFO:tasks.workunit.client.1.smithi026.stdout:4/583: mknod d1/db/de/d4c/d6b/d89/d9c/cbb 0 2024-03-16T10:45:23.572 INFO:tasks.workunit.client.1.smithi026.stdout:0/634: dwrite d1/d12/d2f/d48/d55/d60/f70 [0,4194304] 0 2024-03-16T10:45:23.573 INFO:tasks.workunit.client.1.smithi026.stdout:1/548: creat d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fad x:0 0 0 2024-03-16T10:45:23.578 INFO:tasks.workunit.client.1.smithi026.stdout:8/541: mkdir d5/d5d/d9a 0 2024-03-16T10:45:23.579 INFO:tasks.workunit.client.1.smithi026.stdout:3/600: dwrite d4/f41 [0,4194304] 0 2024-03-16T10:45:23.579 INFO:tasks.workunit.client.1.smithi026.stdout:8/542: fsync d5/d5d/f92 0 2024-03-16T10:45:23.581 INFO:tasks.workunit.client.1.smithi026.stdout:2/538: creat d2/d29/d51/d72/fb8 x:0 0 0 2024-03-16T10:45:23.581 INFO:tasks.workunit.client.1.smithi026.stdout:8/543: chown d5/d7/d2c/d36/d59 106 1 2024-03-16T10:45:23.588 INFO:tasks.workunit.client.1.smithi026.stdout:7/599: unlink d6/d2f/daf/d3f/d65/f89 0 2024-03-16T10:45:23.595 INFO:tasks.workunit.client.1.smithi026.stdout:1/549: rmdir d0/d17/d1f/d20 39 2024-03-16T10:45:23.597 INFO:tasks.workunit.client.1.smithi026.stdout:0/635: unlink d1/d12/d2f/d48/d55/d60/d9b/f3 0 2024-03-16T10:45:23.599 INFO:tasks.workunit.client.1.smithi026.stdout:0/636: write d1/d12/d2f/d48/d55/d60/d9b/d51/fcb [972843,101453] 0 2024-03-16T10:45:23.600 INFO:tasks.workunit.client.1.smithi026.stdout:0/637: chown d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/fc3 0 1 2024-03-16T10:45:23.602 INFO:tasks.workunit.client.1.smithi026.stdout:9/575: creat d0/d5f/fc3 x:0 0 0 2024-03-16T10:45:23.603 INFO:tasks.workunit.client.1.smithi026.stdout:9/576: chown d0/da/d16/d17/c47 0 1 2024-03-16T10:45:23.604 INFO:tasks.workunit.client.1.smithi026.stdout:3/601: creat d4/dc/dd/d47/d53/fca x:0 0 0 2024-03-16T10:45:23.605 INFO:tasks.workunit.client.1.smithi026.stdout:2/539: dread d2/d29/d36/f3e [0,4194304] 0 2024-03-16T10:45:23.606 INFO:tasks.workunit.client.1.smithi026.stdout:2/540: read - d2/d29/d17/d38/d64/fa2 zero size 2024-03-16T10:45:23.610 INFO:tasks.workunit.client.1.smithi026.stdout:0/638: creat d1/d12/d2f/d48/d55/d60/d9b/d57/fdb x:0 0 0 2024-03-16T10:45:23.612 INFO:tasks.workunit.client.1.smithi026.stdout:9/577: unlink d0/d5f/fc3 0 2024-03-16T10:45:23.613 INFO:tasks.workunit.client.1.smithi026.stdout:9/578: readlink d0/da/d16/d17/l4a 0 2024-03-16T10:45:23.614 INFO:tasks.workunit.client.1.smithi026.stdout:8/544: creat d5/d5d/d9a/f9b x:0 0 0 2024-03-16T10:45:23.617 INFO:tasks.workunit.client.1.smithi026.stdout:9/579: dread d0/da/d13/f69 [0,4194304] 0 2024-03-16T10:45:23.620 INFO:tasks.workunit.client.1.smithi026.stdout:2/541: truncate d2/d29/d17/d38/f7d 430253 0 2024-03-16T10:45:23.626 INFO:tasks.workunit.client.1.smithi026.stdout:0/639: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/d5f/cdc 0 2024-03-16T10:45:23.628 INFO:tasks.workunit.client.1.smithi026.stdout:0/640: write d1/d12/d2f/d48/d55/d60/d9b/d11/fc0 [805005,68479] 0 2024-03-16T10:45:23.632 INFO:tasks.workunit.client.1.smithi026.stdout:0/641: dread d1/d12/d2f/d48/d55/d60/d9b/d11/fc0 [0,4194304] 0 2024-03-16T10:45:23.633 INFO:tasks.workunit.client.1.smithi026.stdout:9/580: symlink d0/da/d14/da7/lc4 0 2024-03-16T10:45:23.635 INFO:tasks.workunit.client.1.smithi026.stdout:2/542: creat d2/d29/d51/d55/d6d/fb9 x:0 0 0 2024-03-16T10:45:23.636 INFO:tasks.workunit.client.1.smithi026.stdout:6/450: link d0/d10/f6c d0/d10/d79/d7b/f9d 0 2024-03-16T10:45:23.638 INFO:tasks.workunit.client.1.smithi026.stdout:4/584: dwrite d1/db/de/f3d [0,4194304] 0 2024-03-16T10:45:23.639 INFO:tasks.workunit.client.1.smithi026.stdout:2/543: write d2/d29/d3b/fb1 [199402,73965] 0 2024-03-16T10:45:23.640 INFO:tasks.workunit.client.1.smithi026.stdout:4/585: read - d1/db/de/d4c/d79/daa/d61/f7c zero size 2024-03-16T10:45:23.642 INFO:tasks.workunit.client.1.smithi026.stdout:4/586: fsync d1/db/de/d4c/d6b/d7a/fba 0 2024-03-16T10:45:23.642 INFO:tasks.workunit.client.1.smithi026.stdout:4/587: fdatasync d1/f2d 0 2024-03-16T10:45:23.643 INFO:tasks.workunit.client.1.smithi026.stdout:3/602: link d4/dc/dd/d3d/f3e d4/dc/dd/fcb 0 2024-03-16T10:45:23.644 INFO:tasks.workunit.client.1.smithi026.stdout:3/603: write d4/faa [579318,10361] 0 2024-03-16T10:45:23.645 INFO:tasks.workunit.client.1.smithi026.stdout:4/588: truncate d1/db/de/d4c/d6b/fad 800567 0 2024-03-16T10:45:23.645 INFO:tasks.workunit.client.1.smithi026.stdout:7/600: write d6/d35/d3a/d59/f96 [2018588,109350] 0 2024-03-16T10:45:23.646 INFO:tasks.workunit.client.1.smithi026.stdout:8/545: mknod d5/d7/d7b/d96/c9c 0 2024-03-16T10:45:23.648 INFO:tasks.workunit.client.1.smithi026.stdout:9/581: rmdir d0/da/d16/d17/db1 39 2024-03-16T10:45:23.648 INFO:tasks.workunit.client.1.smithi026.stdout:7/601: dread d6/f91 [0,4194304] 0 2024-03-16T10:45:23.655 INFO:tasks.workunit.client.1.smithi026.stdout:6/451: truncate d0/d28/d49/f52 1820486 0 2024-03-16T10:45:23.661 INFO:tasks.workunit.client.1.smithi026.stdout:3/604: mknod d4/dc/dd/d47/d53/ccc 0 2024-03-16T10:45:23.662 INFO:tasks.workunit.client.1.smithi026.stdout:3/605: read - d4/dc/dd/d60/d62/f85 zero size 2024-03-16T10:45:23.664 INFO:tasks.workunit.client.1.smithi026.stdout:4/589: mkdir d1/db/de/d4c/d6b/dbc 0 2024-03-16T10:45:23.665 INFO:tasks.workunit.client.1.smithi026.stdout:4/590: stat d1/db/de/l5e 0 2024-03-16T10:45:23.665 INFO:tasks.workunit.client.1.smithi026.stdout:8/546: rmdir d5/d7/d2c/d36/d59 39 2024-03-16T10:45:23.667 INFO:tasks.workunit.client.1.smithi026.stdout:3/606: dread d4/d89/d97/d7d/f88 [4194304,4194304] 0 2024-03-16T10:45:23.667 INFO:tasks.workunit.client.1.smithi026.stdout:4/591: chown d1/db/de/d4c/d79/daa/d44/d48/f73 0 1 2024-03-16T10:45:23.668 INFO:tasks.workunit.client.1.smithi026.stdout:4/592: chown d1/db/de/d4c/d59/d6a/ca9 657 1 2024-03-16T10:45:23.669 INFO:tasks.workunit.client.1.smithi026.stdout:9/582: mkdir d0/da/d4d/dc5 0 2024-03-16T10:45:23.669 INFO:tasks.workunit.client.1.smithi026.stdout:4/593: chown d1/db/de/d6e/l92 73 1 2024-03-16T10:45:23.671 INFO:tasks.workunit.client.1.smithi026.stdout:4/594: readlink d1/db/de/d4c/d79/daa/d44/d48/l71 0 2024-03-16T10:45:23.672 INFO:tasks.workunit.client.1.smithi026.stdout:7/602: mkdir d6/d1e/dbc 0 2024-03-16T10:45:23.679 INFO:tasks.workunit.client.1.smithi026.stdout:7/603: dwrite d6/d35/d3a/d4e/fac [0,4194304] 0 2024-03-16T10:45:23.681 INFO:tasks.workunit.client.1.smithi026.stdout:8/547: creat d5/d5d/f9d x:0 0 0 2024-03-16T10:45:23.687 INFO:tasks.workunit.client.1.smithi026.stdout:9/583: creat d0/d5f/d7f/d8c/fc6 x:0 0 0 2024-03-16T10:45:23.689 INFO:tasks.workunit.client.1.smithi026.stdout:9/584: fdatasync d0/d63/d6b/fc2 0 2024-03-16T10:45:23.691 INFO:tasks.workunit.client.1.smithi026.stdout:8/548: dwrite d5/d28/d90/f91 [0,4194304] 0 2024-03-16T10:45:23.692 INFO:tasks.workunit.client.1.smithi026.stdout:2/544: creat d2/d29/d17/fba x:0 0 0 2024-03-16T10:45:23.694 INFO:tasks.workunit.client.1.smithi026.stdout:2/545: write d2/d29/d51/d55/d6d/fb9 [792232,31749] 0 2024-03-16T10:45:23.695 INFO:tasks.workunit.client.1.smithi026.stdout:7/604: symlink d6/d35/d3a/d4e/d6c/d85/lbd 0 2024-03-16T10:45:23.696 INFO:tasks.workunit.client.1.smithi026.stdout:7/605: chown d6/d35/d3a/d4e/db0 10318 1 2024-03-16T10:45:23.697 INFO:tasks.workunit.client.1.smithi026.stdout:7/606: read d6/d35/d3a/d59/f92 [260375,87330] 0 2024-03-16T10:45:23.697 INFO:tasks.workunit.client.1.smithi026.stdout:3/607: mknod d4/dc/dd/d60/d62/ccd 0 2024-03-16T10:45:23.703 INFO:tasks.workunit.client.1.smithi026.stdout:0/642: link d1/d12/d2f/d48/d55/l76 d1/d12/d2f/d48/d55/d60/d9b/ldd 0 2024-03-16T10:45:23.707 INFO:tasks.workunit.client.1.smithi026.stdout:8/549: creat d5/d23/d4e/d69/d76/f9e x:0 0 0 2024-03-16T10:45:23.710 INFO:tasks.workunit.client.1.smithi026.stdout:2/546: mkdir d2/d29/d3b/dbb 0 2024-03-16T10:45:23.712 INFO:tasks.workunit.client.1.smithi026.stdout:3/608: rename d4/dc/dd/d3d/c81 to d4/dc/d31/cce 0 2024-03-16T10:45:23.715 INFO:tasks.workunit.client.1.smithi026.stdout:3/609: write d4/dc/dd/d3d/f39 [3903351,124774] 0 2024-03-16T10:45:23.720 INFO:tasks.workunit.client.1.smithi026.stdout:4/595: link d1/db/de/d4c/d79/c8f d1/db/de/d4c/d59/cbd 0 2024-03-16T10:45:23.721 INFO:tasks.workunit.client.1.smithi026.stdout:2/547: dwrite d2/d29/d36/f58 [0,4194304] 0 2024-03-16T10:45:23.732 INFO:tasks.workunit.client.1.smithi026.stdout:9/585: creat d0/da/d16/d17/fc7 x:0 0 0 2024-03-16T10:45:23.735 INFO:tasks.workunit.client.1.smithi026.stdout:2/548: unlink d2/d29/d17/d38/c92 0 2024-03-16T10:45:23.750 INFO:tasks.workunit.client.1.smithi026.stdout:4/596: symlink d1/db/de/d4c/d59/d6a/db2/db3/lbe 0 2024-03-16T10:45:23.751 INFO:tasks.workunit.client.1.smithi026.stdout:4/597: readlink d1/db/de/d4c/d59/d6a/l84 0 2024-03-16T10:45:23.751 INFO:tasks.workunit.client.1.smithi026.stdout:3/610: link d4/dc/dd/d80/c8b d4/dc/d73/d83/ccf 0 2024-03-16T10:45:23.752 INFO:tasks.workunit.client.1.smithi026.stdout:3/611: chown d4/dc/d31/d48/d7f/l84 11 1 2024-03-16T10:45:23.753 INFO:tasks.workunit.client.1.smithi026.stdout:4/598: dread d1/db/de/d4c/f8a [0,4194304] 0 2024-03-16T10:45:23.755 INFO:tasks.workunit.client.1.smithi026.stdout:4/599: dread d1/f77 [0,4194304] 0 2024-03-16T10:45:23.756 INFO:tasks.workunit.client.1.smithi026.stdout:2/549: mknod d2/d29/d17/d1a/d2b/cbc 0 2024-03-16T10:45:23.759 INFO:tasks.workunit.client.1.smithi026.stdout:9/586: rename d0/da/d4d/c88 to d0/da/d16/d17/cc8 0 2024-03-16T10:45:23.761 INFO:tasks.workunit.client.1.smithi026.stdout:2/550: dread d2/d29/d17/f40 [0,4194304] 0 2024-03-16T10:45:23.763 INFO:tasks.workunit.client.1.smithi026.stdout:3/612: mknod d4/d89/d97/d7d/dae/cd0 0 2024-03-16T10:45:23.773 INFO:tasks.workunit.client.1.smithi026.stdout:6/452: dwrite d0/d10/d79/d7b/f9d [0,4194304] 0 2024-03-16T10:45:23.786 INFO:tasks.workunit.client.1.smithi026.stdout:0/643: truncate d1/d12/d2f/d48/d55/d60/d9b/d57/fbb 671629 0 2024-03-16T10:45:23.788 INFO:tasks.workunit.client.1.smithi026.stdout:6/453: creat d0/d10/d79/d7b/f9e x:0 0 0 2024-03-16T10:45:23.790 INFO:tasks.workunit.client.1.smithi026.stdout:6/454: chown d0/d10/d79/d7b/f7e 756697615 1 2024-03-16T10:45:23.792 INFO:tasks.workunit.client.1.smithi026.stdout:6/455: write d0/d10/f6c [1091276,49452] 0 2024-03-16T10:45:23.795 INFO:tasks.workunit.client.1.smithi026.stdout:8/550: dwrite d5/f3e [0,4194304] 0 2024-03-16T10:45:23.797 INFO:tasks.workunit.client.1.smithi026.stdout:6/456: dread d0/d10/f38 [0,4194304] 0 2024-03-16T10:45:23.801 INFO:tasks.workunit.client.1.smithi026.stdout:3/613: creat d4/dc/d31/d48/fd1 x:0 0 0 2024-03-16T10:45:23.813 INFO:tasks.workunit.client.1.smithi026.stdout:4/600: write d1/db/de/d4c/f4d [2276068,76816] 0 2024-03-16T10:45:23.815 INFO:tasks.workunit.client.1.smithi026.stdout:4/601: write d1/db/de/f55 [93367,128753] 0 2024-03-16T10:45:23.816 INFO:tasks.workunit.client.1.smithi026.stdout:9/587: link d0/da/d14/f34 d0/da/fc9 0 2024-03-16T10:45:23.822 INFO:tasks.workunit.client.1.smithi026.stdout:2/551: dwrite d2/f35 [0,4194304] 0 2024-03-16T10:45:23.824 INFO:tasks.workunit.client.1.smithi026.stdout:2/552: chown d2/d29/d36/da7/ca1 1311 1 2024-03-16T10:45:23.830 INFO:tasks.workunit.client.1.smithi026.stdout:4/602: symlink d1/db/de/d4c/d79/lbf 0 2024-03-16T10:45:23.836 INFO:tasks.workunit.client.1.smithi026.stdout:9/588: creat d0/da/d16/d17/d44/fca x:0 0 0 2024-03-16T10:45:23.838 INFO:tasks.workunit.client.1.smithi026.stdout:9/589: write d0/f66 [4779395,28946] 0 2024-03-16T10:45:23.842 INFO:tasks.workunit.client.1.smithi026.stdout:8/551: write d5/dc/d3f/f55 [787953,65199] 0 2024-03-16T10:45:23.844 INFO:tasks.workunit.client.1.smithi026.stdout:6/457: mknod d0/c9f 0 2024-03-16T10:45:23.845 INFO:tasks.workunit.client.1.smithi026.stdout:4/603: dread d1/f3 [0,4194304] 0 2024-03-16T10:45:23.852 INFO:tasks.workunit.client.1.smithi026.stdout:8/552: symlink d5/d7/d7b/l9f 0 2024-03-16T10:45:23.857 INFO:tasks.workunit.client.1.smithi026.stdout:3/614: creat d4/dc/dd/fd2 x:0 0 0 2024-03-16T10:45:23.860 INFO:tasks.workunit.client.1.smithi026.stdout:8/553: dwrite d5/d23/d4e/f7d [0,4194304] 0 2024-03-16T10:45:23.866 INFO:tasks.workunit.client.1.smithi026.stdout:0/644: getdents d1/d12/d2f/daa/dac 0 2024-03-16T10:45:23.869 INFO:tasks.workunit.client.1.smithi026.stdout:9/590: mknod d0/da/d4d/dc5/ccb 0 2024-03-16T10:45:23.869 INFO:tasks.workunit.client.1.smithi026.stdout:2/553: dwrite d2/d29/d17/f40 [0,4194304] 0 2024-03-16T10:45:23.873 INFO:tasks.workunit.client.1.smithi026.stdout:3/615: symlink d4/dc/d73/d83/ld3 0 2024-03-16T10:45:23.873 INFO:tasks.workunit.client.1.smithi026.stdout:8/554: dwrite d5/d28/f31 [0,4194304] 0 2024-03-16T10:45:23.878 INFO:tasks.workunit.client.1.smithi026.stdout:3/616: chown d4/dc/dd/fb4 22911 1 2024-03-16T10:45:23.879 INFO:tasks.workunit.client.1.smithi026.stdout:6/458: dwrite d0/d28/d49/f78 [0,4194304] 0 2024-03-16T10:45:23.879 INFO:tasks.workunit.client.1.smithi026.stdout:9/591: mknod d0/da/d16/ccc 0 2024-03-16T10:45:23.884 INFO:tasks.workunit.client.1.smithi026.stdout:2/554: mknod d2/d29/d36/d98/cbd 0 2024-03-16T10:45:23.888 INFO:tasks.workunit.client.1.smithi026.stdout:4/604: dwrite d1/db/de/d4c/d79/daa/d44/d48/d8e/f93 [0,4194304] 0 2024-03-16T10:45:23.889 INFO:tasks.workunit.client.1.smithi026.stdout:9/592: read d0/d63/f83 [1466262,123950] 0 2024-03-16T10:45:23.890 INFO:tasks.workunit.client.1.smithi026.stdout:6/459: rmdir d0/d8a 39 2024-03-16T10:45:23.897 INFO:tasks.workunit.client.1.smithi026.stdout:8/555: mkdir d5/da0 0 2024-03-16T10:45:23.898 INFO:tasks.workunit.client.1.smithi026.stdout:2/555: creat d2/d29/d17/d1a/d2b/d4e/d67/fbe x:0 0 0 2024-03-16T10:45:23.903 INFO:tasks.workunit.client.1.smithi026.stdout:3/617: mknod d4/dc/dd/d60/d62/d4f/d5a/da6/cd4 0 2024-03-16T10:45:23.905 INFO:tasks.workunit.client.1.smithi026.stdout:6/460: dwrite d0/f44 [0,4194304] 0 2024-03-16T10:45:23.909 INFO:tasks.workunit.client.1.smithi026.stdout:0/645: dread d1/d12/d2f/d48/d55/d60/d9b/fcc [0,4194304] 0 2024-03-16T10:45:23.911 INFO:tasks.workunit.client.1.smithi026.stdout:4/605: dwrite d1/db/f64 [0,4194304] 0 2024-03-16T10:45:23.917 INFO:tasks.workunit.client.1.smithi026.stdout:9/593: truncate d0/da/d16/d5d/d72/d62/f8a 303212 0 2024-03-16T10:45:23.919 INFO:tasks.workunit.client.1.smithi026.stdout:6/461: rmdir d0/d28/d2d 39 2024-03-16T10:45:23.921 INFO:tasks.workunit.client.1.smithi026.stdout:0/646: write d1/d12/d2f/d48/d55/d60/d9b/f67 [5084017,123613] 0 2024-03-16T10:45:23.923 INFO:tasks.workunit.client.1.smithi026.stdout:0/647: fsync d1/d12/d2f/daa/dd1/f97 0 2024-03-16T10:45:23.923 INFO:tasks.workunit.client.1.smithi026.stdout:4/606: symlink d1/db/de/d4c/d6b/d89/d9c/lc0 0 2024-03-16T10:45:23.925 INFO:tasks.workunit.client.1.smithi026.stdout:4/607: stat d1/db/de/d4c/d79/daa 0 2024-03-16T10:45:23.925 INFO:tasks.workunit.client.1.smithi026.stdout:3/618: mkdir d4/dc/d73/d83/dbc/dd5 0 2024-03-16T10:45:23.932 INFO:tasks.workunit.client.1.smithi026.stdout:8/556: getdents d5 0 2024-03-16T10:45:23.936 INFO:tasks.workunit.client.1.smithi026.stdout:5/562: sync 2024-03-16T10:45:23.937 INFO:tasks.workunit.client.1.smithi026.stdout:3/619: creat d4/d89/d97/fd6 x:0 0 0 2024-03-16T10:45:23.938 INFO:tasks.workunit.client.1.smithi026.stdout:9/594: creat d0/da/d16/d17/db1/fcd x:0 0 0 2024-03-16T10:45:23.941 INFO:tasks.workunit.client.1.smithi026.stdout:2/556: rename d2/d29/d3b/fa5 to d2/d29/d17/d1a/d2b/d2e/fbf 0 2024-03-16T10:45:23.942 INFO:tasks.workunit.client.1.smithi026.stdout:6/462: mknod d0/d28/d2d/ca0 0 2024-03-16T10:45:23.943 INFO:tasks.workunit.client.1.smithi026.stdout:8/557: mkdir d5/d23/d4e/d69/d76/da1 0 2024-03-16T10:45:23.944 INFO:tasks.workunit.client.1.smithi026.stdout:8/558: stat d5/d7/f8d 0 2024-03-16T10:45:23.950 INFO:tasks.workunit.client.1.smithi026.stdout:9/595: read d0/da/d16/d17/d44/f2b [293089,31351] 0 2024-03-16T10:45:23.951 INFO:tasks.workunit.client.1.smithi026.stdout:8/559: dwrite d5/d5d/f92 [0,4194304] 0 2024-03-16T10:45:23.953 INFO:tasks.workunit.client.1.smithi026.stdout:6/463: rmdir d0/d41 39 2024-03-16T10:45:23.956 INFO:tasks.workunit.client.1.smithi026.stdout:4/608: link d1/la4 d1/db/de/d6e/lc1 0 2024-03-16T10:45:23.957 INFO:tasks.workunit.client.1.smithi026.stdout:8/560: dread d5/d28/f31 [0,4194304] 0 2024-03-16T10:45:23.964 INFO:tasks.workunit.client.1.smithi026.stdout:2/557: creat d2/d29/d3b/dbb/fc0 x:0 0 0 2024-03-16T10:45:23.966 INFO:tasks.workunit.client.1.smithi026.stdout:0/648: getdents d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96 0 2024-03-16T10:45:23.967 INFO:tasks.workunit.client.1.smithi026.stdout:3/620: truncate d4/faa 385980 0 2024-03-16T10:45:23.968 INFO:tasks.workunit.client.1.smithi026.stdout:3/621: write d4/dc/f11 [5041515,72680] 0 2024-03-16T10:45:23.970 INFO:tasks.workunit.client.1.smithi026.stdout:6/464: symlink d0/d10/d79/d7b/la1 0 2024-03-16T10:45:23.971 INFO:tasks.workunit.client.1.smithi026.stdout:6/465: dread - d0/d28/f5d zero size 2024-03-16T10:45:23.972 INFO:tasks.workunit.client.1.smithi026.stdout:6/466: write d0/d10/ff [4652038,8480] 0 2024-03-16T10:45:23.973 INFO:tasks.workunit.client.1.smithi026.stdout:6/467: fsync d0/d28/d49/d7d/f8c 0 2024-03-16T10:45:23.977 INFO:tasks.workunit.client.1.smithi026.stdout:9/596: mkdir d0/dce 0 2024-03-16T10:45:23.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:23 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:23.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:23 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:23.984 INFO:tasks.workunit.client.1.smithi026.stdout:3/622: rename d4/dc/dd/d60 to d4/dc/d73/d83/d7c/dd7 0 2024-03-16T10:45:23.985 INFO:tasks.workunit.client.1.smithi026.stdout:8/561: dread d5/dc/f4d [0,4194304] 0 2024-03-16T10:45:23.985 INFO:tasks.workunit.client.1.smithi026.stdout:3/623: dread - d4/d89/d97/fd6 zero size 2024-03-16T10:45:23.986 INFO:tasks.workunit.client.1.smithi026.stdout:3/624: dread - d4/dc/d73/d83/f6c zero size 2024-03-16T10:45:23.993 INFO:tasks.workunit.client.1.smithi026.stdout:5/563: mkdir d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/dc6 0 2024-03-16T10:45:24.002 INFO:tasks.workunit.client.1.smithi026.stdout:0/649: dwrite d1/d12/d2f/d48/d55/f7c [0,4194304] 0 2024-03-16T10:45:24.005 INFO:tasks.workunit.client.1.smithi026.stdout:6/468: dread d0/d10/f23 [0,4194304] 0 2024-03-16T10:45:24.005 INFO:tasks.workunit.client.1.smithi026.stdout:2/558: symlink d2/d29/d17/d1a/d2b/d2e/lc1 0 2024-03-16T10:45:24.007 INFO:tasks.workunit.client.1.smithi026.stdout:9/597: mknod d0/dce/ccf 0 2024-03-16T10:45:24.010 INFO:tasks.workunit.client.1.smithi026.stdout:4/609: getdents d1/db/de/d4c/d79/daa/d61 0 2024-03-16T10:45:24.011 INFO:tasks.workunit.client.1.smithi026.stdout:9/598: dread d0/da/d16/d17/fa9 [0,4194304] 0 2024-03-16T10:45:24.012 INFO:tasks.workunit.client.1.smithi026.stdout:9/599: chown d0/da/d16/d17/db1 59764 1 2024-03-16T10:45:24.013 INFO:tasks.workunit.client.1.smithi026.stdout:9/600: readlink d0/da/d14/l52 0 2024-03-16T10:45:24.015 INFO:tasks.workunit.client.1.smithi026.stdout:6/469: chown d0/f9a 9100727 1 2024-03-16T10:45:24.023 INFO:tasks.workunit.client.1.smithi026.stdout:8/562: rename d5/d23/l2f to d5/dc/d40/la2 0 2024-03-16T10:45:24.024 INFO:tasks.workunit.client.1.smithi026.stdout:1/550: sync 2024-03-16T10:45:24.024 INFO:tasks.workunit.client.1.smithi026.stdout:7/607: sync 2024-03-16T10:45:24.024 INFO:tasks.workunit.client.1.smithi026.stdout:3/625: creat d4/dc/dd/fd8 x:0 0 0 2024-03-16T10:45:24.025 INFO:tasks.workunit.client.1.smithi026.stdout:5/564: symlink d1/db/d1f/d61/lc7 0 2024-03-16T10:45:24.026 INFO:tasks.workunit.client.1.smithi026.stdout:8/563: stat d5/d7/d2c 0 2024-03-16T10:45:24.026 INFO:tasks.workunit.client.1.smithi026.stdout:9/601: creat d0/dce/fd0 x:0 0 0 2024-03-16T10:45:24.026 INFO:tasks.workunit.client.1.smithi026.stdout:6/470: truncate d0/d10/f38 818324 0 2024-03-16T10:45:24.029 INFO:tasks.workunit.client.1.smithi026.stdout:3/626: chown d4/dc/dd/d3d/d94/fa4 17339 1 2024-03-16T10:45:24.031 INFO:tasks.workunit.client.1.smithi026.stdout:3/627: stat d4/dc/dd/d47/d53/db1 0 2024-03-16T10:45:24.031 INFO:tasks.workunit.client.1.smithi026.stdout:4/610: dwrite d1/db/f26 [0,4194304] 0 2024-03-16T10:45:24.038 INFO:tasks.workunit.client.1.smithi026.stdout:3/628: dwrite d4/dc/d31/f34 [0,4194304] 0 2024-03-16T10:45:24.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:23 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:24.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:23 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:24.050 INFO:tasks.workunit.client.1.smithi026.stdout:0/650: rename d1/d12/dcd to d1/d12/d2f/d48/d55/d60/d9b/d51/dde 0 2024-03-16T10:45:24.052 INFO:tasks.workunit.client.1.smithi026.stdout:1/551: rmdir d0/d17/d1f/d63/d6a 39 2024-03-16T10:45:24.053 INFO:tasks.workunit.client.1.smithi026.stdout:9/602: chown d0/lf 2258 1 2024-03-16T10:45:24.054 INFO:tasks.workunit.client.1.smithi026.stdout:9/603: dread - d0/da/d16/d17/d44/fa3 zero size 2024-03-16T10:45:24.055 INFO:tasks.workunit.client.1.smithi026.stdout:9/604: chown d0/da/d16/l93 2799318 1 2024-03-16T10:45:24.056 INFO:tasks.workunit.client.1.smithi026.stdout:7/608: write d6/d2f/daf/f5c [1031768,126202] 0 2024-03-16T10:45:24.058 INFO:tasks.workunit.client.1.smithi026.stdout:6/471: creat d0/d10/d79/fa2 x:0 0 0 2024-03-16T10:45:24.062 INFO:tasks.workunit.client.1.smithi026.stdout:3/629: symlink d4/dc/dd/d3d/ld9 0 2024-03-16T10:45:24.065 INFO:tasks.workunit.client.1.smithi026.stdout:2/559: rename d2/d29/d36/f58 to d2/d29/d17/d1a/d2b/d2e/d60/fc2 0 2024-03-16T10:45:24.066 INFO:tasks.workunit.client.1.smithi026.stdout:0/651: creat d1/d12/d2f/daa/fdf x:0 0 0 2024-03-16T10:45:24.069 INFO:tasks.workunit.client.1.smithi026.stdout:5/565: creat d1/d1c/d2a/d55/fc8 x:0 0 0 2024-03-16T10:45:24.070 INFO:tasks.workunit.client.1.smithi026.stdout:9/605: rmdir d0 39 2024-03-16T10:45:24.078 INFO:tasks.workunit.client.1.smithi026.stdout:4/611: rename d1/f97 to d1/db/de/d4c/d59/fc2 0 2024-03-16T10:45:24.079 INFO:tasks.workunit.client.1.smithi026.stdout:0/652: readlink d1/d12/d2f/daa/dd1/d61/l9d 0 2024-03-16T10:45:24.081 INFO:tasks.workunit.client.1.smithi026.stdout:1/552: fsync d0/d17/d1f/d63/d6a/d7c/f6c 0 2024-03-16T10:45:24.084 INFO:tasks.workunit.client.1.smithi026.stdout:1/553: read d0/d17/d1f/d63/d6a/d7c/d2e/d54/f5e [117276,18180] 0 2024-03-16T10:45:24.088 INFO:tasks.workunit.client.1.smithi026.stdout:8/564: mkdir d5/d7/d2c/d36/d59/d93/da3 0 2024-03-16T10:45:24.089 INFO:tasks.workunit.client.1.smithi026.stdout:4/612: rmdir d1/db/de/d4c/d59/d6a/db2 39 2024-03-16T10:45:24.090 INFO:tasks.workunit.client.1.smithi026.stdout:0/653: truncate d1/d12/d2f/daa/dd1/d61/f6b 578168 0 2024-03-16T10:45:24.092 INFO:tasks.workunit.client.1.smithi026.stdout:0/654: write d1/d12/d2f/daa/dd1/d1a/d2b/d4f/fa8 [1022284,110719] 0 2024-03-16T10:45:24.096 INFO:tasks.workunit.client.1.smithi026.stdout:9/606: read d0/da/d16/d5d/d72/d62/f65 [2763565,77971] 0 2024-03-16T10:45:24.097 INFO:tasks.workunit.client.1.smithi026.stdout:9/607: dread d0/da/d16/d17/fa9 [0,4194304] 0 2024-03-16T10:45:24.098 INFO:tasks.workunit.client.1.smithi026.stdout:7/609: creat d6/de/fbe x:0 0 0 2024-03-16T10:45:24.098 INFO:tasks.workunit.client.1.smithi026.stdout:9/608: chown d0/da/d14 587187149 1 2024-03-16T10:45:24.099 INFO:tasks.workunit.client.1.smithi026.stdout:7/610: write d6/f53 [706971,117071] 0 2024-03-16T10:45:24.100 INFO:tasks.workunit.client.1.smithi026.stdout:8/565: symlink d5/la4 0 2024-03-16T10:45:24.103 INFO:tasks.workunit.client.1.smithi026.stdout:0/655: mkdir d1/d12/d2f/d48/d55/d60/de0 0 2024-03-16T10:45:24.104 INFO:tasks.workunit.client.1.smithi026.stdout:6/472: write d0/f5f [6034,77224] 0 2024-03-16T10:45:24.106 INFO:tasks.workunit.client.1.smithi026.stdout:5/566: link d1/db/d1f/d61/fb9 d1/d1c/d2a/fc9 0 2024-03-16T10:45:24.106 INFO:tasks.workunit.client.1.smithi026.stdout:3/630: write d4/dc/d73/d83/d57/f9b [7790084,44283] 0 2024-03-16T10:45:24.109 INFO:tasks.workunit.client.1.smithi026.stdout:5/567: stat d1/d1c/d2a/dab/dc5/fba 0 2024-03-16T10:45:24.111 INFO:tasks.workunit.client.1.smithi026.stdout:9/609: chown d0/l20 442 1 2024-03-16T10:45:24.111 INFO:tasks.workunit.client.1.smithi026.stdout:7/611: mkdir d6/d2f/daf/d73/dbf 0 2024-03-16T10:45:24.115 INFO:tasks.workunit.client.1.smithi026.stdout:6/473: dwrite d0/d10/d79/d91/f94 [0,4194304] 0 2024-03-16T10:45:24.117 INFO:tasks.workunit.client.1.smithi026.stdout:0/656: dwrite d1/d12/d2f/daa/dac/fd6 [0,4194304] 0 2024-03-16T10:45:24.117 INFO:tasks.workunit.client.1.smithi026.stdout:6/474: chown d0/d10/d79/d7b/f82 239 1 2024-03-16T10:45:24.121 INFO:tasks.workunit.client.1.smithi026.stdout:1/554: creat d0/d17/d4d/fae x:0 0 0 2024-03-16T10:45:24.123 INFO:tasks.workunit.client.1.smithi026.stdout:3/631: creat d4/dc/dd/d80/dc2/fda x:0 0 0 2024-03-16T10:45:24.124 INFO:tasks.workunit.client.1.smithi026.stdout:1/555: chown d0/d17/d1f/d63/d6a/d7c/d6b/f87 0 1 2024-03-16T10:45:24.124 INFO:tasks.workunit.client.1.smithi026.stdout:3/632: chown d4/dc/d73/d83/ld3 542 1 2024-03-16T10:45:24.126 INFO:tasks.workunit.client.1.smithi026.stdout:5/568: stat d1/d1c/d2a/f32 0 2024-03-16T10:45:24.131 INFO:tasks.workunit.client.1.smithi026.stdout:9/610: creat d0/da/d13/d6e/fd1 x:0 0 0 2024-03-16T10:45:24.133 INFO:tasks.workunit.client.1.smithi026.stdout:2/560: rename d2/d29/d17/c43 to d2/d29/cc3 0 2024-03-16T10:45:24.136 INFO:tasks.workunit.client.1.smithi026.stdout:6/475: chown d0/d10/d79/f8d 883703 1 2024-03-16T10:45:24.139 INFO:tasks.workunit.client.1.smithi026.stdout:1/556: creat d0/d17/d1f/d63/d6a/d7c/d2e/faf x:0 0 0 2024-03-16T10:45:24.141 INFO:tasks.workunit.client.1.smithi026.stdout:8/566: write d5/d23/f82 [669491,109956] 0 2024-03-16T10:45:24.147 INFO:tasks.workunit.client.1.smithi026.stdout:4/613: rename d1/f3 to d1/db/de/d4c/d59/fc3 0 2024-03-16T10:45:24.148 INFO:tasks.workunit.client.1.smithi026.stdout:2/561: mknod d2/d29/d17/d1a/d2b/d2e/d60/d7f/cc4 0 2024-03-16T10:45:24.150 INFO:tasks.workunit.client.1.smithi026.stdout:0/657: getdents d1/d12/d2f/daa/dd1/d1a/d2b/db4 0 2024-03-16T10:45:24.153 INFO:tasks.workunit.client.1.smithi026.stdout:1/557: rmdir d0/d17/d1f/d63/d58 39 2024-03-16T10:45:24.155 INFO:tasks.workunit.client.1.smithi026.stdout:8/567: rmdir d5/d23/d4e/d60 39 2024-03-16T10:45:24.156 INFO:tasks.workunit.client.1.smithi026.stdout:5/569: creat d1/d6e/d9f/fca x:0 0 0 2024-03-16T10:45:24.161 INFO:tasks.workunit.client.1.smithi026.stdout:2/562: creat d2/d29/d17/d1a/d2b/d2e/d60/d7f/fc5 x:0 0 0 2024-03-16T10:45:24.162 INFO:tasks.workunit.client.1.smithi026.stdout:8/568: read d5/dc/d40/f77 [3610843,77021] 0 2024-03-16T10:45:24.163 INFO:tasks.workunit.client.1.smithi026.stdout:5/570: mknod d1/d1c/d2a/d55/d91/ccb 0 2024-03-16T10:45:24.164 INFO:tasks.workunit.client.1.smithi026.stdout:9/611: link d0/da/l29 d0/d5f/da2/ld2 0 2024-03-16T10:45:24.166 INFO:tasks.workunit.client.1.smithi026.stdout:4/614: symlink d1/db/de/d4c/d79/lc4 0 2024-03-16T10:45:24.168 INFO:tasks.workunit.client.1.smithi026.stdout:4/615: truncate d1/db/de/d4c/fa1 1133461 0 2024-03-16T10:45:24.168 INFO:tasks.workunit.client.1.smithi026.stdout:2/563: mkdir d2/d29/d85/dc6 0 2024-03-16T10:45:24.169 INFO:tasks.workunit.client.1.smithi026.stdout:4/616: chown d1/db/de/c36 33699922 1 2024-03-16T10:45:24.172 INFO:tasks.workunit.client.1.smithi026.stdout:5/571: write d1/d6e/f92 [817387,125183] 0 2024-03-16T10:45:24.175 INFO:tasks.workunit.client.1.smithi026.stdout:8/569: dread d5/d7/d2c/f70 [0,4194304] 0 2024-03-16T10:45:24.175 INFO:tasks.workunit.client.1.smithi026.stdout:8/570: fsync d5/d7/f1a 0 2024-03-16T10:45:24.176 INFO:tasks.workunit.client.1.smithi026.stdout:7/612: rename d6/d35/d3a/d4e/da1/fa8 to d6/de/fc0 0 2024-03-16T10:45:24.177 INFO:tasks.workunit.client.1.smithi026.stdout:2/564: creat d2/d29/d85/fc7 x:0 0 0 2024-03-16T10:45:24.178 INFO:tasks.workunit.client.1.smithi026.stdout:2/565: chown d2/d29/d17/d1a/f39 464714499 1 2024-03-16T10:45:24.184 INFO:tasks.workunit.client.1.smithi026.stdout:7/613: creat d6/de/d16/d88/fc1 x:0 0 0 2024-03-16T10:45:24.188 INFO:tasks.workunit.client.1.smithi026.stdout:2/566: rmdir d2/d29/d36/d98 39 2024-03-16T10:45:24.190 INFO:tasks.workunit.client.1.smithi026.stdout:7/614: dwrite d6/d2f/daf/f5c [0,4194304] 0 2024-03-16T10:45:24.190 INFO:tasks.workunit.client.1.smithi026.stdout:2/567: stat d2/d29/d17/d1a/d2b/d4e 0 2024-03-16T10:45:24.192 INFO:tasks.workunit.client.1.smithi026.stdout:7/615: chown d6/de/d7d/f8a 38169875 1 2024-03-16T10:45:24.193 INFO:tasks.workunit.client.1.smithi026.stdout:4/617: symlink d1/db/de/d4c/d6b/dbc/lc5 0 2024-03-16T10:45:24.194 INFO:tasks.workunit.client.1.smithi026.stdout:7/616: read d6/d35/d3a/d59/fb9 [4148097,76898] 0 2024-03-16T10:45:24.199 INFO:tasks.workunit.client.1.smithi026.stdout:4/618: dread d1/f22 [0,4194304] 0 2024-03-16T10:45:24.202 INFO:tasks.workunit.client.1.smithi026.stdout:3/633: rename d4/dc/l32 to d4/dc/d31/d48/d7f/ldb 0 2024-03-16T10:45:24.216 INFO:tasks.workunit.client.1.smithi026.stdout:9/612: getdents d0/da/d16 0 2024-03-16T10:45:24.220 INFO:tasks.workunit.client.1.smithi026.stdout:5/572: link d1/d1c/d4c/d4d/f4e d1/d1c/d4c/d4d/fcc 0 2024-03-16T10:45:24.222 INFO:tasks.workunit.client.1.smithi026.stdout:4/619: unlink d1/db/de/c6d 0 2024-03-16T10:45:24.223 INFO:tasks.workunit.client.1.smithi026.stdout:9/613: dwrite d0/da/d16/d17/fa9 [0,4194304] 0 2024-03-16T10:45:24.223 INFO:tasks.workunit.client.1.smithi026.stdout:9/614: readlink d0/d63/l87 0 2024-03-16T10:45:24.226 INFO:tasks.workunit.client.1.smithi026.stdout:6/476: rename d0/d41/c66 to d0/d10/d79/d91/ca3 0 2024-03-16T10:45:24.230 INFO:tasks.workunit.client.1.smithi026.stdout:8/571: link d5/d23/d4e/d69/c86 d5/d7/d7b/d96/ca5 0 2024-03-16T10:45:24.231 INFO:tasks.workunit.client.1.smithi026.stdout:8/572: readlink d5/d7/d2c/d36/d59/l64 0 2024-03-16T10:45:24.233 INFO:tasks.workunit.client.1.smithi026.stdout:8/573: read d5/d23/f68 [197995,68884] 0 2024-03-16T10:45:24.233 INFO:tasks.workunit.client.1.smithi026.stdout:2/568: chown d2/d29/d36/d98/cb3 5 1 2024-03-16T10:45:24.234 INFO:tasks.workunit.client.1.smithi026.stdout:2/569: write d2/d29/d17/fba [27669,99271] 0 2024-03-16T10:45:24.235 INFO:tasks.workunit.client.1.smithi026.stdout:2/570: chown d2/d29/d51/d55/c7a 85 1 2024-03-16T10:45:24.238 INFO:tasks.workunit.client.1.smithi026.stdout:5/573: creat d1/d1c/d2a/d55/d99/fcd x:0 0 0 2024-03-16T10:45:24.239 INFO:tasks.workunit.client.1.smithi026.stdout:5/574: chown d1/db/d1f/f5c 70 1 2024-03-16T10:45:24.240 INFO:tasks.workunit.client.1.smithi026.stdout:3/634: write d4/d89/f90 [944039,78294] 0 2024-03-16T10:45:24.242 INFO:tasks.workunit.client.1.smithi026.stdout:4/620: symlink d1/db/de/d4c/d79/daa/d44/lc6 0 2024-03-16T10:45:24.245 INFO:tasks.workunit.client.1.smithi026.stdout:7/617: rename d6/d1e/f6a to d6/d35/d3a/d59/d9e/fc2 0 2024-03-16T10:45:24.245 INFO:tasks.workunit.client.1.smithi026.stdout:9/615: dread d0/f4 [0,4194304] 0 2024-03-16T10:45:24.245 INFO:tasks.workunit.client.1.smithi026.stdout:6/477: fdatasync d0/d28/d2d/f55 0 2024-03-16T10:45:24.248 INFO:tasks.workunit.client.1.smithi026.stdout:9/616: fsync d0/d5f/d7f/f9c 0 2024-03-16T10:45:24.249 INFO:tasks.workunit.client.1.smithi026.stdout:4/621: mkdir d1/db/de/d4c/d81/dc7 0 2024-03-16T10:45:24.250 INFO:tasks.workunit.client.1.smithi026.stdout:3/635: dread d4/dc/dd/d3d/f3e [4194304,4194304] 0 2024-03-16T10:45:24.251 INFO:tasks.workunit.client.1.smithi026.stdout:2/571: symlink d2/d29/da4/lc8 0 2024-03-16T10:45:24.251 INFO:tasks.workunit.client.1.smithi026.stdout:9/617: dread d0/d5f/d7f/f9d [0,4194304] 0 2024-03-16T10:45:24.253 INFO:tasks.workunit.client.1.smithi026.stdout:9/618: truncate d0/da/d16/d17/fbc 127376 0 2024-03-16T10:45:24.255 INFO:tasks.workunit.client.1.smithi026.stdout:5/575: dread d1/d1c/d2a/dab/dc5/d69/d51/f54 [0,4194304] 0 2024-03-16T10:45:24.257 INFO:tasks.workunit.client.1.smithi026.stdout:4/622: creat d1/d85/fc8 x:0 0 0 2024-03-16T10:45:24.263 INFO:tasks.workunit.client.1.smithi026.stdout:7/618: mkdir d6/d2f/daf/d3f/dc3 0 2024-03-16T10:45:24.266 INFO:tasks.workunit.client.1.smithi026.stdout:4/623: dwrite d1/db/de/d4c/d79/daa/f43 [0,4194304] 0 2024-03-16T10:45:24.267 INFO:tasks.workunit.client.1.smithi026.stdout:4/624: chown d1/db/de/l23 193045289 1 2024-03-16T10:45:24.271 INFO:tasks.workunit.client.1.smithi026.stdout:5/576: symlink d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/lce 0 2024-03-16T10:45:24.272 INFO:tasks.workunit.client.1.smithi026.stdout:3/636: write d4/dc/dd/f75 [1002357,125134] 0 2024-03-16T10:45:24.274 INFO:tasks.workunit.client.1.smithi026.stdout:2/572: dread d2/d29/d17/d1a/d2b/d4b/f57 [0,4194304] 0 2024-03-16T10:45:24.276 INFO:tasks.workunit.client.1.smithi026.stdout:7/619: rmdir d6/de/d16/d88 39 2024-03-16T10:45:24.277 INFO:tasks.workunit.client.1.smithi026.stdout:9/619: write d0/f1d [198108,52317] 0 2024-03-16T10:45:24.285 INFO:tasks.workunit.client.1.smithi026.stdout:7/620: symlink d6/d35/d3a/dab/lc4 0 2024-03-16T10:45:24.286 INFO:tasks.workunit.client.1.smithi026.stdout:7/621: dread - d6/d2f/daf/d3f/d70/f9f zero size 2024-03-16T10:45:24.288 INFO:tasks.workunit.client.1.smithi026.stdout:9/620: creat d0/da/d4d/dc5/fd3 x:0 0 0 2024-03-16T10:45:24.289 INFO:tasks.workunit.client.1.smithi026.stdout:6/478: getdents d0/d10/d79/d7b 0 2024-03-16T10:45:24.292 INFO:tasks.workunit.client.1.smithi026.stdout:5/577: creat d1/db/d1f/d61/d65/d82/fcf x:0 0 0 2024-03-16T10:45:24.294 INFO:tasks.workunit.client.1.smithi026.stdout:3/637: mknod d4/dc/d31/d48/cdc 0 2024-03-16T10:45:24.295 INFO:tasks.workunit.client.1.smithi026.stdout:5/578: readlink d1/d1c/d2a/dab/dc5/d69/l63 0 2024-03-16T10:45:24.297 INFO:tasks.workunit.client.1.smithi026.stdout:6/479: dwrite d0/d10/d79/d7b/f9e [0,4194304] 0 2024-03-16T10:45:24.304 INFO:tasks.workunit.client.1.smithi026.stdout:7/622: mkdir d6/d2f/daf/d3f/dc5 0 2024-03-16T10:45:24.307 INFO:tasks.workunit.client.1.smithi026.stdout:5/579: dwrite d1/d1c/d2a/dab/dc5/d69/f67 [0,4194304] 0 2024-03-16T10:45:24.309 INFO:tasks.workunit.client.1.smithi026.stdout:2/573: truncate d2/d29/d17/d1a/d2b/d2e/fbf 4561001 0 2024-03-16T10:45:24.312 INFO:tasks.workunit.client.1.smithi026.stdout:9/621: dwrite d0/fae [0,4194304] 0 2024-03-16T10:45:24.313 INFO:tasks.workunit.client.1.smithi026.stdout:6/480: creat d0/d10/d79/d91/fa4 x:0 0 0 2024-03-16T10:45:24.315 INFO:tasks.workunit.client.1.smithi026.stdout:2/574: chown d2/d29/d17/d1a/d2b/d4b 243230 1 2024-03-16T10:45:24.315 INFO:tasks.workunit.client.1.smithi026.stdout:3/638: mkdir d4/dc/dd/d47/d53/db1/ddd 0 2024-03-16T10:45:24.317 INFO:tasks.workunit.client.1.smithi026.stdout:9/622: dread - d0/d63/d6b/fc2 zero size 2024-03-16T10:45:24.319 INFO:tasks.workunit.client.1.smithi026.stdout:7/623: truncate d6/d1e/f32 3078562 0 2024-03-16T10:45:24.324 INFO:tasks.workunit.client.1.smithi026.stdout:2/575: rename d2/fb2 to d2/d29/d51/d55/fc9 0 2024-03-16T10:45:24.326 INFO:tasks.workunit.client.1.smithi026.stdout:3/639: dwrite d4/d89/d97/f99 [0,4194304] 0 2024-03-16T10:45:24.329 INFO:tasks.workunit.client.1.smithi026.stdout:7/624: mknod d6/d2f/daf/cc6 0 2024-03-16T10:45:24.329 INFO:tasks.workunit.client.1.smithi026.stdout:4/625: getdents d1/db/de/d4c 0 2024-03-16T10:45:24.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/625: write d6/d2f/daf/f5c [4184610,20911] 0 2024-03-16T10:45:24.336 INFO:tasks.workunit.client.1.smithi026.stdout:5/580: rename d1/db/d27/f3e to d1/d6e/fd0 0 2024-03-16T10:45:24.340 INFO:tasks.workunit.client.1.smithi026.stdout:4/626: dwrite d1/fac [0,4194304] 0 2024-03-16T10:45:24.341 INFO:tasks.workunit.client.1.smithi026.stdout:4/627: chown d1/db/f64 3 1 2024-03-16T10:45:24.342 INFO:tasks.workunit.client.1.smithi026.stdout:7/626: dwrite d6/d35/d3a/f8e [0,4194304] 0 2024-03-16T10:45:24.345 INFO:tasks.workunit.client.1.smithi026.stdout:4/628: read d1/fac [4129908,42599] 0 2024-03-16T10:45:24.347 INFO:tasks.workunit.client.1.smithi026.stdout:6/481: getdents d0/d28/d49 0 2024-03-16T10:45:24.350 INFO:tasks.workunit.client.1.smithi026.stdout:2/576: symlink d2/lca 0 2024-03-16T10:45:24.355 INFO:tasks.workunit.client.1.smithi026.stdout:9/623: creat d0/da/d16/d5d/fd4 x:0 0 0 2024-03-16T10:45:24.372 INFO:tasks.workunit.client.1.smithi026.stdout:9/624: truncate d0/da/d13/d6e/f9b 664367 0 2024-03-16T10:45:24.374 INFO:tasks.workunit.client.1.smithi026.stdout:9/625: write d0/d63/d6b/fa5 [612274,71533] 0 2024-03-16T10:45:24.377 INFO:tasks.workunit.client.1.smithi026.stdout:7/627: symlink d6/d35/d3a/d4e/db0/lc7 0 2024-03-16T10:45:24.388 INFO:tasks.workunit.client.1.smithi026.stdout:6/482: unlink d0/d10/f80 0 2024-03-16T10:45:24.395 INFO:tasks.workunit.client.1.smithi026.stdout:5/581: dwrite d1/d1c/d2a/fc9 [0,4194304] 0 2024-03-16T10:45:24.401 INFO:tasks.workunit.client.1.smithi026.stdout:6/483: creat d0/d41/d71/fa5 x:0 0 0 2024-03-16T10:45:24.404 INFO:tasks.workunit.client.1.smithi026.stdout:9/626: fdatasync d0/da/d16/d17/d44/f24 0 2024-03-16T10:45:24.408 INFO:tasks.workunit.client.1.smithi026.stdout:7/628: mkdir d6/d1e/dbc/dc8 0 2024-03-16T10:45:24.409 INFO:tasks.workunit.client.1.smithi026.stdout:5/582: unlink d1/d1c/d4c/f88 0 2024-03-16T10:45:24.412 INFO:tasks.workunit.client.1.smithi026.stdout:4/629: rename d1/db/de/d6e/lc1 to d1/lc9 0 2024-03-16T10:45:24.415 INFO:tasks.workunit.client.1.smithi026.stdout:9/627: dwrite d0/da/d16/f3f [0,4194304] 0 2024-03-16T10:45:24.415 INFO:tasks.workunit.client.1.smithi026.stdout:6/484: creat d0/d10/d79/fa6 x:0 0 0 2024-03-16T10:45:24.417 INFO:tasks.workunit.client.1.smithi026.stdout:9/628: stat d0/d63/d6b/caa 0 2024-03-16T10:45:24.420 INFO:tasks.workunit.client.1.smithi026.stdout:9/629: write d0/da/d16/fad [983565,13417] 0 2024-03-16T10:45:24.425 INFO:tasks.workunit.client.1.smithi026.stdout:4/630: dwrite d1/db/de/d4c/d79/daa/f43 [0,4194304] 0 2024-03-16T10:45:24.429 INFO:tasks.workunit.client.1.smithi026.stdout:2/577: rename d2/d29/d51/d55/d96 to d2/d29/d17/d38/d64/dcb 0 2024-03-16T10:45:24.430 INFO:tasks.workunit.client.1.smithi026.stdout:5/583: dread d1/d1c/d2a/dab/dc5/f52 [0,4194304] 0 2024-03-16T10:45:24.432 INFO:tasks.workunit.client.1.smithi026.stdout:6/485: creat d0/d41/fa7 x:0 0 0 2024-03-16T10:45:24.432 INFO:tasks.workunit.client.1.smithi026.stdout:6/486: stat d0/l2b 0 2024-03-16T10:45:24.433 INFO:tasks.workunit.client.1.smithi026.stdout:9/630: rmdir d0/d5f/d7f 39 2024-03-16T10:45:24.435 INFO:tasks.workunit.client.1.smithi026.stdout:5/584: write d1/f9 [1704962,2559] 0 2024-03-16T10:45:24.436 INFO:tasks.workunit.client.1.smithi026.stdout:5/585: fdatasync d1/db/d1f/d61/fb9 0 2024-03-16T10:45:24.438 INFO:tasks.workunit.client.1.smithi026.stdout:6/487: truncate d0/f3 3598296 0 2024-03-16T10:45:24.439 INFO:tasks.workunit.client.1.smithi026.stdout:6/488: write d0/d41/fa7 [574469,41798] 0 2024-03-16T10:45:24.440 INFO:tasks.workunit.client.1.smithi026.stdout:9/631: write d0/d5f/fb6 [3407973,103303] 0 2024-03-16T10:45:24.445 INFO:tasks.workunit.client.1.smithi026.stdout:2/578: rename d2/d29/d17/d1a/d2b/d2e/f3c to d2/d29/d17/d1a/d2b/d4e/fcc 0 2024-03-16T10:45:24.459 INFO:tasks.workunit.client.1.smithi026.stdout:6/489: rename d0/d41/d71/fa5 to d0/d10/fa8 0 2024-03-16T10:45:24.460 INFO:tasks.workunit.client.1.smithi026.stdout:2/579: link d2/d29/d17/d1a/d2b/d4b/f5b d2/d29/d17/d1a/d2b/d2e/d60/fcd 0 2024-03-16T10:45:24.460 INFO:tasks.workunit.client.1.smithi026.stdout:6/490: stat d0/d10/d79/d7b/f9e 0 2024-03-16T10:45:24.468 INFO:tasks.workunit.client.1.smithi026.stdout:6/491: creat d0/d41/d4f/fa9 x:0 0 0 2024-03-16T10:45:24.469 INFO:tasks.workunit.client.1.smithi026.stdout:9/632: dwrite d0/da/f7a [0,4194304] 0 2024-03-16T10:45:24.479 INFO:tasks.workunit.client.1.smithi026.stdout:6/492: rename d0/d10/d79/d7b/la1 to d0/d41/d4f/laa 0 2024-03-16T10:45:24.485 INFO:tasks.workunit.client.1.smithi026.stdout:9/633: dread d0/da/f2a [0,4194304] 0 2024-03-16T10:45:24.487 INFO:tasks.workunit.client.1.smithi026.stdout:9/634: chown d0/c4e 621337255 1 2024-03-16T10:45:24.488 INFO:tasks.workunit.client.1.smithi026.stdout:9/635: chown d0/da/d16/d17/fa9 593875 1 2024-03-16T10:45:24.493 INFO:tasks.workunit.client.1.smithi026.stdout:2/580: write d2/d29/d17/d1a/d2b/d4b/f5b [304400,120201] 0 2024-03-16T10:45:24.494 INFO:tasks.workunit.client.1.smithi026.stdout:2/581: fdatasync d2/d29/d17/d1a/d2b/f91 0 2024-03-16T10:45:24.496 INFO:tasks.workunit.client.1.smithi026.stdout:2/582: write d2/f4d [1797098,12388] 0 2024-03-16T10:45:24.497 INFO:tasks.workunit.client.1.smithi026.stdout:9/636: getdents d0/da/d13/d6e 0 2024-03-16T10:45:24.498 INFO:tasks.workunit.client.1.smithi026.stdout:2/583: write d2/d29/d17/d38/f81 [2086194,62786] 0 2024-03-16T10:45:24.501 INFO:tasks.workunit.client.1.smithi026.stdout:9/637: mkdir d0/dce/dd5 0 2024-03-16T10:45:24.504 INFO:tasks.workunit.client.1.smithi026.stdout:9/638: dread d0/da/d4d/f8f [0,4194304] 0 2024-03-16T10:45:24.509 INFO:tasks.workunit.client.1.smithi026.stdout:2/584: dwrite d2/d29/d17/d1a/d2b/d2e/f31 [0,4194304] 0 2024-03-16T10:45:24.516 INFO:tasks.workunit.client.1.smithi026.stdout:2/585: dread d2/d29/d17/d1a/d2b/f73 [0,4194304] 0 2024-03-16T10:45:24.517 INFO:tasks.workunit.client.1.smithi026.stdout:2/586: fdatasync d2/d29/f3a 0 2024-03-16T10:45:24.517 INFO:tasks.workunit.client.1.smithi026.stdout:9/639: dread d0/da/d13/f38 [0,4194304] 0 2024-03-16T10:45:24.524 INFO:tasks.workunit.client.1.smithi026.stdout:2/587: unlink d2/d29/d17/d1a/f8e 0 2024-03-16T10:45:24.525 INFO:tasks.workunit.client.1.smithi026.stdout:9/640: mknod d0/da/d16/d5d/cd6 0 2024-03-16T10:45:24.530 INFO:tasks.workunit.client.1.smithi026.stdout:2/588: symlink d2/d29/d17/d1a/d2b/d69/lce 0 2024-03-16T10:45:24.534 INFO:tasks.workunit.client.1.smithi026.stdout:2/589: rename d2/d29/d17/d1a/d2b/d2e/d60/d7f/f8d to d2/d29/d17/d1a/d2b/d4e/d6f/fcf 0 2024-03-16T10:45:24.541 INFO:tasks.workunit.client.1.smithi026.stdout:9/641: write d0/da/d13/f69 [1722659,36062] 0 2024-03-16T10:45:24.544 INFO:tasks.workunit.client.1.smithi026.stdout:2/590: write d2/d29/d17/d1a/d2b/d4e/fcc [901323,10946] 0 2024-03-16T10:45:24.549 INFO:tasks.workunit.client.1.smithi026.stdout:2/591: rename d2/d29/d3b/la6 to d2/d29/d17/d1a/d2b/d4b/ld0 0 2024-03-16T10:45:24.552 INFO:tasks.workunit.client.1.smithi026.stdout:9/642: dwrite d0/da/f11 [0,4194304] 0 2024-03-16T10:45:24.553 INFO:tasks.workunit.client.1.smithi026.stdout:9/643: stat d0/da/d13/d6e/f74 0 2024-03-16T10:45:24.556 INFO:tasks.workunit.client.1.smithi026.stdout:2/592: dwrite d2/f35 [0,4194304] 0 2024-03-16T10:45:24.557 INFO:tasks.workunit.client.1.smithi026.stdout:9/644: unlink d0/cb0 0 2024-03-16T10:45:24.559 INFO:tasks.workunit.client.1.smithi026.stdout:9/645: write d0/da/d13/d6e/f8d [19016,102239] 0 2024-03-16T10:45:24.560 INFO:tasks.workunit.client.1.smithi026.stdout:2/593: unlink d2/d29/d51/f5f 0 2024-03-16T10:45:24.560 INFO:tasks.workunit.client.1.smithi026.stdout:9/646: fsync d0/da/d16/d17/fbc 0 2024-03-16T10:45:24.572 INFO:tasks.workunit.client.1.smithi026.stdout:9/647: rename d0/da/d13/d6e to d0/dd7 0 2024-03-16T10:45:24.572 INFO:tasks.workunit.client.1.smithi026.stdout:2/594: rmdir d2/d29/d17/d1a 39 2024-03-16T10:45:24.577 INFO:tasks.workunit.client.1.smithi026.stdout:9/648: truncate d0/da/d16/d17/d44/f1e 943154 0 2024-03-16T10:45:24.578 INFO:tasks.workunit.client.1.smithi026.stdout:2/595: truncate d2/d29/d17/d1a/d2b/d4e/d67/fbe 410657 0 2024-03-16T10:45:24.579 INFO:tasks.workunit.client.1.smithi026.stdout:2/596: dread - d2/d29/d3b/dbb/fc0 zero size 2024-03-16T10:45:24.594 INFO:tasks.workunit.client.1.smithi026.stdout:2/597: truncate d2/d29/d17/d38/f81 470885 0 2024-03-16T10:45:24.878 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:24 smithi002 ceph-mon[29248]: pgmap v7: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 31 MiB/s rd, 85 MiB/s wr, 233 op/s 2024-03-16T10:45:24.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:24 smithi026 ceph-mon[37041]: pgmap v7: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 31 MiB/s rd, 85 MiB/s wr, 233 op/s 2024-03-16T10:45:25.063 INFO:tasks.workunit.client.1.smithi026.stdout:4/631: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:1/558: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:0/658: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:3/640: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:6/493: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:7/629: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:2/598: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:9/649: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:5/586: sync 2024-03-16T10:45:25.064 INFO:tasks.workunit.client.1.smithi026.stdout:8/574: sync 2024-03-16T10:45:25.066 INFO:tasks.workunit.client.1.smithi026.stdout:6/494: fsync d0/d28/f9c 0 2024-03-16T10:45:25.066 INFO:tasks.workunit.client.1.smithi026.stdout:0/659: chown d1/d12/d2f/daa/dac/fb5 2456914 1 2024-03-16T10:45:25.068 INFO:tasks.workunit.client.1.smithi026.stdout:6/495: write d0/d10/d79/d91/f94 [3745058,39335] 0 2024-03-16T10:45:25.071 INFO:tasks.workunit.client.1.smithi026.stdout:6/496: chown d0/d28/d2d/f60 3265608 1 2024-03-16T10:45:25.071 INFO:tasks.workunit.client.1.smithi026.stdout:0/660: truncate d1/d12/d2f/daa/dd1/fca 203545 0 2024-03-16T10:45:25.074 INFO:tasks.workunit.client.1.smithi026.stdout:2/599: dwrite d2/d29/d85/fc7 [0,4194304] 0 2024-03-16T10:45:25.082 INFO:tasks.workunit.client.1.smithi026.stdout:5/587: mkdir d1/db/d27/dd1 0 2024-03-16T10:45:25.082 INFO:tasks.workunit.client.1.smithi026.stdout:3/641: mknod d4/dc/d31/cde 0 2024-03-16T10:45:25.084 INFO:tasks.workunit.client.1.smithi026.stdout:3/642: dread - d4/dc/d73/d83/f6c zero size 2024-03-16T10:45:25.086 INFO:tasks.workunit.client.1.smithi026.stdout:7/630: creat d6/d35/d3a/d4e/d6c/fc9 x:0 0 0 2024-03-16T10:45:25.088 INFO:tasks.workunit.client.1.smithi026.stdout:6/497: chown d0/c13 10 1 2024-03-16T10:45:25.088 INFO:tasks.workunit.client.1.smithi026.stdout:5/588: dread d1/d1c/d2a/fc9 [0,4194304] 0 2024-03-16T10:45:25.089 INFO:tasks.workunit.client.1.smithi026.stdout:6/498: fsync d0/d5b/f18 0 2024-03-16T10:45:25.092 INFO:tasks.workunit.client.1.smithi026.stdout:0/661: read d1/d12/d2f/d48/d55/d60/d9b/d11/fc0 [750702,15320] 0 2024-03-16T10:45:25.095 INFO:tasks.workunit.client.1.smithi026.stdout:6/499: dread d0/d10/d79/d91/f94 [0,4194304] 0 2024-03-16T10:45:25.097 INFO:tasks.workunit.client.1.smithi026.stdout:6/500: dread - d0/d41/d4f/fa9 zero size 2024-03-16T10:45:25.100 INFO:tasks.workunit.client.1.smithi026.stdout:8/575: dread d5/f7f [0,4194304] 0 2024-03-16T10:45:25.105 INFO:tasks.workunit.client.1.smithi026.stdout:6/501: dwrite d0/d41/d4f/fa9 [0,4194304] 0 2024-03-16T10:45:25.115 INFO:tasks.workunit.client.1.smithi026.stdout:3/643: fdatasync d4/dc/dd/d47/f86 0 2024-03-16T10:45:25.119 INFO:tasks.workunit.client.1.smithi026.stdout:4/632: dwrite d1/db/de/d4c/d59/f62 [0,4194304] 0 2024-03-16T10:45:25.119 INFO:tasks.workunit.client.1.smithi026.stdout:9/650: link d0/da/d16/d5d/cd6 d0/da/d4d/cd8 0 2024-03-16T10:45:25.125 INFO:tasks.workunit.client.1.smithi026.stdout:8/576: symlink d5/dc/d3f/la6 0 2024-03-16T10:45:25.127 INFO:tasks.workunit.client.1.smithi026.stdout:2/600: dwrite d2/d29/d17/d1a/d2b/f73 [0,4194304] 0 2024-03-16T10:45:25.129 INFO:tasks.workunit.client.1.smithi026.stdout:8/577: chown d5/d7/d2c/d36/d59/f78 19 1 2024-03-16T10:45:25.133 INFO:tasks.workunit.client.1.smithi026.stdout:1/559: truncate d0/d17/d1f/d20/f43 358447 0 2024-03-16T10:45:25.134 INFO:tasks.workunit.client.1.smithi026.stdout:1/560: fdatasync d0/d17/d1f/d63/f83 0 2024-03-16T10:45:25.134 INFO:tasks.workunit.client.1.smithi026.stdout:5/589: mkdir d1/db/d1f/d61/d65/d82/dbb/dd2 0 2024-03-16T10:45:25.135 INFO:tasks.workunit.client.1.smithi026.stdout:1/561: chown d0/d17/d1f/d63/d6a/d7c/d6b/l72 1299007 1 2024-03-16T10:45:25.136 INFO:tasks.workunit.client.1.smithi026.stdout:1/562: dread - d0/d17/d1f/d63/d6a/fa3 zero size 2024-03-16T10:45:25.138 INFO:tasks.workunit.client.1.smithi026.stdout:1/563: write d0/d17/d4d/d70/f9b [746995,120434] 0 2024-03-16T10:45:25.142 INFO:tasks.workunit.client.1.smithi026.stdout:0/662: mknod d1/d12/d2f/daa/dd1/d1a/d2b/db4/ce1 0 2024-03-16T10:45:25.149 INFO:tasks.workunit.client.1.smithi026.stdout:4/633: mkdir d1/d85/da2/dca 0 2024-03-16T10:45:25.150 INFO:tasks.workunit.client.1.smithi026.stdout:0/663: dwrite d1/d12/d2f/d48/d55/d60/fa0 [0,4194304] 0 2024-03-16T10:45:25.164 INFO:tasks.workunit.client.1.smithi026.stdout:2/601: dwrite d2/d29/d17/d1a/d2b/d4e/d67/f78 [0,4194304] 0 2024-03-16T10:45:25.170 INFO:tasks.workunit.client.1.smithi026.stdout:3/644: mknod d4/dc/d73/cdf 0 2024-03-16T10:45:25.173 INFO:tasks.workunit.client.1.smithi026.stdout:4/634: unlink d1/db/de/d4c/d79/daa/d44/d48/l4a 0 2024-03-16T10:45:25.173 INFO:tasks.workunit.client.1.smithi026.stdout:1/564: dwrite d0/d17/d1f/d63/d6a/d7c/f30 [0,4194304] 0 2024-03-16T10:45:25.173 INFO:tasks.workunit.client.1.smithi026.stdout:0/664: truncate d1/d12/d2f/daa/dd1/d1a/f46 1850382 0 2024-03-16T10:45:25.177 INFO:tasks.workunit.client.1.smithi026.stdout:5/590: dwrite d1/d1c/d2a/dab/dc5/d69/d51/f7f [0,4194304] 0 2024-03-16T10:45:25.178 INFO:tasks.workunit.client.1.smithi026.stdout:4/635: dread - d1/db/de/d4c/d79/daa/d61/f6c zero size 2024-03-16T10:45:25.178 INFO:tasks.workunit.client.1.smithi026.stdout:8/578: mknod d5/d7/d2c/d36/d59/d93/da3/ca7 0 2024-03-16T10:45:25.180 INFO:tasks.workunit.client.1.smithi026.stdout:4/636: chown d1/db/de/d4c/d59/d6a/ca0 200522630 1 2024-03-16T10:45:25.184 INFO:tasks.workunit.client.1.smithi026.stdout:1/565: write d0/d17/d1f/d63/d6a/d7c/d84/fa5 [789214,97005] 0 2024-03-16T10:45:25.185 INFO:tasks.workunit.client.1.smithi026.stdout:6/502: getdents d0/d41 0 2024-03-16T10:45:25.192 INFO:tasks.workunit.client.1.smithi026.stdout:9/651: getdents d0/da/d14 0 2024-03-16T10:45:25.193 INFO:tasks.workunit.client.1.smithi026.stdout:9/652: readlink d0/da/d16/d17/l22 0 2024-03-16T10:45:25.194 INFO:tasks.workunit.client.1.smithi026.stdout:3/645: mkdir d4/dc/d31/dbe/de0 0 2024-03-16T10:45:25.197 INFO:tasks.workunit.client.1.smithi026.stdout:0/665: mknod d1/d12/d2f/ce2 0 2024-03-16T10:45:25.202 INFO:tasks.workunit.client.1.smithi026.stdout:1/566: creat d0/d17/d1f/d63/d6a/d7c/d84/fb0 x:0 0 0 2024-03-16T10:45:25.204 INFO:tasks.workunit.client.1.smithi026.stdout:1/567: write d0/d17/d1f/d63/d6a/d7c/d6b/f87 [1015317,106424] 0 2024-03-16T10:45:25.208 INFO:tasks.workunit.client.1.smithi026.stdout:3/646: mknod d4/dc/d73/d83/ce1 0 2024-03-16T10:45:25.211 INFO:tasks.workunit.client.1.smithi026.stdout:5/591: link d1/c3 d1/db/d27/cd3 0 2024-03-16T10:45:25.212 INFO:tasks.workunit.client.1.smithi026.stdout:5/592: fsync d1/db/d1f/d61/fa7 0 2024-03-16T10:45:25.213 INFO:tasks.workunit.client.1.smithi026.stdout:9/653: write d0/da/d13/f49 [1839646,54735] 0 2024-03-16T10:45:25.215 INFO:tasks.workunit.client.1.smithi026.stdout:9/654: chown d0/d63/d6b 1 1 2024-03-16T10:45:25.217 INFO:tasks.workunit.client.1.smithi026.stdout:6/503: dwrite d0/d28/d49/f5a [0,4194304] 0 2024-03-16T10:45:25.219 INFO:tasks.workunit.client.1.smithi026.stdout:4/637: truncate d1/db/de/d4c/d79/daa/f95 1270372 0 2024-03-16T10:45:25.221 INFO:tasks.workunit.client.1.smithi026.stdout:2/602: getdents d2/d29/d17/d1a/d2b/d4e/d6f 0 2024-03-16T10:45:25.223 INFO:tasks.workunit.client.1.smithi026.stdout:1/568: write d0/d17/f37 [4984804,84331] 0 2024-03-16T10:45:25.223 INFO:tasks.workunit.client.1.smithi026.stdout:8/579: symlink d5/da0/la8 0 2024-03-16T10:45:25.224 INFO:tasks.workunit.client.1.smithi026.stdout:1/569: stat d0/d17/d4d/daa 0 2024-03-16T10:45:25.225 INFO:tasks.workunit.client.1.smithi026.stdout:3/647: truncate d4/dc/f4e 196887 0 2024-03-16T10:45:25.232 INFO:tasks.workunit.client.1.smithi026.stdout:5/593: chown d1/db/l37 42 1 2024-03-16T10:45:25.234 INFO:tasks.workunit.client.1.smithi026.stdout:5/594: truncate d1/d1c/d2a/dab/dc5/f52 4424294 0 2024-03-16T10:45:25.235 INFO:tasks.workunit.client.1.smithi026.stdout:5/595: stat d1/d1c/d2a/dab/dc5/laa 0 2024-03-16T10:45:25.236 INFO:tasks.workunit.client.1.smithi026.stdout:0/666: truncate d1/d12/d2f/d48/d55/d60/fa0 4159919 0 2024-03-16T10:45:25.239 INFO:tasks.workunit.client.1.smithi026.stdout:5/596: dread d1/d1c/d2a/dab/dc5/d69/d51/f54 [0,4194304] 0 2024-03-16T10:45:25.248 INFO:tasks.workunit.client.1.smithi026.stdout:0/667: creat d1/d12/d2f/d48/d55/fe3 x:0 0 0 2024-03-16T10:45:25.249 INFO:tasks.workunit.client.1.smithi026.stdout:2/603: write d2/d29/d17/d1a/d2b/d4b/f57 [156037,32486] 0 2024-03-16T10:45:25.250 INFO:tasks.workunit.client.1.smithi026.stdout:2/604: chown d2/d29/d51/d72/f76 2 1 2024-03-16T10:45:25.252 INFO:tasks.workunit.client.1.smithi026.stdout:8/580: write d5/d23/f4b [454169,100244] 0 2024-03-16T10:45:25.255 INFO:tasks.workunit.client.1.smithi026.stdout:4/638: creat d1/db/de/d4c/d6b/d7a/db1/fcb x:0 0 0 2024-03-16T10:45:25.256 INFO:tasks.workunit.client.1.smithi026.stdout:7/631: sync 2024-03-16T10:45:25.256 INFO:tasks.workunit.client.1.smithi026.stdout:4/639: write d1/db/de/d4c/d6b/fad [1358991,58001] 0 2024-03-16T10:45:25.262 INFO:tasks.workunit.client.1.smithi026.stdout:9/655: rename d0/d63/d6b/caa to d0/da/d13/cd9 0 2024-03-16T10:45:25.264 INFO:tasks.workunit.client.1.smithi026.stdout:9/656: write d0/da/d16/f3f [400589,107208] 0 2024-03-16T10:45:25.264 INFO:tasks.workunit.client.1.smithi026.stdout:0/668: symlink d1/d12/d2f/d48/d55/d60/d9b/d11/le4 0 2024-03-16T10:45:25.271 INFO:tasks.workunit.client.1.smithi026.stdout:0/669: dwrite d1/f40 [4194304,4194304] 0 2024-03-16T10:45:25.274 INFO:tasks.workunit.client.1.smithi026.stdout:3/648: truncate d4/f41 2411471 0 2024-03-16T10:45:25.276 INFO:tasks.workunit.client.1.smithi026.stdout:3/649: fsync d4/d89/d97/fb7 0 2024-03-16T10:45:25.276 INFO:tasks.workunit.client.1.smithi026.stdout:8/581: dwrite d5/d5d/f92 [4194304,4194304] 0 2024-03-16T10:45:25.279 INFO:tasks.workunit.client.1.smithi026.stdout:5/597: write d1/d1c/d68/f74 [2111698,88417] 0 2024-03-16T10:45:25.280 INFO:tasks.workunit.client.1.smithi026.stdout:5/598: fsync d1/db/d27/f23 0 2024-03-16T10:45:25.281 INFO:tasks.workunit.client.1.smithi026.stdout:9/657: mknod d0/d63/d6b/cda 0 2024-03-16T10:45:25.286 INFO:tasks.workunit.client.1.smithi026.stdout:2/605: mkdir d2/dd1 0 2024-03-16T10:45:25.288 INFO:tasks.workunit.client.1.smithi026.stdout:2/606: read d2/d29/d17/d1a/d2b/d4e/fcc [3692385,123591] 0 2024-03-16T10:45:25.289 INFO:tasks.workunit.client.1.smithi026.stdout:7/632: mknod d6/cca 0 2024-03-16T10:45:25.291 INFO:tasks.workunit.client.1.smithi026.stdout:0/670: symlink d1/d12/d2f/daa/le5 0 2024-03-16T10:45:25.294 INFO:tasks.workunit.client.1.smithi026.stdout:1/570: getdents d0 0 2024-03-16T10:45:25.297 INFO:tasks.workunit.client.1.smithi026.stdout:6/504: rename d0/d10/d79/d91/ca3 to d0/d41/d71/cab 0 2024-03-16T10:45:25.298 INFO:tasks.workunit.client.1.smithi026.stdout:6/505: write d0/f35 [561607,49378] 0 2024-03-16T10:45:25.305 INFO:tasks.workunit.client.1.smithi026.stdout:5/599: unlink d1/db/d27/cd3 0 2024-03-16T10:45:25.305 INFO:tasks.workunit.client.1.smithi026.stdout:5/600: readlink d1/db/l37 0 2024-03-16T10:45:25.307 INFO:tasks.workunit.client.1.smithi026.stdout:9/658: unlink d0/d63/c96 0 2024-03-16T10:45:25.313 INFO:tasks.workunit.client.1.smithi026.stdout:7/633: dread d6/f91 [0,4194304] 0 2024-03-16T10:45:25.316 INFO:tasks.workunit.client.1.smithi026.stdout:8/582: dwrite d5/dc/d40/f77 [0,4194304] 0 2024-03-16T10:45:25.317 INFO:tasks.workunit.client.1.smithi026.stdout:8/583: readlink d5/dc/d3f/la6 0 2024-03-16T10:45:25.323 INFO:tasks.workunit.client.1.smithi026.stdout:3/650: rename d4/d89/d97/f6e to d4/dc/dd/d3d/fe2 0 2024-03-16T10:45:25.324 INFO:tasks.workunit.client.1.smithi026.stdout:6/506: mknod d0/d10/d79/cac 0 2024-03-16T10:45:25.325 INFO:tasks.workunit.client.1.smithi026.stdout:3/651: fsync d4/dc/d31/d48/d69/d78/fa2 0 2024-03-16T10:45:25.325 INFO:tasks.workunit.client.1.smithi026.stdout:2/607: dwrite d2/d29/d51/d72/f75 [0,4194304] 0 2024-03-16T10:45:25.327 INFO:tasks.workunit.client.1.smithi026.stdout:2/608: fsync d2/d29/d36/d98/fa3 0 2024-03-16T10:45:25.327 INFO:tasks.workunit.client.1.smithi026.stdout:9/659: mknod d0/dce/cdb 0 2024-03-16T10:45:25.328 INFO:tasks.workunit.client.1.smithi026.stdout:9/660: read - d0/da/d16/d17/fc7 zero size 2024-03-16T10:45:25.329 INFO:tasks.workunit.client.1.smithi026.stdout:4/640: getdents d1/db/de/d4c/d79/daa/d44/d48 0 2024-03-16T10:45:25.330 INFO:tasks.workunit.client.1.smithi026.stdout:9/661: truncate d0/dd7/fd1 56713 0 2024-03-16T10:45:25.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/634: truncate d6/d2f/daf/d3f/d54/f8d 440098 0 2024-03-16T10:45:25.344 INFO:tasks.workunit.client.1.smithi026.stdout:3/652: read d4/dc/d31/f51 [57200,117681] 0 2024-03-16T10:45:25.348 INFO:tasks.workunit.client.1.smithi026.stdout:7/635: symlink d6/d2f/daf/d73/lcb 0 2024-03-16T10:45:25.349 INFO:tasks.workunit.client.1.smithi026.stdout:7/636: chown d6/d35/d3a/d59/f96 52 1 2024-03-16T10:45:25.349 INFO:tasks.workunit.client.1.smithi026.stdout:0/671: creat d1/d12/d2f/daa/dd1/fe6 x:0 0 0 2024-03-16T10:45:25.352 INFO:tasks.workunit.client.1.smithi026.stdout:5/601: mknod d1/d1c/d2a/db0/cd4 0 2024-03-16T10:45:25.354 INFO:tasks.workunit.client.1.smithi026.stdout:4/641: creat d1/db/de/d4c/d81/dc7/fcc x:0 0 0 2024-03-16T10:45:25.356 INFO:tasks.workunit.client.1.smithi026.stdout:9/662: mknod d0/da/cdc 0 2024-03-16T10:45:25.357 INFO:tasks.workunit.client.1.smithi026.stdout:0/672: mknod d1/ce7 0 2024-03-16T10:45:25.360 INFO:tasks.workunit.client.1.smithi026.stdout:1/571: rename d0/d17/d1f/l60 to d0/d17/d1f/d63/lb1 0 2024-03-16T10:45:25.362 INFO:tasks.workunit.client.1.smithi026.stdout:6/507: creat d0/d10/fad x:0 0 0 2024-03-16T10:45:25.363 INFO:tasks.workunit.client.1.smithi026.stdout:1/572: chown d0/d17/c4c 2704217 1 2024-03-16T10:45:25.363 INFO:tasks.workunit.client.1.smithi026.stdout:5/602: creat d1/db/d27/fd5 x:0 0 0 2024-03-16T10:45:25.365 INFO:tasks.workunit.client.1.smithi026.stdout:1/573: stat d0/d17/d1f/d63/l55 0 2024-03-16T10:45:25.366 INFO:tasks.workunit.client.1.smithi026.stdout:0/673: dwrite d1/d12/d2f/daa/dd1/fb3 [0,4194304] 0 2024-03-16T10:45:25.370 INFO:tasks.workunit.client.1.smithi026.stdout:4/642: dread d1/db/de/d4c/d79/daa/f5f [0,4194304] 0 2024-03-16T10:45:25.375 INFO:tasks.workunit.client.1.smithi026.stdout:5/603: dwrite d1/db/d1f/d61/d65/d82/fcf [0,4194304] 0 2024-03-16T10:45:25.377 INFO:tasks.workunit.client.1.smithi026.stdout:8/584: rename d5/dc/d3f/f55 to d5/d7/d2c/d36/d59/d93/da3/fa9 0 2024-03-16T10:45:25.381 INFO:tasks.workunit.client.1.smithi026.stdout:4/643: dwrite d1/db/de/d4c/d79/daa/d61/f7c [0,4194304] 0 2024-03-16T10:45:25.384 INFO:tasks.workunit.client.1.smithi026.stdout:8/585: read - d5/d23/d4e/d69/d76/f9e zero size 2024-03-16T10:45:25.388 INFO:tasks.workunit.client.1.smithi026.stdout:2/609: getdents d2/d29/d3b 0 2024-03-16T10:45:25.390 INFO:tasks.workunit.client.1.smithi026.stdout:7/637: write d6/d2f/daf/d3f/d54/f8d [328951,26543] 0 2024-03-16T10:45:25.391 INFO:tasks.workunit.client.1.smithi026.stdout:3/653: creat d4/dc/d31/d48/fe3 x:0 0 0 2024-03-16T10:45:25.393 INFO:tasks.workunit.client.1.smithi026.stdout:7/638: truncate d6/d2f/daf/d3f/d65/f71 2450818 0 2024-03-16T10:45:25.393 INFO:tasks.workunit.client.1.smithi026.stdout:1/574: unlink d0/d17/d1f/d63/d6a/d7c/d2e/l39 0 2024-03-16T10:45:25.395 INFO:tasks.workunit.client.1.smithi026.stdout:1/575: readlink d0/d17/d1f/l44 0 2024-03-16T10:45:25.401 INFO:tasks.workunit.client.1.smithi026.stdout:1/576: dread d0/d17/d1f/d63/d6a/d7c/d84/fa5 [0,4194304] 0 2024-03-16T10:45:25.403 INFO:tasks.workunit.client.1.smithi026.stdout:0/674: dwrite d1/d12/d2f/daa/dd1/fc2 [0,4194304] 0 2024-03-16T10:45:25.407 INFO:tasks.workunit.client.1.smithi026.stdout:6/508: truncate d0/f3 112130 0 2024-03-16T10:45:25.408 INFO:tasks.workunit.client.1.smithi026.stdout:4/644: readlink d1/db/de/d4c/d59/d6a/l84 0 2024-03-16T10:45:25.411 INFO:tasks.workunit.client.1.smithi026.stdout:8/586: rmdir d5/d7/d2c/d36/d59 39 2024-03-16T10:45:25.413 INFO:tasks.workunit.client.1.smithi026.stdout:2/610: creat d2/d29/d17/d1a/d2b/d4e/d67/fd2 x:0 0 0 2024-03-16T10:45:25.415 INFO:tasks.workunit.client.1.smithi026.stdout:3/654: mknod d4/d89/d97/d7d/dae/ce4 0 2024-03-16T10:45:25.417 INFO:tasks.workunit.client.1.smithi026.stdout:9/663: creat d0/fdd x:0 0 0 2024-03-16T10:45:25.418 INFO:tasks.workunit.client.1.smithi026.stdout:9/664: write d0/da/d16/d5d/d72/d62/f80 [652929,12264] 0 2024-03-16T10:45:25.421 INFO:tasks.workunit.client.1.smithi026.stdout:7/639: rename d6/d2f/daf to d6/d35/d3a/d4e/dcc 0 2024-03-16T10:45:25.427 INFO:tasks.workunit.client.1.smithi026.stdout:1/577: creat d0/d17/fb2 x:0 0 0 2024-03-16T10:45:25.428 INFO:tasks.workunit.client.1.smithi026.stdout:5/604: getdents d1/db/d1f/d61/d65/d82/dbb/dd2 0 2024-03-16T10:45:25.431 INFO:tasks.workunit.client.1.smithi026.stdout:1/578: dread d0/d17/f81 [0,4194304] 0 2024-03-16T10:45:25.432 INFO:tasks.workunit.client.1.smithi026.stdout:0/675: unlink d1/d12/d2f/daa/dd1/d1a/d2b/f32 0 2024-03-16T10:45:25.437 INFO:tasks.workunit.client.1.smithi026.stdout:1/579: dwrite d0/f8f [0,4194304] 0 2024-03-16T10:45:25.438 INFO:tasks.workunit.client.1.smithi026.stdout:1/580: dread - d0/d17/d1f/d63/d6a/d7c/d84/fa7 zero size 2024-03-16T10:45:25.439 INFO:tasks.workunit.client.1.smithi026.stdout:8/587: mknod d5/dc/d40/d53/caa 0 2024-03-16T10:45:25.441 INFO:tasks.workunit.client.1.smithi026.stdout:1/581: write d0/f7f [966157,34804] 0 2024-03-16T10:45:25.443 INFO:tasks.workunit.client.1.smithi026.stdout:3/655: chown d4/dc/dd/d47/l63 38775 1 2024-03-16T10:45:25.443 INFO:tasks.workunit.client.1.smithi026.stdout:1/582: dread - d0/d17/d1f/d63/d6a/fa3 zero size 2024-03-16T10:45:25.445 INFO:tasks.workunit.client.1.smithi026.stdout:9/665: mknod d0/dce/cde 0 2024-03-16T10:45:25.450 INFO:tasks.workunit.client.1.smithi026.stdout:7/640: read d6/d1e/f32 [2892156,108847] 0 2024-03-16T10:45:25.451 INFO:tasks.workunit.client.1.smithi026.stdout:1/583: dread d0/d17/d1f/d63/d6a/d7c/d84/d9d/f4e [0,4194304] 0 2024-03-16T10:45:25.452 INFO:tasks.workunit.client.1.smithi026.stdout:2/611: write d2/d29/d17/d1a/d2b/d4b/f5c [551653,94637] 0 2024-03-16T10:45:25.454 INFO:tasks.workunit.client.1.smithi026.stdout:1/584: write d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fad [1038796,102385] 0 2024-03-16T10:45:25.456 INFO:tasks.workunit.client.1.smithi026.stdout:1/585: chown d0/d17/d1f/d63/f83 1 1 2024-03-16T10:45:25.457 INFO:tasks.workunit.client.1.smithi026.stdout:0/676: symlink d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/le8 0 2024-03-16T10:45:25.460 INFO:tasks.workunit.client.1.smithi026.stdout:5/605: dwrite d1/d1c/d2a/dab/dc5/f97 [0,4194304] 0 2024-03-16T10:45:25.463 INFO:tasks.workunit.client.1.smithi026.stdout:0/677: chown d1/d12/d2f/daa/dd1/d1a/d2b/ca3 14 1 2024-03-16T10:45:25.464 INFO:tasks.workunit.client.1.smithi026.stdout:3/656: stat d4/dc/d73/d83/d7c/dd7/d62/l71 0 2024-03-16T10:45:25.464 INFO:tasks.workunit.client.1.smithi026.stdout:0/678: chown d1/d12/d2f/d48/d55 856302 1 2024-03-16T10:45:25.468 INFO:tasks.workunit.client.1.smithi026.stdout:9/666: chown d0/da/d16/d5d/d72/lac 322883105 1 2024-03-16T10:45:25.475 INFO:tasks.workunit.client.1.smithi026.stdout:9/667: dwrite d0/d5f/fb6 [0,4194304] 0 2024-03-16T10:45:25.477 INFO:tasks.workunit.client.1.smithi026.stdout:2/612: unlink d2/d29/d17/f27 0 2024-03-16T10:45:25.477 INFO:tasks.workunit.client.1.smithi026.stdout:6/509: link d0/c9f d0/d41/d71/cae 0 2024-03-16T10:45:25.479 INFO:tasks.workunit.client.1.smithi026.stdout:9/668: truncate d0/dce/fd0 130501 0 2024-03-16T10:45:25.479 INFO:tasks.workunit.client.1.smithi026.stdout:4/645: dwrite d1/db/de/d4c/d59/fc2 [0,4194304] 0 2024-03-16T10:45:25.481 INFO:tasks.workunit.client.1.smithi026.stdout:4/646: chown d1/db/de/d4c/f8a 9107 1 2024-03-16T10:45:25.483 INFO:tasks.workunit.client.1.smithi026.stdout:8/588: creat d5/d7/d2c/d36/d59/d93/fab x:0 0 0 2024-03-16T10:45:25.484 INFO:tasks.workunit.client.1.smithi026.stdout:3/657: unlink d4/dc/cc9 0 2024-03-16T10:45:25.489 INFO:tasks.workunit.client.1.smithi026.stdout:0/679: write d1/d12/d2f/d48/d55/d60/d9b/fbe [5089935,36143] 0 2024-03-16T10:45:25.492 INFO:tasks.workunit.client.1.smithi026.stdout:6/510: symlink d0/d5b/laf 0 2024-03-16T10:45:25.493 INFO:tasks.workunit.client.1.smithi026.stdout:6/511: chown d0/d28/d2d 0 1 2024-03-16T10:45:25.495 INFO:tasks.workunit.client.1.smithi026.stdout:2/613: symlink d2/d29/d3b/dbb/ld3 0 2024-03-16T10:45:25.497 INFO:tasks.workunit.client.1.smithi026.stdout:9/669: creat d0/da/d16/d17/d44/fdf x:0 0 0 2024-03-16T10:45:25.497 INFO:tasks.workunit.client.1.smithi026.stdout:2/614: write d2/d29/d17/d1a/d2b/d2e/d60/d7f/fc5 [685409,94830] 0 2024-03-16T10:45:25.499 INFO:tasks.workunit.client.1.smithi026.stdout:2/615: dread - d2/d29/d17/d1a/d2b/f9a zero size 2024-03-16T10:45:25.502 INFO:tasks.workunit.client.1.smithi026.stdout:3/658: creat d4/dc/d31/fe5 x:0 0 0 2024-03-16T10:45:25.503 INFO:tasks.workunit.client.1.smithi026.stdout:3/659: fsync d4/dc/dd/fb4 0 2024-03-16T10:45:25.504 INFO:tasks.workunit.client.1.smithi026.stdout:7/641: creat d6/de/fcd x:0 0 0 2024-03-16T10:45:25.508 INFO:tasks.workunit.client.1.smithi026.stdout:6/512: fsync d0/d10/f23 0 2024-03-16T10:45:25.510 INFO:tasks.workunit.client.1.smithi026.stdout:9/670: creat d0/d5f/da2/fe0 x:0 0 0 2024-03-16T10:45:25.512 INFO:tasks.workunit.client.1.smithi026.stdout:9/671: read d0/f82 [3305872,109874] 0 2024-03-16T10:45:25.512 INFO:tasks.workunit.client.1.smithi026.stdout:1/586: truncate d0/d17/d1f/f90 226081 0 2024-03-16T10:45:25.514 INFO:tasks.workunit.client.1.smithi026.stdout:8/589: fdatasync d5/d7/f57 0 2024-03-16T10:45:25.516 INFO:tasks.workunit.client.1.smithi026.stdout:2/616: mknod d2/d29/da4/cd4 0 2024-03-16T10:45:25.520 INFO:tasks.workunit.client.1.smithi026.stdout:3/660: mkdir d4/dc/d73/d83/d7c/dd7/d62/d4f/de6 0 2024-03-16T10:45:25.521 INFO:tasks.workunit.client.1.smithi026.stdout:0/680: write d1/d12/d2f/d48/d55/d60/d9b/f6f [42624,68783] 0 2024-03-16T10:45:25.522 INFO:tasks.workunit.client.1.smithi026.stdout:4/647: dwrite d1/f22 [0,4194304] 0 2024-03-16T10:45:25.522 INFO:tasks.workunit.client.1.smithi026.stdout:7/642: mkdir d6/de/d16/d44/dce 0 2024-03-16T10:45:25.524 INFO:tasks.workunit.client.1.smithi026.stdout:4/648: chown d1/db/l39 70 1 2024-03-16T10:45:25.528 INFO:tasks.workunit.client.1.smithi026.stdout:6/513: mkdir d0/d28/d49/db0 0 2024-03-16T10:45:25.530 INFO:tasks.workunit.client.1.smithi026.stdout:6/514: read d0/f35 [308776,71321] 0 2024-03-16T10:45:25.542 INFO:tasks.workunit.client.1.smithi026.stdout:3/661: creat d4/dc/d73/d83/d57/fe7 x:0 0 0 2024-03-16T10:45:25.545 INFO:tasks.workunit.client.1.smithi026.stdout:9/672: dwrite d0/da/d4d/f5e [4194304,4194304] 0 2024-03-16T10:45:25.546 INFO:tasks.workunit.client.1.smithi026.stdout:6/515: dread d0/d10/d79/d7b/f9d [0,4194304] 0 2024-03-16T10:45:25.547 INFO:tasks.workunit.client.1.smithi026.stdout:0/681: rmdir d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96 39 2024-03-16T10:45:25.554 INFO:tasks.workunit.client.1.smithi026.stdout:7/643: write d6/d2f/f5f [800705,85048] 0 2024-03-16T10:45:25.554 INFO:tasks.workunit.client.1.smithi026.stdout:6/516: dwrite d0/d10/ff [0,4194304] 0 2024-03-16T10:45:25.557 INFO:tasks.workunit.client.1.smithi026.stdout:4/649: write d1/db/de/d4c/f8a [746589,86297] 0 2024-03-16T10:45:25.563 INFO:tasks.workunit.client.1.smithi026.stdout:4/650: write d1/f52 [2490794,88264] 0 2024-03-16T10:45:25.565 INFO:tasks.workunit.client.1.smithi026.stdout:3/662: dread d4/dc/d31/d48/d69/d78/fa2 [0,4194304] 0 2024-03-16T10:45:25.565 INFO:tasks.workunit.client.1.smithi026.stdout:8/590: write d5/d7/d2c/f32 [4876432,4939] 0 2024-03-16T10:45:25.568 INFO:tasks.workunit.client.1.smithi026.stdout:2/617: mkdir d2/d29/d85/dc6/dd5 0 2024-03-16T10:45:25.569 INFO:tasks.workunit.client.1.smithi026.stdout:4/651: truncate d1/db/de/d4c/d59/d6a/f9b 4593474 0 2024-03-16T10:45:25.570 INFO:tasks.workunit.client.1.smithi026.stdout:4/652: write d1/f52 [3136733,577] 0 2024-03-16T10:45:25.571 INFO:tasks.workunit.client.1.smithi026.stdout:3/663: truncate d4/dc/d73/d83/d7c/dd7/d62/f85 138690 0 2024-03-16T10:45:25.573 INFO:tasks.workunit.client.1.smithi026.stdout:0/682: rename d1/d12/d2f/d48/d55/d60/d9b/f59 to d1/d12/d2f/daa/dd1/d1a/d2b/fe9 0 2024-03-16T10:45:25.576 INFO:tasks.workunit.client.1.smithi026.stdout:6/517: creat d0/d41/d71/fb1 x:0 0 0 2024-03-16T10:45:25.577 INFO:tasks.workunit.client.1.smithi026.stdout:6/518: dread - d0/d41/d4f/f96 zero size 2024-03-16T10:45:25.580 INFO:tasks.workunit.client.1.smithi026.stdout:1/587: rmdir d0/d17/d1f/d63/d8d 0 2024-03-16T10:45:25.583 INFO:tasks.workunit.client.1.smithi026.stdout:2/618: readlink d2/d29/d17/d1a/d2b/d4b/ld0 0 2024-03-16T10:45:25.588 INFO:tasks.workunit.client.1.smithi026.stdout:3/664: dread d4/dc/d73/d83/d7c/f5b [0,4194304] 0 2024-03-16T10:45:25.590 INFO:tasks.workunit.client.1.smithi026.stdout:6/519: write d0/f5f [694551,67014] 0 2024-03-16T10:45:25.590 INFO:tasks.workunit.client.1.smithi026.stdout:7/644: mkdir d6/d35/d3a/d4e/dcc/d3f/dc3/dcf 0 2024-03-16T10:45:25.590 INFO:tasks.workunit.client.1.smithi026.stdout:3/665: chown d4/dc/d31/d48/d69/d78/fa2 138532883 1 2024-03-16T10:45:25.598 INFO:tasks.workunit.client.1.smithi026.stdout:1/588: creat d0/fb3 x:0 0 0 2024-03-16T10:45:25.603 INFO:tasks.workunit.client.1.smithi026.stdout:9/673: creat d0/da/fe1 x:0 0 0 2024-03-16T10:45:25.603 INFO:tasks.workunit.client.1.smithi026.stdout:9/674: chown d0/d5f/fb6 775 1 2024-03-16T10:45:25.604 INFO:tasks.workunit.client.1.smithi026.stdout:2/619: creat d2/d29/d17/d1a/d2b/d4e/d6f/fd6 x:0 0 0 2024-03-16T10:45:25.605 INFO:tasks.workunit.client.1.smithi026.stdout:9/675: write d0/dd7/fd1 [965237,81295] 0 2024-03-16T10:45:25.606 INFO:tasks.workunit.client.1.smithi026.stdout:2/620: chown d2/d29/d17/d38/d64/c95 21175752 1 2024-03-16T10:45:25.613 INFO:tasks.workunit.client.1.smithi026.stdout:0/683: getdents d1/d12/d2f/daa/dac 0 2024-03-16T10:45:25.613 INFO:tasks.workunit.client.1.smithi026.stdout:2/621: dwrite d2/d29/d17/d1a/d2b/d2e/d60/fcd [0,4194304] 0 2024-03-16T10:45:25.615 INFO:tasks.workunit.client.1.smithi026.stdout:6/520: dread - d0/d41/d4f/f69 zero size 2024-03-16T10:45:25.618 INFO:tasks.workunit.client.1.smithi026.stdout:3/666: creat d4/dc/d31/dbe/fe8 x:0 0 0 2024-03-16T10:45:25.619 INFO:tasks.workunit.client.1.smithi026.stdout:3/667: chown d4/dc/dd/d3d/d94 584394 1 2024-03-16T10:45:25.623 INFO:tasks.workunit.client.1.smithi026.stdout:9/676: creat d0/d5f/da2/fe2 x:0 0 0 2024-03-16T10:45:25.625 INFO:tasks.workunit.client.1.smithi026.stdout:2/622: fdatasync d2/d29/d17/d1a/d2b/d4b/f5b 0 2024-03-16T10:45:25.629 INFO:tasks.workunit.client.1.smithi026.stdout:7/645: link d6/d1e/f90 d6/d35/d3a/d4e/dcc/d3f/d54/fd0 0 2024-03-16T10:45:25.631 INFO:tasks.workunit.client.1.smithi026.stdout:8/591: getdents d5/d7/d2c/d36/d59/d93/da3 0 2024-03-16T10:45:25.635 INFO:tasks.workunit.client.1.smithi026.stdout:0/684: mkdir d1/d12/d2f/d48/d55/d60/de0/dea 0 2024-03-16T10:45:25.637 INFO:tasks.workunit.client.1.smithi026.stdout:4/653: truncate d1/db/f26 1681192 0 2024-03-16T10:45:25.638 INFO:tasks.workunit.client.1.smithi026.stdout:2/623: truncate d2/d29/d3b/f47 1891082 0 2024-03-16T10:45:25.644 INFO:tasks.workunit.client.1.smithi026.stdout:7/646: creat d6/d2f/dad/d7b/fd1 x:0 0 0 2024-03-16T10:45:25.645 INFO:tasks.workunit.client.1.smithi026.stdout:9/677: dread d0/da/f75 [0,4194304] 0 2024-03-16T10:45:25.646 INFO:tasks.workunit.client.1.smithi026.stdout:9/678: dread - d0/d5f/da2/fe2 zero size 2024-03-16T10:45:25.646 INFO:tasks.workunit.client.1.smithi026.stdout:8/592: fsync d5/d23/d4e/d69/d76/f87 0 2024-03-16T10:45:25.648 INFO:tasks.workunit.client.1.smithi026.stdout:9/679: read d0/da/f3b [886108,38934] 0 2024-03-16T10:45:25.651 INFO:tasks.workunit.client.1.smithi026.stdout:8/593: dread d5/dc/d40/f77 [0,4194304] 0 2024-03-16T10:45:25.651 INFO:tasks.workunit.client.1.smithi026.stdout:0/685: creat d1/d12/d2f/d48/d55/d60/d9b/d57/feb x:0 0 0 2024-03-16T10:45:25.658 INFO:tasks.workunit.client.1.smithi026.stdout:0/686: dwrite d1/d12/d2f/daa/dd1/fe6 [0,4194304] 0 2024-03-16T10:45:25.666 INFO:tasks.workunit.client.1.smithi026.stdout:7/647: dwrite d6/d1e/f90 [0,4194304] 0 2024-03-16T10:45:25.669 INFO:tasks.workunit.client.1.smithi026.stdout:9/680: symlink d0/da/d4d/le3 0 2024-03-16T10:45:25.670 INFO:tasks.workunit.client.1.smithi026.stdout:9/681: truncate d0/da/fe1 140112 0 2024-03-16T10:45:25.673 INFO:tasks.workunit.client.1.smithi026.stdout:4/654: mkdir d1/dcd 0 2024-03-16T10:45:25.675 INFO:tasks.workunit.client.1.smithi026.stdout:6/521: link d0/d10/f27 d0/d28/fb2 0 2024-03-16T10:45:25.680 INFO:tasks.workunit.client.1.smithi026.stdout:3/668: getdents d4/dc/d73/d83/d57 0 2024-03-16T10:45:25.682 INFO:tasks.workunit.client.1.smithi026.stdout:0/687: rmdir d1/d12/d2f/d48/d55/d60/d9b/d51/d5f 39 2024-03-16T10:45:25.694 INFO:tasks.workunit.client.1.smithi026.stdout:2/624: rename d2/d29/d17/d1a/d2b/c16 to d2/d29/d17/d1a/cd7 0 2024-03-16T10:45:25.695 INFO:tasks.workunit.client.1.smithi026.stdout:2/625: write d2/d29/d17/d1a/d2b/f1f [6017150,128041] 0 2024-03-16T10:45:25.697 INFO:tasks.workunit.client.1.smithi026.stdout:2/626: chown d2/d29/d17/d1a/d2b/d4b/f5c 0 1 2024-03-16T10:45:25.704 INFO:tasks.workunit.client.1.smithi026.stdout:7/648: symlink d6/d35/d3a/d4e/dcc/d3f/ld2 0 2024-03-16T10:45:25.704 INFO:tasks.workunit.client.1.smithi026.stdout:1/589: write d0/d17/d1f/d63/d6a/d7c/f6c [5227492,25435] 0 2024-03-16T10:45:25.707 INFO:tasks.workunit.client.1.smithi026.stdout:4/655: symlink d1/d85/da2/da8/lce 0 2024-03-16T10:45:25.719 INFO:tasks.workunit.client.1.smithi026.stdout:0/688: read - d1/d12/d2f/d48/d55/d60/d9b/d51/d5f/f8a zero size 2024-03-16T10:45:25.720 INFO:tasks.workunit.client.1.smithi026.stdout:0/689: dread - d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/fd8 zero size 2024-03-16T10:45:25.722 INFO:tasks.workunit.client.1.smithi026.stdout:7/649: rmdir d6/de/d16 39 2024-03-16T10:45:25.723 INFO:tasks.workunit.client.1.smithi026.stdout:0/690: dread d1/d12/d2f/daa/dd1/d1a/f94 [0,4194304] 0 2024-03-16T10:45:25.728 INFO:tasks.workunit.client.1.smithi026.stdout:1/590: creat d0/d17/d1f/d63/d6a/d7c/d3b/d64/fb4 x:0 0 0 2024-03-16T10:45:25.729 INFO:tasks.workunit.client.1.smithi026.stdout:4/656: symlink d1/db/de/d4c/d6b/lcf 0 2024-03-16T10:45:25.729 INFO:tasks.workunit.client.1.smithi026.stdout:1/591: chown d0/d17/d1f/d20/c45 3820038 1 2024-03-16T10:45:25.733 INFO:tasks.workunit.client.1.smithi026.stdout:8/594: rename d5/d7/f49 to d5/d23/fac 0 2024-03-16T10:45:25.736 INFO:tasks.workunit.client.1.smithi026.stdout:8/595: read d5/d7/d2c/d36/d59/f78 [555632,125970] 0 2024-03-16T10:45:25.739 INFO:tasks.workunit.client.1.smithi026.stdout:0/691: creat d1/d12/d2f/d48/d55/d60/d9b/d57/fec x:0 0 0 2024-03-16T10:45:25.745 INFO:tasks.workunit.client.1.smithi026.stdout:0/692: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/f5e [0,4194304] 0 2024-03-16T10:45:25.749 INFO:tasks.workunit.client.1.smithi026.stdout:8/596: mknod d5/d7/d2c/d36/d59/d93/da3/cad 0 2024-03-16T10:45:25.755 INFO:tasks.workunit.client.1.smithi026.stdout:4/657: getdents d1/d85/da2/dca 0 2024-03-16T10:45:25.755 INFO:tasks.workunit.client.1.smithi026.stdout:4/658: chown d1/db/f42 400061 1 2024-03-16T10:45:25.758 INFO:tasks.workunit.client.1.smithi026.stdout:1/592: creat d0/d17/d4d/daa/fb5 x:0 0 0 2024-03-16T10:45:25.760 INFO:tasks.workunit.client.1.smithi026.stdout:0/693: creat d1/d12/d2f/d48/d55/d60/d9b/d11/fed x:0 0 0 2024-03-16T10:45:25.763 INFO:tasks.workunit.client.1.smithi026.stdout:9/682: dwrite d0/da/fc9 [4194304,4194304] 0 2024-03-16T10:45:25.766 INFO:tasks.workunit.client.1.smithi026.stdout:6/522: write d0/d10/f23 [4500949,85234] 0 2024-03-16T10:45:25.771 INFO:tasks.workunit.client.1.smithi026.stdout:8/597: dwrite d5/d28/d90/f91 [0,4194304] 0 2024-03-16T10:45:25.773 INFO:tasks.workunit.client.1.smithi026.stdout:3/669: write d4/dc/f16 [497566,54331] 0 2024-03-16T10:45:25.773 INFO:tasks.workunit.client.1.smithi026.stdout:4/659: readlink d1/db/de/d4c/d6f/l72 0 2024-03-16T10:45:25.774 INFO:tasks.workunit.client.1.smithi026.stdout:8/598: fsync d5/dc/d3f/f20 0 2024-03-16T10:45:25.776 INFO:tasks.workunit.client.1.smithi026.stdout:0/694: dread d1/d12/d2f/d48/d55/d60/d9b/d57/f83 [0,4194304] 0 2024-03-16T10:45:25.776 INFO:tasks.workunit.client.1.smithi026.stdout:4/660: fdatasync d1/db/de/d4c/d6b/d7a/fba 0 2024-03-16T10:45:25.776 INFO:tasks.workunit.client.1.smithi026.stdout:3/670: fsync d4/dc/d73/d83/d7c/dd7/d62/f85 0 2024-03-16T10:45:25.781 INFO:tasks.workunit.client.1.smithi026.stdout:0/695: fsync d1/d12/d2f/daa/dac/fb5 0 2024-03-16T10:45:25.784 INFO:tasks.workunit.client.1.smithi026.stdout:5/606: sync 2024-03-16T10:45:25.784 INFO:tasks.workunit.client.1.smithi026.stdout:7/650: getdents d6/d35/d3a/d59 0 2024-03-16T10:45:25.786 INFO:tasks.workunit.client.1.smithi026.stdout:8/599: dwrite d5/d23/f68 [0,4194304] 0 2024-03-16T10:45:25.794 INFO:tasks.workunit.client.1.smithi026.stdout:1/593: rename d0/d17/d1f/d63/d6a/d7c/f6c to d0/d8b/fb6 0 2024-03-16T10:45:25.797 INFO:tasks.workunit.client.1.smithi026.stdout:2/627: dwrite d2/d29/d17/d1a/d2b/d2e/fbf [0,4194304] 0 2024-03-16T10:45:25.799 INFO:tasks.workunit.client.1.smithi026.stdout:9/683: mknod d0/d5f/d7f/ce4 0 2024-03-16T10:45:25.801 INFO:tasks.workunit.client.1.smithi026.stdout:8/600: mknod d5/d23/d4e/d69/cae 0 2024-03-16T10:45:25.802 INFO:tasks.workunit.client.1.smithi026.stdout:7/651: mknod d6/d35/d3a/d4e/dcc/d3f/d65/cd3 0 2024-03-16T10:45:25.806 INFO:tasks.workunit.client.1.smithi026.stdout:6/523: dread d0/f89 [0,4194304] 0 2024-03-16T10:45:25.807 INFO:tasks.workunit.client.1.smithi026.stdout:6/524: truncate d0/f35 773093 0 2024-03-16T10:45:25.811 INFO:tasks.workunit.client.1.smithi026.stdout:5/607: mknod d1/db/cd6 0 2024-03-16T10:45:25.812 INFO:tasks.workunit.client.1.smithi026.stdout:5/608: dread - d1/d1c/d2a/dab/dc5/fba zero size 2024-03-16T10:45:25.815 INFO:tasks.workunit.client.1.smithi026.stdout:2/628: rename d2/d29/d51/d55/c7a to d2/d29/d17/d38/cd8 0 2024-03-16T10:45:25.815 INFO:tasks.workunit.client.1.smithi026.stdout:3/671: write d4/dc/dd/fcb [3430728,112125] 0 2024-03-16T10:45:25.816 INFO:tasks.workunit.client.1.smithi026.stdout:2/629: chown d2/d29/da4/lc8 4833506 1 2024-03-16T10:45:25.817 INFO:tasks.workunit.client.1.smithi026.stdout:9/684: symlink d0/da/d16/le5 0 2024-03-16T10:45:25.820 INFO:tasks.workunit.client.1.smithi026.stdout:3/672: write d4/dc/d73/d83/d7c/dd7/d62/d4f/f70 [7564166,104873] 0 2024-03-16T10:45:25.821 INFO:tasks.workunit.client.1.smithi026.stdout:9/685: write d0/d5f/d7f/d8c/fbd [3511325,41844] 0 2024-03-16T10:45:25.825 INFO:tasks.workunit.client.1.smithi026.stdout:7/652: creat d6/d35/d3a/d59/fd4 x:0 0 0 2024-03-16T10:45:25.826 INFO:tasks.workunit.client.1.smithi026.stdout:4/661: dwrite d1/db/de/d6e/fb6 [0,4194304] 0 2024-03-16T10:45:25.827 INFO:tasks.workunit.client.1.smithi026.stdout:6/525: dread d0/d41/d4f/f77 [4194304,4194304] 0 2024-03-16T10:45:25.830 INFO:tasks.workunit.client.1.smithi026.stdout:5/609: unlink d1/f48 0 2024-03-16T10:45:25.831 INFO:tasks.workunit.client.1.smithi026.stdout:5/610: chown d1/d6e/lc3 1 1 2024-03-16T10:45:25.834 INFO:tasks.workunit.client.1.smithi026.stdout:9/686: dread d0/da/d13/f55 [0,4194304] 0 2024-03-16T10:45:25.838 INFO:tasks.workunit.client.1.smithi026.stdout:0/696: link d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/cc8 d1/d12/d2f/d48/d55/d60/d9b/d11/cee 0 2024-03-16T10:45:25.839 INFO:tasks.workunit.client.1.smithi026.stdout:7/653: stat d6/d35/d3a/d4e/dcc/d3f/d65/d68/c75 0 2024-03-16T10:45:25.846 INFO:tasks.workunit.client.1.smithi026.stdout:7/654: dread d6/d35/d3a/d59/d9e/fc2 [0,4194304] 0 2024-03-16T10:45:25.849 INFO:tasks.workunit.client.1.smithi026.stdout:5/611: creat d1/d6e/fd7 x:0 0 0 2024-03-16T10:45:25.850 INFO:tasks.workunit.client.1.smithi026.stdout:5/612: chown d1/d1c/d2a/dab/dc5/d69/d51/f9b 6877862 1 2024-03-16T10:45:25.851 INFO:tasks.workunit.client.1.smithi026.stdout:5/613: readlink d1/l6 0 2024-03-16T10:45:25.854 INFO:tasks.workunit.client.1.smithi026.stdout:8/601: creat d5/dc/faf x:0 0 0 2024-03-16T10:45:25.855 INFO:tasks.workunit.client.1.smithi026.stdout:3/673: mkdir d4/dc/dd/d47/d53/db1/ddd/de9 0 2024-03-16T10:45:25.859 INFO:tasks.workunit.client.1.smithi026.stdout:8/602: dwrite d5/dc/d3f/f20 [0,4194304] 0 2024-03-16T10:45:25.860 INFO:tasks.workunit.client.1.smithi026.stdout:8/603: chown d5/dc/d40/d53 2751 1 2024-03-16T10:45:25.860 INFO:tasks.workunit.client.1.smithi026.stdout:6/526: fdatasync d0/f2c 0 2024-03-16T10:45:25.865 INFO:tasks.workunit.client.1.smithi026.stdout:7/655: creat d6/d35/d3a/d4e/d6c/fd5 x:0 0 0 2024-03-16T10:45:25.866 INFO:tasks.workunit.client.1.smithi026.stdout:2/630: write d2/d29/d17/d38/f81 [338289,114716] 0 2024-03-16T10:45:25.871 INFO:tasks.workunit.client.1.smithi026.stdout:9/687: mknod d0/da/d16/d17/ce6 0 2024-03-16T10:45:25.873 INFO:tasks.workunit.client.1.smithi026.stdout:3/674: fdatasync d4/dc/f20 0 2024-03-16T10:45:25.875 INFO:tasks.workunit.client.1.smithi026.stdout:2/631: dwrite d2/d29/d17/d1a/d2b/d2e/d60/fcd [0,4194304] 0 2024-03-16T10:45:25.881 INFO:tasks.workunit.client.1.smithi026.stdout:2/632: truncate d2/d29/d17/f23 4959689 0 2024-03-16T10:45:25.882 INFO:tasks.workunit.client.1.smithi026.stdout:9/688: dread d0/da/d13/f31 [0,4194304] 0 2024-03-16T10:45:25.883 INFO:tasks.workunit.client.1.smithi026.stdout:8/604: mkdir d5/d7/d2c/d36/d59/d93/db0 0 2024-03-16T10:45:25.884 INFO:tasks.workunit.client.1.smithi026.stdout:4/662: dwrite d1/db/de/d4c/d79/daa/f95 [0,4194304] 0 2024-03-16T10:45:25.885 INFO:tasks.workunit.client.1.smithi026.stdout:2/633: write d2/d29/d51/d55/d6d/fb9 [1846727,104824] 0 2024-03-16T10:45:25.885 INFO:tasks.workunit.client.1.smithi026.stdout:6/527: dread d0/f3 [0,4194304] 0 2024-03-16T10:45:25.887 INFO:tasks.workunit.client.1.smithi026.stdout:5/614: creat d1/db/d27/dd1/fd8 x:0 0 0 2024-03-16T10:45:25.890 INFO:tasks.workunit.client.1.smithi026.stdout:7/656: fdatasync d6/d35/d3a/d4e/dcc/d3f/d9d/f97 0 2024-03-16T10:45:25.892 INFO:tasks.workunit.client.1.smithi026.stdout:0/697: creat d1/d12/d2f/daa/dd1/d1a/d2b/d4f/fef x:0 0 0 2024-03-16T10:45:25.893 INFO:tasks.workunit.client.1.smithi026.stdout:3/675: write d4/dc/f20 [1404151,114541] 0 2024-03-16T10:45:25.894 INFO:tasks.workunit.client.1.smithi026.stdout:7/657: fdatasync d6/de/faa 0 2024-03-16T10:45:25.897 INFO:tasks.workunit.client.1.smithi026.stdout:9/689: read d0/da/d13/f55 [505337,110277] 0 2024-03-16T10:45:25.897 INFO:tasks.workunit.client.1.smithi026.stdout:8/605: unlink d5/d7/d2c/f70 0 2024-03-16T10:45:25.902 INFO:tasks.workunit.client.1.smithi026.stdout:4/663: dwrite d1/db/de/d4c/d81/dc7/fcc [0,4194304] 0 2024-03-16T10:45:25.903 INFO:tasks.workunit.client.1.smithi026.stdout:7/658: fsync d6/d35/d3a/d4e/dcc/d3f/d9d/f97 0 2024-03-16T10:45:25.905 INFO:tasks.workunit.client.1.smithi026.stdout:2/634: dwrite d2/d29/d17/d1a/d2b/f91 [0,4194304] 0 2024-03-16T10:45:25.915 INFO:tasks.workunit.client.1.smithi026.stdout:3/676: dwrite d4/dc/d73/d83/d57/f9b [8388608,4194304] 0 2024-03-16T10:45:25.921 INFO:tasks.workunit.client.1.smithi026.stdout:8/606: symlink d5/d28/lb1 0 2024-03-16T10:45:25.924 INFO:tasks.workunit.client.1.smithi026.stdout:2/635: dwrite d2/d29/d17/d38/d64/fa2 [0,4194304] 0 2024-03-16T10:45:25.925 INFO:tasks.workunit.client.1.smithi026.stdout:9/690: write d0/da/d13/f31 [2639185,54860] 0 2024-03-16T10:45:25.926 INFO:tasks.workunit.client.1.smithi026.stdout:5/615: mknod d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/dc6/cd9 0 2024-03-16T10:45:25.927 INFO:tasks.workunit.client.1.smithi026.stdout:6/528: dwrite d0/d28/d49/f52 [0,4194304] 0 2024-03-16T10:45:25.928 INFO:tasks.workunit.client.1.smithi026.stdout:7/659: stat d6/de/d16/f49 0 2024-03-16T10:45:25.933 INFO:tasks.workunit.client.1.smithi026.stdout:4/664: creat d1/db/de/d4c/d79/daa/fd0 x:0 0 0 2024-03-16T10:45:25.933 INFO:tasks.workunit.client.1.smithi026.stdout:8/607: truncate d5/d7/d2c/d36/d59/d93/fab 541666 0 2024-03-16T10:45:25.934 INFO:tasks.workunit.client.1.smithi026.stdout:4/665: chown d1/db/f64 33 1 2024-03-16T10:45:25.940 INFO:tasks.workunit.client.1.smithi026.stdout:8/608: dwrite d5/d23/d4e/d69/d76/f9e [0,4194304] 0 2024-03-16T10:45:25.946 INFO:tasks.workunit.client.1.smithi026.stdout:3/677: write d4/d89/d97/f3a [1795731,96539] 0 2024-03-16T10:45:25.948 INFO:tasks.workunit.client.1.smithi026.stdout:6/529: mkdir d0/d41/d4f/db3 0 2024-03-16T10:45:25.950 INFO:tasks.workunit.client.1.smithi026.stdout:9/691: creat d0/da/d16/d5d/d72/d62/fe7 x:0 0 0 2024-03-16T10:45:25.951 INFO:tasks.workunit.client.1.smithi026.stdout:7/660: chown d6/de/d16/d88/fc1 5325857 1 2024-03-16T10:45:25.952 INFO:tasks.workunit.client.1.smithi026.stdout:9/692: write d0/d5f/d7f/d8c/fbd [4317395,85647] 0 2024-03-16T10:45:25.953 INFO:tasks.workunit.client.1.smithi026.stdout:5/616: rename d1/d1c/d2a/d55/d91/ccb to d1/d6e/d9f/cda 0 2024-03-16T10:45:25.955 INFO:tasks.workunit.client.1.smithi026.stdout:0/698: creat d1/d12/d2f/d48/d55/d60/d9b/ff0 x:0 0 0 2024-03-16T10:45:25.962 INFO:tasks.workunit.client.1.smithi026.stdout:3/678: creat d4/dc/d31/d48/d69/d78/fea x:0 0 0 2024-03-16T10:45:25.962 INFO:tasks.workunit.client.1.smithi026.stdout:8/609: dread d5/dc/d89/f8a [0,4194304] 0 2024-03-16T10:45:25.963 INFO:tasks.workunit.client.1.smithi026.stdout:6/530: creat d0/d10/d79/fb4 x:0 0 0 2024-03-16T10:45:25.965 INFO:tasks.workunit.client.1.smithi026.stdout:9/693: symlink d0/d63/le8 0 2024-03-16T10:45:25.967 INFO:tasks.workunit.client.1.smithi026.stdout:4/666: creat d1/dcd/fd1 x:0 0 0 2024-03-16T10:45:25.967 INFO:tasks.workunit.client.1.smithi026.stdout:4/667: chown d1/db/f74 1217924 1 2024-03-16T10:45:25.968 INFO:tasks.workunit.client.1.smithi026.stdout:5/617: read d1/f71 [3815066,120466] 0 2024-03-16T10:45:25.969 INFO:tasks.workunit.client.1.smithi026.stdout:0/699: fdatasync d1/d12/d2f/d48/d55/d60/d9b/d57/f83 0 2024-03-16T10:45:25.971 INFO:tasks.workunit.client.1.smithi026.stdout:2/636: creat d2/d29/fd9 x:0 0 0 2024-03-16T10:45:25.971 INFO:tasks.workunit.client.1.smithi026.stdout:0/700: chown d1/d12/d2f/daa/dd1/d1a/d2b/d4f/l6e 14830 1 2024-03-16T10:45:25.973 INFO:tasks.workunit.client.1.smithi026.stdout:3/679: mknod d4/dc/dd/d3d/ceb 0 2024-03-16T10:45:25.975 INFO:tasks.workunit.client.1.smithi026.stdout:4/668: dwrite d1/db/de/d4c/fa1 [0,4194304] 0 2024-03-16T10:45:25.976 INFO:tasks.workunit.client.1.smithi026.stdout:8/610: mkdir d5/d7/d7b/db2 0 2024-03-16T10:45:25.979 INFO:tasks.workunit.client.1.smithi026.stdout:4/669: chown d1/db/de/d4c/d6b/fad 179350148 1 2024-03-16T10:45:25.980 INFO:tasks.workunit.client.1.smithi026.stdout:9/694: unlink d0/d63/d6b/cda 0 2024-03-16T10:45:25.981 INFO:tasks.workunit.client.1.smithi026.stdout:2/637: dread d2/d29/d17/d1a/d2b/d2e/d60/fcd [0,4194304] 0 2024-03-16T10:45:25.981 INFO:tasks.workunit.client.1.smithi026.stdout:7/661: getdents d6/d35/d3a/d4e/dcc/d73/dbf 0 2024-03-16T10:45:25.989 INFO:tasks.workunit.client.1.smithi026.stdout:5/618: symlink d1/ldb 0 2024-03-16T10:45:25.990 INFO:tasks.workunit.client.1.smithi026.stdout:7/662: dread d6/d1e/f29 [0,4194304] 0 2024-03-16T10:45:25.995 INFO:tasks.workunit.client.1.smithi026.stdout:6/531: dwrite d0/d10/fd [0,4194304] 0 2024-03-16T10:45:25.998 INFO:tasks.workunit.client.1.smithi026.stdout:4/670: mknod d1/d85/da2/da8/cd2 0 2024-03-16T10:45:26.001 INFO:tasks.workunit.client.1.smithi026.stdout:9/695: symlink d0/da/d14/da7/le9 0 2024-03-16T10:45:26.004 INFO:tasks.workunit.client.1.smithi026.stdout:0/701: dwrite d1/d12/d2f/daa/dd1/d61/f62 [4194304,4194304] 0 2024-03-16T10:45:26.004 INFO:tasks.workunit.client.1.smithi026.stdout:9/696: chown d0/d63/d6b/fa5 3154522 1 2024-03-16T10:45:26.012 INFO:tasks.workunit.client.1.smithi026.stdout:3/680: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/fec x:0 0 0 2024-03-16T10:45:26.018 INFO:tasks.workunit.client.1.smithi026.stdout:2/638: truncate d2/d29/d17/d38/f7d 574583 0 2024-03-16T10:45:26.020 INFO:tasks.workunit.client.1.smithi026.stdout:5/619: write d1/d1c/d2a/d55/db1/fbc [257460,26052] 0 2024-03-16T10:45:26.026 INFO:tasks.workunit.client.1.smithi026.stdout:9/697: write d0/da/d16/d5d/d72/d62/f65 [1575776,8072] 0 2024-03-16T10:45:26.028 INFO:tasks.workunit.client.1.smithi026.stdout:3/681: unlink d4/dc/dd/d47/d53/cc8 0 2024-03-16T10:45:26.035 INFO:tasks.workunit.client.1.smithi026.stdout:4/671: mknod d1/db/de/d4c/d6b/d89/da5/cd3 0 2024-03-16T10:45:26.036 INFO:tasks.workunit.client.1.smithi026.stdout:2/639: creat d2/d29/d17/d1a/d2b/d4b/fda x:0 0 0 2024-03-16T10:45:26.037 INFO:tasks.workunit.client.1.smithi026.stdout:9/698: read d0/fae [3025444,109872] 0 2024-03-16T10:45:26.037 INFO:tasks.workunit.client.1.smithi026.stdout:5/620: dwrite d1/db/d1f/d61/d65/f9d [0,4194304] 0 2024-03-16T10:45:26.040 INFO:tasks.workunit.client.1.smithi026.stdout:8/611: getdents d5/d23/d4e/d69/d76 0 2024-03-16T10:45:26.042 INFO:tasks.workunit.client.1.smithi026.stdout:6/532: dwrite d0/d28/d2d/f76 [8388608,4194304] 0 2024-03-16T10:45:26.042 INFO:tasks.workunit.client.1.smithi026.stdout:4/672: chown d1/db/de/l5e 30 1 2024-03-16T10:45:26.042 INFO:tasks.workunit.client.1.smithi026.stdout:5/621: dread - d1/d1c/d2a/dab/dc5/fba zero size 2024-03-16T10:45:26.147 INFO:tasks.workunit.client.1.smithi026.stdout:0/702: write d1/d12/d2f/daa/dd1/d1a/d2b/fe9 [2517950,34104] 0 2024-03-16T10:45:26.147 INFO:tasks.workunit.client.1.smithi026.stdout:7/663: getdents d6/d35/d3a/d4e/dcc/d73 0 2024-03-16T10:45:26.148 INFO:tasks.workunit.client.1.smithi026.stdout:3/682: mknod d4/dc/dd/d47/d53/ced 0 2024-03-16T10:45:26.148 INFO:tasks.workunit.client.1.smithi026.stdout:9/699: mkdir d0/da/d16/dea 0 2024-03-16T10:45:26.148 INFO:tasks.workunit.client.1.smithi026.stdout:5/622: fdatasync d1/d1c/d68/f74 0 2024-03-16T10:45:26.148 INFO:tasks.workunit.client.1.smithi026.stdout:8/612: symlink d5/dc/d89/lb3 0 2024-03-16T10:45:26.149 INFO:tasks.workunit.client.1.smithi026.stdout:6/533: symlink d0/d10/d79/d91/lb5 0 2024-03-16T10:45:26.151 INFO:tasks.workunit.client.1.smithi026.stdout:8/613: dread - d5/d5d/f9d zero size 2024-03-16T10:45:26.153 INFO:tasks.workunit.client.1.smithi026.stdout:3/683: creat d4/d89/fee x:0 0 0 2024-03-16T10:45:26.155 INFO:tasks.workunit.client.1.smithi026.stdout:0/703: mkdir d1/d12/d2f/d48/d55/d60/d9b/dcf/df1 0 2024-03-16T10:45:26.157 INFO:tasks.workunit.client.1.smithi026.stdout:5/623: creat d1/db/d1f/d61/fdc x:0 0 0 2024-03-16T10:45:26.157 INFO:tasks.workunit.client.1.smithi026.stdout:6/534: dread d0/d62/f84 [0,4194304] 0 2024-03-16T10:45:26.159 INFO:tasks.workunit.client.1.smithi026.stdout:7/664: rename d6/de/d16/d44/c47 to d6/d35/d3a/d4e/dcc/d3f/d9d/cd6 0 2024-03-16T10:45:26.160 INFO:tasks.workunit.client.1.smithi026.stdout:9/700: mkdir d0/da/d13/deb 0 2024-03-16T10:45:26.163 INFO:tasks.workunit.client.1.smithi026.stdout:5/624: dwrite d1/d1c/d68/fa3 [0,4194304] 0 2024-03-16T10:45:26.166 INFO:tasks.workunit.client.1.smithi026.stdout:3/684: dread d4/dc/f19 [0,4194304] 0 2024-03-16T10:45:26.170 INFO:tasks.workunit.client.1.smithi026.stdout:5/625: unlink d1/db/l37 0 2024-03-16T10:45:26.177 INFO:tasks.workunit.client.1.smithi026.stdout:3/685: creat d4/dc/dd/d80/fef x:0 0 0 2024-03-16T10:45:26.181 INFO:tasks.workunit.client.1.smithi026.stdout:8/614: getdents d5/d7 0 2024-03-16T10:45:26.182 INFO:tasks.workunit.client.1.smithi026.stdout:9/701: rename d0/da/d16/d17/d44/l95 to d0/lec 0 2024-03-16T10:45:26.183 INFO:tasks.workunit.client.1.smithi026.stdout:4/673: truncate d1/db/de/d4c/d79/daa/d44/d48/d8e/f93 2578551 0 2024-03-16T10:45:26.186 INFO:tasks.workunit.client.1.smithi026.stdout:3/686: symlink d4/dc/d31/d48/d69/d78/db3/lf0 0 2024-03-16T10:45:26.188 INFO:tasks.workunit.client.1.smithi026.stdout:7/665: rename d6/d35/d3a/d4e/dcc/d3f/d65/fb1 to d6/d1e/dbc/dc8/fd7 0 2024-03-16T10:45:26.189 INFO:tasks.workunit.client.1.smithi026.stdout:9/702: mkdir d0/d63/d6b/ded 0 2024-03-16T10:45:26.191 INFO:tasks.workunit.client.1.smithi026.stdout:5/626: creat d1/db/d1f/d61/d65/fdd x:0 0 0 2024-03-16T10:45:26.197 INFO:tasks.workunit.client.1.smithi026.stdout:9/703: dwrite d0/d63/d6b/fc2 [0,4194304] 0 2024-03-16T10:45:26.208 INFO:tasks.workunit.client.1.smithi026.stdout:3/687: write d4/dc/d31/d48/f9f [901745,35778] 0 2024-03-16T10:45:26.209 INFO:tasks.workunit.client.1.smithi026.stdout:4/674: getdents d1/d85/da2/da8 0 2024-03-16T10:45:26.209 INFO:tasks.workunit.client.1.smithi026.stdout:7/666: write d6/f15 [794531,81478] 0 2024-03-16T10:45:26.212 INFO:tasks.workunit.client.1.smithi026.stdout:5/627: rename d1/d1c/d2a/dab/lb5 to d1/db/lde 0 2024-03-16T10:45:26.214 INFO:tasks.workunit.client.1.smithi026.stdout:9/704: dread d0/f66 [0,4194304] 0 2024-03-16T10:45:26.215 INFO:tasks.workunit.client.1.smithi026.stdout:7/667: mkdir d6/d35/d3a/d4e/dcc/d3f/d54/dd8 0 2024-03-16T10:45:26.216 INFO:tasks.workunit.client.1.smithi026.stdout:3/688: truncate d4/dc/d31/d48/d69/d78/f87 1536111 0 2024-03-16T10:45:26.221 INFO:tasks.workunit.client.1.smithi026.stdout:7/668: rmdir d6/d35/d3a/d4e/dcc/d3f/d54 39 2024-03-16T10:45:26.224 INFO:tasks.workunit.client.1.smithi026.stdout:3/689: mknod d4/dc/d31/d48/d7f/cf1 0 2024-03-16T10:45:26.232 INFO:tasks.workunit.client.1.smithi026.stdout:5/628: creat d1/d1c/d2a/fdf x:0 0 0 2024-03-16T10:45:26.233 INFO:tasks.workunit.client.1.smithi026.stdout:3/690: creat d4/dc/d31/dbe/ff2 x:0 0 0 2024-03-16T10:45:26.234 INFO:tasks.workunit.client.1.smithi026.stdout:7/669: dwrite d6/d35/d3a/d4e/dcc/d3f/d54/f8b [0,4194304] 0 2024-03-16T10:45:26.236 INFO:tasks.workunit.client.1.smithi026.stdout:9/705: link d0/d63/le8 d0/d5f/d7f/d8c/lee 0 2024-03-16T10:45:26.237 INFO:tasks.workunit.client.1.smithi026.stdout:5/629: mknod d1/d1c/d2a/d55/db1/ce0 0 2024-03-16T10:45:26.239 INFO:tasks.workunit.client.1.smithi026.stdout:3/691: symlink d4/dc/dd/d3d/lf3 0 2024-03-16T10:45:26.243 INFO:tasks.workunit.client.1.smithi026.stdout:5/630: creat d1/d1c/d2a/dab/dc5/fe1 x:0 0 0 2024-03-16T10:45:26.244 INFO:tasks.workunit.client.1.smithi026.stdout:7/670: mknod d6/d35/d3a/d4e/dcc/d73/dbf/cd9 0 2024-03-16T10:45:26.245 INFO:tasks.workunit.client.1.smithi026.stdout:7/671: readlink d6/d35/d3a/d4e/db0/lc7 0 2024-03-16T10:45:26.247 INFO:tasks.workunit.client.1.smithi026.stdout:5/631: dread d1/d1c/d2a/d55/d99/f95 [0,4194304] 0 2024-03-16T10:45:26.250 INFO:tasks.workunit.client.1.smithi026.stdout:3/692: link d4/dc/dd/fd2 d4/dc/dd/d47/d53/db1/ddd/ff4 0 2024-03-16T10:45:26.252 INFO:tasks.workunit.client.1.smithi026.stdout:7/672: rename d6/d1e/fa9 to d6/d35/d3a/fda 0 2024-03-16T10:45:26.253 INFO:tasks.workunit.client.1.smithi026.stdout:5/632: dwrite d1/db/d1f/d61/f6d [0,4194304] 0 2024-03-16T10:45:26.261 INFO:tasks.workunit.client.1.smithi026.stdout:3/693: mknod d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/cf5 0 2024-03-16T10:45:26.263 INFO:tasks.workunit.client.1.smithi026.stdout:9/706: getdents d0/d63 0 2024-03-16T10:45:26.265 INFO:tasks.workunit.client.1.smithi026.stdout:7/673: mknod d6/d35/d3a/d4e/dcc/d73/dbf/cdb 0 2024-03-16T10:45:26.268 INFO:tasks.workunit.client.1.smithi026.stdout:3/694: symlink d4/dc/d73/d83/d7c/lf6 0 2024-03-16T10:45:26.269 INFO:tasks.workunit.client.1.smithi026.stdout:9/707: unlink d0/d63/f83 0 2024-03-16T10:45:26.271 INFO:tasks.workunit.client.1.smithi026.stdout:7/674: mknod d6/d1e/dbc/dc8/cdc 0 2024-03-16T10:45:26.273 INFO:tasks.workunit.client.1.smithi026.stdout:5/633: rename d1/db/d1f/c6a to d1/d1c/d2a/d8e/ce2 0 2024-03-16T10:45:26.275 INFO:tasks.workunit.client.1.smithi026.stdout:5/634: write d1/d1c/d2a/d55/d99/f96 [764102,67169] 0 2024-03-16T10:45:26.276 INFO:tasks.workunit.client.1.smithi026.stdout:9/708: dwrite d0/da/d16/d17/d44/fca [0,4194304] 0 2024-03-16T10:45:26.279 INFO:tasks.workunit.client.1.smithi026.stdout:3/695: creat d4/dc/dd/d47/d53/db1/ddd/ff7 x:0 0 0 2024-03-16T10:45:26.279 INFO:tasks.workunit.client.1.smithi026.stdout:7/675: mknod d6/d35/d3a/d4e/dcc/d3f/d65/d68/cdd 0 2024-03-16T10:45:26.284 INFO:tasks.workunit.client.1.smithi026.stdout:5/635: creat d1/d1c/d4c/fe3 x:0 0 0 2024-03-16T10:45:26.285 INFO:tasks.workunit.client.1.smithi026.stdout:9/709: dwrite d0/d5f/fb6 [0,4194304] 0 2024-03-16T10:45:26.289 INFO:tasks.workunit.client.1.smithi026.stdout:7/676: symlink d6/d35/d3a/lde 0 2024-03-16T10:45:26.290 INFO:tasks.workunit.client.1.smithi026.stdout:3/696: creat d4/dc/dd/d47/d53/ff8 x:0 0 0 2024-03-16T10:45:26.293 INFO:tasks.workunit.client.1.smithi026.stdout:5/636: creat d1/db/d1f/d61/d65/d82/fe4 x:0 0 0 2024-03-16T10:45:26.294 INFO:tasks.workunit.client.1.smithi026.stdout:9/710: write d0/d5f/da2/fa6 [843577,86844] 0 2024-03-16T10:45:26.296 INFO:tasks.workunit.client.1.smithi026.stdout:9/711: read d0/f4 [88602,101963] 0 2024-03-16T10:45:26.297 INFO:tasks.workunit.client.1.smithi026.stdout:9/712: readlink d0/da/d14/l8e 0 2024-03-16T10:45:26.300 INFO:tasks.workunit.client.1.smithi026.stdout:5/637: getdents d1/d1c/d2a/d8e 0 2024-03-16T10:45:26.311 INFO:tasks.workunit.client.1.smithi026.stdout:9/713: write d0/da/d13/f26 [5021613,104757] 0 2024-03-16T10:45:26.314 INFO:tasks.workunit.client.1.smithi026.stdout:9/714: mkdir d0/d63/d6b/def 0 2024-03-16T10:45:26.315 INFO:tasks.workunit.client.1.smithi026.stdout:5/638: dwrite d1/d1c/d2a/dab/dc5/d69/d51/d80/fa6 [0,4194304] 0 2024-03-16T10:45:26.315 INFO:tasks.workunit.client.1.smithi026.stdout:9/715: chown d0/da/d16/d17/lb5 1 1 2024-03-16T10:45:26.317 INFO:tasks.workunit.client.1.smithi026.stdout:5/639: truncate d1/db/d1f/f85 483543 0 2024-03-16T10:45:26.318 INFO:tasks.workunit.client.1.smithi026.stdout:9/716: creat d0/da/d16/d5d/d72/d62/ff0 x:0 0 0 2024-03-16T10:45:26.325 INFO:tasks.workunit.client.1.smithi026.stdout:9/717: dwrite d0/da/d16/d17/d44/fdf [0,4194304] 0 2024-03-16T10:45:26.329 INFO:tasks.workunit.client.1.smithi026.stdout:9/718: symlink d0/da/d16/d5d/d72/lf1 0 2024-03-16T10:45:26.329 INFO:tasks.workunit.client.1.smithi026.stdout:9/719: stat d0/da/d13/c33 0 2024-03-16T10:45:26.330 INFO:tasks.workunit.client.1.smithi026.stdout:9/720: stat d0/da/f3b 0 2024-03-16T10:45:26.334 INFO:tasks.workunit.client.1.smithi026.stdout:9/721: mknod d0/da/d16/dea/cf2 0 2024-03-16T10:45:26.334 INFO:tasks.workunit.client.1.smithi026.stdout:5/640: dwrite d1/db/d1f/f26 [0,4194304] 0 2024-03-16T10:45:26.341 INFO:tasks.workunit.client.1.smithi026.stdout:9/722: creat d0/da/d13/ff3 x:0 0 0 2024-03-16T10:45:26.343 INFO:tasks.workunit.client.1.smithi026.stdout:9/723: symlink d0/da/d16/d17/d44/lf4 0 2024-03-16T10:45:26.346 INFO:tasks.workunit.client.1.smithi026.stdout:9/724: truncate d0/da/d16/d17/d44/f2b 894928 0 2024-03-16T10:45:26.348 INFO:tasks.workunit.client.1.smithi026.stdout:9/725: mknod d0/dd7/cf5 0 2024-03-16T10:45:26.353 INFO:tasks.workunit.client.1.smithi026.stdout:9/726: getdents d0/da/d14/da7 0 2024-03-16T10:45:26.353 INFO:tasks.workunit.client.1.smithi026.stdout:1/594: sync 2024-03-16T10:45:26.355 INFO:tasks.workunit.client.1.smithi026.stdout:5/641: dwrite d1/d1c/d4c/d4d/fcc [0,4194304] 0 2024-03-16T10:45:26.357 INFO:tasks.workunit.client.1.smithi026.stdout:1/595: mkdir d0/d17/db7 0 2024-03-16T10:45:26.360 INFO:tasks.workunit.client.1.smithi026.stdout:9/727: creat d0/dd7/ff6 x:0 0 0 2024-03-16T10:45:26.362 INFO:tasks.workunit.client.1.smithi026.stdout:5/642: unlink d1/db/d1f/d61/d65/d82/fe4 0 2024-03-16T10:45:26.364 INFO:tasks.workunit.client.1.smithi026.stdout:1/596: rmdir d0/d17/d1f/d63/d6a/d7c/d2e/d54 39 2024-03-16T10:45:26.369 INFO:tasks.workunit.client.1.smithi026.stdout:5/643: unlink d1/d1c/d2a/d55/fc8 0 2024-03-16T10:45:26.373 INFO:tasks.workunit.client.1.smithi026.stdout:9/728: rename d0/cbb to d0/d5f/d7f/d8c/cf7 0 2024-03-16T10:45:26.375 INFO:tasks.workunit.client.1.smithi026.stdout:5/644: mknod d1/d1c/d2a/dab/dc5/ce5 0 2024-03-16T10:45:26.376 INFO:tasks.workunit.client.1.smithi026.stdout:9/729: symlink d0/da/d16/d5d/d72/lf8 0 2024-03-16T10:45:26.377 INFO:tasks.workunit.client.1.smithi026.stdout:5/645: write d1/d1c/d2a/dab/dc5/d69/d51/d80/fa6 [1533493,17204] 0 2024-03-16T10:45:26.380 INFO:tasks.workunit.client.1.smithi026.stdout:9/730: creat d0/d5f/d7f/ff9 x:0 0 0 2024-03-16T10:45:26.381 INFO:tasks.workunit.client.1.smithi026.stdout:1/597: dread d0/d17/f37 [0,4194304] 0 2024-03-16T10:45:26.381 INFO:tasks.workunit.client.1.smithi026.stdout:9/731: chown d0/da/d14/c1a 838902 1 2024-03-16T10:45:26.382 INFO:tasks.workunit.client.1.smithi026.stdout:1/598: read - d0/d17/d4d/daa/fb5 zero size 2024-03-16T10:45:26.386 INFO:tasks.workunit.client.1.smithi026.stdout:9/732: mkdir d0/da/d16/d17/db1/dfa 0 2024-03-16T10:45:26.386 INFO:tasks.workunit.client.1.smithi026.stdout:9/733: chown d0/d63/d6b/ded 34816341 1 2024-03-16T10:45:26.392 INFO:tasks.workunit.client.1.smithi026.stdout:5/646: link d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/dc6/cd9 d1/db/d1f/d61/ce6 0 2024-03-16T10:45:26.393 INFO:tasks.workunit.client.1.smithi026.stdout:5/647: write d1/d6e/d9f/fca [75614,76748] 0 2024-03-16T10:45:26.401 INFO:tasks.workunit.client.1.smithi026.stdout:9/734: getdents d0/da/d13 0 2024-03-16T10:45:26.404 INFO:tasks.workunit.client.1.smithi026.stdout:5/648: dread d1/db/d1f/d61/fa7 [0,4194304] 0 2024-03-16T10:45:26.405 INFO:tasks.workunit.client.1.smithi026.stdout:5/649: write d1/f5 [3747665,9813] 0 2024-03-16T10:45:26.413 INFO:tasks.workunit.client.1.smithi026.stdout:5/650: mknod d1/db/d1f/d61/d65/d82/dbb/dd2/ce7 0 2024-03-16T10:45:26.414 INFO:tasks.workunit.client.1.smithi026.stdout:1/599: dwrite d0/d17/f81 [0,4194304] 0 2024-03-16T10:45:26.420 INFO:tasks.workunit.client.1.smithi026.stdout:9/735: write d0/f3c [2401839,28282] 0 2024-03-16T10:45:26.423 INFO:tasks.workunit.client.1.smithi026.stdout:1/600: rename d0/d17/d1f/d20/l2c to d0/d17/d1f/d63/d58/lb8 0 2024-03-16T10:45:26.428 INFO:tasks.workunit.client.1.smithi026.stdout:1/601: unlink d0/d17/d1f/d63/l55 0 2024-03-16T10:45:26.429 INFO:tasks.workunit.client.1.smithi026.stdout:1/602: fdatasync d0/d17/d1f/d20/d65/f8c 0 2024-03-16T10:45:26.431 INFO:tasks.workunit.client.1.smithi026.stdout:9/736: truncate d0/f66 4306482 0 2024-03-16T10:45:26.434 INFO:tasks.workunit.client.1.smithi026.stdout:9/737: unlink d0/d63/c7e 0 2024-03-16T10:45:26.437 INFO:tasks.workunit.client.1.smithi026.stdout:9/738: chown d0/da/d16/d17/d44/f2b 612311 1 2024-03-16T10:45:26.437 INFO:tasks.workunit.client.1.smithi026.stdout:9/739: chown d0/f82 26 1 2024-03-16T10:45:26.440 INFO:tasks.workunit.client.1.smithi026.stdout:9/740: fsync d0/d63/d6b/f86 0 2024-03-16T10:45:26.446 INFO:tasks.workunit.client.1.smithi026.stdout:9/741: dread d0/da/d13/f69 [0,4194304] 0 2024-03-16T10:45:26.449 INFO:tasks.workunit.client.1.smithi026.stdout:9/742: creat d0/d63/ffb x:0 0 0 2024-03-16T10:45:26.451 INFO:tasks.workunit.client.1.smithi026.stdout:9/743: unlink d0/da/d14/f91 0 2024-03-16T10:45:26.452 INFO:tasks.workunit.client.1.smithi026.stdout:9/744: rmdir d0/da/d16/d5d/d72 39 2024-03-16T10:45:26.455 INFO:tasks.workunit.client.1.smithi026.stdout:9/745: mkdir d0/d5f/dfc 0 2024-03-16T10:45:26.457 INFO:tasks.workunit.client.1.smithi026.stdout:9/746: dread - d0/da/d4d/f71 zero size 2024-03-16T10:45:26.459 INFO:tasks.workunit.client.1.smithi026.stdout:9/747: mknod d0/d5f/d7f/d8c/cfd 0 2024-03-16T10:45:26.459 INFO:tasks.workunit.client.1.smithi026.stdout:9/748: stat d0/fdd 0 2024-03-16T10:45:26.612 INFO:tasks.workunit.client.1.smithi026.stdout:2/640: sync 2024-03-16T10:45:26.613 INFO:tasks.workunit.client.1.smithi026.stdout:2/641: chown d2/d29/d17/d38/c8a 4996 1 2024-03-16T10:45:26.613 INFO:tasks.workunit.client.1.smithi026.stdout:2/642: chown d2/d29/d17/d38/l93 754951 1 2024-03-16T10:45:26.645 INFO:tasks.workunit.client.1.smithi026.stdout:2/643: creat d2/d29/d17/d1a/d2b/d69/fdb x:0 0 0 2024-03-16T10:45:26.666 INFO:tasks.workunit.client.1.smithi026.stdout:2/644: fsync d2/d29/d17/d1a/d2b/d4b/f5a 0 2024-03-16T10:45:26.669 INFO:tasks.workunit.client.1.smithi026.stdout:2/645: mkdir d2/d29/d17/d1a/d2b/d2e/d60/d7f/ddc 0 2024-03-16T10:45:26.675 INFO:tasks.workunit.client.1.smithi026.stdout:2/646: creat d2/d29/d36/fdd x:0 0 0 2024-03-16T10:45:26.680 INFO:tasks.workunit.client.1.smithi026.stdout:2/647: dwrite d2/d29/d17/d1a/d2b/f1f [4194304,4194304] 0 2024-03-16T10:45:26.703 INFO:tasks.workunit.client.1.smithi026.stdout:2/648: write d2/d29/d17/d1a/d2b/f80 [254995,52948] 0 2024-03-16T10:45:26.708 INFO:tasks.workunit.client.1.smithi026.stdout:2/649: dread d2/d29/d17/d1a/d2b/d2e/fbf [0,4194304] 0 2024-03-16T10:45:26.723 INFO:tasks.workunit.client.1.smithi026.stdout:2/650: fsync d2/d29/d17/d1a/d2b/d69/fb6 0 2024-03-16T10:45:26.735 INFO:tasks.workunit.client.1.smithi026.stdout:2/651: unlink d2/d29/d17/d1a/d2b/d2e/f45 0 2024-03-16T10:45:26.748 INFO:tasks.workunit.client.1.smithi026.stdout:2/652: link d2/d29/d17/d1a/d2b/d2e/c6e d2/d29/d17/d38/d64/dcb/cde 0 2024-03-16T10:45:26.751 INFO:tasks.workunit.client.1.smithi026.stdout:2/653: stat d2/d29/d17/d1a/cd7 0 2024-03-16T10:45:26.752 INFO:tasks.workunit.client.1.smithi026.stdout:2/654: dread - d2/f8f zero size 2024-03-16T10:45:26.757 INFO:tasks.workunit.client.1.smithi026.stdout:2/655: dwrite d2/d29/d17/d1a/fad [0,4194304] 0 2024-03-16T10:45:26.770 INFO:tasks.workunit.client.1.smithi026.stdout:2/656: rename d2/d29/d36/d98/fa3 to d2/d29/d17/d1a/d2b/d69/fdf 0 2024-03-16T10:45:26.770 INFO:tasks.workunit.client.1.smithi026.stdout:2/657: stat d2/f35 0 2024-03-16T10:45:26.774 INFO:tasks.workunit.client.1.smithi026.stdout:2/658: symlink d2/d29/d17/le0 0 2024-03-16T10:45:26.779 INFO:tasks.workunit.client.1.smithi026.stdout:2/659: dwrite d2/d29/d17/fba [0,4194304] 0 2024-03-16T10:45:26.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: pgmap v8: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 24 MiB/s rd, 66 MiB/s wr, 181 op/s 2024-03-16T10:45:26.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: Standby manager daemon smithi026.omjkpo started 2024-03-16T10:45:26.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:45:26.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:45:26.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:45:26.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:45:26.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:26.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:26 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:26.799 INFO:tasks.workunit.client.1.smithi026.stdout:2/660: dread d2/f35 [0,4194304] 0 2024-03-16T10:45:26.808 INFO:tasks.workunit.client.1.smithi026.stdout:2/661: dwrite d2/d29/d17/d1a/d2b/f84 [0,4194304] 0 2024-03-16T10:45:26.813 INFO:tasks.workunit.client.1.smithi026.stdout:2/662: creat d2/dd1/fe1 x:0 0 0 2024-03-16T10:45:26.828 INFO:tasks.workunit.client.1.smithi026.stdout:2/663: dread d2/d29/d17/d1a/d2b/d4b/fb0 [0,4194304] 0 2024-03-16T10:45:26.829 INFO:tasks.workunit.client.1.smithi026.stdout:2/664: chown d2/d29/d51/d72/l9f 4192493 1 2024-03-16T10:45:26.832 INFO:tasks.workunit.client.1.smithi026.stdout:2/665: rename d2/d29/d17/d1a/cd7 to d2/d29/d36/da7/ce2 0 2024-03-16T10:45:26.834 INFO:tasks.workunit.client.1.smithi026.stdout:2/666: chown d2/d29/d17/d1a/d2b/d4e/d67/c7e 68 1 2024-03-16T10:45:26.834 INFO:tasks.workunit.client.1.smithi026.stdout:2/667: dread - d2/d29/fd9 zero size 2024-03-16T10:45:26.837 INFO:tasks.workunit.client.1.smithi026.stdout:2/668: unlink d2/d29/d3b/fb1 0 2024-03-16T10:45:26.842 INFO:tasks.workunit.client.1.smithi026.stdout:2/669: dwrite d2/dd1/fe1 [0,4194304] 0 2024-03-16T10:45:26.849 INFO:tasks.workunit.client.1.smithi026.stdout:2/670: symlink d2/d29/d17/d1a/d2b/d2e/d60/d7f/ddc/le3 0 2024-03-16T10:45:26.856 INFO:tasks.workunit.client.1.smithi026.stdout:2/671: creat d2/d29/d17/d1a/d2b/d2e/fe4 x:0 0 0 2024-03-16T10:45:26.859 INFO:tasks.workunit.client.1.smithi026.stdout:2/672: rmdir d2/d29/d17/d1a/d2b/d4b 39 2024-03-16T10:45:26.865 INFO:tasks.workunit.client.1.smithi026.stdout:2/673: dwrite d2/d29/d17/d1a/d2b/d4e/d6f/fd6 [0,4194304] 0 2024-03-16T10:45:26.871 INFO:tasks.workunit.client.1.smithi026.stdout:2/674: dread d2/d29/d17/d38/f7d [0,4194304] 0 2024-03-16T10:45:26.875 INFO:tasks.workunit.client.1.smithi026.stdout:2/675: dread d2/d29/d17/d1a/d2b/d4e/d6f/fd6 [0,4194304] 0 2024-03-16T10:45:26.878 INFO:tasks.workunit.client.1.smithi026.stdout:2/676: creat d2/d29/d36/d98/fe5 x:0 0 0 2024-03-16T10:45:26.880 INFO:tasks.workunit.client.1.smithi026.stdout:2/677: write d2/d29/d17/d1a/d2b/d4e/d67/f78 [444903,82441] 0 2024-03-16T10:45:26.881 INFO:tasks.workunit.client.1.smithi026.stdout:2/678: write d2/d29/d17/d1a/d2b/d2e/fe4 [44880,80866] 0 2024-03-16T10:45:26.886 INFO:tasks.workunit.client.1.smithi026.stdout:2/679: rename d2/d29/d51/d72/fb8 to d2/d29/d85/dc6/dd5/fe6 0 2024-03-16T10:45:26.890 INFO:tasks.workunit.client.1.smithi026.stdout:2/680: rename d2/d29/d17/d1a/d2b/d2e/d60/fcd to d2/d29/d51/d55/fe7 0 2024-03-16T10:45:26.896 INFO:tasks.workunit.client.1.smithi026.stdout:0/704: sync 2024-03-16T10:45:26.909 INFO:tasks.workunit.client.1.smithi026.stdout:2/681: write d2/d29/d17/fba [4417541,21591] 0 2024-03-16T10:45:26.911 INFO:tasks.workunit.client.1.smithi026.stdout:0/705: truncate d1/d12/d2f/daa/dd1/fb3 2096725 0 2024-03-16T10:45:26.913 INFO:tasks.workunit.client.1.smithi026.stdout:2/682: mkdir d2/d29/d17/de8 0 2024-03-16T10:45:26.916 INFO:tasks.workunit.client.1.smithi026.stdout:2/683: mknod d2/d29/d17/d1a/d2b/d2e/d60/d7f/ddc/ce9 0 2024-03-16T10:45:26.918 INFO:tasks.workunit.client.1.smithi026.stdout:0/706: rename d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/cbd to d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/cf2 0 2024-03-16T10:45:26.923 INFO:tasks.workunit.client.1.smithi026.stdout:2/684: dwrite d2/d29/d17/d1a/d2b/d69/fdb [0,4194304] 0 2024-03-16T10:45:26.927 INFO:tasks.workunit.client.1.smithi026.stdout:0/707: getdents d1/d12/d2f/d48/d55/d60/d9b/d57/dd7 0 2024-03-16T10:45:26.929 INFO:tasks.workunit.client.1.smithi026.stdout:0/708: write d1/d12/d2f/daa/dd1/d1a/d2b/fe9 [2405426,73721] 0 2024-03-16T10:45:26.934 INFO:tasks.workunit.client.1.smithi026.stdout:0/709: symlink d1/lf3 0 2024-03-16T10:45:26.936 INFO:tasks.workunit.client.1.smithi026.stdout:2/685: dwrite d2/d29/d17/d1a/d2b/d2e/d60/d7f/fb4 [4194304,4194304] 0 2024-03-16T10:45:26.944 INFO:tasks.workunit.client.1.smithi026.stdout:0/710: creat d1/d12/d2f/d48/d55/d60/de0/dea/ff4 x:0 0 0 2024-03-16T10:45:26.945 INFO:tasks.workunit.client.1.smithi026.stdout:0/711: dread - d1/d12/d2f/dba/fc7 zero size 2024-03-16T10:45:26.948 INFO:tasks.workunit.client.1.smithi026.stdout:0/712: mkdir d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/df5 0 2024-03-16T10:45:26.949 INFO:tasks.workunit.client.1.smithi026.stdout:0/713: dread - d1/d12/d2f/d48/d55/d60/d9b/d57/fec zero size 2024-03-16T10:45:26.950 INFO:tasks.workunit.client.1.smithi026.stdout:2/686: link d2/d29/d17/d1a/d2b/d4b/fda d2/d29/d51/d72/fea 0 2024-03-16T10:45:26.957 INFO:tasks.workunit.client.1.smithi026.stdout:2/687: dwrite d2/d29/d17/d1a/d2b/d2e/fe4 [0,4194304] 0 2024-03-16T10:45:26.958 INFO:tasks.workunit.client.1.smithi026.stdout:2/688: readlink d2/d29/l62 0 2024-03-16T10:45:26.965 INFO:tasks.workunit.client.1.smithi026.stdout:0/714: dread d1/d12/d2f/d48/d55/d60/d9b/d11/f54 [0,4194304] 0 2024-03-16T10:45:26.978 INFO:tasks.workunit.client.1.smithi026.stdout:2/689: truncate d2/d29/d17/d1a/f9e 55087 0 2024-03-16T10:45:26.979 INFO:tasks.workunit.client.1.smithi026.stdout:0/715: write d1/d12/d2f/d48/d55/d60/d9b/d57/f3e [1302978,19230] 0 2024-03-16T10:45:26.980 INFO:tasks.workunit.client.1.smithi026.stdout:2/690: fsync d2/d29/d17/d1a/d2b/d4e/fcc 0 2024-03-16T10:45:26.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: pgmap v8: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 24 MiB/s rd, 66 MiB/s wr, 181 op/s 2024-03-16T10:45:26.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: Standby manager daemon smithi026.omjkpo started 2024-03-16T10:45:26.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:45:26.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:45:26.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:45:26.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/799928742' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:45:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:26 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:26.982 INFO:tasks.workunit.client.1.smithi026.stdout:2/691: write d2/d29/d85/fc7 [2682879,26679] 0 2024-03-16T10:45:26.983 INFO:tasks.workunit.client.1.smithi026.stdout:2/692: chown d2/d29/d17/d1a/d2b/f84 37625 1 2024-03-16T10:45:26.988 INFO:tasks.workunit.client.1.smithi026.stdout:0/716: mkdir d1/d12/d2f/d48/d55/d60/d9b/d51/df6 0 2024-03-16T10:45:26.990 INFO:tasks.workunit.client.1.smithi026.stdout:2/693: symlink d2/d29/d85/dc6/dd5/leb 0 2024-03-16T10:45:26.995 INFO:tasks.workunit.client.1.smithi026.stdout:0/717: dwrite d1/d12/d2f/d48/d55/d60/f70 [0,4194304] 0 2024-03-16T10:45:26.995 INFO:tasks.workunit.client.1.smithi026.stdout:2/694: creat d2/dd1/fec x:0 0 0 2024-03-16T10:45:27.003 INFO:tasks.workunit.client.1.smithi026.stdout:0/718: dwrite d1/d12/d2f/d48/d55/fe3 [0,4194304] 0 2024-03-16T10:45:27.004 INFO:tasks.workunit.client.1.smithi026.stdout:2/695: dread d2/d29/d17/d1a/d2b/d2e/d60/d7f/fc5 [0,4194304] 0 2024-03-16T10:45:27.009 INFO:tasks.workunit.client.1.smithi026.stdout:0/719: creat d1/d12/d2f/d48/d55/d60/d9b/d57/ff7 x:0 0 0 2024-03-16T10:45:27.010 INFO:tasks.workunit.client.1.smithi026.stdout:2/696: mknod d2/d29/d3b/dbb/ced 0 2024-03-16T10:45:27.010 INFO:tasks.workunit.client.1.smithi026.stdout:0/720: write d1/d12/d2f/daa/dac/fd6 [3443151,123672] 0 2024-03-16T10:45:27.013 INFO:tasks.workunit.client.1.smithi026.stdout:0/721: write d1/d12/d2f/d48/d55/d60/d9b/fcc [5087911,59364] 0 2024-03-16T10:45:27.014 INFO:tasks.workunit.client.1.smithi026.stdout:2/697: creat d2/d29/da4/fee x:0 0 0 2024-03-16T10:45:27.018 INFO:tasks.workunit.client.1.smithi026.stdout:2/698: mkdir d2/d29/d17/d1a/d2b/d4e/d67/def 0 2024-03-16T10:45:27.019 INFO:tasks.workunit.client.1.smithi026.stdout:0/722: fdatasync d1/d12/d2f/d48/d55/d60/d9b/d57/fda 0 2024-03-16T10:45:27.021 INFO:tasks.workunit.client.1.smithi026.stdout:0/723: truncate d1/d12/d2f/d48/d55/d60/d9b/f67 5739103 0 2024-03-16T10:45:27.023 INFO:tasks.workunit.client.1.smithi026.stdout:2/699: unlink d2/d29/d17/d1a/d2b/f80 0 2024-03-16T10:45:27.023 INFO:tasks.workunit.client.1.smithi026.stdout:2/700: chown d2/d29/d17/f23 424 1 2024-03-16T10:45:27.025 INFO:tasks.workunit.client.1.smithi026.stdout:2/701: write d2/d29/d17/d1a/fad [5041996,38040] 0 2024-03-16T10:45:27.030 INFO:tasks.workunit.client.1.smithi026.stdout:2/702: symlink d2/d29/d17/lf0 0 2024-03-16T10:45:27.031 INFO:tasks.workunit.client.1.smithi026.stdout:2/703: chown d2/d29/d17/d38/d64/l6a 270 1 2024-03-16T10:45:27.040 INFO:tasks.workunit.client.1.smithi026.stdout:0/724: write d1/d12/d2f/daa/dd1/d1a/d2b/f86 [757074,128987] 0 2024-03-16T10:45:27.047 INFO:tasks.workunit.client.1.smithi026.stdout:2/704: dread d2/d29/d17/d1a/d2b/d2e/d60/d7f/fb4 [0,4194304] 0 2024-03-16T10:45:27.047 INFO:tasks.workunit.client.1.smithi026.stdout:2/705: chown d2/d29/d3b 8147019 1 2024-03-16T10:45:27.057 INFO:tasks.workunit.client.1.smithi026.stdout:2/706: rename d2/d29/d17/d1a/d2b/d4b/f57 to d2/d29/d17/d38/ff1 0 2024-03-16T10:45:27.059 INFO:tasks.workunit.client.1.smithi026.stdout:2/707: read d2/d29/d17/d1a/d2b/d2e/d60/fc2 [3996922,7476] 0 2024-03-16T10:45:27.063 INFO:tasks.workunit.client.1.smithi026.stdout:0/725: getdents d1/d12/d2f/d48/d55/d60/d9b/d51/d5f 0 2024-03-16T10:45:27.064 INFO:tasks.workunit.client.1.smithi026.stdout:0/726: readlink d1/d12/d2f/daa/dd1/d1a/lb0 0 2024-03-16T10:45:27.065 INFO:tasks.workunit.client.1.smithi026.stdout:2/708: link d2/d29/d17/f2d d2/d29/d85/dc6/dd5/ff2 0 2024-03-16T10:45:27.067 INFO:tasks.workunit.client.1.smithi026.stdout:2/709: symlink d2/d29/d17/d1a/d2b/d2e/d60/d7f/lf3 0 2024-03-16T10:45:27.069 INFO:tasks.workunit.client.1.smithi026.stdout:2/710: creat d2/d29/d85/ff4 x:0 0 0 2024-03-16T10:45:27.070 INFO:tasks.workunit.client.1.smithi026.stdout:2/711: write d2/d29/d85/ff4 [716597,4155] 0 2024-03-16T10:45:27.071 INFO:tasks.workunit.client.1.smithi026.stdout:2/712: dread - d2/d29/d17/d1a/d2b/d4e/d67/fd2 zero size 2024-03-16T10:45:27.073 INFO:tasks.workunit.client.1.smithi026.stdout:2/713: write d2/d29/d17/d1a/d2b/d2e/d60/fc2 [4426953,82530] 0 2024-03-16T10:45:27.075 INFO:tasks.workunit.client.1.smithi026.stdout:2/714: truncate d2/d29/d36/f3e 1014261 0 2024-03-16T10:45:27.076 INFO:tasks.workunit.client.1.smithi026.stdout:2/715: readlink d2/d29/d17/d1a/d2b/d4b/lae 0 2024-03-16T10:45:27.231 INFO:tasks.workunit.client.1.smithi026.stdout:8/615: sync 2024-03-16T10:45:27.232 INFO:tasks.workunit.client.1.smithi026.stdout:6/535: sync 2024-03-16T10:45:27.239 INFO:tasks.workunit.client.1.smithi026.stdout:8/616: rename d5/d23/l6c to d5/dc/d40/lb4 0 2024-03-16T10:45:27.240 INFO:tasks.workunit.client.1.smithi026.stdout:8/617: truncate d5/d23/f82 1733065 0 2024-03-16T10:45:27.243 INFO:tasks.workunit.client.1.smithi026.stdout:6/536: truncate d0/d41/d4f/f83 474876 0 2024-03-16T10:45:27.247 INFO:tasks.workunit.client.1.smithi026.stdout:8/618: dwrite d5/d23/f4b [0,4194304] 0 2024-03-16T10:45:27.250 INFO:tasks.workunit.client.1.smithi026.stdout:6/537: rmdir d0/d41/d4f/db3 0 2024-03-16T10:45:27.251 INFO:tasks.workunit.client.1.smithi026.stdout:6/538: truncate d0/d41/d4f/f87 536618 0 2024-03-16T10:45:27.267 INFO:tasks.workunit.client.1.smithi026.stdout:8/619: dread d5/d28/f31 [0,4194304] 0 2024-03-16T10:45:27.270 INFO:tasks.workunit.client.1.smithi026.stdout:8/620: unlink d5/d7/f1c 0 2024-03-16T10:45:27.282 INFO:tasks.workunit.client.1.smithi026.stdout:8/621: link d5/d7/f11 d5/d23/fb5 0 2024-03-16T10:45:27.293 INFO:tasks.workunit.client.1.smithi026.stdout:8/622: dread d5/d23/f68 [0,4194304] 0 2024-03-16T10:45:27.350 INFO:tasks.workunit.client.1.smithi026.stdout:8/623: dwrite d5/d23/fac [0,4194304] 0 2024-03-16T10:45:27.355 INFO:tasks.workunit.client.1.smithi026.stdout:8/624: symlink d5/d28/lb6 0 2024-03-16T10:45:27.360 INFO:tasks.workunit.client.1.smithi026.stdout:8/625: unlink d5/dc/d40/l71 0 2024-03-16T10:45:27.374 INFO:tasks.workunit.client.1.smithi026.stdout:8/626: truncate d5/d7/d2c/d36/d59/d93/fab 216155 0 2024-03-16T10:45:27.377 INFO:tasks.workunit.client.1.smithi026.stdout:8/627: creat d5/dc/d40/d53/fb7 x:0 0 0 2024-03-16T10:45:27.379 INFO:tasks.workunit.client.1.smithi026.stdout:8/628: creat d5/d5d/d9a/fb8 x:0 0 0 2024-03-16T10:45:27.380 INFO:tasks.workunit.client.1.smithi026.stdout:8/629: symlink d5/d5d/lb9 0 2024-03-16T10:45:27.380 INFO:tasks.workunit.client.1.smithi026.stdout:8/630: readlink d5/d23/l54 0 2024-03-16T10:45:27.386 INFO:tasks.workunit.client.1.smithi026.stdout:7/677: sync 2024-03-16T10:45:27.387 INFO:tasks.workunit.client.1.smithi026.stdout:4/675: sync 2024-03-16T10:45:27.388 INFO:tasks.workunit.client.1.smithi026.stdout:7/678: readlink d6/d35/d3a/d4e/dcc/d73/lcb 0 2024-03-16T10:45:27.391 INFO:tasks.workunit.client.1.smithi026.stdout:8/631: dread d5/dc/d89/f6e [0,4194304] 0 2024-03-16T10:45:27.393 INFO:tasks.workunit.client.1.smithi026.stdout:8/632: write d5/d23/f82 [2527281,111307] 0 2024-03-16T10:45:27.393 INFO:tasks.workunit.client.1.smithi026.stdout:7/679: unlink d6/cca 0 2024-03-16T10:45:27.403 INFO:tasks.workunit.client.1.smithi026.stdout:7/680: dread d6/d35/d3a/d4e/dcc/d3f/fa0 [0,4194304] 0 2024-03-16T10:45:27.403 INFO:tasks.workunit.client.1.smithi026.stdout:7/681: dread - d6/de/fbe zero size 2024-03-16T10:45:27.409 INFO:tasks.workunit.client.1.smithi026.stdout:4/676: dwrite d1/db/de/f58 [8388608,4194304] 0 2024-03-16T10:45:27.410 INFO:tasks.workunit.client.1.smithi026.stdout:7/682: mknod d6/d35/d3a/d4e/dcc/d3f/d65/d68/cdf 0 2024-03-16T10:45:27.415 INFO:tasks.workunit.client.1.smithi026.stdout:7/683: write d6/de/d7d/f86 [694104,25343] 0 2024-03-16T10:45:27.418 INFO:tasks.workunit.client.1.smithi026.stdout:4/677: rename d1/db/de/d4c/d6b/d7a/db1/fcb to d1/fd4 0 2024-03-16T10:45:27.419 INFO:tasks.workunit.client.1.smithi026.stdout:8/633: dwrite d5/d23/f67 [0,4194304] 0 2024-03-16T10:45:27.423 INFO:tasks.workunit.client.1.smithi026.stdout:7/684: creat d6/d35/d3a/d4e/dcc/d3f/dc3/fe0 x:0 0 0 2024-03-16T10:45:27.429 INFO:tasks.workunit.client.1.smithi026.stdout:4/678: dwrite d1/db/de/d4c/d81/f94 [0,4194304] 0 2024-03-16T10:45:27.430 INFO:tasks.workunit.client.1.smithi026.stdout:7/685: chown d6/d35/d3a/d4e/dcc/d3f/d9d/cd6 6928227 1 2024-03-16T10:45:27.444 INFO:tasks.workunit.client.1.smithi026.stdout:8/634: dwrite d5/d7/f34 [0,4194304] 0 2024-03-16T10:45:27.445 INFO:tasks.workunit.client.1.smithi026.stdout:7/686: dwrite d6/d35/d3a/d4e/dcc/d3f/d70/fa3 [0,4194304] 0 2024-03-16T10:45:27.454 INFO:tasks.workunit.client.1.smithi026.stdout:4/679: truncate d1/db/de/d6e/fb6 4118427 0 2024-03-16T10:45:27.454 INFO:tasks.workunit.client.1.smithi026.stdout:4/680: chown d1/db/l1a 60 1 2024-03-16T10:45:27.455 INFO:tasks.workunit.client.1.smithi026.stdout:8/635: read d5/d7/d2c/d36/d59/f78 [633678,126314] 0 2024-03-16T10:45:27.455 INFO:tasks.workunit.client.1.smithi026.stdout:4/681: fdatasync d1/db/de/d4c/fa1 0 2024-03-16T10:45:27.457 INFO:tasks.workunit.client.1.smithi026.stdout:7/687: mkdir d6/d35/d3a/d4e/dcc/d3f/d54/de1 0 2024-03-16T10:45:27.458 INFO:tasks.workunit.client.1.smithi026.stdout:7/688: fsync d6/de/faa 0 2024-03-16T10:45:27.461 INFO:tasks.workunit.client.1.smithi026.stdout:4/682: mkdir d1/db/de/d4c/d79/daa/d44/dd5 0 2024-03-16T10:45:27.463 INFO:tasks.workunit.client.1.smithi026.stdout:4/683: write d1/db/de/d4c/fa1 [3802358,126173] 0 2024-03-16T10:45:27.464 INFO:tasks.workunit.client.1.smithi026.stdout:7/689: rmdir d6/d35/d3a/d4e/da1 39 2024-03-16T10:45:27.465 INFO:tasks.workunit.client.1.smithi026.stdout:7/690: dread - d6/d35/d3a/d4e/d6c/fc9 zero size 2024-03-16T10:45:27.466 INFO:tasks.workunit.client.1.smithi026.stdout:8/636: mknod d5/d23/d4e/cba 0 2024-03-16T10:45:27.466 INFO:tasks.workunit.client.1.smithi026.stdout:4/684: creat d1/d85/fd6 x:0 0 0 2024-03-16T10:45:27.467 INFO:tasks.workunit.client.1.smithi026.stdout:7/691: fsync d6/d1e/da6/f8f 0 2024-03-16T10:45:27.475 INFO:tasks.workunit.client.1.smithi026.stdout:7/692: dread - d6/d35/d3a/d4e/dcc/d3f/d54/f9b zero size 2024-03-16T10:45:27.477 INFO:tasks.workunit.client.1.smithi026.stdout:4/685: dread d1/db/f26 [0,4194304] 0 2024-03-16T10:45:27.479 INFO:tasks.workunit.client.1.smithi026.stdout:4/686: chown d1/db/de/d4c/d6b/dbc/lc5 78 1 2024-03-16T10:45:27.486 INFO:tasks.workunit.client.1.smithi026.stdout:7/693: getdents d6/d35/d3a/d4e/dcc/d3f/d70 0 2024-03-16T10:45:27.487 INFO:tasks.workunit.client.1.smithi026.stdout:8/637: dwrite d5/d23/f61 [0,4194304] 0 2024-03-16T10:45:27.497 INFO:tasks.workunit.client.1.smithi026.stdout:4/687: getdents d1/db/de/d4c/d59/d6a/db2 0 2024-03-16T10:45:27.499 INFO:tasks.workunit.client.1.smithi026.stdout:4/688: mknod d1/dcd/cd7 0 2024-03-16T10:45:27.499 INFO:tasks.workunit.client.1.smithi026.stdout:7/694: write d6/f24 [3963726,14030] 0 2024-03-16T10:45:27.501 INFO:tasks.workunit.client.1.smithi026.stdout:7/695: stat d6/d2f/dad/f7c 0 2024-03-16T10:45:27.519 INFO:tasks.workunit.client.1.smithi026.stdout:4/689: dread d1/db/de/d4c/d81/dc7/fcc [0,4194304] 0 2024-03-16T10:45:27.519 INFO:tasks.workunit.client.1.smithi026.stdout:7/696: dread d6/de/f64 [0,4194304] 0 2024-03-16T10:45:27.520 INFO:tasks.workunit.client.1.smithi026.stdout:7/697: readlink d6/d35/d3a/d4e/dcc/d73/lcb 0 2024-03-16T10:45:27.525 INFO:tasks.workunit.client.1.smithi026.stdout:4/690: rename d1/fac to d1/d85/fd8 0 2024-03-16T10:45:27.528 INFO:tasks.workunit.client.1.smithi026.stdout:4/691: write d1/db/f42 [1020671,5187] 0 2024-03-16T10:45:27.534 INFO:tasks.workunit.client.1.smithi026.stdout:4/692: truncate d1/db/de/d4c/d59/d6a/f9a 4804374 0 2024-03-16T10:45:27.534 INFO:tasks.workunit.client.1.smithi026.stdout:7/698: dwrite d6/de/d16/f1f [0,4194304] 0 2024-03-16T10:45:27.538 INFO:tasks.workunit.client.1.smithi026.stdout:7/699: symlink d6/d35/d3a/d4e/dcc/d73/le2 0 2024-03-16T10:45:27.540 INFO:tasks.workunit.client.1.smithi026.stdout:4/693: rename d1/db/de/d4c/d59/d6a/ca9 to d1/db/de/d4c/d59/cd9 0 2024-03-16T10:45:27.540 INFO:tasks.workunit.client.1.smithi026.stdout:4/694: readlink d1/l96 0 2024-03-16T10:45:27.542 INFO:tasks.workunit.client.1.smithi026.stdout:7/700: truncate d6/f91 1101256 0 2024-03-16T10:45:27.551 INFO:tasks.workunit.client.1.smithi026.stdout:7/701: link d6/d35/d3a/d4e/dcc/d3f/d54/f8b d6/d35/d3a/d4e/dcc/d3f/dc3/fe3 0 2024-03-16T10:45:27.552 INFO:tasks.workunit.client.1.smithi026.stdout:7/702: write d6/d35/d3a/d4e/dcc/d3f/d9d/f97 [242278,57400] 0 2024-03-16T10:45:27.556 INFO:tasks.workunit.client.1.smithi026.stdout:7/703: mknod d6/d35/d3a/d4e/dcc/d3f/d54/ce4 0 2024-03-16T10:45:27.560 INFO:tasks.workunit.client.1.smithi026.stdout:7/704: dread d6/d35/f7e [0,4194304] 0 2024-03-16T10:45:27.562 INFO:tasks.workunit.client.1.smithi026.stdout:4/695: dwrite d1/db/de/d6e/fb6 [4194304,4194304] 0 2024-03-16T10:45:27.569 INFO:tasks.workunit.client.1.smithi026.stdout:4/696: symlink d1/db/de/d4c/d79/daa/d44/lda 0 2024-03-16T10:45:27.578 INFO:tasks.workunit.client.1.smithi026.stdout:4/697: getdents d1/db/de/d4c/d79/daa/d44 0 2024-03-16T10:45:27.578 INFO:tasks.workunit.client.1.smithi026.stdout:4/698: chown d1/d85/da2/da8 16815 1 2024-03-16T10:45:27.580 INFO:tasks.workunit.client.1.smithi026.stdout:4/699: write d1/f18 [353149,6368] 0 2024-03-16T10:45:27.584 INFO:tasks.workunit.client.1.smithi026.stdout:7/705: dwrite d6/f91 [0,4194304] 0 2024-03-16T10:45:27.586 INFO:tasks.workunit.client.1.smithi026.stdout:4/700: rename d1/db/de/d4c/d79 to d1/db/ddb 0 2024-03-16T10:45:27.589 INFO:tasks.workunit.client.1.smithi026.stdout:7/706: unlink d6/d35/d3a/d4e/db0/lc7 0 2024-03-16T10:45:27.589 INFO:tasks.workunit.client.1.smithi026.stdout:7/707: readlink d6/de/l2c 0 2024-03-16T10:45:27.591 INFO:tasks.workunit.client.1.smithi026.stdout:7/708: symlink d6/d2f/le5 0 2024-03-16T10:45:27.595 INFO:tasks.workunit.client.1.smithi026.stdout:4/701: dread d1/db/de/f5b [0,4194304] 0 2024-03-16T10:45:27.600 INFO:tasks.workunit.client.1.smithi026.stdout:4/702: mkdir d1/db/ddb/ddc 0 2024-03-16T10:45:27.606 INFO:tasks.workunit.client.1.smithi026.stdout:4/703: dwrite d1/db/ddb/daa/d61/f75 [0,4194304] 0 2024-03-16T10:45:27.609 INFO:tasks.workunit.client.1.smithi026.stdout:4/704: dread d1/db/de/d4c/d59/d6a/db2/fb4 [0,4194304] 0 2024-03-16T10:45:27.612 INFO:tasks.workunit.client.1.smithi026.stdout:7/709: dwrite d6/f38 [0,4194304] 0 2024-03-16T10:45:27.615 INFO:tasks.workunit.client.1.smithi026.stdout:4/705: symlink d1/db/de/d4c/d6b/d7a/db1/ldd 0 2024-03-16T10:45:27.617 INFO:tasks.workunit.client.1.smithi026.stdout:4/706: write d1/f52 [5042639,126021] 0 2024-03-16T10:45:27.619 INFO:tasks.workunit.client.1.smithi026.stdout:5/651: sync 2024-03-16T10:45:27.619 INFO:tasks.workunit.client.1.smithi026.stdout:3/697: sync 2024-03-16T10:45:27.619 INFO:tasks.workunit.client.1.smithi026.stdout:1/603: sync 2024-03-16T10:45:27.621 INFO:tasks.workunit.client.1.smithi026.stdout:4/707: write d1/db/ddb/daa/d61/f7c [4223868,84938] 0 2024-03-16T10:45:27.624 INFO:tasks.workunit.client.1.smithi026.stdout:7/710: link d6/l45 d6/d35/d3a/d59/le6 0 2024-03-16T10:45:27.625 INFO:tasks.workunit.client.1.smithi026.stdout:7/711: chown d6/de/d16/d88/l9c 710173061 1 2024-03-16T10:45:27.627 INFO:tasks.workunit.client.1.smithi026.stdout:5/652: mkdir d1/d1c/d2a/dab/de8 0 2024-03-16T10:45:27.630 INFO:tasks.workunit.client.1.smithi026.stdout:1/604: creat d0/d17/d4d/d70/fb9 x:0 0 0 2024-03-16T10:45:27.636 INFO:tasks.workunit.client.1.smithi026.stdout:5/653: dread d1/db/d1f/d61/d65/f7c [0,4194304] 0 2024-03-16T10:45:27.637 INFO:tasks.workunit.client.1.smithi026.stdout:5/654: readlink d1/d6e/lc3 0 2024-03-16T10:45:27.642 INFO:tasks.workunit.client.1.smithi026.stdout:1/605: creat d0/d17/d1f/d63/d6a/d7c/fba x:0 0 0 2024-03-16T10:45:27.662 INFO:tasks.workunit.client.1.smithi026.stdout:3/698: truncate d4/dc/f11 3141797 0 2024-03-16T10:45:27.664 INFO:tasks.workunit.client.1.smithi026.stdout:7/712: truncate d6/f15 1023533 0 2024-03-16T10:45:27.665 INFO:tasks.workunit.client.1.smithi026.stdout:3/699: dread d4/dc/d73/d83/d7c/dd7/d62/f85 [0,4194304] 0 2024-03-16T10:45:27.667 INFO:tasks.workunit.client.1.smithi026.stdout:1/606: write d0/d17/d1f/d63/d6a/d7c/f82 [6673921,106205] 0 2024-03-16T10:45:27.670 INFO:tasks.workunit.client.1.smithi026.stdout:1/607: write d0/d17/d1f/d20/d65/f8c [983103,4338] 0 2024-03-16T10:45:27.671 INFO:tasks.workunit.client.1.smithi026.stdout:3/700: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/ff9 x:0 0 0 2024-03-16T10:45:27.672 INFO:tasks.workunit.client.1.smithi026.stdout:5/655: dwrite d1/d1c/d2a/dab/dc5/f3a [0,4194304] 0 2024-03-16T10:45:27.676 INFO:tasks.workunit.client.1.smithi026.stdout:1/608: mkdir d0/d17/d1f/d63/d6a/d7c/d84/dbb 0 2024-03-16T10:45:27.678 INFO:tasks.workunit.client.1.smithi026.stdout:5/656: mkdir d1/d1c/d2a/dab/dc5/d69/d51/d80/de9 0 2024-03-16T10:45:27.682 INFO:tasks.workunit.client.1.smithi026.stdout:3/701: rename d4/dc/dd/d3d to d4/d89/d97/d7d/dfa 0 2024-03-16T10:45:27.682 INFO:tasks.workunit.client.1.smithi026.stdout:3/702: fsync d4/dc/d31/d48/fd1 0 2024-03-16T10:45:27.701 INFO:tasks.workunit.client.1.smithi026.stdout:3/703: dread d4/d89/d97/f99 [0,4194304] 0 2024-03-16T10:45:27.702 INFO:tasks.workunit.client.1.smithi026.stdout:3/704: write d4/dc/dd/f75 [810750,54833] 0 2024-03-16T10:45:27.712 INFO:tasks.workunit.client.1.smithi026.stdout:3/705: rename d4/dc/d31/d48/d69/d78/db3/fb5 to d4/dc/d31/d48/d7f/ffb 0 2024-03-16T10:45:27.721 INFO:tasks.workunit.client.1.smithi026.stdout:1/609: dwrite d0/d17/d1f/d20/f85 [0,4194304] 0 2024-03-16T10:45:27.723 INFO:tasks.workunit.client.1.smithi026.stdout:5/657: dwrite d1/d1c/d2a/f89 [0,4194304] 0 2024-03-16T10:45:27.726 INFO:tasks.workunit.client.1.smithi026.stdout:3/706: link d4/dc/d31/d48/d69/f6f d4/dc/dd/d47/d53/db1/ddd/ffc 0 2024-03-16T10:45:27.727 INFO:tasks.workunit.client.1.smithi026.stdout:3/707: write d4/fe [5033762,110837] 0 2024-03-16T10:45:27.736 INFO:tasks.workunit.client.1.smithi026.stdout:5/658: rename d1/db/d27/dd1 to d1/db/d1f/d61/d65/d82/dbb/dea 0 2024-03-16T10:45:27.740 INFO:tasks.workunit.client.1.smithi026.stdout:5/659: dread d1/d1c/d4c/d4d/fcc [0,4194304] 0 2024-03-16T10:45:27.741 INFO:tasks.workunit.client.1.smithi026.stdout:6/539: sync 2024-03-16T10:45:27.741 INFO:tasks.workunit.client.1.smithi026.stdout:0/727: sync 2024-03-16T10:45:27.742 INFO:tasks.workunit.client.1.smithi026.stdout:2/716: sync 2024-03-16T10:45:27.742 INFO:tasks.workunit.client.1.smithi026.stdout:9/749: sync 2024-03-16T10:45:27.743 INFO:tasks.workunit.client.1.smithi026.stdout:1/610: mkdir d0/d17/d1f/d63/dbc 0 2024-03-16T10:45:27.745 INFO:tasks.workunit.client.1.smithi026.stdout:0/728: readlink d1/d12/d2f/daa/dd1/d61/l9d 0 2024-03-16T10:45:27.746 INFO:tasks.workunit.client.1.smithi026.stdout:1/611: read - d0/d17/fb2 zero size 2024-03-16T10:45:27.748 INFO:tasks.workunit.client.1.smithi026.stdout:5/660: dwrite d1/d1c/d2a/dab/dc5/f76 [0,4194304] 0 2024-03-16T10:45:27.755 INFO:tasks.workunit.client.1.smithi026.stdout:6/540: symlink d0/d10/d79/lb6 0 2024-03-16T10:45:27.757 INFO:tasks.workunit.client.1.smithi026.stdout:9/750: symlink d0/da/d16/d17/db1/lfe 0 2024-03-16T10:45:27.759 INFO:tasks.workunit.client.1.smithi026.stdout:9/751: read d0/d5f/da2/fa6 [523601,10942] 0 2024-03-16T10:45:27.761 INFO:tasks.workunit.client.1.smithi026.stdout:0/729: unlink d1/d12/d2f/daa/dd1/d1a/d2b/l68 0 2024-03-16T10:45:27.763 INFO:tasks.workunit.client.1.smithi026.stdout:1/612: rename d0/d17/d1f/d63/d6a/fa3 to d0/d17/d1f/d20/fbd 0 2024-03-16T10:45:27.768 INFO:tasks.workunit.client.1.smithi026.stdout:0/730: dwrite d1/d12/d2f/d48/d55/d60/d9b/d57/fdb [0,4194304] 0 2024-03-16T10:45:27.769 INFO:tasks.workunit.client.1.smithi026.stdout:3/708: link d4/dc/l40 d4/dc/dd/d47/lfd 0 2024-03-16T10:45:27.772 INFO:tasks.workunit.client.1.smithi026.stdout:9/752: mkdir d0/dd7/dff 0 2024-03-16T10:45:27.774 INFO:tasks.workunit.client.1.smithi026.stdout:5/661: mknod d1/d1c/d2a/dab/dc5/d69/d51/ceb 0 2024-03-16T10:45:27.776 INFO:tasks.workunit.client.1.smithi026.stdout:6/541: mkdir d0/d8a/db7 0 2024-03-16T10:45:27.778 INFO:tasks.workunit.client.1.smithi026.stdout:3/709: dwrite d4/dc/dd/f24 [0,4194304] 0 2024-03-16T10:45:27.779 INFO:tasks.workunit.client.1.smithi026.stdout:5/662: truncate d1/db/d1f/d61/fdc 836119 0 2024-03-16T10:45:27.779 INFO:tasks.workunit.client.1.smithi026.stdout:6/542: chown d0/d28/f9c 7 1 2024-03-16T10:45:27.780 INFO:tasks.workunit.client.1.smithi026.stdout:2/717: link d2/d29/da4/cd4 d2/d29/d85/dc6/cf5 0 2024-03-16T10:45:27.782 INFO:tasks.workunit.client.1.smithi026.stdout:2/718: write d2/d29/d17/d1a/d2b/f1f [3851683,116370] 0 2024-03-16T10:45:27.787 INFO:tasks.workunit.client.1.smithi026.stdout:0/731: dread d1/d12/d2f/daa/dd1/fe6 [0,4194304] 0 2024-03-16T10:45:27.787 INFO:tasks.workunit.client.1.smithi026.stdout:9/753: fdatasync d0/d63/d6b/f89 0 2024-03-16T10:45:27.792 INFO:tasks.workunit.client.1.smithi026.stdout:1/613: rename d0/f74 to d0/d17/d1f/d63/d6a/d7c/d84/d9d/fbe 0 2024-03-16T10:45:27.793 INFO:tasks.workunit.client.1.smithi026.stdout:3/710: mknod d4/dc/dd/d80/cfe 0 2024-03-16T10:45:27.801 INFO:tasks.workunit.client.1.smithi026.stdout:0/732: dread d1/d12/d2f/daa/dd1/d61/f62 [4194304,4194304] 0 2024-03-16T10:45:27.802 INFO:tasks.workunit.client.1.smithi026.stdout:9/754: chown d0/da/d16/d5d/d72/laf 15641 1 2024-03-16T10:45:27.805 INFO:tasks.workunit.client.1.smithi026.stdout:2/719: rename d2/d29/d85/dc6/dd5/fe6 to d2/d29/d51/ff6 0 2024-03-16T10:45:27.807 INFO:tasks.workunit.client.1.smithi026.stdout:6/543: mknod d0/d8a/db7/cb8 0 2024-03-16T10:45:27.808 INFO:tasks.workunit.client.1.smithi026.stdout:6/544: chown d0/f2c 76934777 1 2024-03-16T10:45:27.810 INFO:tasks.workunit.client.1.smithi026.stdout:9/755: creat d0/da/d16/f100 x:0 0 0 2024-03-16T10:45:27.813 INFO:tasks.workunit.client.1.smithi026.stdout:3/711: rename d4/d89/d97/d7d to d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff 0 2024-03-16T10:45:27.813 INFO:tasks.workunit.client.1.smithi026.stdout:3/712: fdatasync d4/dc/dd/f24 0 2024-03-16T10:45:27.814 INFO:tasks.workunit.client.1.smithi026.stdout:3/713: dread - d4/dc/d73/d83/f6c zero size 2024-03-16T10:45:27.829 INFO:tasks.workunit.client.1.smithi026.stdout:2/720: dwrite d2/d29/d51/d72/f76 [0,4194304] 0 2024-03-16T10:45:27.829 INFO:tasks.workunit.client.1.smithi026.stdout:1/614: symlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/lbf 0 2024-03-16T10:45:27.830 INFO:tasks.workunit.client.1.smithi026.stdout:0/733: truncate d1/d12/d2f/d48/d55/d60/d9b/d51/f5e 2759515 0 2024-03-16T10:45:27.831 INFO:tasks.workunit.client.1.smithi026.stdout:9/756: dwrite d0/da/f27 [0,4194304] 0 2024-03-16T10:45:27.833 INFO:tasks.workunit.client.1.smithi026.stdout:1/615: write d0/d17/d1f/d20/d65/f8c [1799270,100750] 0 2024-03-16T10:45:27.835 INFO:tasks.workunit.client.1.smithi026.stdout:6/545: truncate d0/d28/d49/d7d/f8c 1180787 0 2024-03-16T10:45:27.837 INFO:tasks.workunit.client.1.smithi026.stdout:1/616: write d0/d17/d1f/d20/d65/f8c [2484868,62473] 0 2024-03-16T10:45:27.837 INFO:tasks.workunit.client.1.smithi026.stdout:6/546: read d0/d28/d2d/f55 [80133,69288] 0 2024-03-16T10:45:27.852 INFO:tasks.workunit.client.1.smithi026.stdout:1/617: dwrite d0/d17/f81 [4194304,4194304] 0 2024-03-16T10:45:27.854 INFO:tasks.workunit.client.1.smithi026.stdout:9/757: dwrite d0/da/d13/f69 [0,4194304] 0 2024-03-16T10:45:27.854 INFO:tasks.workunit.client.1.smithi026.stdout:6/547: dwrite d0/d28/d2d/f55 [0,4194304] 0 2024-03-16T10:45:27.855 INFO:tasks.workunit.client.1.smithi026.stdout:3/714: rmdir d4/dc/dd/d47/d53/db1/ddd/de9 0 2024-03-16T10:45:27.861 INFO:tasks.workunit.client.1.smithi026.stdout:2/721: link d2/d29/d17/d1a/d2b/d4b/f5c d2/d29/d17/d1a/d2b/d4e/d6f/ff7 0 2024-03-16T10:45:27.871 INFO:tasks.workunit.client.1.smithi026.stdout:6/548: mknod d0/d8a/cb9 0 2024-03-16T10:45:27.872 INFO:tasks.workunit.client.1.smithi026.stdout:9/758: dread d0/da/d14/f34 [4194304,4194304] 0 2024-03-16T10:45:27.875 INFO:tasks.workunit.client.1.smithi026.stdout:1/618: rename d0/d17/d1f/c34 to d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/cc0 0 2024-03-16T10:45:27.877 INFO:tasks.workunit.client.1.smithi026.stdout:2/722: dread - d2/d29/d17/d38/f97 zero size 2024-03-16T10:45:27.877 INFO:tasks.workunit.client.1.smithi026.stdout:3/715: read d4/dc/dd/d47/f86 [710563,57624] 0 2024-03-16T10:45:27.880 INFO:tasks.workunit.client.1.smithi026.stdout:6/549: mknod d0/d8a/cba 0 2024-03-16T10:45:27.882 INFO:tasks.workunit.client.1.smithi026.stdout:0/734: getdents d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89 0 2024-03-16T10:45:27.884 INFO:tasks.workunit.client.1.smithi026.stdout:0/735: read d1/d12/d2f/daa/dd1/fb3 [30411,51683] 0 2024-03-16T10:45:27.892 INFO:tasks.workunit.client.1.smithi026.stdout:9/759: creat d0/d5f/f101 x:0 0 0 2024-03-16T10:45:27.894 INFO:tasks.workunit.client.1.smithi026.stdout:3/716: write d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f6a [195097,4338] 0 2024-03-16T10:45:27.897 INFO:tasks.workunit.client.1.smithi026.stdout:1/619: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/fbe [3514161,72647] 0 2024-03-16T10:45:27.898 INFO:tasks.workunit.client.1.smithi026.stdout:1/620: dread - d0/d17/d1f/d63/d6a/d7c/d2e/faf zero size 2024-03-16T10:45:27.901 INFO:tasks.workunit.client.1.smithi026.stdout:9/760: mknod d0/da/d16/d5d/d72/d62/c102 0 2024-03-16T10:45:27.904 INFO:tasks.workunit.client.1.smithi026.stdout:3/717: symlink d4/dc/d31/d48/d69/d78/l100 0 2024-03-16T10:45:27.907 INFO:tasks.workunit.client.1.smithi026.stdout:1/621: mkdir d0/d17/d1f/d63/d58/dc1 0 2024-03-16T10:45:27.908 INFO:tasks.workunit.client.1.smithi026.stdout:9/761: chown d0/da/d16/d5d/cd6 0 1 2024-03-16T10:45:27.909 INFO:tasks.workunit.client.1.smithi026.stdout:0/736: creat d1/d12/d2f/d48/d55/d60/d9b/ff8 x:0 0 0 2024-03-16T10:45:27.912 INFO:tasks.workunit.client.1.smithi026.stdout:6/550: link d0/l2f d0/d10/lbb 0 2024-03-16T10:45:27.918 INFO:tasks.workunit.client.1.smithi026.stdout:1/622: dread d0/f7f [0,4194304] 0 2024-03-16T10:45:27.919 INFO:tasks.workunit.client.1.smithi026.stdout:0/737: rmdir d1/d12/d2f/daa/dd1/d1a/d2b 39 2024-03-16T10:45:27.920 INFO:tasks.workunit.client.1.smithi026.stdout:3/718: write d4/dc/d73/d83/d7c/dd7/f64 [602216,94140] 0 2024-03-16T10:45:27.921 INFO:tasks.workunit.client.1.smithi026.stdout:0/738: write d1/d12/d2f/daa/dd1/f97 [3804084,116402] 0 2024-03-16T10:45:27.926 INFO:tasks.workunit.client.1.smithi026.stdout:1/623: fsync d0/d17/d1f/d20/d65/f8e 0 2024-03-16T10:45:27.932 INFO:tasks.workunit.client.1.smithi026.stdout:9/762: creat d0/da/d16/d5d/f103 x:0 0 0 2024-03-16T10:45:27.933 INFO:tasks.workunit.client.1.smithi026.stdout:1/624: dwrite d0/d17/d1f/d63/d58/f9a [0,4194304] 0 2024-03-16T10:45:27.935 INFO:tasks.workunit.client.1.smithi026.stdout:0/739: mknod d1/d12/d2f/daa/dd1/d1a/cf9 0 2024-03-16T10:45:27.935 INFO:tasks.workunit.client.1.smithi026.stdout:6/551: link d0/f1d d0/d28/d49/d7d/fbc 0 2024-03-16T10:45:27.979 INFO:tasks.workunit.client.1.smithi026.stdout:4/708: sync 2024-03-16T10:45:27.979 INFO:tasks.workunit.client.1.smithi026.stdout:8/638: sync 2024-03-16T10:45:27.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:27 smithi026 ceph-mon[37041]: mgrmap e31: smithi002.zevrtm(active, since 11s), standbys: smithi026.omjkpo 2024-03-16T10:45:27.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:27 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:45:27.982 INFO:tasks.workunit.client.1.smithi026.stdout:8/639: dread d5/d23/f68 [0,4194304] 0 2024-03-16T10:45:27.983 INFO:tasks.workunit.client.1.smithi026.stdout:3/719: getdents d4/dc/d73 0 2024-03-16T10:45:27.987 INFO:tasks.workunit.client.1.smithi026.stdout:3/720: dread d4/dc/f19 [0,4194304] 0 2024-03-16T10:45:28.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:27 smithi002 ceph-mon[29248]: mgrmap e31: smithi002.zevrtm(active, since 11s), standbys: smithi026.omjkpo 2024-03-16T10:45:28.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:27 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:45:28.061 INFO:tasks.workunit.client.1.smithi026.stdout:8/640: dwrite d5/d5d/f92 [0,4194304] 0 2024-03-16T10:45:28.061 INFO:tasks.workunit.client.1.smithi026.stdout:4/709: fsync d1/db/ddb/daa/d61/f6c 0 2024-03-16T10:45:28.064 INFO:tasks.workunit.client.1.smithi026.stdout:1/625: write d0/d17/d1f/d20/f43 [1022510,33418] 0 2024-03-16T10:45:28.065 INFO:tasks.workunit.client.1.smithi026.stdout:6/552: write d0/d28/d49/f78 [860702,91105] 0 2024-03-16T10:45:28.068 INFO:tasks.workunit.client.1.smithi026.stdout:6/553: readlink d0/l2b 0 2024-03-16T10:45:28.072 INFO:tasks.workunit.client.1.smithi026.stdout:9/763: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:28.079 INFO:tasks.workunit.client.1.smithi026.stdout:4/710: dread d1/db/fc [0,4194304] 0 2024-03-16T10:45:28.079 INFO:tasks.workunit.client.1.smithi026.stdout:3/721: dread d4/fe [0,4194304] 0 2024-03-16T10:45:28.081 INFO:tasks.workunit.client.1.smithi026.stdout:3/722: write d4/dc/d73/d83/d7c/dd7/d62/d4f/ff9 [992834,120343] 0 2024-03-16T10:45:28.110 INFO:tasks.workunit.client.1.smithi026.stdout:1/626: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/dc2 0 2024-03-16T10:45:28.113 INFO:tasks.workunit.client.1.smithi026.stdout:7/713: sync 2024-03-16T10:45:28.115 INFO:tasks.workunit.client.1.smithi026.stdout:4/711: truncate d1/f77 473020 0 2024-03-16T10:45:28.116 INFO:tasks.workunit.client.1.smithi026.stdout:1/627: dwrite d0/d17/d1f/d63/d6a/d7c/f82 [4194304,4194304] 0 2024-03-16T10:45:28.117 INFO:tasks.workunit.client.1.smithi026.stdout:4/712: write d1/f2d [154906,127755] 0 2024-03-16T10:45:28.128 INFO:tasks.workunit.client.1.smithi026.stdout:7/714: rmdir d6/d2f/dad 39 2024-03-16T10:45:28.130 INFO:tasks.workunit.client.1.smithi026.stdout:4/713: stat d1/db/de/d4c/d6b/d7a/c86 0 2024-03-16T10:45:28.131 INFO:tasks.workunit.client.1.smithi026.stdout:8/641: mkdir d5/d7/d7b/db2/dbb 0 2024-03-16T10:45:28.135 INFO:tasks.workunit.client.1.smithi026.stdout:7/715: creat d6/d35/d3a/d4e/db0/fe7 x:0 0 0 2024-03-16T10:45:28.137 INFO:tasks.workunit.client.1.smithi026.stdout:6/554: link d0/d10/d79/f8d d0/fbd 0 2024-03-16T10:45:28.138 INFO:tasks.workunit.client.1.smithi026.stdout:7/716: truncate d6/de/fcd 110765 0 2024-03-16T10:45:28.138 INFO:tasks.workunit.client.1.smithi026.stdout:8/642: dwrite d5/d5d/d9a/f9b [0,4194304] 0 2024-03-16T10:45:28.138 INFO:tasks.workunit.client.1.smithi026.stdout:9/764: link d0/da/d13/f55 d0/da/d16/d17/f104 0 2024-03-16T10:45:28.141 INFO:tasks.workunit.client.1.smithi026.stdout:1/628: creat d0/d17/d1f/d63/fc3 x:0 0 0 2024-03-16T10:45:28.143 INFO:tasks.workunit.client.1.smithi026.stdout:6/555: mkdir d0/d41/dbe 0 2024-03-16T10:45:28.144 INFO:tasks.workunit.client.1.smithi026.stdout:3/723: write d4/d89/f8f [3274578,130413] 0 2024-03-16T10:45:28.157 INFO:tasks.workunit.client.1.smithi026.stdout:6/556: dread d0/d28/d2d/f4c [0,4194304] 0 2024-03-16T10:45:28.158 INFO:tasks.workunit.client.1.smithi026.stdout:7/717: mkdir d6/d35/d3a/d4e/dcc/d3f/dc5/de8 0 2024-03-16T10:45:28.160 INFO:tasks.workunit.client.1.smithi026.stdout:9/765: write d0/da/d4d/f8f [331202,23038] 0 2024-03-16T10:45:28.162 INFO:tasks.workunit.client.1.smithi026.stdout:9/766: write d0/da/d16/d17/d44/fdf [1529002,87982] 0 2024-03-16T10:45:28.163 INFO:tasks.workunit.client.1.smithi026.stdout:7/718: dread d6/d35/d3a/fda [0,4194304] 0 2024-03-16T10:45:28.165 INFO:tasks.workunit.client.1.smithi026.stdout:8/643: creat d5/fbc x:0 0 0 2024-03-16T10:45:28.166 INFO:tasks.workunit.client.1.smithi026.stdout:3/724: symlink d4/dc/d31/dbe/de0/l101 0 2024-03-16T10:45:28.166 INFO:tasks.workunit.client.1.smithi026.stdout:1/629: write d0/fa6 [490819,81202] 0 2024-03-16T10:45:28.168 INFO:tasks.workunit.client.1.smithi026.stdout:6/557: creat d0/d62/fbf x:0 0 0 2024-03-16T10:45:28.169 INFO:tasks.workunit.client.1.smithi026.stdout:6/558: chown d0/f9a 6 1 2024-03-16T10:45:28.176 INFO:tasks.workunit.client.1.smithi026.stdout:1/630: truncate d0/d8b/fb6 5765160 0 2024-03-16T10:45:28.177 INFO:tasks.workunit.client.1.smithi026.stdout:3/725: rename l2 to d4/dc/d31/dbe/l102 0 2024-03-16T10:45:28.181 INFO:tasks.workunit.client.1.smithi026.stdout:6/559: write d0/d10/d79/d7b/f7e [3710514,21101] 0 2024-03-16T10:45:28.252 INFO:tasks.workunit.client.1.smithi026.stdout:8/644: creat d5/dc/d3f/fbd x:0 0 0 2024-03-16T10:45:28.256 INFO:tasks.workunit.client.1.smithi026.stdout:3/726: mknod d4/dc/dd/d47/d53/c103 0 2024-03-16T10:45:28.257 INFO:tasks.workunit.client.1.smithi026.stdout:3/727: stat d4/dc/dd/d47/l63 0 2024-03-16T10:45:28.267 INFO:tasks.workunit.client.1.smithi026.stdout:7/719: rmdir d6/d35/d3a/d4e/dcc/d3f/d99 0 2024-03-16T10:45:28.268 INFO:tasks.workunit.client.1.smithi026.stdout:6/560: dread - d0/d62/f6d zero size 2024-03-16T10:45:28.274 INFO:tasks.workunit.client.1.smithi026.stdout:6/561: dwrite d0/d5b/f30 [0,4194304] 0 2024-03-16T10:45:28.285 INFO:tasks.workunit.client.1.smithi026.stdout:9/767: getdents d0 0 2024-03-16T10:45:28.290 INFO:tasks.workunit.client.1.smithi026.stdout:9/768: truncate d0/da/d4d/f71 1000872 0 2024-03-16T10:45:28.296 INFO:tasks.workunit.client.1.smithi026.stdout:5/663: sync 2024-03-16T10:45:28.296 INFO:tasks.workunit.client.1.smithi026.stdout:0/740: sync 2024-03-16T10:45:28.297 INFO:tasks.workunit.client.1.smithi026.stdout:4/714: sync 2024-03-16T10:45:28.297 INFO:tasks.workunit.client.1.smithi026.stdout:2/723: sync 2024-03-16T10:45:28.298 INFO:tasks.workunit.client.1.smithi026.stdout:2/724: stat d2/d29/d17/d38/f81 0 2024-03-16T10:45:28.298 INFO:tasks.workunit.client.1.smithi026.stdout:5/664: dread - d1/d6e/fd7 zero size 2024-03-16T10:45:28.299 INFO:tasks.workunit.client.1.smithi026.stdout:5/665: chown d1/d1c/d2a/dab 90346998 1 2024-03-16T10:45:28.300 INFO:tasks.workunit.client.1.smithi026.stdout:2/725: write d2/f8f [667476,107785] 0 2024-03-16T10:45:28.300 INFO:tasks.workunit.client.1.smithi026.stdout:0/741: chown d1/f40 7428 1 2024-03-16T10:45:28.301 INFO:tasks.workunit.client.1.smithi026.stdout:2/726: chown d2/d29/d17/f2d 2323 1 2024-03-16T10:45:28.302 INFO:tasks.workunit.client.1.smithi026.stdout:9/769: mknod d0/da/d14/c105 0 2024-03-16T10:45:28.307 INFO:tasks.workunit.client.1.smithi026.stdout:9/770: creat d0/da/d4d/f106 x:0 0 0 2024-03-16T10:45:28.308 INFO:tasks.workunit.client.1.smithi026.stdout:0/742: dwrite d1/d12/d2f/d48/d55/d60/f70 [0,4194304] 0 2024-03-16T10:45:28.309 INFO:tasks.workunit.client.1.smithi026.stdout:2/727: rename d2/d29/d17/d1a/d2b/d4e/d67/c7e to d2/d29/d17/d38/cf8 0 2024-03-16T10:45:28.310 INFO:tasks.workunit.client.1.smithi026.stdout:2/728: chown d2/d29/d36/d98 99266 1 2024-03-16T10:45:28.315 INFO:tasks.workunit.client.1.smithi026.stdout:7/720: mkdir d6/d2f/dad/de9 0 2024-03-16T10:45:28.316 INFO:tasks.workunit.client.1.smithi026.stdout:9/771: dwrite d0/f4 [0,4194304] 0 2024-03-16T10:45:28.318 INFO:tasks.workunit.client.1.smithi026.stdout:2/729: mknod d2/d29/d36/da7/cf9 0 2024-03-16T10:45:28.321 INFO:tasks.workunit.client.1.smithi026.stdout:7/721: mknod d6/d35/d3a/d4e/dcc/d73/dbf/cea 0 2024-03-16T10:45:28.322 INFO:tasks.workunit.client.1.smithi026.stdout:7/722: dread - d6/d35/d3a/d4e/d6c/fc9 zero size 2024-03-16T10:45:28.323 INFO:tasks.workunit.client.1.smithi026.stdout:9/772: mkdir d0/da/d16/d5d/d107 0 2024-03-16T10:45:28.325 INFO:tasks.workunit.client.1.smithi026.stdout:2/730: creat d2/d29/d36/d98/ffa x:0 0 0 2024-03-16T10:45:28.326 INFO:tasks.workunit.client.1.smithi026.stdout:2/731: write d2/d29/d85/fc7 [1326514,88840] 0 2024-03-16T10:45:28.327 INFO:tasks.workunit.client.1.smithi026.stdout:7/723: truncate d6/d35/d3a/d59/f96 908131 0 2024-03-16T10:45:28.330 INFO:tasks.workunit.client.1.smithi026.stdout:7/724: fdatasync d6/f39 0 2024-03-16T10:45:28.333 INFO:tasks.workunit.client.1.smithi026.stdout:2/732: mkdir d2/d29/d3b/dbb/dfb 0 2024-03-16T10:45:28.338 INFO:tasks.workunit.client.1.smithi026.stdout:2/733: truncate d2/d29/d17/d1a/d2b/d4e/fcc 5096331 0 2024-03-16T10:45:28.345 INFO:tasks.workunit.client.1.smithi026.stdout:6/562: rmdir d0/d10/d79/d91 39 2024-03-16T10:45:28.358 INFO:tasks.workunit.client.1.smithi026.stdout:8/645: rmdir d5 39 2024-03-16T10:45:28.364 INFO:tasks.workunit.client.1.smithi026.stdout:8/646: creat d5/d7/d7b/db2/dbb/fbe x:0 0 0 2024-03-16T10:45:28.366 INFO:tasks.workunit.client.1.smithi026.stdout:7/725: rename d6/d35/f7e to d6/d35/d3a/d4e/dcc/d73/dbf/feb 0 2024-03-16T10:45:28.367 INFO:tasks.workunit.client.1.smithi026.stdout:8/647: symlink d5/d23/d4e/d69/lbf 0 2024-03-16T10:45:28.369 INFO:tasks.workunit.client.1.smithi026.stdout:2/734: rename d2/d29/d17/d1a/d2b/d4e/d6f/ff7 to d2/d29/da4/ffc 0 2024-03-16T10:45:28.370 INFO:tasks.workunit.client.1.smithi026.stdout:7/726: mknod d6/d35/d3a/d59/cec 0 2024-03-16T10:45:28.372 INFO:tasks.workunit.client.1.smithi026.stdout:2/735: write d2/d29/d17/d1a/d2b/d4e/d6f/fcf [717402,54544] 0 2024-03-16T10:45:28.374 INFO:tasks.workunit.client.1.smithi026.stdout:2/736: read d2/d29/d17/f2d [33469,42979] 0 2024-03-16T10:45:28.376 INFO:tasks.workunit.client.1.smithi026.stdout:2/737: chown d2/d29/d17/d38/d64/dcb/la9 1 1 2024-03-16T10:45:28.381 INFO:tasks.workunit.client.1.smithi026.stdout:2/738: creat d2/d29/d51/d72/ffd x:0 0 0 2024-03-16T10:45:28.383 INFO:tasks.workunit.client.1.smithi026.stdout:3/728: mkdir d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/d104 0 2024-03-16T10:45:28.383 INFO:tasks.workunit.client.1.smithi026.stdout:3/729: readlink d4/dc/d31/l7a 0 2024-03-16T10:45:28.384 INFO:tasks.workunit.client.1.smithi026.stdout:3/730: write d4/dc/dd/fb4 [244669,88231] 0 2024-03-16T10:45:28.401 INFO:tasks.workunit.client.1.smithi026.stdout:0/743: dwrite d1/d12/d2f/dba/fc7 [0,4194304] 0 2024-03-16T10:45:28.404 INFO:tasks.workunit.client.1.smithi026.stdout:0/744: write d1/d12/d2f/d48/d55/d60/d9b/d11/fb7 [881645,25983] 0 2024-03-16T10:45:28.409 INFO:tasks.workunit.client.1.smithi026.stdout:0/745: rename d1/d12/d2f/daa/le5 to d1/d12/d2f/daa/dac/lfa 0 2024-03-16T10:45:28.410 INFO:tasks.workunit.client.1.smithi026.stdout:0/746: readlink d1/d12/d2f/daa/dd1/d61/l8c 0 2024-03-16T10:45:28.411 INFO:tasks.workunit.client.1.smithi026.stdout:6/563: write d0/f51 [1060353,56754] 0 2024-03-16T10:45:28.413 INFO:tasks.workunit.client.1.smithi026.stdout:4/715: creat d1/db/de/d4c/d6b/fde x:0 0 0 2024-03-16T10:45:28.414 INFO:tasks.workunit.client.1.smithi026.stdout:4/716: chown d1/db/de/l51 124 1 2024-03-16T10:45:28.418 INFO:tasks.workunit.client.1.smithi026.stdout:5/666: creat d1/db/d1f/d61/fec x:0 0 0 2024-03-16T10:45:28.419 INFO:tasks.workunit.client.1.smithi026.stdout:4/717: creat d1/db/ddb/daa/d44/d48/d8e/fdf x:0 0 0 2024-03-16T10:45:28.423 INFO:tasks.workunit.client.1.smithi026.stdout:1/631: unlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/fb4 0 2024-03-16T10:45:28.425 INFO:tasks.workunit.client.1.smithi026.stdout:4/718: chown d1/db/ddb/c8f 661 1 2024-03-16T10:45:28.426 INFO:tasks.workunit.client.1.smithi026.stdout:0/747: getdents d1/d12/d2f/d48/d55/d60/d9b/d57/dd7 0 2024-03-16T10:45:28.430 INFO:tasks.workunit.client.1.smithi026.stdout:1/632: fsync d0/d17/d1f/d20/d65/f88 0 2024-03-16T10:45:28.434 INFO:tasks.workunit.client.1.smithi026.stdout:4/719: symlink d1/db/le0 0 2024-03-16T10:45:28.436 INFO:tasks.workunit.client.1.smithi026.stdout:8/648: truncate d5/d28/d90/f91 3721924 0 2024-03-16T10:45:28.436 INFO:tasks.workunit.client.1.smithi026.stdout:8/649: chown d5/d28/lb6 65635 1 2024-03-16T10:45:28.438 INFO:tasks.workunit.client.1.smithi026.stdout:5/667: link d1/d1c/d2a/d8e/cb3 d1/d1c/d4c/d4d/ced 0 2024-03-16T10:45:28.439 INFO:tasks.workunit.client.1.smithi026.stdout:1/633: readlink d0/l77 0 2024-03-16T10:45:28.445 INFO:tasks.workunit.client.1.smithi026.stdout:7/727: unlink d6/d2f/f5f 0 2024-03-16T10:45:28.446 INFO:tasks.workunit.client.1.smithi026.stdout:3/731: dwrite d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/fe2 [0,4194304] 0 2024-03-16T10:45:28.447 INFO:tasks.workunit.client.1.smithi026.stdout:3/732: dread - d4/dc/dd/d47/d53/ff8 zero size 2024-03-16T10:45:28.448 INFO:tasks.workunit.client.1.smithi026.stdout:8/650: symlink d5/d7/d7b/db2/lc0 0 2024-03-16T10:45:28.453 INFO:tasks.workunit.client.1.smithi026.stdout:4/720: symlink d1/db/ddb/daa/d44/dd5/le1 0 2024-03-16T10:45:28.456 INFO:tasks.workunit.client.1.smithi026.stdout:2/739: dwrite d2/d29/d17/d1a/d2b/d4e/fcc [0,4194304] 0 2024-03-16T10:45:28.457 INFO:tasks.workunit.client.1.smithi026.stdout:2/740: write d2/d29/d17/d1a/fad [3061371,34536] 0 2024-03-16T10:45:28.460 INFO:tasks.workunit.client.1.smithi026.stdout:7/728: symlink d6/de/d7d/led 0 2024-03-16T10:45:28.461 INFO:tasks.workunit.client.1.smithi026.stdout:4/721: dwrite d1/db/ddb/daa/f95 [0,4194304] 0 2024-03-16T10:45:28.464 INFO:tasks.workunit.client.1.smithi026.stdout:4/722: stat d1/db/c7d 0 2024-03-16T10:45:28.469 INFO:tasks.workunit.client.1.smithi026.stdout:7/729: dread d6/de/d7d/f86 [0,4194304] 0 2024-03-16T10:45:28.469 INFO:tasks.workunit.client.1.smithi026.stdout:6/564: dwrite d0/d28/d2d/f68 [0,4194304] 0 2024-03-16T10:45:28.470 INFO:tasks.workunit.client.1.smithi026.stdout:2/741: dread d2/d29/d51/d55/d6d/fb9 [0,4194304] 0 2024-03-16T10:45:28.473 INFO:tasks.workunit.client.1.smithi026.stdout:5/668: symlink d1/d1c/lee 0 2024-03-16T10:45:28.494 INFO:tasks.workunit.client.1.smithi026.stdout:8/651: unlink d5/d7/f57 0 2024-03-16T10:45:28.495 INFO:tasks.workunit.client.1.smithi026.stdout:8/652: chown d5/dc/d89 20 1 2024-03-16T10:45:28.499 INFO:tasks.workunit.client.1.smithi026.stdout:6/565: mknod d0/d41/d71/cc0 0 2024-03-16T10:45:28.503 INFO:tasks.workunit.client.1.smithi026.stdout:2/742: unlink d2/d29/d17/d1a/d2b/d4e/d6f/fd6 0 2024-03-16T10:45:28.506 INFO:tasks.workunit.client.1.smithi026.stdout:3/733: unlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f4a 0 2024-03-16T10:45:28.507 INFO:tasks.workunit.client.1.smithi026.stdout:1/634: link d0/d17/d1f/c15 d0/d17/d1f/d63/d58/dc1/cc4 0 2024-03-16T10:45:28.509 INFO:tasks.workunit.client.1.smithi026.stdout:0/748: link d1/d12/d2f/daa/dd1/d1a/d2b/c9f d1/d12/d2f/daa/dd1/d1a/cfb 0 2024-03-16T10:45:28.516 INFO:tasks.workunit.client.1.smithi026.stdout:6/566: mknod d0/d10/d79/d7b/cc1 0 2024-03-16T10:45:28.518 INFO:tasks.workunit.client.1.smithi026.stdout:4/723: dwrite d1/db/fc [0,4194304] 0 2024-03-16T10:45:28.521 INFO:tasks.workunit.client.1.smithi026.stdout:2/743: symlink d2/d29/d17/d38/d64/dcb/lfe 0 2024-03-16T10:45:28.524 INFO:tasks.workunit.client.1.smithi026.stdout:0/749: dread d1/d12/d2f/d48/faf [0,4194304] 0 2024-03-16T10:45:28.524 INFO:tasks.workunit.client.1.smithi026.stdout:1/635: unlink d0/d17/d4d/f6d 0 2024-03-16T10:45:28.530 INFO:tasks.workunit.client.1.smithi026.stdout:6/567: rmdir d0/d41/d4f 39 2024-03-16T10:45:28.531 INFO:tasks.workunit.client.1.smithi026.stdout:6/568: write d0/f44 [553742,79989] 0 2024-03-16T10:45:28.532 INFO:tasks.workunit.client.1.smithi026.stdout:6/569: chown d0/d5b/c3b 189740 1 2024-03-16T10:45:28.538 INFO:tasks.workunit.client.1.smithi026.stdout:2/744: creat d2/d29/d17/d1a/d2b/d4e/fff x:0 0 0 2024-03-16T10:45:28.540 INFO:tasks.workunit.client.1.smithi026.stdout:0/750: mknod d1/d12/d2f/d48/d55/d60/cfc 0 2024-03-16T10:45:28.542 INFO:tasks.workunit.client.1.smithi026.stdout:1/636: rename d0/d17/d4d/f98 to d0/d17/d1f/d63/d6a/d7c/d84/fc5 0 2024-03-16T10:45:28.544 INFO:tasks.workunit.client.1.smithi026.stdout:3/734: dwrite d4/dc/f13 [0,4194304] 0 2024-03-16T10:45:28.546 INFO:tasks.workunit.client.1.smithi026.stdout:8/653: write d5/d7/d7b/f94 [1800878,25924] 0 2024-03-16T10:45:28.552 INFO:tasks.workunit.client.1.smithi026.stdout:7/730: getdents d6/d35/d3a/d4e/dcc/d3f/dc5 0 2024-03-16T10:45:28.554 INFO:tasks.workunit.client.1.smithi026.stdout:3/735: dwrite d4/f9d [0,4194304] 0 2024-03-16T10:45:28.557 INFO:tasks.workunit.client.1.smithi026.stdout:2/745: rmdir d2 39 2024-03-16T10:45:28.559 INFO:tasks.workunit.client.1.smithi026.stdout:4/724: dwrite d1/db/de/f3c [0,4194304] 0 2024-03-16T10:45:28.561 INFO:tasks.workunit.client.1.smithi026.stdout:1/637: mkdir d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6 0 2024-03-16T10:45:28.562 INFO:tasks.workunit.client.1.smithi026.stdout:8/654: mknod d5/d23/d4e/d69/d76/cc1 0 2024-03-16T10:45:28.564 INFO:tasks.workunit.client.1.smithi026.stdout:6/570: dwrite d0/d28/f5d [0,4194304] 0 2024-03-16T10:45:28.568 INFO:tasks.workunit.client.1.smithi026.stdout:1/638: write d0/d17/d1f/d63/d6a/d7c/d2e/faf [385544,54889] 0 2024-03-16T10:45:28.570 INFO:tasks.workunit.client.1.smithi026.stdout:3/736: creat d4/dc/d73/d83/f105 x:0 0 0 2024-03-16T10:45:28.572 INFO:tasks.workunit.client.1.smithi026.stdout:6/571: chown d0/d10/f1a 0 1 2024-03-16T10:45:28.572 INFO:tasks.workunit.client.1.smithi026.stdout:8/655: write d5/d7/d2c/d36/d59/d93/da3/fa9 [343299,7921] 0 2024-03-16T10:45:28.572 INFO:tasks.workunit.client.1.smithi026.stdout:2/746: dread - d2/d29/d51/d55/fc9 zero size 2024-03-16T10:45:28.574 INFO:tasks.workunit.client.1.smithi026.stdout:6/572: chown d0/d41/c6e 0 1 2024-03-16T10:45:28.574 INFO:tasks.workunit.client.1.smithi026.stdout:0/751: creat d1/d12/d2f/d48/d55/d60/d9b/ffd x:0 0 0 2024-03-16T10:45:28.575 INFO:tasks.workunit.client.1.smithi026.stdout:1/639: write d0/d17/d1f/d63/d6a/d7c/d6b/f87 [1816348,90823] 0 2024-03-16T10:45:28.578 INFO:tasks.workunit.client.1.smithi026.stdout:1/640: stat d0/d17/d1f/d20/l5c 0 2024-03-16T10:45:28.580 INFO:tasks.workunit.client.1.smithi026.stdout:8/656: chown d5/d7/f11 306815709 1 2024-03-16T10:45:28.582 INFO:tasks.workunit.client.1.smithi026.stdout:0/752: rmdir d1/d12/d2f/daa/dac 39 2024-03-16T10:45:28.583 INFO:tasks.workunit.client.1.smithi026.stdout:4/725: write d1/db/f74 [1096545,101700] 0 2024-03-16T10:45:28.585 INFO:tasks.workunit.client.1.smithi026.stdout:2/747: symlink d2/d29/da4/l100 0 2024-03-16T10:45:28.587 INFO:tasks.workunit.client.1.smithi026.stdout:4/726: dread - d1/fd4 zero size 2024-03-16T10:45:28.588 INFO:tasks.workunit.client.1.smithi026.stdout:3/737: creat d4/dc/dd/f106 x:0 0 0 2024-03-16T10:45:28.590 INFO:tasks.workunit.client.1.smithi026.stdout:7/731: dwrite d6/d35/d3a/d4e/fbb [0,4194304] 0 2024-03-16T10:45:28.591 INFO:tasks.workunit.client.1.smithi026.stdout:1/641: write d0/fd [639664,114052] 0 2024-03-16T10:45:28.592 INFO:tasks.workunit.client.1.smithi026.stdout:6/573: dwrite d0/d28/d2d/f68 [0,4194304] 0 2024-03-16T10:45:28.594 INFO:tasks.workunit.client.1.smithi026.stdout:0/753: mkdir d1/d12/d2f/d48/dfe 0 2024-03-16T10:45:28.594 INFO:tasks.workunit.client.1.smithi026.stdout:6/574: stat d0/d62 0 2024-03-16T10:45:28.597 INFO:tasks.workunit.client.1.smithi026.stdout:2/748: mknod d2/d29/d51/d55/c101 0 2024-03-16T10:45:28.599 INFO:tasks.workunit.client.1.smithi026.stdout:7/732: creat d6/d35/d3a/d4e/dcc/d3f/d70/fee x:0 0 0 2024-03-16T10:45:28.616 INFO:tasks.workunit.client.1.smithi026.stdout:0/754: mknod d1/d12/d2f/cff 0 2024-03-16T10:45:28.617 INFO:tasks.workunit.client.1.smithi026.stdout:8/657: dread d5/d28/d90/f91 [0,4194304] 0 2024-03-16T10:45:28.619 INFO:tasks.workunit.client.1.smithi026.stdout:6/575: write d0/d62/f85 [1644147,43863] 0 2024-03-16T10:45:28.620 INFO:tasks.workunit.client.1.smithi026.stdout:7/733: creat d6/d35/d3a/d4e/dcc/d3f/d65/fef x:0 0 0 2024-03-16T10:45:28.621 INFO:tasks.workunit.client.1.smithi026.stdout:4/727: dread d1/db/de/d6e/fb6 [0,4194304] 0 2024-03-16T10:45:28.621 INFO:tasks.workunit.client.1.smithi026.stdout:6/576: chown d0/d10/fd 9606 1 2024-03-16T10:45:28.621 INFO:tasks.workunit.client.1.smithi026.stdout:1/642: write d0/d17/d1f/d63/d6a/d7c/d2e/f7e [62779,27736] 0 2024-03-16T10:45:28.622 INFO:tasks.workunit.client.1.smithi026.stdout:6/577: chown d0/f9a 9 1 2024-03-16T10:45:28.626 INFO:tasks.workunit.client.1.smithi026.stdout:8/658: rename d5/d7/d2c/d36/l98 to d5/d7/d2c/d36/d8c/lc2 0 2024-03-16T10:45:28.631 INFO:tasks.workunit.client.1.smithi026.stdout:0/755: truncate d1/d12/d2f/daa/dd1/f36 4974198 0 2024-03-16T10:45:28.635 INFO:tasks.workunit.client.1.smithi026.stdout:2/749: truncate d2/d29/d17/d1a/d2b/f1f 7445727 0 2024-03-16T10:45:28.636 INFO:tasks.workunit.client.1.smithi026.stdout:2/750: read - d2/d29/d17/d1a/d2b/d4e/fff zero size 2024-03-16T10:45:28.636 INFO:tasks.workunit.client.1.smithi026.stdout:6/578: chown d0/d41/d4f/c56 161004 1 2024-03-16T10:45:28.637 INFO:tasks.workunit.client.1.smithi026.stdout:6/579: write d0/f3 [1391629,2619] 0 2024-03-16T10:45:28.638 INFO:tasks.workunit.client.1.smithi026.stdout:0/756: unlink d1/d12/d2f/daa/dd1/d1a/f7a 0 2024-03-16T10:45:28.638 INFO:tasks.workunit.client.1.smithi026.stdout:4/728: dread d1/db/de/f2b [4194304,4194304] 0 2024-03-16T10:45:28.640 INFO:tasks.workunit.client.1.smithi026.stdout:8/659: creat d5/d5d/fc3 x:0 0 0 2024-03-16T10:45:28.643 INFO:tasks.workunit.client.1.smithi026.stdout:2/751: creat d2/d29/d36/da7/f102 x:0 0 0 2024-03-16T10:45:28.644 INFO:tasks.workunit.client.1.smithi026.stdout:8/660: dread d5/d5d/d9a/f9b [0,4194304] 0 2024-03-16T10:45:28.647 INFO:tasks.workunit.client.1.smithi026.stdout:4/729: mknod d1/d85/da2/ce2 0 2024-03-16T10:45:28.656 INFO:tasks.workunit.client.1.smithi026.stdout:4/730: creat d1/db/fe3 x:0 0 0 2024-03-16T10:45:28.659 INFO:tasks.workunit.client.1.smithi026.stdout:0/757: dread d1/d12/d2f/d48/d55/d60/f28 [4194304,4194304] 0 2024-03-16T10:45:28.660 INFO:tasks.workunit.client.1.smithi026.stdout:0/758: chown d1/d12/d2f/d48/d55/d60/d9b/d11/fed 1355 1 2024-03-16T10:45:28.664 INFO:tasks.workunit.client.1.smithi026.stdout:4/731: creat d1/db/ddb/daa/d44/fe4 x:0 0 0 2024-03-16T10:45:28.672 INFO:tasks.workunit.client.1.smithi026.stdout:4/732: write d1/fd4 [123602,114860] 0 2024-03-16T10:45:28.672 INFO:tasks.workunit.client.1.smithi026.stdout:2/752: link d2/d29/d3b/dbb/ld3 d2/d29/d36/l103 0 2024-03-16T10:45:28.673 INFO:tasks.workunit.client.1.smithi026.stdout:2/753: dread - d2/d29/d17/d1a/d2b/d4b/fda zero size 2024-03-16T10:45:28.674 INFO:tasks.workunit.client.1.smithi026.stdout:0/759: creat d1/d12/d2f/daa/dac/f100 x:0 0 0 2024-03-16T10:45:28.676 INFO:tasks.workunit.client.1.smithi026.stdout:2/754: read d2/d29/d17/d1a/d2b/d4e/d6f/fcf [71158,101872] 0 2024-03-16T10:45:28.677 INFO:tasks.workunit.client.1.smithi026.stdout:4/733: write d1/d85/fd8 [7504015,130957] 0 2024-03-16T10:45:28.678 INFO:tasks.workunit.client.1.smithi026.stdout:4/734: truncate d1/db/fe3 528847 0 2024-03-16T10:45:28.679 INFO:tasks.workunit.client.1.smithi026.stdout:4/735: chown d1/d85/da2/ce2 5711 1 2024-03-16T10:45:28.681 INFO:tasks.workunit.client.1.smithi026.stdout:4/736: read d1/db/f64 [764828,35306] 0 2024-03-16T10:45:28.684 INFO:tasks.workunit.client.1.smithi026.stdout:6/580: dwrite d0/d5b/f95 [0,4194304] 0 2024-03-16T10:45:28.685 INFO:tasks.workunit.client.1.smithi026.stdout:6/581: chown d0/d10/d79/fa6 4629811 1 2024-03-16T10:45:28.750 INFO:tasks.workunit.client.1.smithi026.stdout:9/773: sync 2024-03-16T10:45:28.751 INFO:tasks.workunit.client.1.smithi026.stdout:9/774: chown d0/da/d16/l93 225201 1 2024-03-16T10:45:28.763 INFO:tasks.workunit.client.1.smithi026.stdout:9/775: dread d0/da/d4d/f5e [4194304,4194304] 0 2024-03-16T10:45:28.764 INFO:tasks.workunit.client.1.smithi026.stdout:9/776: chown d0/da/d16/d17 2 1 2024-03-16T10:45:28.795 INFO:tasks.workunit.client.1.smithi026.stdout:4/737: fsync d1/db/de/d4c/d59/fc3 0 2024-03-16T10:45:28.796 INFO:tasks.workunit.client.1.smithi026.stdout:4/738: write d1/db/fc [2778194,8081] 0 2024-03-16T10:45:28.800 INFO:tasks.workunit.client.1.smithi026.stdout:2/755: mkdir d2/d29/d17/d1a/d104 0 2024-03-16T10:45:28.801 INFO:tasks.workunit.client.1.smithi026.stdout:2/756: chown d2/d29/d17/d1a/d2b/d4e/d6f/fcf 12709 1 2024-03-16T10:45:28.802 INFO:tasks.workunit.client.1.smithi026.stdout:2/757: fdatasync d2/d29/d17/f23 0 2024-03-16T10:45:28.809 INFO:tasks.workunit.client.1.smithi026.stdout:9/777: creat d0/d5f/f108 x:0 0 0 2024-03-16T10:45:28.811 INFO:tasks.workunit.client.1.smithi026.stdout:8/661: truncate d5/d7/d2c/f56 439258 0 2024-03-16T10:45:28.814 INFO:tasks.workunit.client.1.smithi026.stdout:8/662: mkdir d5/d28/d90/dc4 0 2024-03-16T10:45:28.814 INFO:tasks.workunit.client.1.smithi026.stdout:2/758: dread d2/d29/d17/d1a/d2b/d2e/fe4 [0,4194304] 0 2024-03-16T10:45:28.814 INFO:tasks.workunit.client.1.smithi026.stdout:8/663: readlink d5/dc/l73 0 2024-03-16T10:45:28.815 INFO:tasks.workunit.client.1.smithi026.stdout:2/759: write d2/d29/da4/fee [448594,76510] 0 2024-03-16T10:45:28.826 INFO:tasks.workunit.client.1.smithi026.stdout:8/664: dread d5/d7/f43 [0,4194304] 0 2024-03-16T10:45:28.828 INFO:tasks.workunit.client.1.smithi026.stdout:0/760: write d1/d12/d2f/daa/dd1/fb3 [3128819,99961] 0 2024-03-16T10:45:28.830 INFO:tasks.workunit.client.1.smithi026.stdout:0/761: chown d1/d12/d2f/daa/dd1/d1a/d2b/d4f/l90 1281 1 2024-03-16T10:45:28.835 INFO:tasks.workunit.client.1.smithi026.stdout:0/762: unlink d1/d12/d2f/cff 0 2024-03-16T10:45:28.838 INFO:tasks.workunit.client.1.smithi026.stdout:0/763: fsync d1/d12/fa9 0 2024-03-16T10:45:28.840 INFO:tasks.workunit.client.1.smithi026.stdout:0/764: write d1/d12/d2f/daa/dd1/fc2 [5486878,39522] 0 2024-03-16T10:45:28.844 INFO:tasks.workunit.client.1.smithi026.stdout:4/739: write d1/db/ddb/daa/d44/f5a [1796733,129387] 0 2024-03-16T10:45:28.849 INFO:tasks.workunit.client.1.smithi026.stdout:4/740: link d1/db/f42 d1/d85/da2/dca/fe5 0 2024-03-16T10:45:28.849 INFO:tasks.workunit.client.1.smithi026.stdout:0/765: dread d1/d12/d2f/daa/dd1/d1a/f46 [0,4194304] 0 2024-03-16T10:45:28.850 INFO:tasks.workunit.client.1.smithi026.stdout:4/741: chown d1/db/f7e 7679601 1 2024-03-16T10:45:28.853 INFO:tasks.workunit.client.1.smithi026.stdout:4/742: symlink d1/dcd/le6 0 2024-03-16T10:45:28.856 INFO:tasks.workunit.client.1.smithi026.stdout:0/766: dwrite d1/d12/d2f/daa/dd1/fc2 [4194304,4194304] 0 2024-03-16T10:45:28.862 INFO:tasks.workunit.client.1.smithi026.stdout:0/767: chown d1/d12/d2f/d48/d55/d60/d9b/d51/f5e 2043 1 2024-03-16T10:45:28.865 INFO:tasks.workunit.client.1.smithi026.stdout:2/760: write d2/d29/d17/d1a/d2b/d4e/f71 [728848,67585] 0 2024-03-16T10:45:28.869 INFO:tasks.workunit.client.1.smithi026.stdout:0/768: mkdir d1/d12/d2f/d48/d55/d60/de0/dea/d101 0 2024-03-16T10:45:28.871 INFO:tasks.workunit.client.1.smithi026.stdout:4/743: link d1/db/de/d6e/c87 d1/db/ddb/ce7 0 2024-03-16T10:45:28.873 INFO:tasks.workunit.client.1.smithi026.stdout:2/761: dwrite d2/d29/d17/d1a/d2b/d69/fdb [0,4194304] 0 2024-03-16T10:45:28.875 INFO:tasks.workunit.client.1.smithi026.stdout:4/744: write d1/db/de/d4c/f4d [1555312,59658] 0 2024-03-16T10:45:28.876 INFO:tasks.workunit.client.1.smithi026.stdout:0/769: truncate d1/d12/d2f/d48/d55/d60/d9b/d11/fed 222950 0 2024-03-16T10:45:28.877 INFO:tasks.workunit.client.1.smithi026.stdout:2/762: chown d2/d29/d36/d98/cb3 1215 1 2024-03-16T10:45:28.879 INFO:tasks.workunit.client.1.smithi026.stdout:2/763: chown d2/d29/d17/d1a/d2b/d2e/d60/d7f/fb4 1205 1 2024-03-16T10:45:28.884 INFO:tasks.workunit.client.1.smithi026.stdout:4/745: creat d1/db/de/fe8 x:0 0 0 2024-03-16T10:45:28.885 INFO:tasks.workunit.client.1.smithi026.stdout:8/665: write d5/d7/d2c/f50 [1868802,22008] 0 2024-03-16T10:45:28.885 INFO:tasks.workunit.client.1.smithi026.stdout:4/746: write d1/f52 [2428291,928] 0 2024-03-16T10:45:28.886 INFO:tasks.workunit.client.1.smithi026.stdout:0/770: creat d1/d12/d2f/d48/d55/d60/de0/f102 x:0 0 0 2024-03-16T10:45:28.888 INFO:tasks.workunit.client.1.smithi026.stdout:4/747: chown d1/db/de/d4c/d6b/d7a 3807799 1 2024-03-16T10:45:28.892 INFO:tasks.workunit.client.1.smithi026.stdout:8/666: mknod d5/d7/d2c/d36/d59/d93/cc5 0 2024-03-16T10:45:28.893 INFO:tasks.workunit.client.1.smithi026.stdout:4/748: mkdir d1/db/ddb/daa/d44/dd5/de9 0 2024-03-16T10:45:28.893 INFO:tasks.workunit.client.1.smithi026.stdout:4/749: readlink d1/l96 0 2024-03-16T10:45:28.893 INFO:tasks.workunit.client.1.smithi026.stdout:8/667: chown d5/d7/d2c/d36/d59/d93/da3/ca7 265 1 2024-03-16T10:45:28.894 INFO:tasks.workunit.client.1.smithi026.stdout:5/669: sync 2024-03-16T10:45:28.894 INFO:tasks.workunit.client.1.smithi026.stdout:8/668: chown d5/dc/d40 2798 1 2024-03-16T10:45:28.894 INFO:tasks.workunit.client.1.smithi026.stdout:3/738: sync 2024-03-16T10:45:28.896 INFO:tasks.workunit.client.1.smithi026.stdout:5/670: read d1/d6e/f92 [18066,5407] 0 2024-03-16T10:45:28.896 INFO:tasks.workunit.client.1.smithi026.stdout:3/739: write d4/f9d [4067370,10174] 0 2024-03-16T10:45:28.909 INFO:tasks.workunit.client.1.smithi026.stdout:2/764: dread d2/d29/d17/d1a/f39 [0,4194304] 0 2024-03-16T10:45:28.910 INFO:tasks.workunit.client.1.smithi026.stdout:5/671: dwrite d1/db/d27/fd5 [0,4194304] 0 2024-03-16T10:45:28.913 INFO:tasks.workunit.client.1.smithi026.stdout:3/740: rmdir d4/dc/dd 39 2024-03-16T10:45:28.919 INFO:tasks.workunit.client.1.smithi026.stdout:0/771: dread d1/d12/d2f/d48/d55/d60/fa0 [0,4194304] 0 2024-03-16T10:45:28.931 INFO:tasks.workunit.client.1.smithi026.stdout:5/672: unlink d1/d1c/fb8 0 2024-03-16T10:45:28.935 INFO:tasks.workunit.client.1.smithi026.stdout:3/741: write d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f3e [4724279,12309] 0 2024-03-16T10:45:28.941 INFO:tasks.workunit.client.1.smithi026.stdout:2/765: mkdir d2/d29/d17/d1a/d104/d105 0 2024-03-16T10:45:28.945 INFO:tasks.workunit.client.1.smithi026.stdout:3/742: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/f107 x:0 0 0 2024-03-16T10:45:28.946 INFO:tasks.workunit.client.1.smithi026.stdout:4/750: getdents d1/db/de/d4c/d6b/d89/da5 0 2024-03-16T10:45:28.950 INFO:tasks.workunit.client.1.smithi026.stdout:2/766: unlink d2/d29/d17/le0 0 2024-03-16T10:45:28.951 INFO:tasks.workunit.client.1.smithi026.stdout:3/743: read d4/dc/f66 [1961490,59082] 0 2024-03-16T10:45:28.953 INFO:tasks.workunit.client.1.smithi026.stdout:8/669: dwrite d5/d7/f8d [0,4194304] 0 2024-03-16T10:45:28.957 INFO:tasks.workunit.client.1.smithi026.stdout:5/673: write d1/db/f59 [1575550,45418] 0 2024-03-16T10:45:28.959 INFO:tasks.workunit.client.1.smithi026.stdout:2/767: dread d2/d29/d51/d55/d6d/fb9 [0,4194304] 0 2024-03-16T10:45:28.961 INFO:tasks.workunit.client.1.smithi026.stdout:2/768: write d2/d29/d17/d1a/d2b/d4e/d67/f78 [4162233,71185] 0 2024-03-16T10:45:28.964 INFO:tasks.workunit.client.1.smithi026.stdout:4/751: creat d1/db/ddb/daa/d44/dd5/de9/fea x:0 0 0 2024-03-16T10:45:28.965 INFO:tasks.workunit.client.1.smithi026.stdout:4/752: write d1/db/fe3 [927035,126953] 0 2024-03-16T10:45:28.967 INFO:tasks.workunit.client.1.smithi026.stdout:3/744: rename d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/d94/l98 to d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/l108 0 2024-03-16T10:45:28.971 INFO:tasks.workunit.client.1.smithi026.stdout:2/769: symlink d2/d29/d51/d72/l106 0 2024-03-16T10:45:28.972 INFO:tasks.workunit.client.1.smithi026.stdout:5/674: dread d1/d1c/d2a/dab/dc5/d69/d51/d80/fac [0,4194304] 0 2024-03-16T10:45:28.974 INFO:tasks.workunit.client.1.smithi026.stdout:4/753: rmdir d1/db/de/d4c/d6b/d89/da5 39 2024-03-16T10:45:28.975 INFO:tasks.workunit.client.1.smithi026.stdout:4/754: chown d1/db/de/d4c/d59/d6a/ca0 853 1 2024-03-16T10:45:28.976 INFO:tasks.workunit.client.1.smithi026.stdout:4/755: fdatasync d1/db/de/fe8 0 2024-03-16T10:45:28.977 INFO:tasks.workunit.client.1.smithi026.stdout:4/756: chown d1/db/de/d4c/d6b/fde 51903393 1 2024-03-16T10:45:28.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:28 smithi026 ceph-mon[37041]: pgmap v9: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 39 MiB/s rd, 110 MiB/s wr, 288 op/s 2024-03-16T10:45:28.984 INFO:tasks.workunit.client.1.smithi026.stdout:8/670: dwrite d5/d23/d4e/d60/f84 [0,4194304] 0 2024-03-16T10:45:28.986 INFO:tasks.workunit.client.1.smithi026.stdout:2/770: mknod d2/d29/d17/d1a/d2b/d4e/d6f/c107 0 2024-03-16T10:45:28.995 INFO:tasks.workunit.client.1.smithi026.stdout:8/671: mknod d5/d23/d4e/d69/d76/cc6 0 2024-03-16T10:45:28.999 INFO:tasks.workunit.client.1.smithi026.stdout:2/771: mknod d2/d29/d17/d1a/d2b/d4b/c108 0 2024-03-16T10:45:29.002 INFO:tasks.workunit.client.1.smithi026.stdout:3/745: link d4/dc/d73/d83/d57/f9a d4/dc/d73/f109 0 2024-03-16T10:45:29.009 INFO:tasks.workunit.client.1.smithi026.stdout:2/772: creat d2/d29/d17/d1a/d104/f109 x:0 0 0 2024-03-16T10:45:29.012 INFO:tasks.workunit.client.1.smithi026.stdout:5/675: getdents d1/db/d1f/d61/d65 0 2024-03-16T10:45:29.013 INFO:tasks.workunit.client.1.smithi026.stdout:8/672: mknod d5/d23/d4e/d69/d76/da1/cc7 0 2024-03-16T10:45:29.014 INFO:tasks.workunit.client.1.smithi026.stdout:4/757: dwrite d1/db/ddb/daf/fb9 [0,4194304] 0 2024-03-16T10:45:29.016 INFO:tasks.workunit.client.1.smithi026.stdout:3/746: read - d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/f107 zero size 2024-03-16T10:45:29.018 INFO:tasks.workunit.client.1.smithi026.stdout:2/773: creat d2/d29/d17/d1a/d2b/d2e/d60/f10a x:0 0 0 2024-03-16T10:45:29.019 INFO:tasks.workunit.client.1.smithi026.stdout:8/673: rmdir d5/d7 39 2024-03-16T10:45:29.022 INFO:tasks.workunit.client.1.smithi026.stdout:5/676: read - d1/db/d27/f41 zero size 2024-03-16T10:45:29.024 INFO:tasks.workunit.client.1.smithi026.stdout:4/758: read - d1/db/de/d4c/d59/d6a/f82 zero size 2024-03-16T10:45:29.031 INFO:tasks.workunit.client.1.smithi026.stdout:8/674: write d5/d7/d2c/f50 [712056,21340] 0 2024-03-16T10:45:29.033 INFO:tasks.workunit.client.1.smithi026.stdout:8/675: read - d5/dc/d3f/fbd zero size 2024-03-16T10:45:29.037 INFO:tasks.workunit.client.1.smithi026.stdout:4/759: link d1/f22 d1/db/de/d6e/feb 0 2024-03-16T10:45:29.042 INFO:tasks.workunit.client.1.smithi026.stdout:5/677: dread d1/d1c/d2a/dab/dc5/f97 [0,4194304] 0 2024-03-16T10:45:29.043 INFO:tasks.workunit.client.1.smithi026.stdout:3/747: dread d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/fbf [0,4194304] 0 2024-03-16T10:45:29.044 INFO:tasks.workunit.client.1.smithi026.stdout:4/760: rename d1/db/ddb/daa/d44/lc6 to d1/db/ddb/daa/d44/dd5/lec 0 2024-03-16T10:45:29.046 INFO:tasks.workunit.client.1.smithi026.stdout:8/676: write d5/d23/d4e/f7c [1384461,36519] 0 2024-03-16T10:45:29.047 INFO:tasks.workunit.client.1.smithi026.stdout:2/774: dwrite d2/d29/d17/d1a/d2b/d4e/d6f/f9c [0,4194304] 0 2024-03-16T10:45:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:28 smithi002 ceph-mon[29248]: pgmap v9: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 39 MiB/s rd, 110 MiB/s wr, 288 op/s 2024-03-16T10:45:29.050 INFO:tasks.workunit.client.1.smithi026.stdout:3/748: creat d4/dc/d31/dbe/f10a x:0 0 0 2024-03-16T10:45:29.051 INFO:tasks.workunit.client.1.smithi026.stdout:5/678: dread d1/d6e/fb7 [0,4194304] 0 2024-03-16T10:45:29.051 INFO:tasks.workunit.client.1.smithi026.stdout:4/761: rename d1/fd4 to d1/d85/da2/da8/fed 0 2024-03-16T10:45:29.065 INFO:tasks.workunit.client.1.smithi026.stdout:2/775: creat d2/d29/d17/d1a/d2b/f10b x:0 0 0 2024-03-16T10:45:29.066 INFO:tasks.workunit.client.1.smithi026.stdout:3/749: write d4/dc/f19 [479380,76350] 0 2024-03-16T10:45:29.083 INFO:tasks.workunit.client.1.smithi026.stdout:1/643: sync 2024-03-16T10:45:29.083 INFO:tasks.workunit.client.1.smithi026.stdout:7/734: sync 2024-03-16T10:45:29.088 INFO:tasks.workunit.client.1.smithi026.stdout:3/750: mkdir d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d10b 0 2024-03-16T10:45:29.088 INFO:tasks.workunit.client.1.smithi026.stdout:3/751: read d4/f9d [1110529,102780] 0 2024-03-16T10:45:29.089 INFO:tasks.workunit.client.1.smithi026.stdout:5/679: getdents d1/d1c/d4c 0 2024-03-16T10:45:29.091 INFO:tasks.workunit.client.1.smithi026.stdout:1/644: mkdir d0/d17/d1f/dc7 0 2024-03-16T10:45:29.092 INFO:tasks.workunit.client.1.smithi026.stdout:1/645: write d0/d17/d4d/fae [719716,13493] 0 2024-03-16T10:45:29.095 INFO:tasks.workunit.client.1.smithi026.stdout:5/680: mknod d1/d6e/cef 0 2024-03-16T10:45:29.152 INFO:tasks.workunit.client.1.smithi026.stdout:3/752: truncate d4/d89/d97/fd6 947067 0 2024-03-16T10:45:29.155 INFO:tasks.workunit.client.1.smithi026.stdout:1/646: dwrite d0/fd [0,4194304] 0 2024-03-16T10:45:29.155 INFO:tasks.workunit.client.1.smithi026.stdout:5/681: dread d1/d1c/d2a/d55/f58 [0,4194304] 0 2024-03-16T10:45:29.157 INFO:tasks.workunit.client.1.smithi026.stdout:1/647: write d0/d17/d1f/d63/d58/f9a [5204776,120259] 0 2024-03-16T10:45:29.159 INFO:tasks.workunit.client.1.smithi026.stdout:1/648: chown d0/d17/d1f/l44 33 1 2024-03-16T10:45:29.172 INFO:tasks.workunit.client.1.smithi026.stdout:5/682: dread d1/d1c/d68/fa3 [0,4194304] 0 2024-03-16T10:45:29.173 INFO:tasks.workunit.client.1.smithi026.stdout:3/753: dwrite d4/dc/dd/d47/f86 [0,4194304] 0 2024-03-16T10:45:29.179 INFO:tasks.workunit.client.1.smithi026.stdout:3/754: chown d4/dc/d31/d48/c76 2 1 2024-03-16T10:45:29.180 INFO:tasks.workunit.client.1.smithi026.stdout:1/649: dwrite d0/d17/d1f/d63/d6a/d7c/d3b/f2f [0,4194304] 0 2024-03-16T10:45:29.180 INFO:tasks.workunit.client.1.smithi026.stdout:5/683: truncate d1/db/f28 3496177 0 2024-03-16T10:45:29.182 INFO:tasks.workunit.client.1.smithi026.stdout:5/684: stat d1/d1c/d4c 0 2024-03-16T10:45:29.189 INFO:tasks.workunit.client.1.smithi026.stdout:1/650: rename d0/d17/d4d to d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8 0 2024-03-16T10:45:29.189 INFO:tasks.workunit.client.1.smithi026.stdout:3/755: dwrite d4/dc/dd/d80/dc2/fda [0,4194304] 0 2024-03-16T10:45:29.190 INFO:tasks.workunit.client.1.smithi026.stdout:5/685: mknod d1/d1c/cf0 0 2024-03-16T10:45:29.191 INFO:tasks.workunit.client.1.smithi026.stdout:5/686: chown d1/d1c/lc1 486 1 2024-03-16T10:45:29.192 INFO:tasks.workunit.client.1.smithi026.stdout:1/651: dread d0/d17/d1f/d63/d6a/d7c/f59 [0,4194304] 0 2024-03-16T10:45:29.198 INFO:tasks.workunit.client.1.smithi026.stdout:3/756: dwrite d4/d89/d97/f3a [0,4194304] 0 2024-03-16T10:45:29.201 INFO:tasks.workunit.client.1.smithi026.stdout:1/652: write d0/f8f [4535211,119822] 0 2024-03-16T10:45:29.202 INFO:tasks.workunit.client.1.smithi026.stdout:1/653: chown d0/d17/d1f/d63/d6a/d7c/d84/fb0 88498567 1 2024-03-16T10:45:29.208 INFO:tasks.workunit.client.1.smithi026.stdout:5/687: creat d1/d1c/d2a/ff1 x:0 0 0 2024-03-16T10:45:29.213 INFO:tasks.workunit.client.1.smithi026.stdout:5/688: dread d1/d1c/d2a/d55/d99/f96 [0,4194304] 0 2024-03-16T10:45:29.215 INFO:tasks.workunit.client.1.smithi026.stdout:3/757: creat d4/dc/dd/d47/f10c x:0 0 0 2024-03-16T10:45:29.217 INFO:tasks.workunit.client.1.smithi026.stdout:1/654: creat d0/d17/d1f/fc9 x:0 0 0 2024-03-16T10:45:29.221 INFO:tasks.workunit.client.1.smithi026.stdout:3/758: dread d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f49 [4194304,4194304] 0 2024-03-16T10:45:29.224 INFO:tasks.workunit.client.1.smithi026.stdout:1/655: rmdir d0/d17/d1f/d63/d6a/d7c/d2e 39 2024-03-16T10:45:29.228 INFO:tasks.workunit.client.1.smithi026.stdout:3/759: dwrite d4/dc/d31/d48/d69/d78/fea [0,4194304] 0 2024-03-16T10:45:29.234 INFO:tasks.workunit.client.1.smithi026.stdout:3/760: dwrite d4/dc/dd/f75 [0,4194304] 0 2024-03-16T10:45:29.237 INFO:tasks.workunit.client.1.smithi026.stdout:1/656: getdents d0/d17/d1f/d63/d6a/d7c/d84/d9d 0 2024-03-16T10:45:29.237 INFO:tasks.workunit.client.1.smithi026.stdout:5/689: write d1/db/d1f/f5c [6922476,115067] 0 2024-03-16T10:45:29.238 INFO:tasks.workunit.client.1.smithi026.stdout:3/761: mknod d4/dc/d73/d83/c10d 0 2024-03-16T10:45:29.244 INFO:tasks.workunit.client.1.smithi026.stdout:1/657: dread d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/f9b [0,4194304] 0 2024-03-16T10:45:29.244 INFO:tasks.workunit.client.1.smithi026.stdout:5/690: creat d1/d1c/d2a/ff2 x:0 0 0 2024-03-16T10:45:29.245 INFO:tasks.workunit.client.1.smithi026.stdout:3/762: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/l10e 0 2024-03-16T10:45:29.246 INFO:tasks.workunit.client.1.smithi026.stdout:3/763: readlink d4/dc/dd/l26 0 2024-03-16T10:45:29.249 INFO:tasks.workunit.client.1.smithi026.stdout:3/764: symlink d4/dc/d73/d83/dbc/dd5/l10f 0 2024-03-16T10:45:29.250 INFO:tasks.workunit.client.1.smithi026.stdout:3/765: read - d4/dc/d31/d48/fe3 zero size 2024-03-16T10:45:29.252 INFO:tasks.workunit.client.1.smithi026.stdout:5/691: creat d1/d1c/ff3 x:0 0 0 2024-03-16T10:45:29.257 INFO:tasks.workunit.client.1.smithi026.stdout:5/692: symlink d1/db/d1f/d61/d65/lf4 0 2024-03-16T10:45:29.260 INFO:tasks.workunit.client.1.smithi026.stdout:3/766: getdents d4/dc/d31/d48/d69/d78/db3 0 2024-03-16T10:45:29.265 INFO:tasks.workunit.client.1.smithi026.stdout:3/767: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/d104/f110 x:0 0 0 2024-03-16T10:45:29.266 INFO:tasks.workunit.client.1.smithi026.stdout:3/768: stat d4/dc/d31/d48/d69/cc3 0 2024-03-16T10:45:29.268 INFO:tasks.workunit.client.1.smithi026.stdout:3/769: chown d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/d94/fa4 21352 1 2024-03-16T10:45:29.269 INFO:tasks.workunit.client.1.smithi026.stdout:5/693: dwrite d1/d1c/f50 [0,4194304] 0 2024-03-16T10:45:29.270 INFO:tasks.workunit.client.1.smithi026.stdout:3/770: write d4/dc/d31/d48/f9f [1861254,50838] 0 2024-03-16T10:45:29.274 INFO:tasks.workunit.client.1.smithi026.stdout:5/694: symlink d1/d1c/d2a/d55/db1/lf5 0 2024-03-16T10:45:29.276 INFO:tasks.workunit.client.1.smithi026.stdout:0/772: sync 2024-03-16T10:45:29.276 INFO:tasks.workunit.client.1.smithi026.stdout:9/778: sync 2024-03-16T10:45:29.276 INFO:tasks.workunit.client.1.smithi026.stdout:6/582: sync 2024-03-16T10:45:29.277 INFO:tasks.workunit.client.1.smithi026.stdout:9/779: write d0/da/f27 [1231177,102343] 0 2024-03-16T10:45:29.279 INFO:tasks.workunit.client.1.smithi026.stdout:0/773: dread d1/d12/d2f/daa/dd1/d1a/f46 [0,4194304] 0 2024-03-16T10:45:29.279 INFO:tasks.workunit.client.1.smithi026.stdout:6/583: mkdir d0/d28/d49/dc2 0 2024-03-16T10:45:29.280 INFO:tasks.workunit.client.1.smithi026.stdout:5/695: mkdir d1/db/d1f/d61/d65/df6 0 2024-03-16T10:45:29.286 INFO:tasks.workunit.client.1.smithi026.stdout:9/780: mkdir d0/dd7/dff/d109 0 2024-03-16T10:45:29.286 INFO:tasks.workunit.client.1.smithi026.stdout:0/774: symlink d1/d12/d2f/d48/d55/d60/d9b/dcf/df1/l103 0 2024-03-16T10:45:29.287 INFO:tasks.workunit.client.1.smithi026.stdout:9/781: write d0/d63/d6b/f77 [1458311,118675] 0 2024-03-16T10:45:29.290 INFO:tasks.workunit.client.1.smithi026.stdout:0/775: rmdir d1/d12/d2f/d48/d55/d60/d9b/d51 39 2024-03-16T10:45:29.292 INFO:tasks.workunit.client.1.smithi026.stdout:9/782: unlink d0/da/d16/d17/la4 0 2024-03-16T10:45:29.292 INFO:tasks.workunit.client.1.smithi026.stdout:3/771: dread d4/dc/f96 [0,4194304] 0 2024-03-16T10:45:29.294 INFO:tasks.workunit.client.1.smithi026.stdout:0/776: mkdir d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d104 0 2024-03-16T10:45:29.296 INFO:tasks.workunit.client.1.smithi026.stdout:9/783: creat d0/d63/d6b/f10a x:0 0 0 2024-03-16T10:45:29.299 INFO:tasks.workunit.client.1.smithi026.stdout:9/784: dread - d0/da/d16/d17/d44/fa3 zero size 2024-03-16T10:45:29.302 INFO:tasks.workunit.client.1.smithi026.stdout:6/584: dread d0/f1d [0,4194304] 0 2024-03-16T10:45:29.302 INFO:tasks.workunit.client.1.smithi026.stdout:0/777: dread d1/d12/d2f/d48/d55/d60/d9b/d57/fbb [0,4194304] 0 2024-03-16T10:45:29.304 INFO:tasks.workunit.client.1.smithi026.stdout:9/785: read d0/da/d16/d17/d44/f1e [534786,89066] 0 2024-03-16T10:45:29.309 INFO:tasks.workunit.client.1.smithi026.stdout:9/786: chown d0/d5f/da2/fa6 59 1 2024-03-16T10:45:29.310 INFO:tasks.workunit.client.1.smithi026.stdout:5/696: dwrite d1/d1c/d2a/dab/dc5/d69/d51/fa2 [0,4194304] 0 2024-03-16T10:45:29.311 INFO:tasks.workunit.client.1.smithi026.stdout:3/772: dread d4/dc/dd/f2e [0,4194304] 0 2024-03-16T10:45:29.312 INFO:tasks.workunit.client.1.smithi026.stdout:6/585: dread d0/d5b/f95 [0,4194304] 0 2024-03-16T10:45:29.313 INFO:tasks.workunit.client.1.smithi026.stdout:9/787: stat d0/d5f/d7f/d8c/fbd 0 2024-03-16T10:45:29.314 INFO:tasks.workunit.client.1.smithi026.stdout:9/788: chown d0/da/d16/d17/c47 2022 1 2024-03-16T10:45:29.315 INFO:tasks.workunit.client.1.smithi026.stdout:0/778: truncate d1/d12/d2f/d48/d55/d60/d9b/f82 547033 0 2024-03-16T10:45:29.316 INFO:tasks.workunit.client.1.smithi026.stdout:0/779: chown d1/d12/d2f/d48/d55/d60/f70 112095 1 2024-03-16T10:45:29.320 INFO:tasks.workunit.client.1.smithi026.stdout:3/773: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/l111 0 2024-03-16T10:45:29.323 INFO:tasks.workunit.client.1.smithi026.stdout:5/697: fdatasync d1/db/f44 0 2024-03-16T10:45:29.324 INFO:tasks.workunit.client.1.smithi026.stdout:5/698: chown d1/db/d1f/lbf 80302 1 2024-03-16T10:45:29.325 INFO:tasks.workunit.client.1.smithi026.stdout:9/789: rename d0/d5f/d7f/d8c/l97 to d0/d5f/d7f/d8c/l10b 0 2024-03-16T10:45:29.328 INFO:tasks.workunit.client.1.smithi026.stdout:3/774: unlink d4/dc/dd/l2b 0 2024-03-16T10:45:29.335 INFO:tasks.workunit.client.1.smithi026.stdout:9/790: creat d0/da/d14/da7/f10c x:0 0 0 2024-03-16T10:45:29.336 INFO:tasks.workunit.client.1.smithi026.stdout:3/775: dwrite d4/dc/dd/d80/fef [0,4194304] 0 2024-03-16T10:45:29.339 INFO:tasks.workunit.client.1.smithi026.stdout:6/586: creat d0/d10/fc3 x:0 0 0 2024-03-16T10:45:29.339 INFO:tasks.workunit.client.1.smithi026.stdout:0/780: dread d1/d12/d2f/f7b [0,4194304] 0 2024-03-16T10:45:29.343 INFO:tasks.workunit.client.1.smithi026.stdout:5/699: mknod d1/d1c/cf7 0 2024-03-16T10:45:29.347 INFO:tasks.workunit.client.1.smithi026.stdout:0/781: dread d1/d12/d2f/daa/dd1/d1a/f46 [0,4194304] 0 2024-03-16T10:45:29.347 INFO:tasks.workunit.client.1.smithi026.stdout:3/776: truncate d4/dc/d31/f51 1165202 0 2024-03-16T10:45:29.348 INFO:tasks.workunit.client.1.smithi026.stdout:6/587: dwrite d0/f44 [0,4194304] 0 2024-03-16T10:45:29.350 INFO:tasks.workunit.client.1.smithi026.stdout:6/588: chown d0/d62/l73 16128 1 2024-03-16T10:45:29.351 INFO:tasks.workunit.client.1.smithi026.stdout:6/589: truncate d0/f5f 1144584 0 2024-03-16T10:45:29.354 INFO:tasks.workunit.client.1.smithi026.stdout:5/700: symlink d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/lf8 0 2024-03-16T10:45:29.359 INFO:tasks.workunit.client.1.smithi026.stdout:9/791: dread d0/dd7/f74 [0,4194304] 0 2024-03-16T10:45:29.360 INFO:tasks.workunit.client.1.smithi026.stdout:9/792: readlink d0/lf 0 2024-03-16T10:45:29.361 INFO:tasks.workunit.client.1.smithi026.stdout:5/701: mknod d1/d6e/cf9 0 2024-03-16T10:45:29.362 INFO:tasks.workunit.client.1.smithi026.stdout:5/702: chown d1/d1c/d2a/dab/dc5/d69/d51/d80/fa6 502201 1 2024-03-16T10:45:29.364 INFO:tasks.workunit.client.1.smithi026.stdout:0/782: rename d1/d12/d2f/d48/d55/d60/d9b/d11/cee to d1/c105 0 2024-03-16T10:45:29.367 INFO:tasks.workunit.client.1.smithi026.stdout:6/590: symlink d0/d10/d79/d8e/lc4 0 2024-03-16T10:45:29.369 INFO:tasks.workunit.client.1.smithi026.stdout:9/793: unlink d0/da/d4d/f8f 0 2024-03-16T10:45:29.373 INFO:tasks.workunit.client.1.smithi026.stdout:3/777: write d4/dc/d31/d48/d69/f6f [3714326,116432] 0 2024-03-16T10:45:29.375 INFO:tasks.workunit.client.1.smithi026.stdout:5/703: mknod d1/d1c/d2a/dab/de8/cfa 0 2024-03-16T10:45:29.377 INFO:tasks.workunit.client.1.smithi026.stdout:0/783: link d1/d12/d2f/daa/dd1/f97 d1/d12/d2f/d48/d55/d60/f106 0 2024-03-16T10:45:29.378 INFO:tasks.workunit.client.1.smithi026.stdout:0/784: stat d1/d12/d2f/d48/d55/d60/d9b/d11/c5b 0 2024-03-16T10:45:29.379 INFO:tasks.workunit.client.1.smithi026.stdout:9/794: creat d0/d63/d6b/ded/f10d x:0 0 0 2024-03-16T10:45:29.380 INFO:tasks.workunit.client.1.smithi026.stdout:9/795: read d0/da/d16/d17/fbc [55493,119749] 0 2024-03-16T10:45:29.396 INFO:tasks.workunit.client.1.smithi026.stdout:0/785: symlink d1/d12/d2f/d48/d55/d60/d9b/d51/d88/l107 0 2024-03-16T10:45:29.398 INFO:tasks.workunit.client.1.smithi026.stdout:5/704: dwrite d1/d1c/d2a/dab/dc5/d69/d51/f9b [0,4194304] 0 2024-03-16T10:45:29.398 INFO:tasks.workunit.client.1.smithi026.stdout:0/786: readlink d1/d12/d2f/daa/dd1/d1a/d2b/l30 0 2024-03-16T10:45:29.399 INFO:tasks.workunit.client.1.smithi026.stdout:5/705: chown d1/d1c/d2a/ff1 75156293 1 2024-03-16T10:45:29.401 INFO:tasks.workunit.client.1.smithi026.stdout:3/778: getdents d4/dc/d73/d83/dbc 0 2024-03-16T10:45:29.406 INFO:tasks.workunit.client.1.smithi026.stdout:0/787: dread d1/d12/d2f/daa/dd1/d1a/f2c [4194304,4194304] 0 2024-03-16T10:45:29.408 INFO:tasks.workunit.client.1.smithi026.stdout:5/706: dread d1/db/f7d [0,4194304] 0 2024-03-16T10:45:29.411 INFO:tasks.workunit.client.1.smithi026.stdout:9/796: write d0/da/d16/d17/d44/f2b [562812,107590] 0 2024-03-16T10:45:29.415 INFO:tasks.workunit.client.1.smithi026.stdout:9/797: symlink d0/da/d4d/l10e 0 2024-03-16T10:45:29.418 INFO:tasks.workunit.client.1.smithi026.stdout:5/707: link d1/db/f44 d1/d1c/d2a/d55/db1/ffb 0 2024-03-16T10:45:29.419 INFO:tasks.workunit.client.1.smithi026.stdout:0/788: dwrite d1/d12/d2f/d48/d55/d60/d9b/d57/fbb [0,4194304] 0 2024-03-16T10:45:29.420 INFO:tasks.workunit.client.1.smithi026.stdout:3/779: dread d4/dc/d73/d83/d7c/dd7/f64 [0,4194304] 0 2024-03-16T10:45:29.421 INFO:tasks.workunit.client.1.smithi026.stdout:9/798: symlink d0/d5f/da2/l10f 0 2024-03-16T10:45:29.426 INFO:tasks.workunit.client.1.smithi026.stdout:9/799: symlink d0/da/d16/l110 0 2024-03-16T10:45:29.429 INFO:tasks.workunit.client.1.smithi026.stdout:5/708: dwrite d1/d1c/d2a/dab/dc5/d69/d51/f7f [0,4194304] 0 2024-03-16T10:45:29.434 INFO:tasks.workunit.client.1.smithi026.stdout:9/800: rmdir d0/d63 39 2024-03-16T10:45:29.437 INFO:tasks.workunit.client.1.smithi026.stdout:5/709: creat d1/db/d27/ffc x:0 0 0 2024-03-16T10:45:29.437 INFO:tasks.workunit.client.1.smithi026.stdout:3/780: dread d4/f41 [0,4194304] 0 2024-03-16T10:45:29.438 INFO:tasks.workunit.client.1.smithi026.stdout:5/710: chown d1/d1c/d2a/d8e 2 1 2024-03-16T10:45:29.442 INFO:tasks.workunit.client.1.smithi026.stdout:5/711: creat d1/d1c/d2a/d55/d91/ffd x:0 0 0 2024-03-16T10:45:29.442 INFO:tasks.workunit.client.1.smithi026.stdout:9/801: mknod d0/da/d14/da7/c111 0 2024-03-16T10:45:29.445 INFO:tasks.workunit.client.1.smithi026.stdout:0/789: dwrite d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/fc3 [0,4194304] 0 2024-03-16T10:45:29.446 INFO:tasks.workunit.client.1.smithi026.stdout:0/790: chown d1/ce7 214 1 2024-03-16T10:45:29.449 INFO:tasks.workunit.client.1.smithi026.stdout:0/791: fsync d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/fd8 0 2024-03-16T10:45:29.451 INFO:tasks.workunit.client.1.smithi026.stdout:3/781: dread d4/dc/d73/d83/d7c/dd7/d62/d4f/ff9 [0,4194304] 0 2024-03-16T10:45:29.452 INFO:tasks.workunit.client.1.smithi026.stdout:0/792: write d1/d12/d2f/daa/dd1/d1a/d2b/d4f/fef [362937,48974] 0 2024-03-16T10:45:29.458 INFO:tasks.workunit.client.1.smithi026.stdout:9/802: mkdir d0/da/d16/d17/db1/dfa/d112 0 2024-03-16T10:45:29.461 INFO:tasks.workunit.client.1.smithi026.stdout:3/782: mknod d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/c112 0 2024-03-16T10:45:29.464 INFO:tasks.workunit.client.1.smithi026.stdout:5/712: getdents d1/d1c/d68 0 2024-03-16T10:45:29.468 INFO:tasks.workunit.client.1.smithi026.stdout:5/713: dread d1/d1c/d2a/d55/d99/fa5 [0,4194304] 0 2024-03-16T10:45:29.471 INFO:tasks.workunit.client.1.smithi026.stdout:9/803: creat d0/f113 x:0 0 0 2024-03-16T10:45:29.472 INFO:tasks.workunit.client.1.smithi026.stdout:0/793: link d1/d12/d2f/d48/d55/d60/d9b/f82 d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/f108 0 2024-03-16T10:45:29.479 INFO:tasks.workunit.client.1.smithi026.stdout:3/783: getdents d4/dc/d31/d48/d7f 0 2024-03-16T10:45:29.481 INFO:tasks.workunit.client.1.smithi026.stdout:3/784: fdatasync d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/fec 0 2024-03-16T10:45:29.483 INFO:tasks.workunit.client.1.smithi026.stdout:5/714: creat d1/d1c/d2a/ffe x:0 0 0 2024-03-16T10:45:29.485 INFO:tasks.workunit.client.1.smithi026.stdout:0/794: creat d1/d12/d2f/daa/f109 x:0 0 0 2024-03-16T10:45:29.487 INFO:tasks.workunit.client.1.smithi026.stdout:3/785: rename d4/lb to d4/dc/d31/l113 0 2024-03-16T10:45:29.491 INFO:tasks.workunit.client.1.smithi026.stdout:0/795: mknod d1/d12/d2f/d48/d55/d60/d9b/d57/c10a 0 2024-03-16T10:45:29.493 INFO:tasks.workunit.client.1.smithi026.stdout:0/796: dread d1/d12/d2f/d48/d55/d60/d9b/d11/fed [0,4194304] 0 2024-03-16T10:45:29.498 INFO:tasks.workunit.client.1.smithi026.stdout:3/786: dwrite d4/dc/d31/d48/d7f/ffb [0,4194304] 0 2024-03-16T10:45:29.500 INFO:tasks.workunit.client.1.smithi026.stdout:8/677: sync 2024-03-16T10:45:29.502 INFO:tasks.workunit.client.1.smithi026.stdout:8/678: write d5/dc/faf [484450,13091] 0 2024-03-16T10:45:29.505 INFO:tasks.workunit.client.1.smithi026.stdout:0/797: rename d1/d12/d2f/d48/d55/d60/d9b/dcf to d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b 0 2024-03-16T10:45:29.505 INFO:tasks.workunit.client.1.smithi026.stdout:8/679: dread - d5/d5d/f9d zero size 2024-03-16T10:45:29.507 INFO:tasks.workunit.client.1.smithi026.stdout:9/804: write d0/dd7/f9b [201765,69492] 0 2024-03-16T10:45:29.507 INFO:tasks.workunit.client.1.smithi026.stdout:3/787: dwrite d4/dc/d31/d48/d69/d78/fa2 [0,4194304] 0 2024-03-16T10:45:29.508 INFO:tasks.workunit.client.1.smithi026.stdout:8/680: fsync d5/d7/f8d 0 2024-03-16T10:45:29.510 INFO:tasks.workunit.client.1.smithi026.stdout:3/788: fdatasync d4/dc/d31/d48/fe3 0 2024-03-16T10:45:29.524 INFO:tasks.workunit.client.1.smithi026.stdout:8/681: rename d5/d28/d90 to d5/d23/d4e/d60/dc8 0 2024-03-16T10:45:29.525 INFO:tasks.workunit.client.1.smithi026.stdout:9/805: mkdir d0/da/d114 0 2024-03-16T10:45:29.527 INFO:tasks.workunit.client.1.smithi026.stdout:9/806: dread - d0/da/d16/d5d/f103 zero size 2024-03-16T10:45:29.527 INFO:tasks.workunit.client.1.smithi026.stdout:0/798: write d1/d12/d2f/d48/d55/f7c [4541654,112424] 0 2024-03-16T10:45:29.530 INFO:tasks.workunit.client.1.smithi026.stdout:8/682: rename d5/d23/d4e/cba to d5/d23/d4e/d60/dc8/cc9 0 2024-03-16T10:45:29.530 INFO:tasks.workunit.client.1.smithi026.stdout:0/799: write d1/d12/d2f/d48/d55/d60/d9b/d11/fb7 [228703,62608] 0 2024-03-16T10:45:29.531 INFO:tasks.workunit.client.1.smithi026.stdout:3/789: dread d4/dc/f5f [0,4194304] 0 2024-03-16T10:45:29.542 INFO:tasks.workunit.client.1.smithi026.stdout:8/683: dwrite d5/d23/f68 [0,4194304] 0 2024-03-16T10:45:29.543 INFO:tasks.workunit.client.1.smithi026.stdout:9/807: dread d0/da/d16/d17/d44/fdf [0,4194304] 0 2024-03-16T10:45:29.550 INFO:tasks.workunit.client.1.smithi026.stdout:8/684: rename d5/d7/d7b/db2/dbb/fbe to d5/d7/d2c/d36/d59/d93/db0/fca 0 2024-03-16T10:45:29.551 INFO:tasks.workunit.client.1.smithi026.stdout:0/800: write d1/d12/d2f/daa/dd1/d1a/d2b/d4f/f80 [3360282,67216] 0 2024-03-16T10:45:29.552 INFO:tasks.workunit.client.1.smithi026.stdout:3/790: write d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/fbf [3177773,18966] 0 2024-03-16T10:45:29.553 INFO:tasks.workunit.client.1.smithi026.stdout:3/791: chown d4/dc/d31/l37 1257186 1 2024-03-16T10:45:29.553 INFO:tasks.workunit.client.1.smithi026.stdout:9/808: truncate d0/da/d16/d5d/d72/d62/f8a 102463 0 2024-03-16T10:45:29.556 INFO:tasks.workunit.client.1.smithi026.stdout:8/685: creat d5/d23/d4e/d69/d76/fcb x:0 0 0 2024-03-16T10:45:29.558 INFO:tasks.workunit.client.1.smithi026.stdout:0/801: dread d1/d12/d2f/daa/dd1/d1a/d2b/d4f/fa8 [0,4194304] 0 2024-03-16T10:45:29.560 INFO:tasks.workunit.client.1.smithi026.stdout:9/809: truncate d0/d5f/da2/fe2 932265 0 2024-03-16T10:45:29.560 INFO:tasks.workunit.client.1.smithi026.stdout:3/792: mkdir d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/d114 0 2024-03-16T10:45:29.563 INFO:tasks.workunit.client.1.smithi026.stdout:8/686: creat d5/dc/d40/d53/fcc x:0 0 0 2024-03-16T10:45:29.567 INFO:tasks.workunit.client.1.smithi026.stdout:9/810: dread - d0/d63/ffb zero size 2024-03-16T10:45:29.568 INFO:tasks.workunit.client.1.smithi026.stdout:3/793: creat d4/dc/d73/d83/d7c/f115 x:0 0 0 2024-03-16T10:45:29.572 INFO:tasks.workunit.client.1.smithi026.stdout:0/802: unlink d1/d12/d2f/d48/d55/d60/d9b/l6c 0 2024-03-16T10:45:29.574 INFO:tasks.workunit.client.1.smithi026.stdout:9/811: write d0/da/d16/d17/d44/f24 [5264616,26433] 0 2024-03-16T10:45:29.574 INFO:tasks.workunit.client.1.smithi026.stdout:9/812: stat d0/da/d16/c23 0 2024-03-16T10:45:29.577 INFO:tasks.workunit.client.1.smithi026.stdout:3/794: fsync d4/dc/d73/f109 0 2024-03-16T10:45:29.578 INFO:tasks.workunit.client.1.smithi026.stdout:3/795: write d4/d89/d97/f3a [2420643,113996] 0 2024-03-16T10:45:29.580 INFO:tasks.workunit.client.1.smithi026.stdout:0/803: symlink d1/d12/l10c 0 2024-03-16T10:45:29.581 INFO:tasks.workunit.client.1.smithi026.stdout:0/804: fsync d1/d12/d2f/daa/dd1/d1a/d2b/fe9 0 2024-03-16T10:45:29.585 INFO:tasks.workunit.client.1.smithi026.stdout:8/687: rename d5/d7/d2c/d36/f4c to d5/fcd 0 2024-03-16T10:45:29.588 INFO:tasks.workunit.client.1.smithi026.stdout:3/796: chown d4/dc/d73/cdf 0 1 2024-03-16T10:45:29.588 INFO:tasks.workunit.client.1.smithi026.stdout:3/797: dread - d4/dc/dd/f106 zero size 2024-03-16T10:45:29.590 INFO:tasks.workunit.client.1.smithi026.stdout:9/813: mknod d0/d63/d6b/def/c115 0 2024-03-16T10:45:29.591 INFO:tasks.workunit.client.1.smithi026.stdout:9/814: fdatasync d0/d5f/da2/fe2 0 2024-03-16T10:45:29.592 INFO:tasks.workunit.client.1.smithi026.stdout:3/798: dread d4/dc/f5f [0,4194304] 0 2024-03-16T10:45:29.598 INFO:tasks.workunit.client.1.smithi026.stdout:3/799: creat d4/dc/d73/d83/d7c/dd7/f116 x:0 0 0 2024-03-16T10:45:29.600 INFO:tasks.workunit.client.1.smithi026.stdout:0/805: link d1/d12/d2f/daa/dd1/d1a/d2b/d4f/l6e d1/d12/d2f/d48/d55/d60/d9b/d51/df6/l10d 0 2024-03-16T10:45:29.600 INFO:tasks.workunit.client.1.smithi026.stdout:3/800: fdatasync d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f6a 0 2024-03-16T10:45:29.602 INFO:tasks.workunit.client.1.smithi026.stdout:8/688: write d5/d5d/d9a/f9b [4995853,123963] 0 2024-03-16T10:45:29.604 INFO:tasks.workunit.client.1.smithi026.stdout:9/815: rename d0/da/d16/d5d/d72/lac to d0/d5f/l116 0 2024-03-16T10:45:29.607 INFO:tasks.workunit.client.1.smithi026.stdout:3/801: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/f117 x:0 0 0 2024-03-16T10:45:29.608 INFO:tasks.workunit.client.1.smithi026.stdout:8/689: creat d5/dc/d3f/fce x:0 0 0 2024-03-16T10:45:29.609 INFO:tasks.workunit.client.1.smithi026.stdout:0/806: dread d1/d12/d2f/daa/dd1/d61/f6b [0,4194304] 0 2024-03-16T10:45:29.611 INFO:tasks.workunit.client.1.smithi026.stdout:3/802: mkdir d4/d89/d118 0 2024-03-16T10:45:29.612 INFO:tasks.workunit.client.1.smithi026.stdout:0/807: creat d1/d12/d2f/d48/d55/d60/d9b/d51/d5f/f10e x:0 0 0 2024-03-16T10:45:29.614 INFO:tasks.workunit.client.1.smithi026.stdout:3/803: mknod d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/c119 0 2024-03-16T10:45:29.615 INFO:tasks.workunit.client.1.smithi026.stdout:3/804: chown d4/d89/f8f 33407 1 2024-03-16T10:45:29.616 INFO:tasks.workunit.client.1.smithi026.stdout:0/808: readlink d1/d12/d2f/d48/lc1 0 2024-03-16T10:45:29.620 INFO:tasks.workunit.client.1.smithi026.stdout:8/690: link d5/d5d/f97 d5/d5d/d9a/fcf 0 2024-03-16T10:45:29.621 INFO:tasks.workunit.client.1.smithi026.stdout:3/805: mknod d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/d94/c11a 0 2024-03-16T10:45:29.625 INFO:tasks.workunit.client.1.smithi026.stdout:3/806: fdatasync d4/f41 0 2024-03-16T10:45:29.627 INFO:tasks.workunit.client.1.smithi026.stdout:9/816: write d0/da/d14/f39 [5681276,26450] 0 2024-03-16T10:45:29.629 INFO:tasks.workunit.client.1.smithi026.stdout:8/691: truncate d5/f3e 2582649 0 2024-03-16T10:45:29.630 INFO:tasks.workunit.client.1.smithi026.stdout:8/692: readlink d5/dc/l88 0 2024-03-16T10:45:29.633 INFO:tasks.workunit.client.1.smithi026.stdout:3/807: dwrite d4/dc/d31/dbe/f10a [0,4194304] 0 2024-03-16T10:45:29.635 INFO:tasks.workunit.client.1.smithi026.stdout:3/808: readlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/ld9 0 2024-03-16T10:45:29.638 INFO:tasks.workunit.client.1.smithi026.stdout:0/809: write d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/fb2 [832127,106164] 0 2024-03-16T10:45:29.639 INFO:tasks.workunit.client.1.smithi026.stdout:0/810: dread - d1/d12/d2f/d48/d55/d60/d9b/d57/fec zero size 2024-03-16T10:45:29.640 INFO:tasks.workunit.client.1.smithi026.stdout:9/817: mkdir d0/da/d16/d5d/d117 0 2024-03-16T10:45:29.643 INFO:tasks.workunit.client.1.smithi026.stdout:8/693: mknod d5/dc/cd0 0 2024-03-16T10:45:29.646 INFO:tasks.workunit.client.1.smithi026.stdout:9/818: mknod d0/da/d16/d17/db1/c118 0 2024-03-16T10:45:29.647 INFO:tasks.workunit.client.1.smithi026.stdout:8/694: dread d5/d7/f43 [4194304,4194304] 0 2024-03-16T10:45:29.649 INFO:tasks.workunit.client.1.smithi026.stdout:8/695: read d5/d7/d7b/f94 [1317227,53769] 0 2024-03-16T10:45:29.650 INFO:tasks.workunit.client.1.smithi026.stdout:3/809: creat d4/dc/d73/f11b x:0 0 0 2024-03-16T10:45:29.651 INFO:tasks.workunit.client.1.smithi026.stdout:3/810: dread - d4/dc/dd/d47/f10c zero size 2024-03-16T10:45:29.653 INFO:tasks.workunit.client.1.smithi026.stdout:0/811: link d1/d12/d2f/daa/dd1/fca d1/d12/d2f/dba/f10f 0 2024-03-16T10:45:29.657 INFO:tasks.workunit.client.1.smithi026.stdout:9/819: rename d0/dce/cdb to d0/d63/d6b/c119 0 2024-03-16T10:45:29.661 INFO:tasks.workunit.client.1.smithi026.stdout:3/811: rename d4/d89/d97/c65 to d4/dc/d31/d48/d69/c11c 0 2024-03-16T10:45:29.661 INFO:tasks.workunit.client.1.smithi026.stdout:4/762: sync 2024-03-16T10:45:29.662 INFO:tasks.workunit.client.1.smithi026.stdout:9/820: symlink d0/da/d16/d17/d44/l11a 0 2024-03-16T10:45:29.662 INFO:tasks.workunit.client.1.smithi026.stdout:4/763: chown d1/db/l1f 10572860 1 2024-03-16T10:45:29.665 INFO:tasks.workunit.client.1.smithi026.stdout:9/821: rename d0/dd7/cb3 to d0/dd7/dff/d109/c11b 0 2024-03-16T10:45:29.667 INFO:tasks.workunit.client.1.smithi026.stdout:4/764: symlink d1/db/ddb/daa/d44/d48/lee 0 2024-03-16T10:45:29.674 INFO:tasks.workunit.client.1.smithi026.stdout:4/765: dread d1/f56 [0,4194304] 0 2024-03-16T10:45:29.675 INFO:tasks.workunit.client.1.smithi026.stdout:3/812: link d4/dc/dd/d47/d53/ccc d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/d94/c11d 0 2024-03-16T10:45:29.680 INFO:tasks.workunit.client.1.smithi026.stdout:9/822: link d0/lec d0/d5f/dfc/l11c 0 2024-03-16T10:45:29.682 INFO:tasks.workunit.client.1.smithi026.stdout:8/696: dwrite d5/d23/d4e/d69/d76/f87 [0,4194304] 0 2024-03-16T10:45:29.688 INFO:tasks.workunit.client.1.smithi026.stdout:9/823: creat d0/d63/f11d x:0 0 0 2024-03-16T10:45:29.688 INFO:tasks.workunit.client.1.smithi026.stdout:4/766: dread d1/db/de/d6e/fb6 [4194304,4194304] 0 2024-03-16T10:45:29.689 INFO:tasks.workunit.client.1.smithi026.stdout:8/697: rmdir d5 39 2024-03-16T10:45:29.692 INFO:tasks.workunit.client.1.smithi026.stdout:8/698: chown d5/d23 644266 1 2024-03-16T10:45:29.694 INFO:tasks.workunit.client.1.smithi026.stdout:9/824: creat d0/da/d16/d17/db1/f11e x:0 0 0 2024-03-16T10:45:29.696 INFO:tasks.workunit.client.1.smithi026.stdout:8/699: chown d5/c45 1167 1 2024-03-16T10:45:29.712 INFO:tasks.workunit.client.1.smithi026.stdout:4/767: link d1/db/de/d4c/d59/cbd d1/db/de/d4c/d59/d6a/cef 0 2024-03-16T10:45:29.717 INFO:tasks.workunit.client.1.smithi026.stdout:4/768: write d1/db/de/d4c/d81/dc7/fcc [4378877,97308] 0 2024-03-16T10:45:29.722 INFO:tasks.workunit.client.1.smithi026.stdout:4/769: mkdir d1/d85/df0 0 2024-03-16T10:45:29.726 INFO:tasks.workunit.client.1.smithi026.stdout:4/770: fsync d1/db/de/d4c/d6b/d89/d9c/fa6 0 2024-03-16T10:45:29.734 INFO:tasks.workunit.client.1.smithi026.stdout:4/771: mkdir d1/db/de/d4c/d59/d6a/df1 0 2024-03-16T10:45:29.736 INFO:tasks.workunit.client.1.smithi026.stdout:4/772: mknod d1/dcd/cf2 0 2024-03-16T10:45:29.739 INFO:tasks.workunit.client.1.smithi026.stdout:4/773: mkdir d1/db/de/d4c/d6f/df3 0 2024-03-16T10:45:29.739 INFO:tasks.workunit.client.1.smithi026.stdout:3/813: dwrite d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:29.743 INFO:tasks.workunit.client.1.smithi026.stdout:4/774: mkdir d1/d85/da2/da8/df4 0 2024-03-16T10:45:29.745 INFO:tasks.workunit.client.1.smithi026.stdout:3/814: creat d4/dc/d31/d48/d69/d78/db3/f11e x:0 0 0 2024-03-16T10:45:29.746 INFO:tasks.workunit.client.1.smithi026.stdout:8/700: write d5/dc/d40/d53/f58 [1796747,39951] 0 2024-03-16T10:45:29.747 INFO:tasks.workunit.client.1.smithi026.stdout:4/775: write d1/db/de/d4c/d59/fc3 [6695016,30691] 0 2024-03-16T10:45:29.751 INFO:tasks.workunit.client.1.smithi026.stdout:3/815: creat d4/dc/d31/d48/d69/f11f x:0 0 0 2024-03-16T10:45:29.752 INFO:tasks.workunit.client.1.smithi026.stdout:9/825: dwrite d0/da/d4d/f5e [8388608,4194304] 0 2024-03-16T10:45:29.753 INFO:tasks.workunit.client.1.smithi026.stdout:9/826: fsync d0/da/d16/d17/db1/f11e 0 2024-03-16T10:45:29.759 INFO:tasks.workunit.client.1.smithi026.stdout:8/701: rename d5/d23/l54 to d5/d23/d4e/ld1 0 2024-03-16T10:45:29.765 INFO:tasks.workunit.client.1.smithi026.stdout:4/776: rename d1/db/ddb/daa/d44/d48/fa7 to d1/db/de/d4c/d6b/ff5 0 2024-03-16T10:45:29.765 INFO:tasks.workunit.client.1.smithi026.stdout:8/702: dwrite d5/d23/f4b [0,4194304] 0 2024-03-16T10:45:29.768 INFO:tasks.workunit.client.1.smithi026.stdout:9/827: link d0/dce/fd0 d0/da/d4d/dc5/f11f 0 2024-03-16T10:45:29.771 INFO:tasks.workunit.client.1.smithi026.stdout:9/828: write d0/da/f11 [2507920,112999] 0 2024-03-16T10:45:29.771 INFO:tasks.workunit.client.1.smithi026.stdout:9/829: dread - d0/dd7/ff6 zero size 2024-03-16T10:45:29.776 INFO:tasks.workunit.client.1.smithi026.stdout:3/816: getdents d4/dc/d73/d83/d7c/dd7 0 2024-03-16T10:45:29.777 INFO:tasks.workunit.client.1.smithi026.stdout:3/817: write d4/dc/d31/d48/d69/d78/fea [516385,118604] 0 2024-03-16T10:45:29.780 INFO:tasks.workunit.client.1.smithi026.stdout:4/777: symlink d1/d85/df0/lf6 0 2024-03-16T10:45:29.782 INFO:tasks.workunit.client.1.smithi026.stdout:9/830: rmdir d0/d5f/d7f/d8c 39 2024-03-16T10:45:29.783 INFO:tasks.workunit.client.1.smithi026.stdout:9/831: chown d0/d63/d6b/f10a 118258 1 2024-03-16T10:45:29.786 INFO:tasks.workunit.client.1.smithi026.stdout:8/703: truncate d5/d7/d2c/d36/d59/d93/da3/fa9 1418214 0 2024-03-16T10:45:29.794 INFO:tasks.workunit.client.1.smithi026.stdout:4/778: dread d1/db/de/d4c/d59/fc2 [0,4194304] 0 2024-03-16T10:45:29.800 INFO:tasks.workunit.client.1.smithi026.stdout:3/818: dwrite d4/f9d [0,4194304] 0 2024-03-16T10:45:29.802 INFO:tasks.workunit.client.1.smithi026.stdout:4/779: mknod d1/db/de/d4c/d59/d6a/cf7 0 2024-03-16T10:45:29.804 INFO:tasks.workunit.client.1.smithi026.stdout:4/780: chown d1/db/fe3 130 1 2024-03-16T10:45:29.808 INFO:tasks.workunit.client.1.smithi026.stdout:3/819: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/l120 0 2024-03-16T10:45:29.809 INFO:tasks.workunit.client.1.smithi026.stdout:8/704: dread d5/f3e [0,4194304] 0 2024-03-16T10:45:29.811 INFO:tasks.workunit.client.1.smithi026.stdout:9/832: dwrite d0/f1d [0,4194304] 0 2024-03-16T10:45:29.812 INFO:tasks.workunit.client.1.smithi026.stdout:9/833: fdatasync d0/d5f/da2/fe0 0 2024-03-16T10:45:29.814 INFO:tasks.workunit.client.1.smithi026.stdout:4/781: dwrite d1/db/de/d4c/fa1 [0,4194304] 0 2024-03-16T10:45:29.815 INFO:tasks.workunit.client.1.smithi026.stdout:3/820: symlink d4/dc/d31/d48/l121 0 2024-03-16T10:45:29.818 INFO:tasks.workunit.client.1.smithi026.stdout:9/834: dread d0/da/d16/d17/d44/fdf [0,4194304] 0 2024-03-16T10:45:29.820 INFO:tasks.workunit.client.1.smithi026.stdout:9/835: write d0/da/fe1 [1126691,105214] 0 2024-03-16T10:45:29.821 INFO:tasks.workunit.client.1.smithi026.stdout:3/821: mknod d4/dc/dd/d47/c122 0 2024-03-16T10:45:29.821 INFO:tasks.workunit.client.1.smithi026.stdout:3/822: chown d4/dc/d73/d83/cb0 16 1 2024-03-16T10:45:29.822 INFO:tasks.workunit.client.1.smithi026.stdout:4/782: creat d1/db/de/d4c/d6f/df3/ff8 x:0 0 0 2024-03-16T10:45:29.825 INFO:tasks.workunit.client.1.smithi026.stdout:2/776: sync 2024-03-16T10:45:29.825 INFO:tasks.workunit.client.1.smithi026.stdout:6/591: sync 2024-03-16T10:45:29.825 INFO:tasks.workunit.client.1.smithi026.stdout:5/715: sync 2024-03-16T10:45:29.825 INFO:tasks.workunit.client.1.smithi026.stdout:1/658: sync 2024-03-16T10:45:29.826 INFO:tasks.workunit.client.1.smithi026.stdout:7/735: sync 2024-03-16T10:45:29.828 INFO:tasks.workunit.client.1.smithi026.stdout:6/592: chown d0/d10 7 1 2024-03-16T10:45:29.830 INFO:tasks.workunit.client.1.smithi026.stdout:3/823: fsync d4/dc/dd/d47/d53/fca 0 2024-03-16T10:45:29.835 INFO:tasks.workunit.client.1.smithi026.stdout:4/783: mkdir d1/db/ddb/daf/df9 0 2024-03-16T10:45:29.835 INFO:tasks.workunit.client.1.smithi026.stdout:5/716: creat d1/d1c/d2a/db0/fff x:0 0 0 2024-03-16T10:45:29.835 INFO:tasks.workunit.client.1.smithi026.stdout:9/836: dwrite d0/d63/d6b/fc2 [0,4194304] 0 2024-03-16T10:45:29.842 INFO:tasks.workunit.client.1.smithi026.stdout:2/777: dwrite d2/d29/d17/d1a/d2b/d2e/fe4 [0,4194304] 0 2024-03-16T10:45:29.844 INFO:tasks.workunit.client.1.smithi026.stdout:3/824: mkdir d4/dc/dd/d47/d53/db1/d123 0 2024-03-16T10:45:29.846 INFO:tasks.workunit.client.1.smithi026.stdout:2/778: truncate d2/d29/d17/d1a/d2b/d69/fdf 18816 0 2024-03-16T10:45:29.849 INFO:tasks.workunit.client.1.smithi026.stdout:5/717: chown d1/db/d1f/d61/d65/df6 554846 1 2024-03-16T10:45:29.851 INFO:tasks.workunit.client.1.smithi026.stdout:3/825: truncate d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/fbf 4222263 0 2024-03-16T10:45:29.852 INFO:tasks.workunit.client.1.smithi026.stdout:9/837: creat d0/da/d16/d17/db1/f120 x:0 0 0 2024-03-16T10:45:29.852 INFO:tasks.workunit.client.1.smithi026.stdout:6/593: mkdir d0/d41/dc5 0 2024-03-16T10:45:29.854 INFO:tasks.workunit.client.1.smithi026.stdout:5/718: fsync d1/d1c/d2a/dab/dc5/d69/d51/fa2 0 2024-03-16T10:45:29.854 INFO:tasks.workunit.client.1.smithi026.stdout:9/838: fsync d0/da/d13/f31 0 2024-03-16T10:45:29.856 INFO:tasks.workunit.client.1.smithi026.stdout:9/839: chown d0/d63 15 1 2024-03-16T10:45:29.856 INFO:tasks.workunit.client.1.smithi026.stdout:1/659: creat d0/d17/db7/fca x:0 0 0 2024-03-16T10:45:29.857 INFO:tasks.workunit.client.1.smithi026.stdout:4/784: dwrite d1/d85/fd8 [0,4194304] 0 2024-03-16T10:45:29.859 INFO:tasks.workunit.client.1.smithi026.stdout:5/719: readlink d1/d1c/d2a/dab/dc5/l7e 0 2024-03-16T10:45:29.860 INFO:tasks.workunit.client.1.smithi026.stdout:3/826: readlink d4/d89/d97/l2c 0 2024-03-16T10:45:29.862 INFO:tasks.workunit.client.1.smithi026.stdout:1/660: write d0/d17/fb2 [212677,81826] 0 2024-03-16T10:45:29.862 INFO:tasks.workunit.client.1.smithi026.stdout:2/779: write d2/f35 [2069012,126584] 0 2024-03-16T10:45:29.865 INFO:tasks.workunit.client.1.smithi026.stdout:9/840: fdatasync d0/da/f2a 0 2024-03-16T10:45:29.865 INFO:tasks.workunit.client.1.smithi026.stdout:3/827: truncate d4/dc/f19 1451519 0 2024-03-16T10:45:29.871 INFO:tasks.workunit.client.1.smithi026.stdout:7/736: dwrite d6/d35/d3a/d59/fb9 [0,4194304] 0 2024-03-16T10:45:29.875 INFO:tasks.workunit.client.1.smithi026.stdout:3/828: truncate d4/dc/dd/f106 21094 0 2024-03-16T10:45:29.878 INFO:tasks.workunit.client.1.smithi026.stdout:7/737: dread - d6/d35/d3a/d4e/dcc/d3f/d65/fef zero size 2024-03-16T10:45:29.878 INFO:tasks.workunit.client.1.smithi026.stdout:9/841: write d0/da/d13/f5a [1202390,40709] 0 2024-03-16T10:45:29.878 INFO:tasks.workunit.client.1.smithi026.stdout:6/594: getdents d0/d5b 0 2024-03-16T10:45:29.880 INFO:tasks.workunit.client.1.smithi026.stdout:4/785: write d1/db/f26 [1162407,13113] 0 2024-03-16T10:45:29.884 INFO:tasks.workunit.client.1.smithi026.stdout:5/720: dread d1/f71 [0,4194304] 0 2024-03-16T10:45:29.884 INFO:tasks.workunit.client.1.smithi026.stdout:1/661: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/fbe [0,4194304] 0 2024-03-16T10:45:29.885 INFO:tasks.workunit.client.1.smithi026.stdout:2/780: dwrite d2/d29/d17/d38/f81 [0,4194304] 0 2024-03-16T10:45:29.887 INFO:tasks.workunit.client.1.smithi026.stdout:4/786: write d1/db/ddb/daa/d61/f7c [4564727,65453] 0 2024-03-16T10:45:29.887 INFO:tasks.workunit.client.1.smithi026.stdout:6/595: symlink d0/d28/lc6 0 2024-03-16T10:45:29.889 INFO:tasks.workunit.client.1.smithi026.stdout:3/829: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d10b/l124 0 2024-03-16T10:45:29.890 INFO:tasks.workunit.client.1.smithi026.stdout:4/787: fsync d1/db/de/d4c/d6f/df3/ff8 0 2024-03-16T10:45:29.892 INFO:tasks.workunit.client.1.smithi026.stdout:3/830: write d4/dc/f16 [1572869,107736] 0 2024-03-16T10:45:29.897 INFO:tasks.workunit.client.1.smithi026.stdout:1/662: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/f9b [635894,48263] 0 2024-03-16T10:45:29.901 INFO:tasks.workunit.client.1.smithi026.stdout:9/842: creat d0/da/d16/d17/f121 x:0 0 0 2024-03-16T10:45:29.903 INFO:tasks.workunit.client.1.smithi026.stdout:5/721: symlink d1/db/l100 0 2024-03-16T10:45:29.905 INFO:tasks.workunit.client.1.smithi026.stdout:7/738: creat d6/d35/d3a/d4e/dcc/d3f/d54/de1/ff0 x:0 0 0 2024-03-16T10:45:29.909 INFO:tasks.workunit.client.1.smithi026.stdout:6/596: unlink d0/d41/d71/cab 0 2024-03-16T10:45:29.911 INFO:tasks.workunit.client.1.smithi026.stdout:4/788: rename d1/f77 to d1/db/de/d4c/d6b/d7a/ffa 0 2024-03-16T10:45:29.915 INFO:tasks.workunit.client.1.smithi026.stdout:9/843: creat d0/da/d16/f122 x:0 0 0 2024-03-16T10:45:29.915 INFO:tasks.workunit.client.1.smithi026.stdout:9/844: read - d0/da/d16/d5d/fd4 zero size 2024-03-16T10:45:29.916 INFO:tasks.workunit.client.1.smithi026.stdout:7/739: mkdir d6/d35/d3a/d59/df1 0 2024-03-16T10:45:29.917 INFO:tasks.workunit.client.1.smithi026.stdout:9/845: chown d0/da/d16/d5d/d72/d62/f65 148507743 1 2024-03-16T10:45:29.922 INFO:tasks.workunit.client.1.smithi026.stdout:5/722: rename d1/db/d27/c46 to d1/d1c/d2a/d55/db1/c101 0 2024-03-16T10:45:29.923 INFO:tasks.workunit.client.1.smithi026.stdout:4/789: mkdir d1/db/ddb/daa/dfb 0 2024-03-16T10:45:29.934 INFO:tasks.workunit.client.1.smithi026.stdout:2/781: truncate d2/f35 3431603 0 2024-03-16T10:45:29.939 INFO:tasks.workunit.client.1.smithi026.stdout:5/723: mknod d1/db/d27/c102 0 2024-03-16T10:45:29.940 INFO:tasks.workunit.client.1.smithi026.stdout:4/790: fsync d1/f8 0 2024-03-16T10:45:29.944 INFO:tasks.workunit.client.1.smithi026.stdout:4/791: dread d1/db/de/d4c/f8a [0,4194304] 0 2024-03-16T10:45:29.946 INFO:tasks.workunit.client.1.smithi026.stdout:2/782: rmdir d2/d29/d17/d1a/d2b/d4e/d6f 39 2024-03-16T10:45:29.947 INFO:tasks.workunit.client.1.smithi026.stdout:3/831: link d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/d94/c11d d4/d89/d118/c125 0 2024-03-16T10:45:29.947 INFO:tasks.workunit.client.1.smithi026.stdout:2/783: read d2/d29/d17/d38/f81 [1504020,11051] 0 2024-03-16T10:45:29.950 INFO:tasks.workunit.client.1.smithi026.stdout:6/597: creat d0/d41/fc7 x:0 0 0 2024-03-16T10:45:29.953 INFO:tasks.workunit.client.1.smithi026.stdout:4/792: dwrite d1/db/de/fe8 [0,4194304] 0 2024-03-16T10:45:29.955 INFO:tasks.workunit.client.1.smithi026.stdout:5/724: mknod d1/d1c/d2a/d8e/c103 0 2024-03-16T10:45:29.956 INFO:tasks.workunit.client.1.smithi026.stdout:7/740: dwrite d6/d2f/f4a [0,4194304] 0 2024-03-16T10:45:29.958 INFO:tasks.workunit.client.1.smithi026.stdout:7/741: fdatasync d6/d35/d3a/d4e/db0/fe7 0 2024-03-16T10:45:29.965 INFO:tasks.workunit.client.1.smithi026.stdout:6/598: dread d0/f5f [0,4194304] 0 2024-03-16T10:45:29.965 INFO:tasks.workunit.client.1.smithi026.stdout:9/846: rename d0/d5f/d7f/d8c/l10b to d0/dd7/l123 0 2024-03-16T10:45:29.967 INFO:tasks.workunit.client.1.smithi026.stdout:9/847: write d0/da/f3b [1684346,79635] 0 2024-03-16T10:45:29.973 INFO:tasks.workunit.client.1.smithi026.stdout:2/784: chown d2/d29/d17/d1a/d2b/d4e/c46 323917 1 2024-03-16T10:45:29.973 INFO:tasks.workunit.client.1.smithi026.stdout:3/832: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/f126 x:0 0 0 2024-03-16T10:45:29.974 INFO:tasks.workunit.client.1.smithi026.stdout:6/599: dwrite d0/d41/f70 [0,4194304] 0 2024-03-16T10:45:29.974 INFO:tasks.workunit.client.1.smithi026.stdout:4/793: dread - d1/db/de/d4c/d6b/ff5 zero size 2024-03-16T10:45:29.975 INFO:tasks.workunit.client.1.smithi026.stdout:2/785: fdatasync d2/d29/d36/da7/f102 0 2024-03-16T10:45:29.976 INFO:tasks.workunit.client.1.smithi026.stdout:3/833: chown d4/dc/dd/d47/d53/db1/ddd/ff4 310 1 2024-03-16T10:45:29.981 INFO:tasks.workunit.client.1.smithi026.stdout:7/742: dread d6/d35/d3a/d4e/dcc/d3f/d54/f8b [0,4194304] 0 2024-03-16T10:45:29.982 INFO:tasks.workunit.client.1.smithi026.stdout:7/743: fdatasync d6/d2f/f4a 0 2024-03-16T10:45:29.985 INFO:tasks.workunit.client.1.smithi026.stdout:6/600: dwrite d0/d10/d79/fb4 [0,4194304] 0 2024-03-16T10:45:29.994 INFO:tasks.workunit.client.1.smithi026.stdout:5/725: mknod d1/db/d1f/d61/d65/df6/c104 0 2024-03-16T10:45:29.995 INFO:tasks.workunit.client.1.smithi026.stdout:2/786: dread d2/d29/d17/f23 [0,4194304] 0 2024-03-16T10:45:29.997 INFO:tasks.workunit.client.1.smithi026.stdout:7/744: fsync d6/d35/d3a/d4e/dcc/d3f/d54/f8b 0 2024-03-16T10:45:29.999 INFO:tasks.workunit.client.1.smithi026.stdout:7/745: chown d6/d35/d3a/d4e/dcc/d3f/d65/d68/c75 774 1 2024-03-16T10:45:30.001 INFO:tasks.workunit.client.1.smithi026.stdout:9/848: symlink d0/d5f/d7f/l124 0 2024-03-16T10:45:30.007 INFO:tasks.workunit.client.1.smithi026.stdout:3/834: dread d4/f9 [0,4194304] 0 2024-03-16T10:45:30.008 INFO:tasks.workunit.client.1.smithi026.stdout:6/601: symlink d0/d28/d49/dc2/lc8 0 2024-03-16T10:45:30.008 INFO:tasks.workunit.client.1.smithi026.stdout:7/746: mknod d6/d35/d3a/d4e/dcc/cf2 0 2024-03-16T10:45:30.011 INFO:tasks.workunit.client.1.smithi026.stdout:9/849: rename d0/d5f/da2/fe0 to d0/dd7/dff/d109/f125 0 2024-03-16T10:45:30.012 INFO:tasks.workunit.client.1.smithi026.stdout:9/850: chown d0/da/d114 54792 1 2024-03-16T10:45:30.012 INFO:tasks.workunit.client.1.smithi026.stdout:5/726: truncate d1/db/f38 4683107 0 2024-03-16T10:45:30.013 INFO:tasks.workunit.client.1.smithi026.stdout:2/787: symlink d2/d29/d17/d1a/l10c 0 2024-03-16T10:45:30.015 INFO:tasks.workunit.client.1.smithi026.stdout:6/602: creat d0/d28/d49/d7d/fc9 x:0 0 0 2024-03-16T10:45:30.020 INFO:tasks.workunit.client.1.smithi026.stdout:9/851: read d0/f66 [2376964,20050] 0 2024-03-16T10:45:30.022 INFO:tasks.workunit.client.1.smithi026.stdout:2/788: symlink d2/d29/d17/d38/l10d 0 2024-03-16T10:45:30.023 INFO:tasks.workunit.client.1.smithi026.stdout:7/747: dwrite d6/d35/d3a/d59/d9e/fc2 [0,4194304] 0 2024-03-16T10:45:30.023 INFO:tasks.workunit.client.1.smithi026.stdout:9/852: dread d0/da/d16/d17/fbc [0,4194304] 0 2024-03-16T10:45:30.026 INFO:tasks.workunit.client.1.smithi026.stdout:7/748: write d6/d35/d3a/d4e/dcc/d3f/d70/fa3 [4512353,15802] 0 2024-03-16T10:45:30.031 INFO:tasks.workunit.client.1.smithi026.stdout:6/603: mkdir d0/d10/d79/dca 0 2024-03-16T10:45:30.036 INFO:tasks.workunit.client.1.smithi026.stdout:5/727: dwrite d1/d1c/d2a/f32 [4194304,4194304] 0 2024-03-16T10:45:30.036 INFO:tasks.workunit.client.1.smithi026.stdout:9/853: rename d0/dce to d0/da/d14/da7/d126 0 2024-03-16T10:45:30.037 INFO:tasks.workunit.client.1.smithi026.stdout:9/854: chown d0/da/d16/d17 23191 1 2024-03-16T10:45:30.041 INFO:tasks.workunit.client.1.smithi026.stdout:7/749: symlink d6/d35/d3a/d4e/dcc/d3f/d70/lf3 0 2024-03-16T10:45:30.041 INFO:tasks.workunit.client.1.smithi026.stdout:3/835: getdents d4/dc/d31/dbe/de0 0 2024-03-16T10:45:30.049 INFO:tasks.workunit.client.1.smithi026.stdout:7/750: rename d6/d35/d3a/lde to d6/d35/d3a/d4e/dcc/d3f/d54/lf4 0 2024-03-16T10:45:30.052 INFO:tasks.workunit.client.1.smithi026.stdout:3/836: dwrite d4/dc/d73/d83/d7c/dd7/d62/f85 [0,4194304] 0 2024-03-16T10:45:30.052 INFO:tasks.workunit.client.1.smithi026.stdout:6/604: mkdir d0/d28/dcb 0 2024-03-16T10:45:30.053 INFO:tasks.workunit.client.1.smithi026.stdout:2/789: write d2/d29/d17/d1a/f9e [612601,97836] 0 2024-03-16T10:45:30.058 INFO:tasks.workunit.client.1.smithi026.stdout:7/751: truncate d6/d1e/da6/f95 238894 0 2024-03-16T10:45:30.060 INFO:tasks.workunit.client.1.smithi026.stdout:2/790: mknod d2/d29/d51/d55/c10e 0 2024-03-16T10:45:30.061 INFO:tasks.workunit.client.1.smithi026.stdout:5/728: dread d1/db/d1f/d61/d65/f83 [0,4194304] 0 2024-03-16T10:45:30.066 INFO:tasks.workunit.client.1.smithi026.stdout:6/605: dread d0/d62/f84 [0,4194304] 0 2024-03-16T10:45:30.071 INFO:tasks.workunit.client.1.smithi026.stdout:7/752: creat d6/de/d7d/ff5 x:0 0 0 2024-03-16T10:45:30.074 INFO:tasks.workunit.client.1.smithi026.stdout:9/855: getdents d0/da/d4d 0 2024-03-16T10:45:30.076 INFO:tasks.workunit.client.1.smithi026.stdout:3/837: write d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/d94/fa4 [173996,102436] 0 2024-03-16T10:45:30.086 INFO:tasks.workunit.client.1.smithi026.stdout:7/753: dread d6/d35/d3a/d4e/dcc/d3f/d70/fa3 [0,4194304] 0 2024-03-16T10:45:30.098 INFO:tasks.workunit.client.1.smithi026.stdout:2/791: rename d2/d29/d17/d1a/l6b to d2/d29/d3b/l10f 0 2024-03-16T10:45:30.100 INFO:tasks.workunit.client.1.smithi026.stdout:5/729: truncate d1/d1c/d2a/f3b 968915 0 2024-03-16T10:45:30.106 INFO:tasks.workunit.client.1.smithi026.stdout:3/838: dread d4/dc/d31/d48/d69/d78/f87 [0,4194304] 0 2024-03-16T10:45:30.107 INFO:tasks.workunit.client.1.smithi026.stdout:3/839: chown d4/dc/d73/d83/d7c/dd7/d62/f85 2788 1 2024-03-16T10:45:30.108 INFO:tasks.workunit.client.1.smithi026.stdout:3/840: chown d4/dc/dd/d47/f86 0 1 2024-03-16T10:45:30.109 INFO:tasks.workunit.client.1.smithi026.stdout:5/730: mknod d1/db/d1f/c105 0 2024-03-16T10:45:30.152 INFO:tasks.workunit.client.1.smithi026.stdout:0/812: sync 2024-03-16T10:45:30.153 INFO:tasks.workunit.client.1.smithi026.stdout:9/856: symlink d0/da/d16/d5d/d107/l127 0 2024-03-16T10:45:30.153 INFO:tasks.workunit.client.1.smithi026.stdout:2/792: mkdir d2/d29/d110 0 2024-03-16T10:45:30.153 INFO:tasks.workunit.client.1.smithi026.stdout:3/841: unlink d4/dc/dd/d80/dc2/fda 0 2024-03-16T10:45:30.157 INFO:tasks.workunit.client.1.smithi026.stdout:5/731: symlink d1/d1c/l106 0 2024-03-16T10:45:30.158 INFO:tasks.workunit.client.1.smithi026.stdout:9/857: chown d0/d63/d6b/f10a 4302106 1 2024-03-16T10:45:30.159 INFO:tasks.workunit.client.1.smithi026.stdout:6/606: dwrite d0/d10/f67 [0,4194304] 0 2024-03-16T10:45:30.163 INFO:tasks.workunit.client.1.smithi026.stdout:2/793: mknod d2/d29/d17/d1a/c111 0 2024-03-16T10:45:30.164 INFO:tasks.workunit.client.1.smithi026.stdout:3/842: dread d4/dc/f13 [4194304,4194304] 0 2024-03-16T10:45:30.165 INFO:tasks.workunit.client.1.smithi026.stdout:7/754: dwrite d6/d1e/f32 [0,4194304] 0 2024-03-16T10:45:30.166 INFO:tasks.workunit.client.1.smithi026.stdout:9/858: symlink d0/da/d14/l128 0 2024-03-16T10:45:30.170 INFO:tasks.workunit.client.1.smithi026.stdout:5/732: rename d1/d1c/d2a/ff2 to d1/d1c/d2a/dab/f107 0 2024-03-16T10:45:30.174 INFO:tasks.workunit.client.1.smithi026.stdout:0/813: creat d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f110 x:0 0 0 2024-03-16T10:45:30.174 INFO:tasks.workunit.client.1.smithi026.stdout:3/843: dwrite d4/dc/d31/dbe/f10a [0,4194304] 0 2024-03-16T10:45:30.178 INFO:tasks.workunit.client.1.smithi026.stdout:5/733: dread - d1/d1c/d2a/d55/d99/fcd zero size 2024-03-16T10:45:30.178 INFO:tasks.workunit.client.1.smithi026.stdout:6/607: getdents d0/d41/dc5 0 2024-03-16T10:45:30.181 INFO:tasks.workunit.client.1.smithi026.stdout:0/814: unlink d1/d12/fa9 0 2024-03-16T10:45:30.183 INFO:tasks.workunit.client.1.smithi026.stdout:7/755: dread d6/de/fc0 [0,4194304] 0 2024-03-16T10:45:30.184 INFO:tasks.workunit.client.1.smithi026.stdout:0/815: stat d1/d12/d2f/daa/dd1/fc2 0 2024-03-16T10:45:30.184 INFO:tasks.workunit.client.1.smithi026.stdout:3/844: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d10b/l127 0 2024-03-16T10:45:30.186 INFO:tasks.workunit.client.1.smithi026.stdout:3/845: chown d4/dc/d31/d48/d69/d78/db3/lb8 5679 1 2024-03-16T10:45:30.186 INFO:tasks.workunit.client.1.smithi026.stdout:5/734: creat d1/db/d1f/d61/d65/df6/f108 x:0 0 0 2024-03-16T10:45:30.187 INFO:tasks.workunit.client.1.smithi026.stdout:3/846: fsync d4/d89/f8f 0 2024-03-16T10:45:30.188 INFO:tasks.workunit.client.1.smithi026.stdout:7/756: mkdir d6/de/d16/d88/df6 0 2024-03-16T10:45:30.198 INFO:tasks.workunit.client.1.smithi026.stdout:6/608: rename d0/d41/fc7 to d0/d41/fcc 0 2024-03-16T10:45:30.198 INFO:tasks.workunit.client.1.smithi026.stdout:2/794: write d2/d29/d17/d1a/d2b/d69/fb6 [696323,98909] 0 2024-03-16T10:45:30.199 INFO:tasks.workunit.client.1.smithi026.stdout:6/609: fsync d0/d10/d79/d7b/f7e 0 2024-03-16T10:45:30.200 INFO:tasks.workunit.client.1.smithi026.stdout:6/610: readlink d0/l3c 0 2024-03-16T10:45:30.201 INFO:tasks.workunit.client.1.smithi026.stdout:3/847: dread d4/dc/dd/d47/f86 [0,4194304] 0 2024-03-16T10:45:30.202 INFO:tasks.workunit.client.1.smithi026.stdout:3/848: chown d4/dc/d31/d48/d7f/l84 0 1 2024-03-16T10:45:30.203 INFO:tasks.workunit.client.1.smithi026.stdout:0/816: write d1/d12/d2f/daa/dd1/d1a/f2c [8777561,20655] 0 2024-03-16T10:45:30.203 INFO:tasks.workunit.client.1.smithi026.stdout:3/849: read d4/f9d [3107702,42045] 0 2024-03-16T10:45:30.206 INFO:tasks.workunit.client.1.smithi026.stdout:5/735: rename d1/db/l6c to d1/d1c/d2a/db0/l109 0 2024-03-16T10:45:30.209 INFO:tasks.workunit.client.1.smithi026.stdout:7/757: unlink d6/d35/d3a/d59/le6 0 2024-03-16T10:45:30.211 INFO:tasks.workunit.client.1.smithi026.stdout:6/611: symlink d0/d8a/lcd 0 2024-03-16T10:45:30.211 INFO:tasks.workunit.client.1.smithi026.stdout:6/612: dread - d0/d10/fad zero size 2024-03-16T10:45:30.214 INFO:tasks.workunit.client.1.smithi026.stdout:0/817: unlink d1/d12/d2f/d48/d55/f71 0 2024-03-16T10:45:30.215 INFO:tasks.workunit.client.1.smithi026.stdout:3/850: creat d4/dc/d73/d83/f128 x:0 0 0 2024-03-16T10:45:30.230 INFO:tasks.workunit.client.1.smithi026.stdout:6/613: rename d0/d62/f84 to d0/d28/fce 0 2024-03-16T10:45:30.233 INFO:tasks.workunit.client.1.smithi026.stdout:0/818: rename d1/d12/d2f/d48/d55/d60/d9b/d57/fda to d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/d85/f111 0 2024-03-16T10:45:30.234 INFO:tasks.workunit.client.1.smithi026.stdout:6/614: truncate d0/f2c 4716537 0 2024-03-16T10:45:30.237 INFO:tasks.workunit.client.1.smithi026.stdout:6/615: write d0/d10/d79/d7b/f7e [4558972,118155] 0 2024-03-16T10:45:30.238 INFO:tasks.workunit.client.1.smithi026.stdout:6/616: chown d0/d62 0 1 2024-03-16T10:45:30.238 INFO:tasks.workunit.client.1.smithi026.stdout:7/758: rename d6/d35/d3a/d4e/dcc/d3f/d9d/f97 to d6/d35/d3a/d4e/da1/ff7 0 2024-03-16T10:45:30.240 INFO:tasks.workunit.client.1.smithi026.stdout:2/795: dwrite d2/d29/d17/d38/f59 [0,4194304] 0 2024-03-16T10:45:30.242 INFO:tasks.workunit.client.1.smithi026.stdout:3/851: truncate d4/dc/f11 1143753 0 2024-03-16T10:45:30.243 INFO:tasks.workunit.client.1.smithi026.stdout:3/852: chown d4/f9 163 1 2024-03-16T10:45:30.244 INFO:tasks.workunit.client.1.smithi026.stdout:6/617: creat d0/d28/d49/fcf x:0 0 0 2024-03-16T10:45:30.245 INFO:tasks.workunit.client.1.smithi026.stdout:6/618: read d0/f89 [2260813,38882] 0 2024-03-16T10:45:30.250 INFO:tasks.workunit.client.1.smithi026.stdout:0/819: dread d1/d12/d2f/dba/fc7 [0,4194304] 0 2024-03-16T10:45:30.256 INFO:tasks.workunit.client.1.smithi026.stdout:3/853: dwrite d4/dc/f13 [0,4194304] 0 2024-03-16T10:45:30.263 INFO:tasks.workunit.client.1.smithi026.stdout:0/820: rename d1/d12/d2f/d48/d55/d60/d9b/d11/fb7 to d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/f112 0 2024-03-16T10:45:30.265 INFO:tasks.workunit.client.1.smithi026.stdout:2/796: truncate d2/d29/d51/d72/f75 3019496 0 2024-03-16T10:45:30.265 INFO:tasks.workunit.client.1.smithi026.stdout:7/759: dwrite d6/d35/d3a/dab/fb3 [0,4194304] 0 2024-03-16T10:45:30.266 INFO:tasks.workunit.client.1.smithi026.stdout:7/760: dread - d6/de/d7d/ff5 zero size 2024-03-16T10:45:30.268 INFO:tasks.workunit.client.1.smithi026.stdout:3/854: symlink d4/dc/d73/d83/dbc/dd5/l129 0 2024-03-16T10:45:30.271 INFO:tasks.workunit.client.1.smithi026.stdout:0/821: readlink d1/d12/d2f/d48/d55/d60/d9b/l3f 0 2024-03-16T10:45:30.272 INFO:tasks.workunit.client.1.smithi026.stdout:0/822: stat d1/d12/d2f/daa/dd1/d61 0 2024-03-16T10:45:30.273 INFO:tasks.workunit.client.1.smithi026.stdout:7/761: read - d6/de/f94 zero size 2024-03-16T10:45:30.275 INFO:tasks.workunit.client.1.smithi026.stdout:3/855: unlink d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/l61 0 2024-03-16T10:45:30.275 INFO:tasks.workunit.client.1.smithi026.stdout:6/619: getdents d0/d41 0 2024-03-16T10:45:30.278 INFO:tasks.workunit.client.1.smithi026.stdout:2/797: fsync d2/d29/d17/d1a/d2b/f1f 0 2024-03-16T10:45:30.280 INFO:tasks.workunit.client.1.smithi026.stdout:0/823: rmdir d1/d12/d2f/d48/da5 39 2024-03-16T10:45:30.284 INFO:tasks.workunit.client.1.smithi026.stdout:7/762: creat d6/d35/d3a/d4e/dcc/d3f/d54/ff8 x:0 0 0 2024-03-16T10:45:30.288 INFO:tasks.workunit.client.1.smithi026.stdout:2/798: chown d2/d29/d17/d1a/d2b/d2e/c86 26083651 1 2024-03-16T10:45:30.291 INFO:tasks.workunit.client.1.smithi026.stdout:3/856: symlink d4/l12a 0 2024-03-16T10:45:30.297 INFO:tasks.workunit.client.1.smithi026.stdout:0/824: mkdir d1/d12/d2f/d113 0 2024-03-16T10:45:30.298 INFO:tasks.workunit.client.1.smithi026.stdout:6/620: dread d0/d10/d79/d7b/f9e [0,4194304] 0 2024-03-16T10:45:30.299 INFO:tasks.workunit.client.1.smithi026.stdout:7/763: rmdir d6/d35/d3a/d4e/d6c/d85 39 2024-03-16T10:45:30.302 INFO:tasks.workunit.client.1.smithi026.stdout:2/799: rename d2/d29/d17/d1a/d2b/d69/f9d to d2/d29/d85/dc6/dd5/f112 0 2024-03-16T10:45:30.305 INFO:tasks.workunit.client.1.smithi026.stdout:7/764: fdatasync d6/d35/d3a/d59/f92 0 2024-03-16T10:45:30.307 INFO:tasks.workunit.client.1.smithi026.stdout:2/800: creat d2/d29/d51/f113 x:0 0 0 2024-03-16T10:45:30.307 INFO:tasks.workunit.client.1.smithi026.stdout:7/765: read d6/f83 [455357,80427] 0 2024-03-16T10:45:30.314 INFO:tasks.workunit.client.1.smithi026.stdout:0/825: dread d1/d12/d2f/daa/dac/fd6 [0,4194304] 0 2024-03-16T10:45:30.314 INFO:tasks.workunit.client.1.smithi026.stdout:1/663: sync 2024-03-16T10:45:30.314 INFO:tasks.workunit.client.1.smithi026.stdout:8/705: sync 2024-03-16T10:45:30.315 INFO:tasks.workunit.client.1.smithi026.stdout:2/801: rename d2/d29/d51/d72/f76 to d2/d29/d85/dc6/dd5/f114 0 2024-03-16T10:45:30.317 INFO:tasks.workunit.client.1.smithi026.stdout:3/857: dread d4/dc/dd/f12 [0,4194304] 0 2024-03-16T10:45:30.322 INFO:tasks.workunit.client.1.smithi026.stdout:3/858: chown d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/c119 3841578 1 2024-03-16T10:45:30.324 INFO:tasks.workunit.client.1.smithi026.stdout:0/826: dwrite d1/d12/d2f/daa/dd1/d1a/d2b/fe9 [0,4194304] 0 2024-03-16T10:45:30.331 INFO:tasks.workunit.client.1.smithi026.stdout:8/706: dread d5/d23/f67 [0,4194304] 0 2024-03-16T10:45:30.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/766: creat d6/d35/d3a/d4e/ff9 x:0 0 0 2024-03-16T10:45:30.336 INFO:tasks.workunit.client.1.smithi026.stdout:2/802: mkdir d2/d29/d36/d115 0 2024-03-16T10:45:30.338 INFO:tasks.workunit.client.1.smithi026.stdout:0/827: creat d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/df1/f114 x:0 0 0 2024-03-16T10:45:30.340 INFO:tasks.workunit.client.1.smithi026.stdout:7/767: creat d6/d1e/dbc/dc8/ffa x:0 0 0 2024-03-16T10:45:30.341 INFO:tasks.workunit.client.1.smithi026.stdout:7/768: chown d6/d1e/f29 58250 1 2024-03-16T10:45:30.342 INFO:tasks.workunit.client.1.smithi026.stdout:7/769: write d6/de/f64 [341553,92537] 0 2024-03-16T10:45:30.343 INFO:tasks.workunit.client.1.smithi026.stdout:3/859: unlink d4/dc/d31/dbe/l102 0 2024-03-16T10:45:30.343 INFO:tasks.workunit.client.1.smithi026.stdout:7/770: write d6/f91 [4529015,128780] 0 2024-03-16T10:45:30.344 INFO:tasks.workunit.client.1.smithi026.stdout:7/771: chown d6/de/l2c 16 1 2024-03-16T10:45:30.345 INFO:tasks.workunit.client.1.smithi026.stdout:1/664: chown d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/lac 56108 1 2024-03-16T10:45:30.347 INFO:tasks.workunit.client.1.smithi026.stdout:1/665: write d0/d17/d1f/d63/d6a/d7c/f82 [1153943,52301] 0 2024-03-16T10:45:30.347 INFO:tasks.workunit.client.1.smithi026.stdout:0/828: creat d1/d12/d2f/d48/d55/d60/f115 x:0 0 0 2024-03-16T10:45:30.349 INFO:tasks.workunit.client.1.smithi026.stdout:8/707: symlink d5/ld2 0 2024-03-16T10:45:30.358 INFO:tasks.workunit.client.1.smithi026.stdout:3/860: write d4/dc/dd/fd8 [781791,2268] 0 2024-03-16T10:45:30.360 INFO:tasks.workunit.client.1.smithi026.stdout:3/861: chown d4/dc/d73/d83/cac 431 1 2024-03-16T10:45:30.360 INFO:tasks.workunit.client.1.smithi026.stdout:1/666: chown d0/d17/d1f/f90 11514058 1 2024-03-16T10:45:30.360 INFO:tasks.workunit.client.1.smithi026.stdout:8/708: mkdir d5/d7/d2c/d36/d8c/dd3 0 2024-03-16T10:45:30.363 INFO:tasks.workunit.client.1.smithi026.stdout:1/667: readlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/l91 0 2024-03-16T10:45:30.365 INFO:tasks.workunit.client.1.smithi026.stdout:2/803: creat d2/f116 x:0 0 0 2024-03-16T10:45:30.369 INFO:tasks.workunit.client.1.smithi026.stdout:7/772: dread d6/d1e/f20 [0,4194304] 0 2024-03-16T10:45:30.370 INFO:tasks.workunit.client.1.smithi026.stdout:3/862: dwrite d4/dc/d31/d48/d69/d78/fa2 [4194304,4194304] 0 2024-03-16T10:45:30.371 INFO:tasks.workunit.client.1.smithi026.stdout:2/804: mkdir d2/d29/d36/d98/d117 0 2024-03-16T10:45:30.372 INFO:tasks.workunit.client.1.smithi026.stdout:0/829: link d1/d12/d2f/d48/d55/d60/fa0 d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d104/f116 0 2024-03-16T10:45:30.376 INFO:tasks.workunit.client.1.smithi026.stdout:8/709: dread d5/d23/fac [0,4194304] 0 2024-03-16T10:45:30.379 INFO:tasks.workunit.client.1.smithi026.stdout:3/863: fsync d4/dc/d73/d83/d7c/dd7/d62/d4f/f7e 0 2024-03-16T10:45:30.382 INFO:tasks.workunit.client.1.smithi026.stdout:8/710: creat d5/dc/d40/d53/fd4 x:0 0 0 2024-03-16T10:45:30.384 INFO:tasks.workunit.client.1.smithi026.stdout:0/830: mknod d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/df5/c117 0 2024-03-16T10:45:30.386 INFO:tasks.workunit.client.1.smithi026.stdout:3/864: symlink d4/d89/d97/l12b 0 2024-03-16T10:45:30.386 INFO:tasks.workunit.client.1.smithi026.stdout:2/805: fsync d2/f116 0 2024-03-16T10:45:30.388 INFO:tasks.workunit.client.1.smithi026.stdout:8/711: mkdir d5/d23/d4e/d69/dd5 0 2024-03-16T10:45:30.392 INFO:tasks.workunit.client.1.smithi026.stdout:2/806: creat d2/d29/d85/dc6/f118 x:0 0 0 2024-03-16T10:45:30.392 INFO:tasks.workunit.client.1.smithi026.stdout:0/831: dwrite d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/fb2 [0,4194304] 0 2024-03-16T10:45:30.396 INFO:tasks.workunit.client.1.smithi026.stdout:1/668: write d0/fab [419122,103820] 0 2024-03-16T10:45:30.398 INFO:tasks.workunit.client.1.smithi026.stdout:7/773: write d6/de/f11 [4420846,85307] 0 2024-03-16T10:45:30.400 INFO:tasks.workunit.client.1.smithi026.stdout:8/712: mkdir d5/d5d/d9a/dd6 0 2024-03-16T10:45:30.408 INFO:tasks.workunit.client.1.smithi026.stdout:3/865: write d4/dc/d31/d48/d69/d78/f87 [2420771,117732] 0 2024-03-16T10:45:30.411 INFO:tasks.workunit.client.1.smithi026.stdout:1/669: dread d0/d17/d1f/d63/d6a/d7c/d84/d9d/f6e [0,4194304] 0 2024-03-16T10:45:30.412 INFO:tasks.workunit.client.1.smithi026.stdout:0/832: mknod d1/d12/d2f/daa/c118 0 2024-03-16T10:45:30.413 INFO:tasks.workunit.client.1.smithi026.stdout:0/833: write d1/d12/d2f/d48/faf [9106925,35001] 0 2024-03-16T10:45:30.415 INFO:tasks.workunit.client.1.smithi026.stdout:2/807: mkdir d2/d29/d17/d1a/d2b/d4e/d119 0 2024-03-16T10:45:30.420 INFO:tasks.workunit.client.1.smithi026.stdout:1/670: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/fcb x:0 0 0 2024-03-16T10:45:30.421 INFO:tasks.workunit.client.1.smithi026.stdout:0/834: mkdir d1/d12/d2f/d48/d55/d60/de0/dea/d119 0 2024-03-16T10:45:30.425 INFO:tasks.workunit.client.1.smithi026.stdout:7/774: creat d6/d2f/ffb x:0 0 0 2024-03-16T10:45:30.427 INFO:tasks.workunit.client.1.smithi026.stdout:8/713: link d5/dc/d40/d53/f63 d5/d7/d2c/d36/d59/d93/fd7 0 2024-03-16T10:45:30.429 INFO:tasks.workunit.client.1.smithi026.stdout:1/671: mkdir d0/d17/d1f/d20/dcc 0 2024-03-16T10:45:30.434 INFO:tasks.workunit.client.1.smithi026.stdout:7/775: readlink d6/d35/d3a/d4e/dcc/d3f/d54/lf4 0 2024-03-16T10:45:30.436 INFO:tasks.workunit.client.1.smithi026.stdout:1/672: dwrite d0/fd [4194304,4194304] 0 2024-03-16T10:45:30.438 INFO:tasks.workunit.client.1.smithi026.stdout:1/673: dread - d0/d17/d1f/d20/fbd zero size 2024-03-16T10:45:30.439 INFO:tasks.workunit.client.1.smithi026.stdout:8/714: rmdir d5/d23/d4e/d69/d76 39 2024-03-16T10:45:30.543 INFO:tasks.workunit.client.1.smithi026.stdout:0/835: creat d1/d12/d2f/d48/d55/d60/de0/dea/d101/f11a x:0 0 0 2024-03-16T10:45:30.544 INFO:tasks.workunit.client.1.smithi026.stdout:4/794: sync 2024-03-16T10:45:30.546 INFO:tasks.workunit.client.1.smithi026.stdout:8/715: read - d5/dc/d40/d53/fcc zero size 2024-03-16T10:45:30.546 INFO:tasks.workunit.client.1.smithi026.stdout:2/808: write d2/d29/d51/d55/fe7 [13822,32978] 0 2024-03-16T10:45:30.549 INFO:tasks.workunit.client.1.smithi026.stdout:7/776: dwrite d6/d35/d3a/d4e/dcc/d3f/dc3/fe3 [0,4194304] 0 2024-03-16T10:45:30.552 INFO:tasks.workunit.client.1.smithi026.stdout:1/674: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa/fb5 [534318,119533] 0 2024-03-16T10:45:30.554 INFO:tasks.workunit.client.1.smithi026.stdout:2/809: chown d2/d29/d17/d1a/d2b/d2e/d60/d7f/cc4 29630266 1 2024-03-16T10:45:30.554 INFO:tasks.workunit.client.1.smithi026.stdout:0/836: chown d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d104/f116 1859084 1 2024-03-16T10:45:30.559 INFO:tasks.workunit.client.1.smithi026.stdout:8/716: rename d5/d23/fb5 to d5/d5d/fd8 0 2024-03-16T10:45:30.560 INFO:tasks.workunit.client.1.smithi026.stdout:1/675: truncate d0/d17/d1f/d20/d65/f88 534589 0 2024-03-16T10:45:30.569 INFO:tasks.workunit.client.1.smithi026.stdout:1/676: rename d0/d17/d1f/d63/d6a/d7c/d84/d9d/fbe to d0/d17/d1f/d20/d75/fcd 0 2024-03-16T10:45:30.574 INFO:tasks.workunit.client.1.smithi026.stdout:7/777: creat d6/d2f/ffc x:0 0 0 2024-03-16T10:45:30.577 INFO:tasks.workunit.client.1.smithi026.stdout:8/717: truncate d5/d7/d2c/d36/d59/d93/da3/fa9 543739 0 2024-03-16T10:45:30.578 INFO:tasks.workunit.client.1.smithi026.stdout:1/677: rmdir d0/d17/d1f/d20/d65 39 2024-03-16T10:45:30.579 INFO:tasks.workunit.client.1.smithi026.stdout:1/678: readlink d0/d17/d1f/l14 0 2024-03-16T10:45:30.582 INFO:tasks.workunit.client.1.smithi026.stdout:8/718: creat d5/d7/d2c/d36/d59/d93/da3/fd9 x:0 0 0 2024-03-16T10:45:30.584 INFO:tasks.workunit.client.1.smithi026.stdout:7/778: dread d6/d1e/f90 [0,4194304] 0 2024-03-16T10:45:30.588 INFO:tasks.workunit.client.1.smithi026.stdout:7/779: dread d6/d35/d3a/d59/d9e/fc2 [0,4194304] 0 2024-03-16T10:45:30.593 INFO:tasks.workunit.client.1.smithi026.stdout:7/780: mknod d6/d1e/cfd 0 2024-03-16T10:45:30.596 INFO:tasks.workunit.client.1.smithi026.stdout:1/679: getdents d0/d17/d1f/d63/d6a/d7c/d3b/d64 0 2024-03-16T10:45:30.605 INFO:tasks.workunit.client.1.smithi026.stdout:0/837: write d1/fd4 [400614,122565] 0 2024-03-16T10:45:30.608 INFO:tasks.workunit.client.1.smithi026.stdout:4/795: dwrite d1/d85/fc8 [0,4194304] 0 2024-03-16T10:45:30.618 INFO:tasks.workunit.client.1.smithi026.stdout:0/838: dread d1/d12/d2f/d48/d55/d60/f28 [4194304,4194304] 0 2024-03-16T10:45:30.618 INFO:tasks.workunit.client.1.smithi026.stdout:4/796: symlink d1/d85/da2/dca/lfc 0 2024-03-16T10:45:30.624 INFO:tasks.workunit.client.1.smithi026.stdout:8/719: dwrite d5/dc/f4d [4194304,4194304] 0 2024-03-16T10:45:30.626 INFO:tasks.workunit.client.1.smithi026.stdout:0/839: creat d1/d12/d2f/d48/d55/d60/de0/dea/f11b x:0 0 0 2024-03-16T10:45:30.628 INFO:tasks.workunit.client.1.smithi026.stdout:4/797: creat d1/db/ddb/daa/d61/ffd x:0 0 0 2024-03-16T10:45:30.629 INFO:tasks.workunit.client.1.smithi026.stdout:7/781: write d6/d1e/da6/f95 [414854,111929] 0 2024-03-16T10:45:30.632 INFO:tasks.workunit.client.1.smithi026.stdout:7/782: truncate d6/de/faa 504564 0 2024-03-16T10:45:30.641 INFO:tasks.workunit.client.1.smithi026.stdout:4/798: dread d1/db/de/d4c/d81/f94 [0,4194304] 0 2024-03-16T10:45:30.644 INFO:tasks.workunit.client.1.smithi026.stdout:8/720: unlink d5/fcd 0 2024-03-16T10:45:30.645 INFO:tasks.workunit.client.1.smithi026.stdout:8/721: readlink d5/dc/d3f/la6 0 2024-03-16T10:45:30.647 INFO:tasks.workunit.client.1.smithi026.stdout:7/783: rename d6/d35/d3a/d4e/dcc/d3f/d65/d68/cdd to d6/d1e/dbc/cfe 0 2024-03-16T10:45:30.648 INFO:tasks.workunit.client.1.smithi026.stdout:7/784: write d6/d1e/da6/f8f [34892,64475] 0 2024-03-16T10:45:30.651 INFO:tasks.workunit.client.1.smithi026.stdout:8/722: dread d5/d23/f82 [0,4194304] 0 2024-03-16T10:45:30.652 INFO:tasks.workunit.client.1.smithi026.stdout:8/723: chown d5/d23/fac 343154508 1 2024-03-16T10:45:30.654 INFO:tasks.workunit.client.1.smithi026.stdout:4/799: creat d1/db/de/d4c/d81/dc7/ffe x:0 0 0 2024-03-16T10:45:30.656 INFO:tasks.workunit.client.1.smithi026.stdout:4/800: truncate d1/db/de/d4c/d59/d6a/f9a 5355899 0 2024-03-16T10:45:30.656 INFO:tasks.workunit.client.1.smithi026.stdout:4/801: readlink d1/l35 0 2024-03-16T10:45:30.658 INFO:tasks.workunit.client.1.smithi026.stdout:8/724: dread - d5/d7/d2c/d36/d59/d93/fd7 zero size 2024-03-16T10:45:30.663 INFO:tasks.workunit.client.1.smithi026.stdout:0/840: write d1/d12/d2f/daa/dd1/d1a/f25 [2064081,5828] 0 2024-03-16T10:45:30.666 INFO:tasks.workunit.client.1.smithi026.stdout:0/841: chown d1/d12/d2f/daa/dd1/d1a/d2b/c9f 3672126 1 2024-03-16T10:45:30.666 INFO:tasks.workunit.client.1.smithi026.stdout:8/725: dwrite d5/dc/d40/d53/fd4 [0,4194304] 0 2024-03-16T10:45:30.667 INFO:tasks.workunit.client.1.smithi026.stdout:0/842: stat d1/d12 0 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 -- 172.21.15.2:0/32376467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf0101e50 msgr2=0x7fbaf01022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/32376467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf0101e50 0x7fbaf01022b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009910 tx=0x7fbae4009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 -- 172.21.15.2:0/32376467 shutdown_connections 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/32376467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf0101e50 0x7fbaf01022b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/32376467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaf00fedd0 0x7fbaf00ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 -- 172.21.15.2:0/32376467 >> 172.21.15.2:0/32376467 conn(0x7fbaf00fa730 msgr2=0x7fbaf00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 -- 172.21.15.2:0/32376467 shutdown_connections 2024-03-16T10:45:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 -- 172.21.15.2:0/32376467 wait complete. 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.669+0000 7fbaf56a0700 1 Processor -- start 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaf56a0700 1 -- start start 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaf56a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf00fedd0 0x7fbaf0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaf56a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaf0101e50 0x7fbaf0196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaf56a0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf0197180 con 0x7fbaf00fedd0 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaf56a0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf01972c0 con 0x7fbaf0101e50 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaeeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf00fedd0 0x7fbaf0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaeeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf00fedd0 0x7fbaf0196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58196/0 (socket says 172.21.15.2:58196) 2024-03-16T10:45:30.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaeeffd700 1 -- 172.21.15.2:0/2331847546 learned_addr learned my addr 172.21.15.2:0/2331847546 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaeeffd700 1 -- 172.21.15.2:0/2331847546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaf0101e50 msgr2=0x7fbaf0196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaeeffd700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaf0101e50 0x7fbaf0196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.670+0000 7fbaeeffd700 1 -- 172.21.15.2:0/2331847546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae40095f0 con 0x7fbaf00fedd0 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.671+0000 7fbaeeffd700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf00fedd0 0x7fbaf0196620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b9c0 tx=0x7fbae000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.671+0000 7fbad7fff700 1 -- 172.21.15.2:0/2331847546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae000c980 con 0x7fbaf00fedd0 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.671+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf019bcd0 con 0x7fbaf00fedd0 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.671+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf019c240 con 0x7fbaf00fedd0 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.671+0000 7fbad7fff700 1 -- 172.21.15.2:0/2331847546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbae0014440 con 0x7fbaf00fedd0 2024-03-16T10:45:30.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.671+0000 7fbad7fff700 1 -- 172.21.15.2:0/2331847546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbae0012570 con 0x7fbaf00fedd0 2024-03-16T10:45:30.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.673+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc005320 con 0x7fbaf00fedd0 2024-03-16T10:45:30.674 INFO:tasks.workunit.client.1.smithi026.stdout:4/802: link d1/d85/da2/da8/lce d1/db/de/d6e/lff 0 2024-03-16T10:45:30.674 INFO:tasks.workunit.client.1.smithi026.stdout:7/785: dread d6/d35/d3a/d4e/dcc/d3f/f87 [0,4194304] 0 2024-03-16T10:45:30.674 INFO:tasks.workunit.client.1.smithi026.stdout:0/843: mkdir d1/d12/d2f/d48/d55/d60/d9b/d57/d11c 0 2024-03-16T10:45:30.674 INFO:tasks.workunit.client.1.smithi026.stdout:4/803: read d1/db/ddb/daa/d44/d48/f73 [2944009,88171] 0 2024-03-16T10:45:30.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.673+0000 7fbad7fff700 1 -- 172.21.15.2:0/2331847546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 31) v1 ==== 91794+0+0 (secure 0 0 0) 0x7fbae000cae0 con 0x7fbaf00fedd0 2024-03-16T10:45:30.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.674+0000 7fbad7fff700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fbad806dd50 0x7fbad8070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:30.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.674+0000 7fbad7fff700 1 -- 172.21.15.2:0/2331847546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbae0006e40 con 0x7fbaf00fedd0 2024-03-16T10:45:30.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.674+0000 7fbaee7fc700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fbad806dd50 0x7fbad8070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:30.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.674+0000 7fbaee7fc700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fbad806dd50 0x7fbad8070210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbae400b5c0 tx=0x7fbae4005160 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:30.679 INFO:tasks.workunit.client.1.smithi026.stdout:0/844: symlink d1/d12/l11d 0 2024-03-16T10:45:30.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.679+0000 7fbad7fff700 1 -- 172.21.15.2:0/2331847546 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fbae005a140 con 0x7fbaf00fedd0 2024-03-16T10:45:30.680 INFO:tasks.workunit.client.1.smithi026.stdout:4/804: read d1/db/de/f55 [30731,27017] 0 2024-03-16T10:45:30.683 INFO:tasks.workunit.client.1.smithi026.stdout:4/805: unlink d1/db/l1a 0 2024-03-16T10:45:30.685 INFO:tasks.workunit.client.1.smithi026.stdout:7/786: rename d6/de/d7d/c81 to d6/d35/d3a/d4e/d6c/d85/cff 0 2024-03-16T10:45:30.688 INFO:tasks.workunit.client.1.smithi026.stdout:0/845: rename d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/c95 to d1/d12/d2f/d48/d55/d60/d9b/d51/d88/c11e 0 2024-03-16T10:45:30.690 INFO:tasks.workunit.client.1.smithi026.stdout:7/787: symlink d6/de/d16/d88/l100 0 2024-03-16T10:45:30.692 INFO:tasks.workunit.client.1.smithi026.stdout:0/846: mknod d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/d85/c11f 0 2024-03-16T10:45:30.693 INFO:tasks.workunit.client.1.smithi026.stdout:0/847: dread - d1/d12/d2f/daa/f109 zero size 2024-03-16T10:45:30.694 INFO:tasks.workunit.client.1.smithi026.stdout:7/788: symlink d6/d35/d3a/d4e/dcc/d3f/dc5/de8/l101 0 2024-03-16T10:45:30.695 INFO:tasks.workunit.client.1.smithi026.stdout:4/806: dread d1/db/de/f58 [4194304,4194304] 0 2024-03-16T10:45:30.697 INFO:tasks.workunit.client.1.smithi026.stdout:7/789: rmdir d6/d35/d3a/d59/d9e 39 2024-03-16T10:45:30.698 INFO:tasks.workunit.client.1.smithi026.stdout:0/848: creat d1/d12/d2f/d48/d55/d60/d9b/d57/d11c/f120 x:0 0 0 2024-03-16T10:45:30.699 INFO:tasks.workunit.client.1.smithi026.stdout:4/807: rmdir d1/d85/df0 39 2024-03-16T10:45:30.700 INFO:tasks.workunit.client.1.smithi026.stdout:7/790: creat d6/d1e/da6/f102 x:0 0 0 2024-03-16T10:45:30.701 INFO:tasks.workunit.client.1.smithi026.stdout:7/791: truncate d6/f38 5232405 0 2024-03-16T10:45:30.702 INFO:tasks.workunit.client.1.smithi026.stdout:7/792: readlink d6/de/d16/d44/l48 0 2024-03-16T10:45:30.711 INFO:tasks.workunit.client.1.smithi026.stdout:7/793: mkdir d6/d35/d3a/d4e/dcc/d3f/d54/dd8/d103 0 2024-03-16T10:45:30.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:30 smithi026 ceph-mon[37041]: pgmap v10: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 36 MiB/s rd, 101 MiB/s wr, 264 op/s 2024-03-16T10:45:30.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:30 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:30.734 INFO:tasks.workunit.client.1.smithi026.stdout:0/849: write d1/d12/d2f/daa/dd1/d1a/d2b/fd5 [443721,91600] 0 2024-03-16T10:45:30.737 INFO:tasks.workunit.client.1.smithi026.stdout:4/808: dwrite d1/f56 [0,4194304] 0 2024-03-16T10:45:30.770 INFO:tasks.workunit.client.1.smithi026.stdout:0/850: creat d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/f121 x:0 0 0 2024-03-16T10:45:30.774 INFO:tasks.workunit.client.1.smithi026.stdout:4/809: dwrite d1/db/ddb/daa/fd0 [0,4194304] 0 2024-03-16T10:45:30.778 INFO:tasks.workunit.client.1.smithi026.stdout:7/794: dwrite d6/d35/d3a/d4e/f6d [0,4194304] 0 2024-03-16T10:45:30.780 INFO:tasks.workunit.client.1.smithi026.stdout:7/795: chown d6/d35/d3a/d4e/dcc/d73/ca2 13731 1 2024-03-16T10:45:30.781 INFO:tasks.workunit.client.1.smithi026.stdout:4/810: dread - d1/db/de/d4c/d6b/f98 zero size 2024-03-16T10:45:30.786 INFO:tasks.workunit.client.1.smithi026.stdout:7/796: mknod d6/d35/d3a/d4e/dcc/d3f/d70/c104 0 2024-03-16T10:45:30.799 INFO:tasks.workunit.client.1.smithi026.stdout:0/851: dread d1/d12/d2f/d48/d55/d60/f70 [0,4194304] 0 2024-03-16T10:45:30.804 INFO:tasks.workunit.client.1.smithi026.stdout:0/852: mkdir d1/d12/d2f/d48/d55/d60/d9b/d51/d122 0 2024-03-16T10:45:30.807 INFO:tasks.workunit.client.1.smithi026.stdout:0/853: dread - d1/d12/d2f/d48/d55/d60/de0/dea/d101/f11a zero size 2024-03-16T10:45:30.810 INFO:tasks.workunit.client.1.smithi026.stdout:4/811: dwrite d1/db/de/d4c/d6b/f98 [0,4194304] 0 2024-03-16T10:45:30.814 INFO:tasks.workunit.client.1.smithi026.stdout:7/797: dwrite d6/d35/d3a/d4e/dcc/d3f/fa0 [0,4194304] 0 2024-03-16T10:45:30.818 INFO:tasks.workunit.client.1.smithi026.stdout:0/854: truncate d1/d12/d2f/daa/fdf 363098 0 2024-03-16T10:45:30.824 INFO:tasks.workunit.client.1.smithi026.stdout:7/798: dwrite d6/d35/d3a/d4e/dcc/d3f/d54/f8d [0,4194304] 0 2024-03-16T10:45:30.832 INFO:tasks.workunit.client.1.smithi026.stdout:4/812: dwrite d1/db/f67 [0,4194304] 0 2024-03-16T10:45:30.834 INFO:tasks.workunit.client.1.smithi026.stdout:7/799: creat d6/f105 x:0 0 0 2024-03-16T10:45:30.835 INFO:tasks.workunit.client.1.smithi026.stdout:7/800: stat d6/l17 0 2024-03-16T10:45:30.841 INFO:tasks.workunit.client.1.smithi026.stdout:0/855: dread d1/d12/d2f/f7b [0,4194304] 0 2024-03-16T10:45:30.847 INFO:tasks.workunit.client.1.smithi026.stdout:4/813: write d1/db/de/d4c/d59/d6a/f82 [483456,127436] 0 2024-03-16T10:45:30.849 INFO:tasks.workunit.client.1.smithi026.stdout:4/814: chown d1/db/ddb/daf/fb9 125451 1 2024-03-16T10:45:30.853 INFO:tasks.workunit.client.1.smithi026.stdout:7/801: dwrite d6/d35/d3a/d4e/dcc/d3f/d54/f9b [0,4194304] 0 2024-03-16T10:45:30.853 INFO:tasks.workunit.client.1.smithi026.stdout:4/815: read d1/db/de/d4c/fa1 [2797196,108444] 0 2024-03-16T10:45:30.855 INFO:tasks.workunit.client.1.smithi026.stdout:7/802: chown d6/de/d16/f18 184 1 2024-03-16T10:45:30.857 INFO:tasks.workunit.client.1.smithi026.stdout:4/816: unlink d1/db/de/d4c/d6b/d89/d9c/cbb 0 2024-03-16T10:45:30.858 INFO:tasks.workunit.client.1.smithi026.stdout:0/856: dread d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f9c [0,4194304] 0 2024-03-16T10:45:30.862 INFO:tasks.workunit.client.1.smithi026.stdout:7/803: symlink d6/de/d16/l106 0 2024-03-16T10:45:30.866 INFO:tasks.workunit.client.1.smithi026.stdout:4/817: dwrite d1/db/de/fe8 [0,4194304] 0 2024-03-16T10:45:30.869 INFO:tasks.workunit.client.1.smithi026.stdout:0/857: write d1/d12/d2f/daa/dd1/d1a/d2b/f4e [480436,112997] 0 2024-03-16T10:45:30.870 INFO:tasks.workunit.client.1.smithi026.stdout:4/818: creat d1/db/de/f100 x:0 0 0 2024-03-16T10:45:30.871 INFO:tasks.workunit.client.1.smithi026.stdout:0/858: creat d1/f123 x:0 0 0 2024-03-16T10:45:30.875 INFO:tasks.workunit.client.1.smithi026.stdout:0/859: symlink d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/l124 0 2024-03-16T10:45:30.876 INFO:tasks.workunit.client.1.smithi026.stdout:7/804: read d6/de/f74 [9968,115189] 0 2024-03-16T10:45:30.878 INFO:tasks.workunit.client.1.smithi026.stdout:7/805: readlink d6/d35/d3a/d4e/l58 0 2024-03-16T10:45:30.878 INFO:tasks.workunit.client.1.smithi026.stdout:4/819: dwrite d1/db/ddb/daa/d44/d48/f73 [4194304,4194304] 0 2024-03-16T10:45:30.882 INFO:tasks.workunit.client.1.smithi026.stdout:7/806: write d6/d1e/dbc/dc8/ffa [734003,107502] 0 2024-03-16T10:45:30.882 INFO:tasks.workunit.client.1.smithi026.stdout:0/860: symlink d1/d12/d2f/d48/d55/d60/de0/dea/d119/l125 0 2024-03-16T10:45:30.884 INFO:tasks.workunit.client.1.smithi026.stdout:0/861: dread - d1/d12/d2f/d48/d55/d60/d9b/ff0 zero size 2024-03-16T10:45:30.885 INFO:tasks.workunit.client.1.smithi026.stdout:4/820: rename d1/db/ddb/daa/d44/lda to d1/db/ddb/daa/d44/d48/l101 0 2024-03-16T10:45:30.886 INFO:tasks.workunit.client.1.smithi026.stdout:7/807: creat d6/d35/d3a/d4e/db0/f107 x:0 0 0 2024-03-16T10:45:30.889 INFO:tasks.workunit.client.1.smithi026.stdout:7/808: read d6/d35/d3a/fda [615550,25340] 0 2024-03-16T10:45:30.891 INFO:tasks.workunit.client.1.smithi026.stdout:4/821: dwrite d1/db/de/d4c/d59/d6a/f9b [0,4194304] 0 2024-03-16T10:45:30.892 INFO:tasks.workunit.client.1.smithi026.stdout:4/822: chown d1/lc9 26877 1 2024-03-16T10:45:30.898 INFO:tasks.workunit.client.1.smithi026.stdout:4/823: dwrite d1/db/f74 [0,4194304] 0 2024-03-16T10:45:30.901 INFO:tasks.workunit.client.1.smithi026.stdout:0/862: truncate d1/d12/d2f/d48/d55/d60/d9b/d57/fdb 3409584 0 2024-03-16T10:45:30.902 INFO:tasks.workunit.client.1.smithi026.stdout:7/809: write d6/f22 [2662505,67380] 0 2024-03-16T10:45:30.903 INFO:tasks.workunit.client.1.smithi026.stdout:4/824: dread d1/db/f7e [0,4194304] 0 2024-03-16T10:45:30.903 INFO:tasks.workunit.client.1.smithi026.stdout:7/810: chown d6/d2f/le5 8172 1 2024-03-16T10:45:30.908 INFO:tasks.workunit.client.1.smithi026.stdout:0/863: mkdir d1/d12/d2f/daa/dd1/d61/d126 0 2024-03-16T10:45:30.911 INFO:tasks.workunit.client.1.smithi026.stdout:4/825: symlink d1/d85/df0/l102 0 2024-03-16T10:45:30.912 INFO:tasks.workunit.client.1.smithi026.stdout:4/826: chown d1/db/de/d4c/d59/d6a/f9a 336712 1 2024-03-16T10:45:30.914 INFO:tasks.workunit.client.1.smithi026.stdout:0/864: mkdir d1/d12/d2f/d48/d127 0 2024-03-16T10:45:30.914 INFO:tasks.workunit.client.1.smithi026.stdout:7/811: getdents d6/d35/d3a/dab 0 2024-03-16T10:45:30.916 INFO:tasks.workunit.client.1.smithi026.stdout:4/827: unlink d1/db/fc 0 2024-03-16T10:45:30.918 INFO:tasks.workunit.client.1.smithi026.stdout:0/865: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/c128 0 2024-03-16T10:45:30.919 INFO:tasks.workunit.client.1.smithi026.stdout:4/828: symlink d1/db/ddb/daa/d44/dd5/de9/l103 0 2024-03-16T10:45:30.926 INFO:tasks.workunit.client.1.smithi026.stdout:0/866: dwrite d1/fd4 [0,4194304] 0 2024-03-16T10:45:30.928 INFO:tasks.workunit.client.1.smithi026.stdout:0/867: readlink d1/d12/d2f/daa/dd1/d1a/lb0 0 2024-03-16T10:45:30.928 INFO:tasks.workunit.client.1.smithi026.stdout:9/859: sync 2024-03-16T10:45:30.929 INFO:tasks.workunit.client.1.smithi026.stdout:4/829: creat d1/f104 x:0 0 0 2024-03-16T10:45:30.930 INFO:tasks.workunit.client.1.smithi026.stdout:0/868: stat d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/f121 0 2024-03-16T10:45:30.930 INFO:tasks.workunit.client.1.smithi026.stdout:7/812: dread d6/f39 [0,4194304] 0 2024-03-16T10:45:30.933 INFO:tasks.workunit.client.1.smithi026.stdout:0/869: write d1/d12/d2f/d48/d55/d60/d9b/ff0 [246761,101493] 0 2024-03-16T10:45:30.936 INFO:tasks.workunit.client.1.smithi026.stdout:7/813: chown d6/d1e/c82 156 1 2024-03-16T10:45:30.938 INFO:tasks.workunit.client.1.smithi026.stdout:4/830: dwrite d1/db/de/f100 [0,4194304] 0 2024-03-16T10:45:30.940 INFO:tasks.workunit.client.1.smithi026.stdout:9/860: creat d0/da/d14/da7/d126/dd5/f129 x:0 0 0 2024-03-16T10:45:30.949 INFO:tasks.workunit.client.1.smithi026.stdout:7/814: unlink d6/d35/d3a/d4e/db0/fe7 0 2024-03-16T10:45:30.953 INFO:tasks.workunit.client.1.smithi026.stdout:9/861: creat d0/da/d16/d5d/d107/f12a x:0 0 0 2024-03-16T10:45:30.955 INFO:tasks.workunit.client.1.smithi026.stdout:0/870: write d1/d12/d2f/daa/dd1/d1a/f46 [1335422,24259] 0 2024-03-16T10:45:30.961 INFO:tasks.workunit.client.1.smithi026.stdout:9/862: creat d0/d63/d6b/def/f12b x:0 0 0 2024-03-16T10:45:30.964 INFO:tasks.workunit.client.1.smithi026.stdout:7/815: truncate d6/d35/d3a/d4e/dcc/d73/dbf/feb 1338174 0 2024-03-16T10:45:30.965 INFO:tasks.workunit.client.1.smithi026.stdout:0/871: getdents d1/d12/d2f/daa/dd1/d61/d126 0 2024-03-16T10:45:30.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.965+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbadc000bf0 con 0x7fbad806dd50 2024-03-16T10:45:30.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:30 smithi002 ceph-mon[29248]: pgmap v10: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 36 MiB/s rd, 101 MiB/s wr, 264 op/s 2024-03-16T10:45:30.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:30 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:30.966 INFO:tasks.workunit.client.1.smithi026.stdout:4/831: truncate d1/d85/fc8 541299 0 2024-03-16T10:45:30.967 INFO:tasks.workunit.client.1.smithi026.stdout:0/872: write d1/d12/d2f/daa/dac/fb5 [80647,90395] 0 2024-03-16T10:45:30.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.967+0000 7fbad7fff700 1 -- 172.21.15.2:0/2331847546 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fbadc000bf0 con 0x7fbad806dd50 2024-03-16T10:45:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fbad806dd50 msgr2=0x7fbad8070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fbad806dd50 0x7fbad8070210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbae400b5c0 tx=0x7fbae4005160 comp rx=0 tx=0).stop 2024-03-16T10:45:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf00fedd0 msgr2=0x7fbaf0196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf00fedd0 0x7fbaf0196620 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbae000b9c0 tx=0x7fbae000bd80 comp rx=0 tx=0).stop 2024-03-16T10:45:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 shutdown_connections 2024-03-16T10:45:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fbad806dd50 0x7fbad8070210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaf00fedd0 0x7fbaf0196620 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 --2- 172.21.15.2:0/2331847546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbaf0101e50 0x7fbaf0196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.969+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 >> 172.21.15.2:0/2331847546 conn(0x7fbaf00fa730 msgr2=0x7fbaf01054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.970+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 shutdown_connections 2024-03-16T10:45:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:30.970+0000 7fbaf56a0700 1 -- 172.21.15.2:0/2331847546 wait complete. 2024-03-16T10:45:30.971 INFO:tasks.workunit.client.1.smithi026.stdout:9/863: creat d0/da/d16/d5d/d117/f12c x:0 0 0 2024-03-16T10:45:30.972 INFO:tasks.workunit.client.1.smithi026.stdout:9/864: stat d0/da/d13/f31 0 2024-03-16T10:45:30.972 INFO:tasks.workunit.client.1.smithi026.stdout:9/865: readlink d0/da/d14/l37 0 2024-03-16T10:45:30.973 INFO:tasks.workunit.client.1.smithi026.stdout:7/816: write d6/d35/d3a/f8e [5803336,61004] 0 2024-03-16T10:45:30.982 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:45:30.982 INFO:tasks.workunit.client.1.smithi026.stdout:4/832: rename d1/d85/df0 to d1/db/de/d4c/d6b/d89/d105 0 2024-03-16T10:45:30.983 INFO:tasks.workunit.client.1.smithi026.stdout:4/833: readlink d1/l41 0 2024-03-16T10:45:30.985 INFO:tasks.workunit.client.1.smithi026.stdout:9/866: link d0/da/d16/d5d/d72/d62/fe7 d0/da/d14/da7/d126/f12d 0 2024-03-16T10:45:30.987 INFO:tasks.workunit.client.1.smithi026.stdout:9/867: write d0/da/d16/d17/d44/fca [3031370,24235] 0 2024-03-16T10:45:30.989 INFO:tasks.workunit.client.1.smithi026.stdout:0/873: getdents d1/d12 0 2024-03-16T10:45:30.991 INFO:tasks.workunit.client.1.smithi026.stdout:4/834: rmdir d1/db/ddb/daa 39 2024-03-16T10:45:30.991 INFO:tasks.workunit.client.1.smithi026.stdout:7/817: dread d6/d35/d3a/d4e/dcc/f40 [0,4194304] 0 2024-03-16T10:45:30.993 INFO:tasks.workunit.client.1.smithi026.stdout:9/868: creat d0/da/d16/d17/d44/f12e x:0 0 0 2024-03-16T10:45:30.994 INFO:tasks.workunit.client.1.smithi026.stdout:9/869: dread - d0/da/d13/ff3 zero size 2024-03-16T10:45:30.995 INFO:tasks.workunit.client.1.smithi026.stdout:7/818: creat d6/d1e/f108 x:0 0 0 2024-03-16T10:45:30.996 INFO:tasks.workunit.client.1.smithi026.stdout:4/835: mknod d1/db/de/d4c/d6b/d7a/c106 0 2024-03-16T10:45:30.997 INFO:tasks.workunit.client.1.smithi026.stdout:4/836: chown d1/db/le0 1 1 2024-03-16T10:45:31.000 INFO:tasks.workunit.client.1.smithi026.stdout:9/870: mknod d0/dd7/dff/c12f 0 2024-03-16T10:45:31.001 INFO:tasks.workunit.client.1.smithi026.stdout:7/819: dwrite d6/d1e/da6/f102 [0,4194304] 0 2024-03-16T10:45:31.006 INFO:tasks.workunit.client.1.smithi026.stdout:9/871: unlink d0/da/d4d/f71 0 2024-03-16T10:45:31.007 INFO:tasks.workunit.client.1.smithi026.stdout:0/874: link d1/d12/d2f/daa/dd1/d1a/d2b/cd9 d1/d12/d2f/c129 0 2024-03-16T10:45:31.007 INFO:tasks.workunit.client.1.smithi026.stdout:9/872: readlink d0/da/d14/da7/la8 0 2024-03-16T10:45:31.009 INFO:tasks.workunit.client.1.smithi026.stdout:0/875: truncate d1/d12/d2f/daa/f109 537826 0 2024-03-16T10:45:31.012 INFO:tasks.workunit.client.1.smithi026.stdout:2/810: sync 2024-03-16T10:45:31.012 INFO:tasks.workunit.client.1.smithi026.stdout:3/866: sync 2024-03-16T10:45:31.012 INFO:tasks.workunit.client.1.smithi026.stdout:6/621: sync 2024-03-16T10:45:31.012 INFO:tasks.workunit.client.1.smithi026.stdout:1/680: sync 2024-03-16T10:45:31.012 INFO:tasks.workunit.client.1.smithi026.stdout:8/726: sync 2024-03-16T10:45:31.012 INFO:tasks.workunit.client.1.smithi026.stdout:5/736: sync 2024-03-16T10:45:31.016 INFO:tasks.workunit.client.1.smithi026.stdout:3/867: stat d4/dc/dd/d47/d53 0 2024-03-16T10:45:31.019 INFO:tasks.workunit.client.1.smithi026.stdout:2/811: mknod d2/d29/d3b/c11a 0 2024-03-16T10:45:31.020 INFO:tasks.workunit.client.1.smithi026.stdout:4/837: dread d1/db/ddb/daa/f95 [0,4194304] 0 2024-03-16T10:45:31.023 INFO:tasks.workunit.client.1.smithi026.stdout:9/873: write d0/da/d16/d5d/f7c [4407293,115674] 0 2024-03-16T10:45:31.023 INFO:tasks.workunit.client.1.smithi026.stdout:7/820: link d6/de/l2c d6/d35/l109 0 2024-03-16T10:45:31.027 INFO:tasks.workunit.client.1.smithi026.stdout:8/727: dwrite d5/d23/d4e/d69/d76/fcb [0,4194304] 0 2024-03-16T10:45:31.027 INFO:tasks.workunit.client.1.smithi026.stdout:3/868: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dae/l12c 0 2024-03-16T10:45:31.027 INFO:tasks.workunit.client.1.smithi026.stdout:5/737: write d1/db/d1f/d61/d65/f9d [2177667,12714] 0 2024-03-16T10:45:31.030 INFO:tasks.workunit.client.1.smithi026.stdout:8/728: fsync d5/fbc 0 2024-03-16T10:45:31.032 INFO:tasks.workunit.client.1.smithi026.stdout:8/729: stat d5/d7/d2c/f32 0 2024-03-16T10:45:31.032 INFO:tasks.workunit.client.1.smithi026.stdout:0/876: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/d5f/f8a [0,4194304] 0 2024-03-16T10:45:31.033 INFO:tasks.workunit.client.1.smithi026.stdout:5/738: write d1/d1c/d2a/fc9 [969796,4277] 0 2024-03-16T10:45:31.035 INFO:tasks.workunit.client.1.smithi026.stdout:3/869: stat d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/ld9 0 2024-03-16T10:45:31.035 INFO:tasks.workunit.client.1.smithi026.stdout:0/877: chown d1/d12/l43 4989302 1 2024-03-16T10:45:31.036 INFO:tasks.workunit.client.1.smithi026.stdout:3/870: fsync d4/dc/d73/d83/d7c/f115 0 2024-03-16T10:45:31.037 INFO:tasks.workunit.client.1.smithi026.stdout:6/622: getdents d0/d41/dbe 0 2024-03-16T10:45:31.042 INFO:tasks.workunit.client.1.smithi026.stdout:4/838: creat d1/db/de/d4c/d6f/df3/f107 x:0 0 0 2024-03-16T10:45:31.044 INFO:tasks.workunit.client.1.smithi026.stdout:9/874: chown d0/d63/d6b/c119 15 1 2024-03-16T10:45:31.046 INFO:tasks.workunit.client.1.smithi026.stdout:1/681: creat d0/d17/d1f/d63/dbc/fce x:0 0 0 2024-03-16T10:45:31.048 INFO:tasks.workunit.client.1.smithi026.stdout:2/812: dwrite d2/d29/d51/d72/fea [0,4194304] 0 2024-03-16T10:45:31.054 INFO:tasks.workunit.client.1.smithi026.stdout:6/623: creat d0/d41/d71/fd0 x:0 0 0 2024-03-16T10:45:31.054 INFO:tasks.workunit.client.1.smithi026.stdout:9/875: dwrite d0/da/f11 [0,4194304] 0 2024-03-16T10:45:31.059 INFO:tasks.workunit.client.1.smithi026.stdout:7/821: creat d6/d2f/dad/de9/f10a x:0 0 0 2024-03-16T10:45:31.063 INFO:tasks.workunit.client.1.smithi026.stdout:1/682: fdatasync d0/d17/d1f/d63/d6a/d7c/d84/d9d/f4e 0 2024-03-16T10:45:31.066 INFO:tasks.workunit.client.1.smithi026.stdout:5/739: truncate d1/db/d1f/d61/d65/f7c 1279699 0 2024-03-16T10:45:31.068 INFO:tasks.workunit.client.1.smithi026.stdout:6/624: readlink d0/d41/d71/l93 0 2024-03-16T10:45:31.070 INFO:tasks.workunit.client.1.smithi026.stdout:9/876: chown d0/da/d16/d17/f104 33339180 1 2024-03-16T10:45:31.078 INFO:tasks.workunit.client.1.smithi026.stdout:8/730: creat d5/d7/fda x:0 0 0 2024-03-16T10:45:31.078 INFO:tasks.workunit.client.1.smithi026.stdout:7/822: chown d6/de/l2c 3 1 2024-03-16T10:45:31.079 INFO:tasks.workunit.client.1.smithi026.stdout:9/877: read - d0/d63/d6b/f10a zero size 2024-03-16T10:45:31.080 INFO:tasks.workunit.client.1.smithi026.stdout:4/839: truncate d1/db/ddb/daf/fb9 125037 0 2024-03-16T10:45:31.084 INFO:tasks.workunit.client.1.smithi026.stdout:9/878: read d0/da/d13/f38 [2098010,38846] 0 2024-03-16T10:45:31.085 INFO:tasks.workunit.client.1.smithi026.stdout:5/740: symlink d1/db/d1f/d61/l10a 0 2024-03-16T10:45:31.086 INFO:tasks.workunit.client.1.smithi026.stdout:3/871: dwrite d4/dc/f11 [0,4194304] 0 2024-03-16T10:45:31.088 INFO:tasks.workunit.client.1.smithi026.stdout:3/872: dread - d4/dc/dd/d47/d53/db1/ddd/ff7 zero size 2024-03-16T10:45:31.094 INFO:tasks.workunit.client.1.smithi026.stdout:2/813: truncate d2/d29/d51/d72/fea 3733719 0 2024-03-16T10:45:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 -- 172.21.15.2:0/2516531450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 msgr2=0x7f4cf0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 --2- 172.21.15.2:0/2516531450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101030 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009910 tx=0x7f4ce4009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 -- 172.21.15.2:0/2516531450 shutdown_connections 2024-03-16T10:45:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 --2- 172.21.15.2:0/2516531450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101030 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 --2- 172.21.15.2:0/2516531450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cf00ff9d0 0x7f4cf00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 -- 172.21.15.2:0/2516531450 >> 172.21.15.2:0/2516531450 conn(0x7f4cf00faf70 msgr2=0x7f4cf00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 -- 172.21.15.2:0/2516531450 shutdown_connections 2024-03-16T10:45:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.097+0000 7f4cf6350700 1 -- 172.21.15.2:0/2516531450 wait complete. 2024-03-16T10:45:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.098+0000 7f4cf6350700 1 Processor -- start 2024-03-16T10:45:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.098+0000 7f4cf6350700 1 -- start start 2024-03-16T10:45:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.098+0000 7f4cf6350700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cf00ff9d0 0x7f4cf01038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.098+0000 7f4cf6350700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.098+0000 7f4cf6350700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf01024f0 con 0x7f4cf0100bd0 2024-03-16T10:45:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.098+0000 7f4cf6350700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0102630 con 0x7f4cf00ff9d0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cef7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58228/0 (socket says 172.21.15.2:58228) 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cef7fe700 1 -- 172.21.15.2:0/3392967277 learned_addr learned my addr 172.21.15.2:0/3392967277 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4ceffff700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cf00ff9d0 0x7f4cf01038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cef7fe700 1 -- 172.21.15.2:0/3392967277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cf00ff9d0 msgr2=0x7f4cf01038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cef7fe700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cf00ff9d0 0x7f4cf01038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cef7fe700 1 -- 172.21.15.2:0/3392967277 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce40095f0 con 0x7f4cf0100bd0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cef7fe700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101fb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4006010 tx=0x7f4ce400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4ced7fa700 1 -- 172.21.15.2:0/3392967277 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce401d070 con 0x7f4cf0100bd0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf01028b0 con 0x7f4cf0100bd0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4ced7fa700 1 -- 172.21.15.2:0/3392967277 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ce400f460 con 0x7f4cf0100bd0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4ced7fa700 1 -- 172.21.15.2:0/3392967277 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce4021620 con 0x7f4cf0100bd0 2024-03-16T10:45:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.099+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf0102e20 con 0x7f4cf0100bd0 2024-03-16T10:45:31.102 INFO:tasks.workunit.client.1.smithi026.stdout:2/814: dwrite d2/f116 [0,4194304] 0 2024-03-16T10:45:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.100+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf004ea80 con 0x7f4cf0100bd0 2024-03-16T10:45:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.101+0000 7f4ced7fa700 1 -- 172.21.15.2:0/3392967277 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 31) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f4ce400f5d0 con 0x7f4cf0100bd0 2024-03-16T10:45:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.101+0000 7f4ced7fa700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4cd806dd60 0x7f4cd8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.102+0000 7f4ced7fa700 1 -- 172.21.15.2:0/3392967277 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4ce408eaf0 con 0x7f4cf0100bd0 2024-03-16T10:45:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.102+0000 7f4ceffff700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4cd806dd60 0x7f4cd8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.102+0000 7f4ceffff700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4cd806dd60 0x7f4cd8070220 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0100a30 tx=0x7f4ce000b410 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:31.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.107+0000 7f4ced7fa700 1 -- 172.21.15.2:0/3392967277 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f4ce405bd50 con 0x7f4cf0100bd0 2024-03-16T10:45:31.145 INFO:tasks.workunit.client.1.smithi026.stdout:7/823: read - d6/d2f/dad/d7b/fd1 zero size 2024-03-16T10:45:31.147 INFO:tasks.workunit.client.1.smithi026.stdout:8/731: creat d5/d7/d7b/d96/fdb x:0 0 0 2024-03-16T10:45:31.147 INFO:tasks.workunit.client.1.smithi026.stdout:0/878: getdents d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7 0 2024-03-16T10:45:31.154 INFO:tasks.workunit.client.1.smithi026.stdout:4/840: symlink d1/db/de/d4c/d6b/dbc/l108 0 2024-03-16T10:45:31.162 INFO:tasks.workunit.client.1.smithi026.stdout:5/741: dread d1/d1c/d2a/dab/dc5/f49 [0,4194304] 0 2024-03-16T10:45:31.164 INFO:tasks.workunit.client.1.smithi026.stdout:4/841: dwrite d1/db/ddb/daa/d44/dd5/de9/fea [0,4194304] 0 2024-03-16T10:45:31.169 INFO:tasks.workunit.client.1.smithi026.stdout:4/842: write d1/db/de/f3d [3891734,80440] 0 2024-03-16T10:45:31.170 INFO:tasks.workunit.client.1.smithi026.stdout:9/879: dread d0/da/f12 [0,4194304] 0 2024-03-16T10:45:31.171 INFO:tasks.workunit.client.1.smithi026.stdout:3/873: dwrite d4/dc/d73/d83/d7c/dd7/d62/d4f/ff9 [0,4194304] 0 2024-03-16T10:45:31.182 INFO:tasks.workunit.client.1.smithi026.stdout:1/683: link d0/d17/d1f/d20/c50 d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa/ccf 0 2024-03-16T10:45:31.183 INFO:tasks.workunit.client.1.smithi026.stdout:1/684: chown d0/d17/d1f/d63/d6a/d7c/d6b/l72 3375721 1 2024-03-16T10:45:31.187 INFO:tasks.workunit.client.1.smithi026.stdout:2/815: mknod d2/d29/d3b/dbb/dfb/c11b 0 2024-03-16T10:45:31.189 INFO:tasks.workunit.client.1.smithi026.stdout:9/880: mkdir d0/da/d14/da7/d126/d130 0 2024-03-16T10:45:31.189 INFO:tasks.workunit.client.1.smithi026.stdout:2/816: dread - d2/d29/d17/d1a/d2b/d4e/d67/fd2 zero size 2024-03-16T10:45:31.191 INFO:tasks.workunit.client.1.smithi026.stdout:7/824: getdents d6/d35/d3a/d4e/dcc/d3f/d54/dd8/d103 0 2024-03-16T10:45:31.193 INFO:tasks.workunit.client.1.smithi026.stdout:8/732: creat d5/d7/d2c/d36/d8c/dd3/fdc x:0 0 0 2024-03-16T10:45:31.193 INFO:tasks.workunit.client.1.smithi026.stdout:5/742: dwrite d1/d1c/d4c/d4d/fcc [0,4194304] 0 2024-03-16T10:45:31.193 INFO:tasks.workunit.client.1.smithi026.stdout:7/825: chown d6/de/d16/l6b 60674 1 2024-03-16T10:45:31.193 INFO:tasks.workunit.client.1.smithi026.stdout:8/733: chown d5/d7/d7b 19 1 2024-03-16T10:45:31.194 INFO:tasks.workunit.client.1.smithi026.stdout:7/826: chown d6/de/d7d/f86 31164929 1 2024-03-16T10:45:31.197 INFO:tasks.workunit.client.1.smithi026.stdout:9/881: creat d0/d5f/d7f/d8c/f131 x:0 0 0 2024-03-16T10:45:31.198 INFO:tasks.workunit.client.1.smithi026.stdout:2/817: write d2/d29/d51/ff6 [743334,99495] 0 2024-03-16T10:45:31.200 INFO:tasks.workunit.client.1.smithi026.stdout:0/879: rename d1/ca to d1/d12/d2f/daa/dd1/d1a/d2b/d4f/c12a 0 2024-03-16T10:45:31.202 INFO:tasks.workunit.client.1.smithi026.stdout:9/882: rmdir d0/da/d16/d17/d44 39 2024-03-16T10:45:31.205 INFO:tasks.workunit.client.1.smithi026.stdout:3/874: rename d4/dc/f16 to d4/dc/d73/d83/d7c/f12d 0 2024-03-16T10:45:31.207 INFO:tasks.workunit.client.1.smithi026.stdout:5/743: symlink d1/d1c/l10b 0 2024-03-16T10:45:31.211 INFO:tasks.workunit.client.1.smithi026.stdout:6/625: write d0/d10/f38 [1213162,26853] 0 2024-03-16T10:45:31.215 INFO:tasks.workunit.client.1.smithi026.stdout:3/875: mknod d4/dc/d31/c12e 0 2024-03-16T10:45:31.216 INFO:tasks.workunit.client.1.smithi026.stdout:3/876: readlink d4/dc/d73/d83/d7c/dd7/la5 0 2024-03-16T10:45:31.217 INFO:tasks.workunit.client.1.smithi026.stdout:0/880: mkdir d1/d12/d2f/daa/dd1/d61/d126/d12b 0 2024-03-16T10:45:31.220 INFO:tasks.workunit.client.1.smithi026.stdout:5/744: creat d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/f10c x:0 0 0 2024-03-16T10:45:31.221 INFO:tasks.workunit.client.1.smithi026.stdout:2/818: dread d2/d29/d17/d1a/d2b/d4b/fda [0,4194304] 0 2024-03-16T10:45:31.223 INFO:tasks.workunit.client.1.smithi026.stdout:9/883: mkdir d0/da/d114/d132 0 2024-03-16T10:45:31.225 INFO:tasks.workunit.client.1.smithi026.stdout:6/626: dread d0/d10/f67 [0,4194304] 0 2024-03-16T10:45:31.228 INFO:tasks.workunit.client.1.smithi026.stdout:1/685: rename d0/l2d to d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/ld0 0 2024-03-16T10:45:31.231 INFO:tasks.workunit.client.1.smithi026.stdout:4/843: truncate d1/db/de/d4c/d59/d6a/f9a 1459145 0 2024-03-16T10:45:31.234 INFO:tasks.workunit.client.1.smithi026.stdout:3/877: symlink d4/dc/dd/d47/d53/l12f 0 2024-03-16T10:45:31.237 INFO:tasks.workunit.client.1.smithi026.stdout:9/884: dread d0/da/d13/f26 [0,4194304] 0 2024-03-16T10:45:31.238 INFO:tasks.workunit.client.1.smithi026.stdout:3/878: dread - d4/dc/d31/d48/d69/d78/db3/f11e zero size 2024-03-16T10:45:31.238 INFO:tasks.workunit.client.1.smithi026.stdout:0/881: dread - d1/f92 zero size 2024-03-16T10:45:31.239 INFO:tasks.workunit.client.1.smithi026.stdout:4/844: dwrite d1/db/de/d4c/d59/d6a/f9b [4194304,4194304] 0 2024-03-16T10:45:31.239 INFO:tasks.workunit.client.1.smithi026.stdout:3/879: readlink d4/dc/l42 0 2024-03-16T10:45:31.241 INFO:tasks.workunit.client.1.smithi026.stdout:3/880: chown d4/dc/dd/fd8 113745630 1 2024-03-16T10:45:31.242 INFO:tasks.workunit.client.1.smithi026.stdout:0/882: dread - d1/d12/d2f/daa/dd1/d1a/d2b/d4f/da7/f121 zero size 2024-03-16T10:45:31.243 INFO:tasks.workunit.client.1.smithi026.stdout:3/881: write d4/d89/f8f [2606977,119368] 0 2024-03-16T10:45:31.244 INFO:tasks.workunit.client.1.smithi026.stdout:2/819: chown d2/d29/d3b/dbb/ld3 1046 1 2024-03-16T10:45:31.257 INFO:tasks.workunit.client.1.smithi026.stdout:8/734: rename d5/dc/d3f to d5/d5d/d9a/ddd 0 2024-03-16T10:45:31.260 INFO:tasks.workunit.client.1.smithi026.stdout:5/745: write d1/db/d27/f41 [407032,53804] 0 2024-03-16T10:45:31.260 INFO:tasks.workunit.client.1.smithi026.stdout:9/885: readlink d0/da/l4c 0 2024-03-16T10:45:31.263 INFO:tasks.workunit.client.1.smithi026.stdout:0/883: fdatasync d1/d12/d2f/d48/d55/d60/d9b/d11/fed 0 2024-03-16T10:45:31.267 INFO:tasks.workunit.client.1.smithi026.stdout:2/820: creat d2/d29/d51/d55/d6d/f11c x:0 0 0 2024-03-16T10:45:31.268 INFO:tasks.workunit.client.1.smithi026.stdout:4/845: write d1/db/f7e [103914,22406] 0 2024-03-16T10:45:31.271 INFO:tasks.workunit.client.1.smithi026.stdout:6/627: symlink d0/d41/dbe/ld1 0 2024-03-16T10:45:31.273 INFO:tasks.workunit.client.1.smithi026.stdout:7/827: rename d6/de/d7d/led to d6/d2f/dad/d7b/l10b 0 2024-03-16T10:45:31.274 INFO:tasks.workunit.client.1.smithi026.stdout:8/735: read - d5/d5d/d9a/fcf zero size 2024-03-16T10:45:31.275 INFO:tasks.workunit.client.1.smithi026.stdout:5/746: symlink d1/l10d 0 2024-03-16T10:45:31.276 INFO:tasks.workunit.client.1.smithi026.stdout:5/747: fdatasync d1/db/d1f/d61/fa7 0 2024-03-16T10:45:31.276 INFO:tasks.workunit.client.1.smithi026.stdout:8/736: write d5/d7/d7b/d96/fdb [1018976,39933] 0 2024-03-16T10:45:31.281 INFO:tasks.workunit.client.1.smithi026.stdout:3/882: creat d4/dc/dd/d47/d53/db1/d123/f130 x:0 0 0 2024-03-16T10:45:31.282 INFO:tasks.workunit.client.1.smithi026.stdout:3/883: dread - d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/f117 zero size 2024-03-16T10:45:31.285 INFO:tasks.workunit.client.1.smithi026.stdout:1/686: creat d0/d17/d1f/d63/d6a/d7c/d2e/d54/fd1 x:0 0 0 2024-03-16T10:45:31.289 INFO:tasks.workunit.client.1.smithi026.stdout:2/821: rename d2/f8f to d2/d29/d85/dc6/dd5/f11d 0 2024-03-16T10:45:31.290 INFO:tasks.workunit.client.1.smithi026.stdout:7/828: mknod d6/d35/d3a/d4e/dcc/d3f/dc5/de8/c10c 0 2024-03-16T10:45:31.291 INFO:tasks.workunit.client.1.smithi026.stdout:0/884: write d1/d12/d2f/f77 [414065,114016] 0 2024-03-16T10:45:31.292 INFO:tasks.workunit.client.1.smithi026.stdout:7/829: chown d6/d35/d3a/d4e/dcc/d3f/dc5 3882023 1 2024-03-16T10:45:31.295 INFO:tasks.workunit.client.1.smithi026.stdout:5/748: mknod d1/d1c/d2a/d55/d91/c10e 0 2024-03-16T10:45:31.297 INFO:tasks.workunit.client.1.smithi026.stdout:1/687: dread d0/d17/d1f/d63/d6a/fa2 [0,4194304] 0 2024-03-16T10:45:31.301 INFO:tasks.workunit.client.1.smithi026.stdout:4/846: creat d1/db/de/d4c/d59/d6a/df1/f109 x:0 0 0 2024-03-16T10:45:31.301 INFO:tasks.workunit.client.1.smithi026.stdout:6/628: mkdir d0/d10/dd2 0 2024-03-16T10:45:31.306 INFO:tasks.workunit.client.1.smithi026.stdout:5/749: dread d1/d1c/f50 [0,4194304] 0 2024-03-16T10:45:31.307 INFO:tasks.workunit.client.1.smithi026.stdout:7/830: fsync d6/de/d16/d88/fc1 0 2024-03-16T10:45:31.308 INFO:tasks.workunit.client.1.smithi026.stdout:5/750: chown d1/d1c/d4c/d4d/f4e 222216 1 2024-03-16T10:45:31.314 INFO:tasks.workunit.client.1.smithi026.stdout:1/688: fdatasync d0/d17/d1f/f90 0 2024-03-16T10:45:31.316 INFO:tasks.workunit.client.1.smithi026.stdout:4/847: creat d1/db/de/d4c/d59/f10a x:0 0 0 2024-03-16T10:45:31.318 INFO:tasks.workunit.client.1.smithi026.stdout:4/848: write d1/db/ddb/daa/d44/d48/d8e/fdf [252132,16584] 0 2024-03-16T10:45:31.319 INFO:tasks.workunit.client.1.smithi026.stdout:3/884: unlink d4/d89/d118/c125 0 2024-03-16T10:45:31.325 INFO:tasks.workunit.client.1.smithi026.stdout:2/822: mknod d2/d29/d17/d1a/d2b/c11e 0 2024-03-16T10:45:31.327 INFO:tasks.workunit.client.1.smithi026.stdout:8/737: dwrite d5/d7/f11 [0,4194304] 0 2024-03-16T10:45:31.327 INFO:tasks.workunit.client.1.smithi026.stdout:7/831: creat d6/d1e/f10d x:0 0 0 2024-03-16T10:45:31.330 INFO:tasks.workunit.client.1.smithi026.stdout:9/886: getdents d0/d63/d6b/ded 0 2024-03-16T10:45:31.332 INFO:tasks.workunit.client.1.smithi026.stdout:1/689: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/cd2 0 2024-03-16T10:45:31.334 INFO:tasks.workunit.client.1.smithi026.stdout:9/887: read - d0/da/d16/d5d/d107/f12a zero size 2024-03-16T10:45:31.335 INFO:tasks.workunit.client.1.smithi026.stdout:4/849: chown d1/db/ddb/la3 9 1 2024-03-16T10:45:31.336 INFO:tasks.workunit.client.1.smithi026.stdout:6/629: dwrite d0/d41/d4f/f96 [0,4194304] 0 2024-03-16T10:45:31.337 INFO:tasks.workunit.client.1.smithi026.stdout:9/888: stat d0/da/d16/d17/db1/f11e 0 2024-03-16T10:45:31.337 INFO:tasks.workunit.client.1.smithi026.stdout:3/885: rename d4/dc/d73/d83/d7c/dd7/d62/f6b to d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/f131 0 2024-03-16T10:45:31.338 INFO:tasks.workunit.client.1.smithi026.stdout:4/850: fsync d1/db/de/d4c/d6f/df3/f107 0 2024-03-16T10:45:31.340 INFO:tasks.workunit.client.1.smithi026.stdout:3/886: chown d4/dc/d31/d48/d69/f6f 76244640 1 2024-03-16T10:45:31.342 INFO:tasks.workunit.client.1.smithi026.stdout:9/889: dread - d0/da/d16/d5d/f103 zero size 2024-03-16T10:45:31.342 INFO:tasks.workunit.client.1.smithi026.stdout:5/751: mkdir d1/d1c/d2a/dab/dc5/d69/d10f 0 2024-03-16T10:45:31.347 INFO:tasks.workunit.client.1.smithi026.stdout:6/630: creat d0/d41/dbe/fd3 x:0 0 0 2024-03-16T10:45:31.349 INFO:tasks.workunit.client.1.smithi026.stdout:2/823: symlink d2/d29/d17/d1a/d2b/d69/l11f 0 2024-03-16T10:45:31.350 INFO:tasks.workunit.client.1.smithi026.stdout:9/890: dwrite d0/da/d16/f122 [0,4194304] 0 2024-03-16T10:45:31.351 INFO:tasks.workunit.client.1.smithi026.stdout:8/738: fsync d5/d23/d4e/f5b 0 2024-03-16T10:45:31.353 INFO:tasks.workunit.client.1.smithi026.stdout:7/832: getdents d6/de/d16/d88/df6 0 2024-03-16T10:45:31.355 INFO:tasks.workunit.client.1.smithi026.stdout:3/887: mkdir d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d132 0 2024-03-16T10:45:31.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.365+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cf019e5e0 con 0x7f4cd806dd60 2024-03-16T10:45:31.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.366+0000 7f4ced7fa700 1 -- 172.21.15.2:0/3392967277 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f4cf019e5e0 con 0x7f4cd806dd60 2024-03-16T10:45:31.370 INFO:tasks.workunit.client.1.smithi026.stdout:8/739: unlink d5/d23/d4e/d69/d76/f9e 0 2024-03-16T10:45:31.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4cd806dd60 msgr2=0x7f4cd8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4cd806dd60 0x7f4cd8070220 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0100a30 tx=0x7f4ce000b410 comp rx=0 tx=0).stop 2024-03-16T10:45:31.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 msgr2=0x7f4cf0101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101fb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4006010 tx=0x7f4ce400bd70 comp rx=0 tx=0).stop 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 shutdown_connections 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4cd806dd60 0x7f4cd8070220 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cf00ff9d0 0x7f4cf01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 --2- 172.21.15.2:0/3392967277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cf0100bd0 0x7f4cf0101fb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 >> 172.21.15.2:0/3392967277 conn(0x7f4cf00faf70 msgr2=0x7f4cf00fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 shutdown_connections 2024-03-16T10:45:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.369+0000 7f4cf6350700 1 -- 172.21.15.2:0/3392967277 wait complete. 2024-03-16T10:45:31.371 INFO:tasks.workunit.client.1.smithi026.stdout:1/690: write d0/f7f [2013,47229] 0 2024-03-16T10:45:31.373 INFO:tasks.workunit.client.1.smithi026.stdout:5/752: unlink d1/d1c/d2a/d55/d99/c9a 0 2024-03-16T10:45:31.374 INFO:tasks.workunit.client.1.smithi026.stdout:7/833: truncate d6/d35/d3a/d4e/dcc/d3f/d9d/fa7 2585234 0 2024-03-16T10:45:31.378 INFO:tasks.workunit.client.1.smithi026.stdout:2/824: dread d2/d29/d17/d1a/d2b/f15 [0,4194304] 0 2024-03-16T10:45:31.378 INFO:tasks.workunit.client.1.smithi026.stdout:4/851: symlink d1/db/de/d6e/l10b 0 2024-03-16T10:45:31.379 INFO:tasks.workunit.client.1.smithi026.stdout:6/631: rename d0/d10/d79/f8d to d0/d10/d79/d8e/fd4 0 2024-03-16T10:45:31.384 INFO:tasks.workunit.client.1.smithi026.stdout:7/834: dwrite d6/f91 [4194304,4194304] 0 2024-03-16T10:45:31.387 INFO:tasks.workunit.client.1.smithi026.stdout:9/891: dwrite d0/d5f/da2/fa6 [0,4194304] 0 2024-03-16T10:45:31.391 INFO:tasks.workunit.client.1.smithi026.stdout:4/852: dread d1/db/ddb/daa/d44/dd5/de9/fea [0,4194304] 0 2024-03-16T10:45:31.392 INFO:tasks.workunit.client.1.smithi026.stdout:3/888: rename d4/dc/d73/d83/dbc to d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d10b/d133 0 2024-03-16T10:45:31.394 INFO:tasks.workunit.client.1.smithi026.stdout:3/889: write d4/dc/d31/d48/f9f [1393376,120958] 0 2024-03-16T10:45:31.400 INFO:tasks.workunit.client.1.smithi026.stdout:5/753: dwrite d1/d1c/f3c [0,4194304] 0 2024-03-16T10:45:31.408 INFO:tasks.workunit.client.1.smithi026.stdout:8/740: truncate d5/d23/d4e/f7c 1132987 0 2024-03-16T10:45:31.414 INFO:tasks.workunit.client.1.smithi026.stdout:9/892: unlink d0/da/d4d/l10e 0 2024-03-16T10:45:31.416 INFO:tasks.workunit.client.1.smithi026.stdout:4/853: fsync d1/db/de/f5b 0 2024-03-16T10:45:31.422 INFO:tasks.workunit.client.1.smithi026.stdout:7/835: rename d6/de/d16/d88/l9c to d6/d35/d3a/d4e/dcc/d3f/d65/l10e 0 2024-03-16T10:45:31.422 INFO:tasks.workunit.client.1.smithi026.stdout:8/741: dread d5/d7/f43 [4194304,4194304] 0 2024-03-16T10:45:31.426 INFO:tasks.workunit.client.1.smithi026.stdout:3/890: creat d4/dc/d73/d83/d7c/dd7/f134 x:0 0 0 2024-03-16T10:45:31.427 INFO:tasks.workunit.client.1.smithi026.stdout:2/825: mknod d2/d29/d17/d1a/d2b/d2e/c120 0 2024-03-16T10:45:31.429 INFO:tasks.workunit.client.1.smithi026.stdout:2/826: chown d2/d29/d17/c1e 4 1 2024-03-16T10:45:31.429 INFO:tasks.workunit.client.1.smithi026.stdout:7/836: dwrite d6/d35/d3a/d4e/dcc/d3f/d54/de1/ff0 [0,4194304] 0 2024-03-16T10:45:31.429 INFO:tasks.workunit.client.1.smithi026.stdout:5/754: mkdir d1/db/d1f/d61/d110 0 2024-03-16T10:45:31.430 INFO:tasks.workunit.client.1.smithi026.stdout:2/827: chown d2/d29/d51/d55/c10e 8 1 2024-03-16T10:45:31.431 INFO:tasks.workunit.client.1.smithi026.stdout:5/755: chown d1/d1c/d68/f74 1581832116 1 2024-03-16T10:45:31.432 INFO:tasks.workunit.client.1.smithi026.stdout:5/756: readlink d1/db/l100 0 2024-03-16T10:45:31.432 INFO:tasks.workunit.client.1.smithi026.stdout:2/828: chown d2/d29/d17/d1a/d2b/d4e/d67/fd2 512085175 1 2024-03-16T10:45:31.434 INFO:tasks.workunit.client.1.smithi026.stdout:4/854: symlink d1/db/de/d4c/d59/d6a/db2/db3/l10c 0 2024-03-16T10:45:31.435 INFO:tasks.workunit.client.1.smithi026.stdout:4/855: chown d1/l35 763971429 1 2024-03-16T10:45:31.436 INFO:tasks.workunit.client.1.smithi026.stdout:8/742: mkdir d5/d7/d2c/d36/d59/d93/dde 0 2024-03-16T10:45:31.441 INFO:tasks.workunit.client.1.smithi026.stdout:6/632: creat d0/d10/d79/fd5 x:0 0 0 2024-03-16T10:45:31.442 INFO:tasks.workunit.client.1.smithi026.stdout:8/743: dwrite d5/d5d/d9a/f9b [0,4194304] 0 2024-03-16T10:45:31.443 INFO:tasks.workunit.client.1.smithi026.stdout:8/744: chown d5/d7/d7b 0 1 2024-03-16T10:45:31.449 INFO:tasks.workunit.client.1.smithi026.stdout:6/633: dwrite d0/d28/d2d/f68 [0,4194304] 0 2024-03-16T10:45:31.456 INFO:tasks.workunit.client.1.smithi026.stdout:3/891: mkdir d4/dc/dd/d135 0 2024-03-16T10:45:31.459 INFO:tasks.workunit.client.1.smithi026.stdout:4/856: dread d1/db/de/f3d [0,4194304] 0 2024-03-16T10:45:31.459 INFO:tasks.workunit.client.1.smithi026.stdout:3/892: fdatasync d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/f126 0 2024-03-16T10:45:31.460 INFO:tasks.workunit.client.1.smithi026.stdout:7/837: mknod d6/d35/d3a/d59/df1/c10f 0 2024-03-16T10:45:31.460 INFO:tasks.workunit.client.1.smithi026.stdout:4/857: chown d1/db/de 13426749 1 2024-03-16T10:45:31.462 INFO:tasks.workunit.client.1.smithi026.stdout:7/838: write d6/d1e/da6/f102 [2309542,96100] 0 2024-03-16T10:45:31.464 INFO:tasks.workunit.client.1.smithi026.stdout:8/745: rename d5/d7/f43 to d5/d28/fdf 0 2024-03-16T10:45:31.464 INFO:tasks.workunit.client.1.smithi026.stdout:2/829: write d2/d29/f3a [1348151,78563] 0 2024-03-16T10:45:31.465 INFO:tasks.workunit.client.1.smithi026.stdout:5/757: mknod d1/d1c/d2a/dab/dc5/d69/d10f/c111 0 2024-03-16T10:45:31.470 INFO:tasks.workunit.client.1.smithi026.stdout:6/634: truncate d0/d10/fd 135257 0 2024-03-16T10:45:31.471 INFO:tasks.workunit.client.1.smithi026.stdout:7/839: dread d6/d35/d3a/d4e/dcc/d3f/fa0 [0,4194304] 0 2024-03-16T10:45:31.473 INFO:tasks.workunit.client.1.smithi026.stdout:5/758: dread d1/d1c/d2a/dab/dc5/fa9 [0,4194304] 0 2024-03-16T10:45:31.475 INFO:tasks.workunit.client.1.smithi026.stdout:3/893: unlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dae/lbb 0 2024-03-16T10:45:31.475 INFO:tasks.workunit.client.1.smithi026.stdout:3/894: dread - d4/dc/d73/d83/f128 zero size 2024-03-16T10:45:31.489 INFO:tasks.workunit.client.1.smithi026.stdout:3/895: symlink d4/dc/d73/d83/l136 0 2024-03-16T10:45:31.491 INFO:tasks.workunit.client.1.smithi026.stdout:3/896: write d4/dc/d73/d83/d7c/dd7/f134 [465341,41748] 0 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 -- 172.21.15.2:0/662769441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 msgr2=0x7fe7100ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 --2- 172.21.15.2:0/662769441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe7100ffc50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe700009910 tx=0x7fe700009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 -- 172.21.15.2:0/662769441 shutdown_connections 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 --2- 172.21.15.2:0/662769441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710100a30 0x7fe710100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 --2- 172.21.15.2:0/662769441 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe7100ffc50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 -- 172.21.15.2:0/662769441 >> 172.21.15.2:0/662769441 conn(0x7fe7100fadf0 msgr2=0x7fe7100fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 -- 172.21.15.2:0/662769441 shutdown_connections 2024-03-16T10:45:31.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.491+0000 7fe7167a2700 1 -- 172.21.15.2:0/662769441 wait complete. 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7167a2700 1 Processor -- start 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7167a2700 1 -- start start 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7167a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe710196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7167a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710100a30 0x7fe710196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7167a2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710196ea0 con 0x7fe7100ff830 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7167a2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710196fe0 con 0x7fe710100a30 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7157a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe710196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7157a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe710196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58254/0 (socket says 172.21.15.2:58254) 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7157a0700 1 -- 172.21.15.2:0/3690643528 learned_addr learned my addr 172.21.15.2:0/3690643528 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe714f9f700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710100a30 0x7fe710196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7157a0700 1 -- 172.21.15.2:0/3690643528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710100a30 msgr2=0x7fe710196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.492+0000 7fe7157a0700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710100a30 0x7fe710196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.493+0000 7fe7157a0700 1 -- 172.21.15.2:0/3690643528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000095f0 con 0x7fe7100ff830 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.493+0000 7fe7157a0700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe710196340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe700005f00 tx=0x7fe70000b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.493+0000 7fe70e7fc700 1 -- 172.21.15.2:0/3690643528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70001d070 con 0x7fe7100ff830 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.493+0000 7fe70e7fc700 1 -- 172.21.15.2:0/3690643528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7000052a0 con 0x7fe7100ff830 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.493+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71019b9e0 con 0x7fe7100ff830 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.493+0000 7fe70e7fc700 1 -- 172.21.15.2:0/3690643528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe700017400 con 0x7fe7100ff830 2024-03-16T10:45:31.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.493+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71019bf50 con 0x7fe7100ff830 2024-03-16T10:45:31.496 INFO:tasks.workunit.client.1.smithi026.stdout:8/746: write d5/d7/d7b/f94 [3613139,45728] 0 2024-03-16T10:45:31.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.494+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71004f2d0 con 0x7fe7100ff830 2024-03-16T10:45:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.494+0000 7fe70e7fc700 1 -- 172.21.15.2:0/3690643528 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 31) v1 ==== 91794+0+0 (secure 0 0 0) 0x7fe700003680 con 0x7fe7100ff830 2024-03-16T10:45:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.495+0000 7fe70e7fc700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe6f806dd60 0x7fe6f8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.495+0000 7fe70e7fc700 1 -- 172.21.15.2:0/3690643528 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe70008ee60 con 0x7fe7100ff830 2024-03-16T10:45:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.495+0000 7fe714f9f700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe6f806dd60 0x7fe6f8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.495+0000 7fe714f9f700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe6f806dd60 0x7fe6f8070220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe704009c60 tx=0x7fe704009260 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:31.498 INFO:tasks.workunit.client.1.smithi026.stdout:4/858: creat d1/db/ddb/f10d x:0 0 0 2024-03-16T10:45:31.499 INFO:tasks.workunit.client.1.smithi026.stdout:4/859: stat d1/db/de/d4c/d59/f10a 0 2024-03-16T10:45:31.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.500+0000 7fe70e7fc700 1 -- 172.21.15.2:0/3690643528 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fe70005c0c0 con 0x7fe7100ff830 2024-03-16T10:45:31.500 INFO:tasks.workunit.client.1.smithi026.stdout:4/860: chown d1/db/de/d4c/d6b/dbc/l108 14267 1 2024-03-16T10:45:31.506 INFO:tasks.workunit.client.1.smithi026.stdout:5/759: symlink d1/d1c/d2a/l112 0 2024-03-16T10:45:31.508 INFO:tasks.workunit.client.1.smithi026.stdout:3/897: truncate d4/dc/dd/f21 2573420 0 2024-03-16T10:45:31.509 INFO:tasks.workunit.client.1.smithi026.stdout:3/898: chown d4/dc/d31/d48/fd1 15 1 2024-03-16T10:45:31.509 INFO:tasks.workunit.client.1.smithi026.stdout:3/899: readlink d4/d89/d97/lba 0 2024-03-16T10:45:31.514 INFO:tasks.workunit.client.1.smithi026.stdout:6/635: dwrite d0/fbd [0,4194304] 0 2024-03-16T10:45:31.516 INFO:tasks.workunit.client.1.smithi026.stdout:2/830: truncate d2/d29/d17/d1a/d2b/d2e/d60/d7f/fb4 1406497 0 2024-03-16T10:45:31.517 INFO:tasks.workunit.client.1.smithi026.stdout:7/840: write d6/d35/d3a/d4e/dcc/d3f/d9d/fa7 [3298952,6208] 0 2024-03-16T10:45:31.521 INFO:tasks.workunit.client.1.smithi026.stdout:3/900: symlink d4/dc/d31/l137 0 2024-03-16T10:45:31.522 INFO:tasks.workunit.client.1.smithi026.stdout:8/747: mkdir d5/d7/d2c/d36/d59/d93/dde/de0 0 2024-03-16T10:45:31.522 INFO:tasks.workunit.client.1.smithi026.stdout:3/901: readlink d4/dc/d73/d83/d7c/dd7/d62/l71 0 2024-03-16T10:45:31.526 INFO:tasks.workunit.client.1.smithi026.stdout:6/636: symlink d0/d28/d49/d7d/ld6 0 2024-03-16T10:45:31.527 INFO:tasks.workunit.client.1.smithi026.stdout:6/637: write d0/d28/d49/f78 [1069485,92800] 0 2024-03-16T10:45:31.528 INFO:tasks.workunit.client.1.smithi026.stdout:2/831: creat d2/d29/d17/d38/d64/f121 x:0 0 0 2024-03-16T10:45:31.530 INFO:tasks.workunit.client.1.smithi026.stdout:7/841: mknod d6/d35/d3a/d4e/db0/c110 0 2024-03-16T10:45:31.533 INFO:tasks.workunit.client.1.smithi026.stdout:5/760: symlink d1/db/l113 0 2024-03-16T10:45:31.623 INFO:tasks.workunit.client.1.smithi026.stdout:3/902: chown d4/d89/cb2 161217 1 2024-03-16T10:45:31.630 INFO:tasks.workunit.client.1.smithi026.stdout:4/861: rmdir d1/db/ddb/daf/df9 0 2024-03-16T10:45:31.634 INFO:tasks.workunit.client.1.smithi026.stdout:2/832: truncate d2/d29/d17/d1a/d2b/d4b/f5a 989016 0 2024-03-16T10:45:31.636 INFO:tasks.workunit.client.1.smithi026.stdout:2/833: chown d2/d29/d17/d1a/d2b/d4b/fb0 2042786 1 2024-03-16T10:45:31.637 INFO:tasks.workunit.client.1.smithi026.stdout:6/638: dread d0/d28/fce [0,4194304] 0 2024-03-16T10:45:31.646 INFO:tasks.workunit.client.1.smithi026.stdout:2/834: mkdir d2/d29/d36/d98/d122 0 2024-03-16T10:45:31.647 INFO:tasks.workunit.client.1.smithi026.stdout:4/862: dwrite d1/db/ddb/daa/d61/f75 [0,4194304] 0 2024-03-16T10:45:31.652 INFO:tasks.workunit.client.1.smithi026.stdout:8/748: link d5/la4 d5/d7/d2c/d36/d8c/dd3/le1 0 2024-03-16T10:45:31.653 INFO:tasks.workunit.client.1.smithi026.stdout:8/749: write d5/d5d/d9a/ddd/fce [327951,118761] 0 2024-03-16T10:45:31.656 INFO:tasks.workunit.client.1.smithi026.stdout:7/842: link d6/d35/d3a/d4e/dcc/d3f/d65/cd3 d6/d35/d3a/d4e/db0/c111 0 2024-03-16T10:45:31.658 INFO:tasks.workunit.client.1.smithi026.stdout:7/843: chown d6/d35/d3a/d4e/dcc/d3f/d9d/fba 2 1 2024-03-16T10:45:31.668 INFO:tasks.workunit.client.1.smithi026.stdout:8/750: rmdir d5/d28 39 2024-03-16T10:45:31.671 INFO:tasks.workunit.client.1.smithi026.stdout:5/761: rename d1/db/c4a to d1/d1c/d2a/dab/dc5/d69/c114 0 2024-03-16T10:45:31.671 INFO:tasks.workunit.client.1.smithi026.stdout:8/751: chown d5/d7/f8d 1261617593 1 2024-03-16T10:45:31.673 INFO:tasks.workunit.client.1.smithi026.stdout:4/863: dwrite d1/db/de/f58 [0,4194304] 0 2024-03-16T10:45:31.675 INFO:tasks.workunit.client.1.smithi026.stdout:3/903: dwrite d4/dc/dd/d47/d53/fca [0,4194304] 0 2024-03-16T10:45:31.675 INFO:tasks.workunit.client.1.smithi026.stdout:7/844: creat d6/d35/d3a/d4e/dcc/d3f/d54/dd8/f112 x:0 0 0 2024-03-16T10:45:31.685 INFO:tasks.workunit.client.1.smithi026.stdout:7/845: dwrite d6/d2f/ffc [0,4194304] 0 2024-03-16T10:45:31.687 INFO:tasks.workunit.client.1.smithi026.stdout:2/835: link d2/f116 d2/d29/d36/f123 0 2024-03-16T10:45:31.689 INFO:tasks.workunit.client.1.smithi026.stdout:5/762: dread d1/d6e/f92 [0,4194304] 0 2024-03-16T10:45:31.691 INFO:tasks.workunit.client.1.smithi026.stdout:4/864: mkdir d1/d10e 0 2024-03-16T10:45:31.696 INFO:tasks.workunit.client.1.smithi026.stdout:8/752: creat d5/d23/d4e/d69/dd5/fe2 x:0 0 0 2024-03-16T10:45:31.699 INFO:tasks.workunit.client.1.smithi026.stdout:6/639: dwrite d0/d10/d79/d7b/f82 [0,4194304] 0 2024-03-16T10:45:31.699 INFO:tasks.workunit.client.1.smithi026.stdout:3/904: mknod d4/dc/dd/c138 0 2024-03-16T10:45:31.699 INFO:tasks.workunit.client.1.smithi026.stdout:8/753: read - d5/d7/d2c/d36/d59/d93/da3/fd9 zero size 2024-03-16T10:45:31.701 INFO:tasks.workunit.client.1.smithi026.stdout:8/754: dread - d5/d5d/fc3 zero size 2024-03-16T10:45:31.706 INFO:tasks.workunit.client.1.smithi026.stdout:2/836: creat d2/d29/d110/f124 x:0 0 0 2024-03-16T10:45:31.709 INFO:tasks.workunit.client.1.smithi026.stdout:7/846: rename d6/d2f/ffb to d6/d35/d3a/d59/d9e/f113 0 2024-03-16T10:45:31.710 INFO:tasks.workunit.client.1.smithi026.stdout:3/905: truncate d4/dc/d73/d83/d57/fe7 130549 0 2024-03-16T10:45:31.711 INFO:tasks.workunit.client.1.smithi026.stdout:6/640: mkdir d0/d10/d79/dca/dd7 0 2024-03-16T10:45:31.712 INFO:tasks.workunit.client.1.smithi026.stdout:6/641: chown d0/d41/d4f/f69 103854489 1 2024-03-16T10:45:31.713 INFO:tasks.workunit.client.1.smithi026.stdout:2/837: creat d2/d29/da4/f125 x:0 0 0 2024-03-16T10:45:31.716 INFO:tasks.workunit.client.1.smithi026.stdout:3/906: mknod d4/dc/dd/d80/c139 0 2024-03-16T10:45:31.718 INFO:tasks.workunit.client.1.smithi026.stdout:6/642: truncate d0/d28/f9c 20047 0 2024-03-16T10:45:31.718 INFO:tasks.workunit.client.1.smithi026.stdout:6/643: fsync d0/d10/ff 0 2024-03-16T10:45:31.719 INFO:tasks.workunit.client.1.smithi026.stdout:5/763: getdents d1/db/d1f/d61/d65 0 2024-03-16T10:45:31.721 INFO:tasks.workunit.client.1.smithi026.stdout:2/838: creat d2/d29/d85/dc6/dd5/f126 x:0 0 0 2024-03-16T10:45:31.725 INFO:tasks.workunit.client.1.smithi026.stdout:6/644: creat d0/d41/fd8 x:0 0 0 2024-03-16T10:45:31.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.730+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe710068c40 con 0x7fe6f806dd60 2024-03-16T10:45:31.731 INFO:tasks.workunit.client.1.smithi026.stdout:6/645: creat d0/d8a/fd9 x:0 0 0 2024-03-16T10:45:31.732 INFO:tasks.workunit.client.1.smithi026.stdout:6/646: truncate d0/d62/fbf 844173 0 2024-03-16T10:45:31.734 INFO:tasks.workunit.client.1.smithi026.stdout:5/764: link d1/db/d1f/d61/d65/f83 d1/d1c/d2a/dab/dc5/d69/d51/d80/de9/f115 0 2024-03-16T10:45:31.734 INFO:tasks.workunit.client.1.smithi026.stdout:5/765: readlink d1/d1c/l36 0 2024-03-16T10:45:31.737 INFO:tasks.workunit.client.1.smithi026.stdout:3/907: link d4/d89/d97/c44 d4/dc/d73/d83/d7c/c13a 0 2024-03-16T10:45:31.740 INFO:tasks.workunit.client.1.smithi026.stdout:3/908: dread - d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/f117 zero size 2024-03-16T10:45:31.741 INFO:tasks.workunit.client.1.smithi026.stdout:3/909: stat d4/dc/d31/d48/d69/f6f 0 2024-03-16T10:45:31.742 INFO:tasks.workunit.client.1.smithi026.stdout:3/910: fsync d4/d89/f8f 0 2024-03-16T10:45:31.743 INFO:tasks.workunit.client.1.smithi026.stdout:2/839: link d2/d29/d17/d38/cac d2/d29/d3b/c127 0 2024-03-16T10:45:31.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.745+0000 7fe70e7fc700 1 -- 172.21.15.2:0/3690643528 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe710068c40 con 0x7fe6f806dd60 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (7m) 10s ago 9m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (9m) 10s ago 9m 7671k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (8m) 75s ago 8m 7596k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (9m) 10s ago 9m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (8m) 75s ago 8m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (7m) 10s ago 8m 85.0M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (3m) 10s ago 3m 16.4M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (3m) 10s ago 3m 13.6M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (3m) 75s ago 3m 63.7M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (3m) 75s ago 3m 14.6M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:45:31.746 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (44s) 10s ago 10m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (102s) 75s ago 8m 492M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5df99c885681 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (10m) 10s ago 10m 51.8M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (8m) 75s ago 8m 39.1M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (9m) 10s ago 9m 16.7M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (8m) 75s ago 8m 15.5M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (7m) 10s ago 7m 169M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (6m) 10s ago 6m 175M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (6m) 10s ago 6m 148M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (5m) 75s ago 5m 60.3M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (5m) 75s ago 5m 55.9M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (5m) 75s ago 5m 57.2M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:45:31.747 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (50s) 10s ago 8m 48.5M - 2.43.0 a07b618ecd1d 0ac6508be9de 2024-03-16T10:45:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.748+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe6f806dd60 msgr2=0x7fe6f8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.748+0000 7fe7167a2700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe6f806dd60 0x7fe6f8070220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe704009c60 tx=0x7fe704009260 comp rx=0 tx=0).stop 2024-03-16T10:45:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.748+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 msgr2=0x7fe710196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.748+0000 7fe7167a2700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe710196340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe700005f00 tx=0x7fe70000b5e0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.749+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 shutdown_connections 2024-03-16T10:45:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.749+0000 7fe7167a2700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe6f806dd60 0x7fe6f8070220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.749+0000 7fe7167a2700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7100ff830 0x7fe710196340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.749+0000 7fe7167a2700 1 --2- 172.21.15.2:0/3690643528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710100a30 0x7fe710196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.749+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 >> 172.21.15.2:0/3690643528 conn(0x7fe7100fadf0 msgr2=0x7fe710068530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.750+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 shutdown_connections 2024-03-16T10:45:31.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.750+0000 7fe7167a2700 1 -- 172.21.15.2:0/3690643528 wait complete. 2024-03-16T10:45:31.751 INFO:tasks.workunit.client.1.smithi026.stdout:6/647: creat d0/d28/dcb/fda x:0 0 0 2024-03-16T10:45:31.759 INFO:tasks.workunit.client.1.smithi026.stdout:2/840: write d2/d29/d17/d1a/d2b/d4e/d6f/f9c [2225986,111865] 0 2024-03-16T10:45:31.762 INFO:tasks.workunit.client.1.smithi026.stdout:6/648: chown d0/d5b/c1e 50799519 1 2024-03-16T10:45:31.763 INFO:tasks.workunit.client.1.smithi026.stdout:3/911: fdatasync d4/faa 0 2024-03-16T10:45:31.778 INFO:tasks.workunit.client.1.smithi026.stdout:7/847: dwrite d6/de/d7d/fae [0,4194304] 0 2024-03-16T10:45:31.786 INFO:tasks.workunit.client.1.smithi026.stdout:7/848: dwrite d6/d35/d3a/d4e/dcc/d3f/d54/dd8/f112 [0,4194304] 0 2024-03-16T10:45:31.790 INFO:tasks.workunit.client.1.smithi026.stdout:7/849: creat d6/de/d16/d44/f114 x:0 0 0 2024-03-16T10:45:31.795 INFO:tasks.workunit.client.1.smithi026.stdout:7/850: symlink d6/d35/d3a/d4e/dcc/d3f/d54/l115 0 2024-03-16T10:45:31.808 INFO:tasks.workunit.client.1.smithi026.stdout:5/766: dwrite d1/d1c/d2a/d55/d99/fb4 [0,4194304] 0 2024-03-16T10:45:31.810 INFO:tasks.workunit.client.1.smithi026.stdout:2/841: dwrite d2/d29/da4/ffc [0,4194304] 0 2024-03-16T10:45:31.811 INFO:tasks.workunit.client.1.smithi026.stdout:7/851: write d6/f39 [2436380,5814] 0 2024-03-16T10:45:31.822 INFO:tasks.workunit.client.1.smithi026.stdout:2/842: link d2/d29/d17/d38/d64/dcb/l99 d2/d29/d17/d1a/d2b/d2e/d60/d7f/l128 0 2024-03-16T10:45:31.826 INFO:tasks.workunit.client.1.smithi026.stdout:5/767: dwrite d1/d1c/d2a/d55/d91/fb2 [0,4194304] 0 2024-03-16T10:45:31.829 INFO:tasks.workunit.client.1.smithi026.stdout:7/852: dwrite d6/fc [0,4194304] 0 2024-03-16T10:45:31.830 INFO:tasks.workunit.client.1.smithi026.stdout:5/768: write d1/d1c/d2a/d55/d91/ffd [942157,54815] 0 2024-03-16T10:45:31.832 INFO:tasks.workunit.client.1.smithi026.stdout:5/769: write d1/d1c/d2a/d55/d91/ffd [566423,61134] 0 2024-03-16T10:45:31.841 INFO:tasks.workunit.client.1.smithi026.stdout:5/770: getdents d1/d1c/d2a 0 2024-03-16T10:45:31.847 INFO:tasks.workunit.client.1.smithi026.stdout:2/843: dread d2/d29/d17/d38/d64/f90 [0,4194304] 0 2024-03-16T10:45:31.849 INFO:tasks.workunit.client.1.smithi026.stdout:7/853: dread d6/d2f/f4a [0,4194304] 0 2024-03-16T10:45:31.851 INFO:tasks.workunit.client.1.smithi026.stdout:2/844: unlink d2/d29/d17/d1a/d2b/d4e/d6f/c107 0 2024-03-16T10:45:31.853 INFO:tasks.workunit.client.1.smithi026.stdout:7/854: symlink d6/d35/l116 0 2024-03-16T10:45:31.853 INFO:tasks.workunit.client.1.smithi026.stdout:7/855: chown d6/de/d16/d88/df6 411412583 1 2024-03-16T10:45:31.854 INFO:tasks.workunit.client.1.smithi026.stdout:5/771: rmdir d1/d1c/d2a/dbe 0 2024-03-16T10:45:31.855 INFO:tasks.workunit.client.1.smithi026.stdout:5/772: chown d1/d6e/f70 145181 1 2024-03-16T10:45:31.856 INFO:tasks.workunit.client.1.smithi026.stdout:2/845: mknod d2/d29/d17/d1a/d104/c129 0 2024-03-16T10:45:31.857 INFO:tasks.workunit.client.1.smithi026.stdout:7/856: creat d6/de/d16/d44/f117 x:0 0 0 2024-03-16T10:45:31.859 INFO:tasks.workunit.client.1.smithi026.stdout:5/773: rename d1/db/d27 to d1/db/d1f/d61/d116 0 2024-03-16T10:45:31.866 INFO:tasks.workunit.client.1.smithi026.stdout:2/846: mknod d2/d29/d17/d1a/d2b/c12a 0 2024-03-16T10:45:31.871 INFO:tasks.workunit.client.1.smithi026.stdout:5/774: rmdir d1/d1c/d2a/d55 39 2024-03-16T10:45:31.872 INFO:tasks.workunit.client.1.smithi026.stdout:7/857: getdents d6/de/d16 0 2024-03-16T10:45:31.875 INFO:tasks.workunit.client.1.smithi026.stdout:7/858: creat d6/d2f/dad/d7b/f118 x:0 0 0 2024-03-16T10:45:31.875 INFO:tasks.workunit.client.1.smithi026.stdout:2/847: link d2/d29/d51/d55/l8b d2/d29/d85/dc6/l12b 0 2024-03-16T10:45:31.878 INFO:tasks.workunit.client.1.smithi026.stdout:5/775: chown d1/d1c/d2a/d55/d99/f96 0 1 2024-03-16T10:45:31.878 INFO:tasks.workunit.client.1.smithi026.stdout:7/859: symlink d6/d35/d3a/d4e/dcc/d3f/d54/dd8/l119 0 2024-03-16T10:45:31.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.883+0000 7f4e3955c700 1 -- 172.21.15.2:0/2395865975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 msgr2=0x7f4e34100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.883+0000 7f4e3955c700 1 --2- 172.21.15.2:0/2395865975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e34100110 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009910 tx=0x7f4e28009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:31.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.883+0000 7f4e3955c700 1 -- 172.21.15.2:0/2395865975 shutdown_connections 2024-03-16T10:45:31.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.883+0000 7f4e3955c700 1 --2- 172.21.15.2:0/2395865975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e34100110 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.883+0000 7f4e3955c700 1 --2- 172.21.15.2:0/2395865975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e34100fe0 0x7f4e34101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.883+0000 7f4e3955c700 1 -- 172.21.15.2:0/2395865975 >> 172.21.15.2:0/2395865975 conn(0x7f4e340fa700 msgr2=0x7f4e340fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:31.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.884+0000 7f4e3955c700 1 -- 172.21.15.2:0/2395865975 shutdown_connections 2024-03-16T10:45:31.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.884+0000 7f4e3955c700 1 -- 172.21.15.2:0/2395865975 wait complete. 2024-03-16T10:45:31.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.884+0000 7f4e3955c700 1 Processor -- start 2024-03-16T10:45:31.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.884+0000 7f4e3955c700 1 -- start start 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e3955c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e341039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e3955c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e34100fe0 0x7f4e34102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e3955c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e341025c0 con 0x7f4e340ffc90 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e3955c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e34102700 con 0x7f4e34100fe0 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e32ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e341039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e32ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e341039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58276/0 (socket says 172.21.15.2:58276) 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e32ffd700 1 -- 172.21.15.2:0/3049296225 learned_addr learned my addr 172.21.15.2:0/3049296225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e327fc700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e34100fe0 0x7f4e34102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e32ffd700 1 -- 172.21.15.2:0/3049296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e34100fe0 msgr2=0x7f4e34102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e32ffd700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e34100fe0 0x7f4e34102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e32ffd700 1 -- 172.21.15.2:0/3049296225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e280095f0 con 0x7f4e340ffc90 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e32ffd700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e341039e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400e8f0 tx=0x7f4e2400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e1bfff700 1 -- 172.21.15.2:0/3049296225 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e24009790 con 0x7f4e340ffc90 2024-03-16T10:45:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e1bfff700 1 -- 172.21.15.2:0/3049296225 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e2400c810 con 0x7f4e340ffc90 2024-03-16T10:45:31.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.885+0000 7f4e1bfff700 1 -- 172.21.15.2:0/3049296225 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e24003d20 con 0x7f4e340ffc90 2024-03-16T10:45:31.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.886+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e341029e0 con 0x7f4e340ffc90 2024-03-16T10:45:31.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.886+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3419e640 con 0x7f4e340ffc90 2024-03-16T10:45:31.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.887+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3404ea80 con 0x7f4e340ffc90 2024-03-16T10:45:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.887+0000 7f4e1bfff700 1 -- 172.21.15.2:0/3049296225 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 31) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f4e240041f0 con 0x7f4e340ffc90 2024-03-16T10:45:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.888+0000 7f4e1bfff700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4e1c06dd60 0x7f4e1c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.888+0000 7f4e1bfff700 1 -- 172.21.15.2:0/3049296225 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4e24014070 con 0x7f4e340ffc90 2024-03-16T10:45:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.888+0000 7f4e327fc700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4e1c06dd60 0x7f4e1c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:31.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.888+0000 7f4e327fc700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4e1c06dd60 0x7f4e1c070220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b800 tx=0x7f4e28009f70 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:31.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:31.892+0000 7f4e1bfff700 1 -- 172.21.15.2:0/3049296225 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f4e2405aff0 con 0x7f4e340ffc90 2024-03-16T10:45:31.969 INFO:tasks.workunit.client.1.smithi026.stdout:5/776: write d1/d1c/f50 [2369,10077] 0 2024-03-16T10:45:31.972 INFO:tasks.workunit.client.1.smithi026.stdout:2/848: getdents d2/d29/d51 0 2024-03-16T10:45:31.974 INFO:tasks.workunit.client.1.smithi026.stdout:5/777: rmdir d1/d1c/d2a/d55/d91 39 2024-03-16T10:45:31.977 INFO:tasks.workunit.client.1.smithi026.stdout:5/778: creat d1/d1c/d2a/d55/d99/f117 x:0 0 0 2024-03-16T10:45:31.978 INFO:tasks.workunit.client.1.smithi026.stdout:7/860: getdents d6/d35/d3a/d4e/dcc/d73 0 2024-03-16T10:45:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:31 smithi026 ceph-mon[37041]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:31.982 INFO:tasks.workunit.client.1.smithi026.stdout:5/779: chown d1/d1c/d2a/db0 8 1 2024-03-16T10:45:31.982 INFO:tasks.workunit.client.1.smithi026.stdout:2/849: getdents d2/d29/d51/d55 0 2024-03-16T10:45:31.985 INFO:tasks.workunit.client.1.smithi026.stdout:7/861: dwrite d6/f39 [0,4194304] 0 2024-03-16T10:45:31.989 INFO:tasks.workunit.client.1.smithi026.stdout:7/862: truncate d6/d35/d3a/d59/d9e/fc2 2242713 0 2024-03-16T10:45:31.990 INFO:tasks.workunit.client.1.smithi026.stdout:7/863: write d6/d35/d3a/d4e/ff9 [263644,48035] 0 2024-03-16T10:45:31.991 INFO:tasks.workunit.client.1.smithi026.stdout:2/850: getdents d2/dd1 0 2024-03-16T10:45:31.993 INFO:tasks.workunit.client.1.smithi026.stdout:2/851: fsync d2/f4d 0 2024-03-16T10:45:32.000 INFO:tasks.workunit.client.1.smithi026.stdout:2/852: dwrite d2/d29/d17/fba [0,4194304] 0 2024-03-16T10:45:32.002 INFO:tasks.workunit.client.1.smithi026.stdout:7/864: dread d6/f38 [0,4194304] 0 2024-03-16T10:45:32.006 INFO:tasks.workunit.client.1.smithi026.stdout:2/853: creat d2/d29/d17/de8/f12c x:0 0 0 2024-03-16T10:45:32.009 INFO:tasks.workunit.client.1.smithi026.stdout:7/865: fdatasync d6/d35/d3a/d4e/d6c/fa4 0 2024-03-16T10:45:32.014 INFO:tasks.workunit.client.1.smithi026.stdout:2/854: symlink d2/d29/d36/l12d 0 2024-03-16T10:45:32.015 INFO:tasks.workunit.client.1.smithi026.stdout:2/855: chown d2/d29/d36/d98/d117 380464721 1 2024-03-16T10:45:32.017 INFO:tasks.workunit.client.1.smithi026.stdout:7/866: creat d6/d35/d3a/d4e/dcc/d3f/f11a x:0 0 0 2024-03-16T10:45:32.018 INFO:tasks.workunit.client.1.smithi026.stdout:2/856: creat d2/d29/d17/d1a/d2b/d2e/d60/f12e x:0 0 0 2024-03-16T10:45:32.025 INFO:tasks.workunit.client.1.smithi026.stdout:7/867: dwrite d6/d35/d3a/d4e/fbb [4194304,4194304] 0 2024-03-16T10:45:32.026 INFO:tasks.workunit.client.1.smithi026.stdout:7/868: truncate d6/de/f11 5539347 0 2024-03-16T10:45:32.027 INFO:tasks.workunit.client.1.smithi026.stdout:7/869: readlink d6/de/d16/l77 0 2024-03-16T10:45:32.028 INFO:tasks.workunit.client.1.smithi026.stdout:7/870: mknod d6/d35/d3a/dab/c11b 0 2024-03-16T10:45:32.031 INFO:tasks.workunit.client.1.smithi026.stdout:7/871: creat d6/d35/d3a/d4e/dcc/f11c x:0 0 0 2024-03-16T10:45:32.035 INFO:tasks.workunit.client.1.smithi026.stdout:7/872: mknod d6/d35/d3a/d4e/dcc/d3f/d70/c11d 0 2024-03-16T10:45:32.036 INFO:tasks.workunit.client.1.smithi026.stdout:2/857: dwrite d2/d29/d17/d1a/d2b/f91 [0,4194304] 0 2024-03-16T10:45:32.037 INFO:tasks.workunit.client.1.smithi026.stdout:7/873: dread - d6/de/d16/d44/f117 zero size 2024-03-16T10:45:32.042 INFO:tasks.workunit.client.1.smithi026.stdout:2/858: symlink d2/d29/d17/d1a/l12f 0 2024-03-16T10:45:32.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:31 smithi002 ceph-mon[29248]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:32.112 INFO:tasks.workunit.client.1.smithi026.stdout:0/885: sync 2024-03-16T10:45:32.120 INFO:tasks.workunit.client.1.smithi026.stdout:7/874: dread d6/f31 [0,4194304] 0 2024-03-16T10:45:32.124 INFO:tasks.workunit.client.1.smithi026.stdout:2/859: dwrite d2/d29/d17/d1a/d2b/d2e/fe4 [0,4194304] 0 2024-03-16T10:45:32.126 INFO:tasks.workunit.client.1.smithi026.stdout:2/860: chown d2/d29/d17/d1a/d2b/c7b 5 1 2024-03-16T10:45:32.126 INFO:tasks.workunit.client.1.smithi026.stdout:0/886: dwrite d1/d12/d2f/d48/d55/d60/de0/dea/f11b [0,4194304] 0 2024-03-16T10:45:32.149 INFO:tasks.workunit.client.1.smithi026.stdout:0/887: symlink d1/d12/d2f/d48/dfe/l12c 0 2024-03-16T10:45:32.151 INFO:tasks.workunit.client.1.smithi026.stdout:2/861: link d2/l1c d2/d29/d51/d55/l130 0 2024-03-16T10:45:32.153 INFO:tasks.workunit.client.1.smithi026.stdout:0/888: mkdir d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d104/d12d 0 2024-03-16T10:45:32.154 INFO:tasks.workunit.client.1.smithi026.stdout:2/862: readlink d2/d29/d17/d1a/d2b/d2e/d60/d7f/l128 0 2024-03-16T10:45:32.155 INFO:tasks.workunit.client.1.smithi026.stdout:0/889: stat d1/d12/d2f/d48/d55/d60/d9b/d51/d88/f9c 0 2024-03-16T10:45:32.157 INFO:tasks.workunit.client.1.smithi026.stdout:0/890: mknod d1/d12/c12e 0 2024-03-16T10:45:32.158 INFO:tasks.workunit.client.1.smithi026.stdout:0/891: read - d1/d12/d2f/d48/d55/d60/d9b/ffd zero size 2024-03-16T10:45:32.163 INFO:tasks.workunit.client.1.smithi026.stdout:0/892: read d1/d12/d2f/daa/dd1/f36 [1172527,63411] 0 2024-03-16T10:45:32.166 INFO:tasks.workunit.client.1.smithi026.stdout:0/893: creat d1/d12/d2f/d48/dfe/f12f x:0 0 0 2024-03-16T10:45:32.167 INFO:tasks.workunit.client.1.smithi026.stdout:0/894: chown d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/f112 3965938 1 2024-03-16T10:45:32.168 INFO:tasks.workunit.client.1.smithi026.stdout:7/875: write d6/d35/f66 [844416,5152] 0 2024-03-16T10:45:32.168 INFO:tasks.workunit.client.1.smithi026.stdout:0/895: fdatasync d1/d12/d2f/daa/dd1/d1a/f46 0 2024-03-16T10:45:32.174 INFO:tasks.workunit.client.1.smithi026.stdout:2/863: rename d2/d29/d51/d55 to d2/d29/d17/d1a/d2b/d2e/d60/d7f/d131 0 2024-03-16T10:45:32.175 INFO:tasks.workunit.client.1.smithi026.stdout:0/896: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/d122/c130 0 2024-03-16T10:45:32.177 INFO:tasks.workunit.client.1.smithi026.stdout:7/876: rename d6/d35/d3a/d4e/dcc/d3f/d65/f71 to d6/d1e/dbc/f11e 0 2024-03-16T10:45:32.181 INFO:tasks.workunit.client.1.smithi026.stdout:2/864: rename d2/d29/d36/d98/fe5 to d2/d29/d85/dc6/dd5/f132 0 2024-03-16T10:45:32.184 INFO:tasks.workunit.client.1.smithi026.stdout:7/877: dwrite d6/de/fbe [0,4194304] 0 2024-03-16T10:45:32.188 INFO:tasks.workunit.client.1.smithi026.stdout:7/878: truncate d6/de/d16/d88/fc1 358411 0 2024-03-16T10:45:32.189 INFO:tasks.workunit.client.1.smithi026.stdout:7/879: readlink d6/d35/d3a/d4e/l58 0 2024-03-16T10:45:32.193 INFO:tasks.workunit.client.1.smithi026.stdout:7/880: dread d6/fc [0,4194304] 0 2024-03-16T10:45:32.196 INFO:tasks.workunit.client.1.smithi026.stdout:2/865: creat d2/d29/d17/d1a/f133 x:0 0 0 2024-03-16T10:45:32.201 INFO:tasks.workunit.client.1.smithi026.stdout:7/881: rename d6/d35/d3a/d4e/dcc/d3f/fa0 to d6/de/d16/f11f 0 2024-03-16T10:45:32.201 INFO:tasks.workunit.client.1.smithi026.stdout:0/897: dwrite d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/f108 [0,4194304] 0 2024-03-16T10:45:32.205 INFO:tasks.workunit.client.1.smithi026.stdout:2/866: fsync d2/f4d 0 2024-03-16T10:45:32.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.207+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4e3419e930 con 0x7f4e340ffc90 2024-03-16T10:45:32.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.208+0000 7f4e1bfff700 1 -- 172.21.15.2:0/3049296225 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f4e2405ab80 con 0x7f4e340ffc90 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:45:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:45:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:45:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 12, 2024-03-16T10:45:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:45:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:45:32.210 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:45:32.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.210+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4e1c06dd60 msgr2=0x7f4e1c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.210+0000 7f4e3955c700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4e1c06dd60 0x7f4e1c070220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b800 tx=0x7f4e28009f70 comp rx=0 tx=0).stop 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.210+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 msgr2=0x7f4e341039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.210+0000 7f4e3955c700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e341039e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400e8f0 tx=0x7f4e2400ec00 comp rx=0 tx=0).stop 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.211+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 shutdown_connections 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.211+0000 7f4e3955c700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f4e1c06dd60 0x7f4e1c070220 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.211+0000 7f4e3955c700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e340ffc90 0x7f4e341039e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.211+0000 7f4e3955c700 1 --2- 172.21.15.2:0/3049296225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e34100fe0 0x7f4e34102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.211+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 >> 172.21.15.2:0/3049296225 conn(0x7f4e340fa700 msgr2=0x7f4e340fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.211+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 shutdown_connections 2024-03-16T10:45:32.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.211+0000 7f4e3955c700 1 -- 172.21.15.2:0/3049296225 wait complete. 2024-03-16T10:45:32.212 INFO:tasks.workunit.client.1.smithi026.stdout:2/867: creat d2/d29/d17/d1a/d2b/d2e/d60/d7f/ddc/f134 x:0 0 0 2024-03-16T10:45:32.214 INFO:tasks.workunit.client.1.smithi026.stdout:0/898: mkdir d1/d12/d2f/d113/d131 0 2024-03-16T10:45:32.223 INFO:tasks.workunit.client.1.smithi026.stdout:7/882: dwrite d6/d35/d3a/d4e/d6c/fd5 [0,4194304] 0 2024-03-16T10:45:32.227 INFO:tasks.workunit.client.1.smithi026.stdout:0/899: creat d1/d12/d2f/daa/dd1/d1a/d2b/d4f/f132 x:0 0 0 2024-03-16T10:45:32.227 INFO:tasks.workunit.client.1.smithi026.stdout:2/868: write d2/d29/d17/d1a/d2b/d4e/d6f/fcf [1265777,120668] 0 2024-03-16T10:45:32.229 INFO:tasks.workunit.client.1.smithi026.stdout:2/869: read - d2/d29/d17/d1a/d2b/d2e/d60/f10a zero size 2024-03-16T10:45:32.231 INFO:tasks.workunit.client.1.smithi026.stdout:2/870: chown d2/d29/d17/d1a/d2b/d2e/d60/d7f/l128 83140 1 2024-03-16T10:45:32.234 INFO:tasks.workunit.client.1.smithi026.stdout:0/900: mkdir d1/d12/d2f/d48/d55/d60/d9b/d51/df6/d133 0 2024-03-16T10:45:32.237 INFO:tasks.workunit.client.1.smithi026.stdout:7/883: mknod d6/de/d16/d44/c120 0 2024-03-16T10:45:32.240 INFO:tasks.workunit.client.1.smithi026.stdout:0/901: rmdir d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89 39 2024-03-16T10:45:32.241 INFO:tasks.workunit.client.1.smithi026.stdout:0/902: readlink d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/l124 0 2024-03-16T10:45:32.244 INFO:tasks.workunit.client.1.smithi026.stdout:2/871: creat d2/f135 x:0 0 0 2024-03-16T10:45:32.248 INFO:tasks.workunit.client.1.smithi026.stdout:7/884: symlink d6/de/d16/d88/l121 0 2024-03-16T10:45:32.251 INFO:tasks.workunit.client.1.smithi026.stdout:2/872: dwrite d2/d29/d17/d1a/d2b/d4e/d67/fd2 [0,4194304] 0 2024-03-16T10:45:32.252 INFO:tasks.workunit.client.1.smithi026.stdout:7/885: creat d6/d35/d3a/d4e/dcc/d3f/d54/f122 x:0 0 0 2024-03-16T10:45:32.254 INFO:tasks.workunit.client.1.smithi026.stdout:7/886: truncate d6/d35/d3a/dab/fb3 4436147 0 2024-03-16T10:45:32.254 INFO:tasks.workunit.client.1.smithi026.stdout:2/873: chown d2/d29/d17/d1a/d2b/d2e/f42 28 1 2024-03-16T10:45:32.257 INFO:tasks.workunit.client.1.smithi026.stdout:2/874: write d2/d29/d3b/f9b [326066,46857] 0 2024-03-16T10:45:32.260 INFO:tasks.workunit.client.1.smithi026.stdout:0/903: write d1/f92 [410343,103659] 0 2024-03-16T10:45:32.260 INFO:tasks.workunit.client.1.smithi026.stdout:2/875: truncate d2/d29/d17/d38/d64/fa2 4432594 0 2024-03-16T10:45:32.262 INFO:tasks.workunit.client.1.smithi026.stdout:0/904: fdatasync d1/d12/d2f/d48/d55/d60/d9b/fcc 0 2024-03-16T10:45:32.265 INFO:tasks.workunit.client.1.smithi026.stdout:2/876: creat d2/d29/d51/d72/f136 x:0 0 0 2024-03-16T10:45:32.266 INFO:tasks.workunit.client.1.smithi026.stdout:7/887: write d6/d35/f52 [961109,90244] 0 2024-03-16T10:45:32.269 INFO:tasks.workunit.client.1.smithi026.stdout:0/905: symlink d1/d12/l134 0 2024-03-16T10:45:32.271 INFO:tasks.workunit.client.1.smithi026.stdout:7/888: dread d6/de/fbe [0,4194304] 0 2024-03-16T10:45:32.276 INFO:tasks.workunit.client.1.smithi026.stdout:7/889: rmdir d6/d35/d3a/d4e/dcc/d3f/d54/de1 39 2024-03-16T10:45:32.279 INFO:tasks.workunit.client.1.smithi026.stdout:2/877: write d2/d29/d17/f40 [626129,19238] 0 2024-03-16T10:45:32.280 INFO:tasks.workunit.client.1.smithi026.stdout:7/890: creat d6/d35/d3a/d4e/dcc/d3f/d65/f123 x:0 0 0 2024-03-16T10:45:32.283 INFO:tasks.workunit.client.1.smithi026.stdout:0/906: getdents d1/d12/d2f/d48/da5 0 2024-03-16T10:45:32.284 INFO:tasks.workunit.client.1.smithi026.stdout:2/878: mkdir d2/d29/d85/dc6/dd5/d137 0 2024-03-16T10:45:32.287 INFO:tasks.workunit.client.1.smithi026.stdout:7/891: dwrite d6/d1e/da6/f102 [4194304,4194304] 0 2024-03-16T10:45:32.289 INFO:tasks.workunit.client.1.smithi026.stdout:0/907: creat d1/d12/d2f/daa/dd1/d1a/d2b/f135 x:0 0 0 2024-03-16T10:45:32.291 INFO:tasks.workunit.client.1.smithi026.stdout:2/879: dread d2/d29/d51/d72/fea [0,4194304] 0 2024-03-16T10:45:32.296 INFO:tasks.workunit.client.1.smithi026.stdout:2/880: mkdir d2/d29/d17/d38/d64/dcb/d138 0 2024-03-16T10:45:32.297 INFO:tasks.workunit.client.1.smithi026.stdout:0/908: link d1/d12/d2f/daa/dac/fb5 d1/d12/d2f/daa/dac/f136 0 2024-03-16T10:45:32.297 INFO:tasks.workunit.client.1.smithi026.stdout:2/881: truncate d2/d29/d51/f113 172043 0 2024-03-16T10:45:32.300 INFO:tasks.workunit.client.1.smithi026.stdout:2/882: creat d2/d29/d51/d72/f139 x:0 0 0 2024-03-16T10:45:32.302 INFO:tasks.workunit.client.1.smithi026.stdout:0/909: truncate d1/d12/d2f/d48/d55/d60/d9b/d57/f83 893140 0 2024-03-16T10:45:32.306 INFO:tasks.workunit.client.1.smithi026.stdout:0/910: symlink d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/d85/l137 0 2024-03-16T10:45:32.306 INFO:tasks.workunit.client.1.smithi026.stdout:7/892: dread d6/d35/d3a/d4e/dcc/d3f/d54/f9b [0,4194304] 0 2024-03-16T10:45:32.310 INFO:tasks.workunit.client.1.smithi026.stdout:0/911: dread d1/d12/d2f/daa/dd1/d1a/d2b/f91 [0,4194304] 0 2024-03-16T10:45:32.312 INFO:tasks.workunit.client.1.smithi026.stdout:0/912: chown d1/d12/d2f/d48/d55/d60/de0/dea/d119/l125 45 1 2024-03-16T10:45:32.317 INFO:tasks.workunit.client.1.smithi026.stdout:2/883: dwrite d2/d29/d36/f123 [4194304,4194304] 0 2024-03-16T10:45:32.321 INFO:tasks.workunit.client.1.smithi026.stdout:0/913: getdents d1/d12/d2f/d48/d55/d60/d9b/d51/df6/d133 0 2024-03-16T10:45:32.323 INFO:tasks.workunit.client.1.smithi026.stdout:2/884: mkdir d2/d29/d36/d98/d122/d13a 0 2024-03-16T10:45:32.325 INFO:tasks.workunit.client.1.smithi026.stdout:7/893: dwrite d6/f15 [0,4194304] 0 2024-03-16T10:45:32.327 INFO:tasks.workunit.client.1.smithi026.stdout:0/914: creat d1/d12/d2f/d48/d55/d60/d9b/d11/f138 x:0 0 0 2024-03-16T10:45:32.330 INFO:tasks.workunit.client.1.smithi026.stdout:2/885: readlink d2/d29/d17/d1a/d2b/d2e/d60/d7f/d131/l130 0 2024-03-16T10:45:32.331 INFO:tasks.workunit.client.1.smithi026.stdout:7/894: symlink d6/d35/d3a/d4e/dcc/d3f/d54/dd8/l124 0 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/2766752239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 msgr2=0x7fb0f0106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/2766752239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f0106280 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009910 tx=0x7fb0dc009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/2766752239 shutdown_connections 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/2766752239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f0106280 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/2766752239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0f0101560 0x7fb0f0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/2766752239 >> 172.21.15.2:0/2766752239 conn(0x7fb0f00faf70 msgr2=0x7fb0f00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/2766752239 shutdown_connections 2024-03-16T10:45:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.331+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/2766752239 wait complete. 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0f6c3b700 1 Processor -- start 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0f6c3b700 1 -- start start 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0f6c3b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0f0101560 0x7fb0f0100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0f6c3b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f00ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0f6c3b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00ffa50 con 0x7fb0f0103e90 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0f6c3b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00ffb90 con 0x7fb0f0101560 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f00ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f00ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58304/0 (socket says 172.21.15.2:58304) 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.332+0000 7fb0effff700 1 -- 172.21.15.2:0/917773004 learned_addr learned my addr 172.21.15.2:0/917773004 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0f49d7700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0f0101560 0x7fb0f0100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0effff700 1 -- 172.21.15.2:0/917773004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0f0101560 msgr2=0x7fb0f0100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0effff700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0f0101560 0x7fb0f0100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0effff700 1 -- 172.21.15.2:0/917773004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0095f0 con 0x7fb0f0103e90 2024-03-16T10:45:32.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0effff700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f00ff510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0092e0 tx=0x7fb0dc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:32.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0edffb700 1 -- 172.21.15.2:0/917773004 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0dc01d070 con 0x7fb0f0103e90 2024-03-16T10:45:32.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f00ffe10 con 0x7fb0f0103e90 2024-03-16T10:45:32.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0edffb700 1 -- 172.21.15.2:0/917773004 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0dc00f460 con 0x7fb0f0103e90 2024-03-16T10:45:32.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.333+0000 7fb0edffb700 1 -- 172.21.15.2:0/917773004 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0dc021640 con 0x7fb0f0103e90 2024-03-16T10:45:32.337 INFO:tasks.workunit.client.1.smithi026.stdout:2/886: mknod d2/d29/d85/dc6/c13b 0 2024-03-16T10:45:32.337 INFO:tasks.workunit.client.1.smithi026.stdout:7/895: mknod d6/d35/d3a/c125 0 2024-03-16T10:45:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.334+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f0100380 con 0x7fb0f0103e90 2024-03-16T10:45:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.335+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f004ea80 con 0x7fb0f0103e90 2024-03-16T10:45:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.336+0000 7fb0edffb700 1 -- 172.21.15.2:0/917773004 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 31) v1 ==== 91794+0+0 (secure 0 0 0) 0x7fb0dc00f5d0 con 0x7fb0f0103e90 2024-03-16T10:45:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.336+0000 7fb0edffb700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fb0e006dd60 0x7fb0e0070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.336+0000 7fb0edffb700 1 -- 172.21.15.2:0/917773004 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb0dc08f010 con 0x7fb0f0103e90 2024-03-16T10:45:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.336+0000 7fb0f49d7700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fb0e006dd60 0x7fb0e0070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.337+0000 7fb0f49d7700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fb0e006dd60 0x7fb0e0070220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009a10 tx=0x7fb0e4009190 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:32.340 INFO:tasks.workunit.client.1.smithi026.stdout:2/887: mkdir d2/d29/d17/d1a/d2b/d2e/d60/d7f/d131/d6d/d13c 0 2024-03-16T10:45:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.340+0000 7fb0edffb700 1 -- 172.21.15.2:0/917773004 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fb0dc05c2f0 con 0x7fb0f0103e90 2024-03-16T10:45:32.345 INFO:tasks.workunit.client.1.smithi026.stdout:2/888: write d2/d29/d17/d1a/d2b/d2e/fe4 [373776,61620] 0 2024-03-16T10:45:32.347 INFO:tasks.workunit.client.1.smithi026.stdout:7/896: dwrite d6/d35/d3a/d4e/dcc/d3f/d54/dd8/f112 [0,4194304] 0 2024-03-16T10:45:32.348 INFO:tasks.workunit.client.1.smithi026.stdout:0/915: dwrite d1/d12/d2f/d48/d55/d60/d9b/d57/ff7 [0,4194304] 0 2024-03-16T10:45:32.351 INFO:tasks.workunit.client.1.smithi026.stdout:7/897: truncate d6/fb7 39864 0 2024-03-16T10:45:32.356 INFO:tasks.workunit.client.1.smithi026.stdout:0/916: rename d1/d12/d2f/d48/d55/ca1 to d1/d12/d2f/d48/d55/d60/d9b/d51/d122/c139 0 2024-03-16T10:45:32.363 INFO:tasks.workunit.client.1.smithi026.stdout:0/917: truncate d1/d12/d2f/d48/d55/d60/d9b/fbe 2222667 0 2024-03-16T10:45:32.364 INFO:tasks.workunit.client.1.smithi026.stdout:7/898: mknod d6/d35/d3a/d59/c126 0 2024-03-16T10:45:32.366 INFO:tasks.workunit.client.1.smithi026.stdout:7/899: write d6/d35/d3a/d4e/db0/f107 [1024199,95834] 0 2024-03-16T10:45:32.368 INFO:tasks.workunit.client.1.smithi026.stdout:0/918: unlink d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/fd8 0 2024-03-16T10:45:32.373 INFO:tasks.workunit.client.1.smithi026.stdout:7/900: dwrite d6/d2f/ffc [0,4194304] 0 2024-03-16T10:45:32.378 INFO:tasks.workunit.client.1.smithi026.stdout:7/901: symlink d6/d35/d3a/d4e/da1/l127 0 2024-03-16T10:45:32.378 INFO:tasks.workunit.client.1.smithi026.stdout:0/919: mkdir d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/d13a 0 2024-03-16T10:45:32.380 INFO:tasks.workunit.client.1.smithi026.stdout:7/902: write d6/d35/d3a/d4e/dcc/d3f/d54/f8b [1386070,68036] 0 2024-03-16T10:45:32.386 INFO:tasks.workunit.client.1.smithi026.stdout:7/903: write d6/d35/d3a/d4e/da1/ff7 [4849436,122065] 0 2024-03-16T10:45:32.387 INFO:tasks.workunit.client.1.smithi026.stdout:0/920: symlink d1/d12/d2f/d113/d131/l13b 0 2024-03-16T10:45:32.389 INFO:tasks.workunit.client.1.smithi026.stdout:0/921: chown d1/d12/d2f/daa/dd1/d1a/d2b/db4 66577789 1 2024-03-16T10:45:32.390 INFO:tasks.workunit.client.1.smithi026.stdout:7/904: mknod d6/d35/d3a/dab/c128 0 2024-03-16T10:45:32.396 INFO:tasks.workunit.client.1.smithi026.stdout:7/905: link d6/d35/d3a/d4e/dcc/d3f/dc5/de8/c10c d6/d35/d3a/d4e/dcc/d3f/d65/d68/c129 0 2024-03-16T10:45:32.398 INFO:tasks.workunit.client.1.smithi026.stdout:7/906: creat d6/d35/d3a/d4e/dcc/d3f/d70/f12a x:0 0 0 2024-03-16T10:45:32.399 INFO:tasks.workunit.client.1.smithi026.stdout:7/907: symlink d6/d35/d3a/d4e/db0/l12b 0 2024-03-16T10:45:32.404 INFO:tasks.workunit.client.1.smithi026.stdout:7/908: truncate d6/d2f/f4a 3344323 0 2024-03-16T10:45:32.405 INFO:tasks.workunit.client.1.smithi026.stdout:7/909: truncate d6/d35/f52 1946667 0 2024-03-16T10:45:32.408 INFO:tasks.workunit.client.1.smithi026.stdout:7/910: stat d6/d35/d3a/d4e/dcc/d73/dbf/feb 0 2024-03-16T10:45:32.412 INFO:tasks.workunit.client.1.smithi026.stdout:7/911: mknod d6/d35/d3a/d4e/dcc/d3f/c12c 0 2024-03-16T10:45:32.414 INFO:tasks.workunit.client.1.smithi026.stdout:7/912: chown d6/d35/d3a/d4e/dcc/l3d 110645 1 2024-03-16T10:45:32.422 INFO:tasks.workunit.client.1.smithi026.stdout:7/913: rename d6/d1e/l2b to d6/d35/d3a/d4e/dcc/d3f/d54/l12d 0 2024-03-16T10:45:32.423 INFO:tasks.workunit.client.1.smithi026.stdout:7/914: write d6/d35/d3a/d4e/dcc/d3f/d65/fef [480675,20320] 0 2024-03-16T10:45:32.425 INFO:tasks.workunit.client.1.smithi026.stdout:7/915: fdatasync d6/d35/d3a/d4e/dcc/d3f/d9d/fa7 0 2024-03-16T10:45:32.444 INFO:tasks.workunit.client.1.smithi026.stdout:7/916: dwrite d6/de/d16/d88/fc1 [0,4194304] 0 2024-03-16T10:45:32.445 INFO:tasks.workunit.client.1.smithi026.stdout:7/917: chown d6/d35/d3a/d4e/dcc 0 1 2024-03-16T10:45:32.449 INFO:tasks.workunit.client.1.smithi026.stdout:7/918: creat d6/d35/d3a/d4e/dcc/d3f/dc3/f12e x:0 0 0 2024-03-16T10:45:32.458 INFO:tasks.workunit.client.1.smithi026.stdout:7/919: getdents d6/d2f 0 2024-03-16T10:45:32.458 INFO:tasks.workunit.client.1.smithi026.stdout:7/920: dread - d6/de/d7d/ff5 zero size 2024-03-16T10:45:32.465 INFO:tasks.workunit.client.1.smithi026.stdout:7/921: dwrite d6/de/d16/d44/f114 [0,4194304] 0 2024-03-16T10:45:32.469 INFO:tasks.workunit.client.1.smithi026.stdout:7/922: mknod d6/d1e/dbc/dc8/c12f 0 2024-03-16T10:45:32.473 INFO:tasks.workunit.client.1.smithi026.stdout:7/923: dread d6/d2f/f4a [0,4194304] 0 2024-03-16T10:45:32.478 INFO:tasks.workunit.client.1.smithi026.stdout:7/924: write d6/d35/d3a/d4e/dcc/d73/dbf/feb [1067165,73300] 0 2024-03-16T10:45:32.479 INFO:tasks.workunit.client.1.smithi026.stdout:7/925: readlink d6/d35/d3a/d4e/dcc/d3f/ld2 0 2024-03-16T10:45:32.486 INFO:tasks.workunit.client.1.smithi026.stdout:7/926: write d6/d2f/dad/d7b/fd1 [272230,76770] 0 2024-03-16T10:45:32.488 INFO:tasks.workunit.client.1.smithi026.stdout:7/927: dread - d6/d35/d3a/d59/d9e/f113 zero size 2024-03-16T10:45:32.491 INFO:tasks.workunit.client.1.smithi026.stdout:7/928: creat d6/d35/d3a/d4e/dcc/d3f/dc5/f130 x:0 0 0 2024-03-16T10:45:32.493 INFO:tasks.workunit.client.1.smithi026.stdout:7/929: write d6/d35/d3a/d4e/fac [4795407,43288] 0 2024-03-16T10:45:32.504 INFO:tasks.workunit.client.1.smithi026.stdout:7/930: dwrite d6/f38 [4194304,4194304] 0 2024-03-16T10:45:32.508 INFO:tasks.workunit.client.1.smithi026.stdout:7/931: symlink d6/d35/d3a/d59/d9e/l131 0 2024-03-16T10:45:32.509 INFO:tasks.workunit.client.1.smithi026.stdout:7/932: stat d6/d35/d3a/dab/c11b 0 2024-03-16T10:45:32.513 INFO:tasks.workunit.client.1.smithi026.stdout:7/933: creat d6/de/d16/f132 x:0 0 0 2024-03-16T10:45:32.515 INFO:tasks.workunit.client.1.smithi026.stdout:7/934: write d6/d2f/dad/de9/f10a [909369,48272] 0 2024-03-16T10:45:32.517 INFO:tasks.workunit.client.1.smithi026.stdout:7/935: fsync d6/d35/d3a/d4e/dcc/d3f/d54/f9b 0 2024-03-16T10:45:32.517 INFO:tasks.workunit.client.1.smithi026.stdout:7/936: dread - d6/f105 zero size 2024-03-16T10:45:32.528 INFO:tasks.workunit.client.1.smithi026.stdout:7/937: dread d6/d35/d3a/d4e/dcc/d3f/dc3/fe3 [0,4194304] 0 2024-03-16T10:45:32.531 INFO:tasks.workunit.client.1.smithi026.stdout:7/938: creat d6/d35/f133 x:0 0 0 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.624+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb0f0100380 con 0x7fb0f0103e90 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.624+0000 7fb0edffb700 1 -- 172.21.15.2:0/917773004 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb0f0100380 con 0x7fb0f0103e90 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:45:32.625 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:45:32.626 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:45:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:45:32.628 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:45:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:45:32.628 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:45:32.628 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:45:32.628 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fb0e006dd60 msgr2=0x7fb0e0070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fb0e006dd60 0x7fb0e0070220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009a10 tx=0x7fb0e4009190 comp rx=0 tx=0).stop 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 msgr2=0x7fb0f00ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f00ff510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0092e0 tx=0x7fb0dc00bd70 comp rx=0 tx=0).stop 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 shutdown_connections 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fb0e006dd60 0x7fb0e0070220 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0f0101560 0x7fb0f0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 --2- 172.21.15.2:0/917773004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0f0103e90 0x7fb0f00ff510 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 >> 172.21.15.2:0/917773004 conn(0x7fb0f00faf70 msgr2=0x7fb0f00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.626+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 shutdown_connections 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.627+0000 7fb0f6c3b700 1 -- 172.21.15.2:0/917773004 wait complete. 2024-03-16T10:45:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:45:32.722 INFO:tasks.workunit.client.1.smithi026.stdout:1/691: sync 2024-03-16T10:45:32.752 INFO:tasks.workunit.client.1.smithi026.stdout:1/692: truncate d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/fae 247764 0 2024-03-16T10:45:32.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.754+0000 7efd22a7c700 1 -- 172.21.15.2:0/757991881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c100c10 msgr2=0x7efd1c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.754+0000 7efd22a7c700 1 --2- 172.21.15.2:0/757991881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c100c10 0x7efd1c101070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efd08009910 tx=0x7efd08009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:32.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 -- 172.21.15.2:0/757991881 shutdown_connections 2024-03-16T10:45:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 --2- 172.21.15.2:0/757991881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c100c10 0x7efd1c101070 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 --2- 172.21.15.2:0/757991881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c0ffa10 0x7efd1c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 -- 172.21.15.2:0/757991881 >> 172.21.15.2:0/757991881 conn(0x7efd1c0faf70 msgr2=0x7efd1c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 -- 172.21.15.2:0/757991881 shutdown_connections 2024-03-16T10:45:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 -- 172.21.15.2:0/757991881 wait complete. 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 Processor -- start 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.755+0000 7efd22a7c700 1 -- start start 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd22a7c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c0ffa10 0x7efd1c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd22a7c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100c10 0x7efd1c196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd22a7c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c197050 con 0x7efd1c0ffa10 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd22a7c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c197190 con 0x7efd1c100c10 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd20818700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c0ffa10 0x7efd1c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd20818700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c0ffa10 0x7efd1c1964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45254/0 (socket says 172.21.15.2:45254) 2024-03-16T10:45:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd20818700 1 -- 172.21.15.2:0/4049616806 learned_addr learned my addr 172.21.15.2:0/4049616806 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd1bfff700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100c10 0x7efd1c196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd20818700 1 -- 172.21.15.2:0/4049616806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100c10 msgr2=0x7efd1c196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd20818700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100c10 0x7efd1c196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd20818700 1 -- 172.21.15.2:0/4049616806 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd080095f0 con 0x7efd1c0ffa10 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd20818700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c0ffa10 0x7efd1c1964f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd1000e8f0 tx=0x7efd1000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd19ffb700 1 -- 172.21.15.2:0/4049616806 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd10009790 con 0x7efd1c0ffa10 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c19bb80 con 0x7efd1c0ffa10 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd19ffb700 1 -- 172.21.15.2:0/4049616806 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd1000c810 con 0x7efd1c0ffa10 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.756+0000 7efd19ffb700 1 -- 172.21.15.2:0/4049616806 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd10003d20 con 0x7efd1c0ffa10 2024-03-16T10:45:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.757+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c19c0d0 con 0x7efd1c0ffa10 2024-03-16T10:45:32.760 INFO:tasks.workunit.client.1.smithi026.stdout:1/693: symlink d0/d17/d1f/d63/d6a/d7c/d84/d9d/ld3 0 2024-03-16T10:45:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.758+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c04ea80 con 0x7efd1c0ffa10 2024-03-16T10:45:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.758+0000 7efd19ffb700 1 -- 172.21.15.2:0/4049616806 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 31) v1 ==== 91794+0+0 (secure 0 0 0) 0x7efd100041f0 con 0x7efd1c0ffa10 2024-03-16T10:45:32.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.758+0000 7efd19ffb700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7efd0c06dd60 0x7efd0c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:32.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.758+0000 7efd19ffb700 1 -- 172.21.15.2:0/4049616806 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efd10014070 con 0x7efd1c0ffa10 2024-03-16T10:45:32.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.758+0000 7efd1bfff700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7efd0c06dd60 0x7efd0c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:32.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.759+0000 7efd1bfff700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7efd0c06dd60 0x7efd0c070220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd0800b800 tx=0x7efd08009f70 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:32.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:32.763+0000 7efd19ffb700 1 -- 172.21.15.2:0/4049616806 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7efd10094080 con 0x7efd1c0ffa10 2024-03-16T10:45:32.767 INFO:tasks.workunit.client.1.smithi026.stdout:1/694: link d0/d17/d1f/d63/d6a/d7c/d6b/f7a d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/fd4 0 2024-03-16T10:45:32.770 INFO:tasks.workunit.client.1.smithi026.stdout:1/695: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/fd5 x:0 0 0 2024-03-16T10:45:32.777 INFO:tasks.workunit.client.1.smithi026.stdout:1/696: rmdir d0/d17/d1f/d63/d6a/d7c/d84/d9d 39 2024-03-16T10:45:32.786 INFO:tasks.workunit.client.1.smithi026.stdout:1/697: dread d0/d17/d1f/d20/d65/f8c [0,4194304] 0 2024-03-16T10:45:32.789 INFO:tasks.workunit.client.1.smithi026.stdout:1/698: symlink d0/d17/d1f/d63/d6a/d7c/d2e/ld6 0 2024-03-16T10:45:32.798 INFO:tasks.workunit.client.1.smithi026.stdout:1/699: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/fd5 [0,4194304] 0 2024-03-16T10:45:32.799 INFO:tasks.workunit.client.1.smithi026.stdout:1/700: readlink d0/d17/d1f/l44 0 2024-03-16T10:45:32.804 INFO:tasks.workunit.client.1.smithi026.stdout:1/701: symlink d0/d8b/ld7 0 2024-03-16T10:45:32.807 INFO:tasks.workunit.client.1.smithi026.stdout:1/702: rmdir d0/d17/d1f/d63/d6a/d7c/d2e/d54 39 2024-03-16T10:45:32.812 INFO:tasks.workunit.client.1.smithi026.stdout:1/703: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/cd8 0 2024-03-16T10:45:32.816 INFO:tasks.workunit.client.1.smithi026.stdout:1/704: rename d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/f9b to d0/d17/d1f/d20/dcc/fd9 0 2024-03-16T10:45:32.821 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:32 smithi026 ceph-mon[37041]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:32.821 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:32 smithi026 ceph-mon[37041]: pgmap v11: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 116 MiB/s wr, 295 op/s 2024-03-16T10:45:32.821 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:32 smithi026 ceph-mon[37041]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:32.821 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:32 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3049296225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:45:32.821 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:32 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/917773004' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:45:32.830 INFO:tasks.workunit.client.1.smithi026.stdout:9/893: sync 2024-03-16T10:45:32.834 INFO:tasks.workunit.client.1.smithi026.stdout:1/705: dwrite d0/d17/d1f/d63/d6a/d7c/d84/fa7 [0,4194304] 0 2024-03-16T10:45:32.840 INFO:tasks.workunit.client.1.smithi026.stdout:9/894: creat d0/d5f/d7f/f133 x:0 0 0 2024-03-16T10:45:32.842 INFO:tasks.workunit.client.1.smithi026.stdout:1/706: link d0/d17/d1f/fc9 d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fda 0 2024-03-16T10:45:32.849 INFO:tasks.workunit.client.1.smithi026.stdout:1/707: dwrite d0/d17/fb2 [0,4194304] 0 2024-03-16T10:45:32.852 INFO:tasks.workunit.client.1.smithi026.stdout:9/895: getdents d0/d5f/d7f/d8c 0 2024-03-16T10:45:32.856 INFO:tasks.workunit.client.1.smithi026.stdout:1/708: symlink d0/d17/d1f/d63/ldb 0 2024-03-16T10:45:32.859 INFO:tasks.workunit.client.1.smithi026.stdout:9/896: creat d0/da/f134 x:0 0 0 2024-03-16T10:45:32.862 INFO:tasks.workunit.client.1.smithi026.stdout:9/897: mkdir d0/da/d4d/d135 0 2024-03-16T10:45:32.864 INFO:tasks.workunit.client.1.smithi026.stdout:9/898: read - d0/da/d14/da7/d126/dd5/f129 zero size 2024-03-16T10:45:32.864 INFO:tasks.workunit.client.1.smithi026.stdout:1/709: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/fdc x:0 0 0 2024-03-16T10:45:32.864 INFO:tasks.workunit.client.1.smithi026.stdout:9/899: readlink d0/d63/d6b/l9a 0 2024-03-16T10:45:32.871 INFO:tasks.workunit.client.1.smithi026.stdout:1/710: dwrite d0/d17/fb2 [0,4194304] 0 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:7/939: sync 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:4/865: sync 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:5/780: sync 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:3/912: sync 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:8/755: sync 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:2/889: sync 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:6/649: sync 2024-03-16T10:45:32.874 INFO:tasks.workunit.client.1.smithi026.stdout:0/922: sync 2024-03-16T10:45:32.875 INFO:tasks.workunit.client.1.smithi026.stdout:8/756: stat d5/dc/d89 0 2024-03-16T10:45:32.875 INFO:tasks.workunit.client.1.smithi026.stdout:4/866: read - d1/f104 zero size 2024-03-16T10:45:32.879 INFO:tasks.workunit.client.1.smithi026.stdout:2/890: dread - d2/d29/d17/d1a/f133 zero size 2024-03-16T10:45:32.880 INFO:tasks.workunit.client.1.smithi026.stdout:6/650: stat d0/d28/fb2 0 2024-03-16T10:45:32.883 INFO:tasks.workunit.client.1.smithi026.stdout:3/913: symlink d4/dc/d73/l13b 0 2024-03-16T10:45:32.886 INFO:tasks.workunit.client.1.smithi026.stdout:4/867: unlink d1/db/de/d4c/d59/f10a 0 2024-03-16T10:45:32.886 INFO:tasks.workunit.client.1.smithi026.stdout:9/900: dwrite d0/d63/d6b/f86 [0,4194304] 0 2024-03-16T10:45:32.890 INFO:tasks.workunit.client.1.smithi026.stdout:1/711: rename d0/d17/d1f/d63/d6a/d7c/d84/fb0 to d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/fdd 0 2024-03-16T10:45:32.891 INFO:tasks.workunit.client.1.smithi026.stdout:0/923: fsync d1/d12/d2f/daa/dd1/d61/f62 0 2024-03-16T10:45:32.893 INFO:tasks.workunit.client.1.smithi026.stdout:0/924: chown d1/d12/d2f/d48/d55/d60/d9b/d51/df6 688940 1 2024-03-16T10:45:32.894 INFO:tasks.workunit.client.1.smithi026.stdout:1/712: dread - d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fda zero size 2024-03-16T10:45:32.896 INFO:tasks.workunit.client.1.smithi026.stdout:6/651: mknod d0/d62/cdb 0 2024-03-16T10:45:32.899 INFO:tasks.workunit.client.1.smithi026.stdout:2/891: creat d2/d29/d17/d1a/d2b/d2e/d60/d7f/ddc/f13d x:0 0 0 2024-03-16T10:45:32.905 INFO:tasks.workunit.client.1.smithi026.stdout:4/868: rmdir d1/db/de/d4c/d6b/dbc 39 2024-03-16T10:45:32.909 INFO:tasks.workunit.client.1.smithi026.stdout:9/901: write d0/d5f/d7f/ff9 [54535,121095] 0 2024-03-16T10:45:32.911 INFO:tasks.workunit.client.1.smithi026.stdout:8/757: mkdir d5/d7/d2c/d36/d59/d93/dde/de0/de3 0 2024-03-16T10:45:32.913 INFO:tasks.workunit.client.1.smithi026.stdout:8/758: read d5/dc/d40/d53/f58 [3280577,78172] 0 2024-03-16T10:45:32.914 INFO:tasks.workunit.client.1.smithi026.stdout:8/759: fsync d5/d23/d4e/d69/d76/f87 0 2024-03-16T10:45:32.914 INFO:tasks.workunit.client.1.smithi026.stdout:5/781: creat d1/d1c/d2a/f118 x:0 0 0 2024-03-16T10:45:32.915 INFO:tasks.workunit.client.1.smithi026.stdout:1/713: mkdir d0/d17/d1f/d20/d65/dde 0 2024-03-16T10:45:32.917 INFO:tasks.workunit.client.1.smithi026.stdout:0/925: mkdir d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/df5/d13c 0 2024-03-16T10:45:32.918 INFO:tasks.workunit.client.1.smithi026.stdout:6/652: fsync d0/d41/d4f/fa9 0 2024-03-16T10:45:32.922 INFO:tasks.workunit.client.1.smithi026.stdout:5/782: dwrite d1/d1c/d2a/d55/d99/f117 [0,4194304] 0 2024-03-16T10:45:32.923 INFO:tasks.workunit.client.1.smithi026.stdout:2/892: symlink d2/d29/d3b/l13e 0 2024-03-16T10:45:32.925 INFO:tasks.workunit.client.1.smithi026.stdout:2/893: fdatasync d2/d29/d17/d1a/d2b/d4e/d6f/fcf 0 2024-03-16T10:45:32.929 INFO:tasks.workunit.client.1.smithi026.stdout:9/902: creat d0/d5f/da2/f136 x:0 0 0 2024-03-16T10:45:32.936 INFO:tasks.workunit.client.1.smithi026.stdout:8/760: rename d5/d23/f4b to d5/d7/d2c/d36/d59/d93/dde/fe4 0 2024-03-16T10:45:32.942 INFO:tasks.workunit.client.1.smithi026.stdout:9/903: dread d0/da/fc9 [0,4194304] 0 2024-03-16T10:45:32.945 INFO:tasks.workunit.client.1.smithi026.stdout:6/653: creat d0/d10/d79/d7b/fdc x:0 0 0 2024-03-16T10:45:32.945 INFO:tasks.workunit.client.1.smithi026.stdout:6/654: fsync d0/f6 0 2024-03-16T10:45:32.948 INFO:tasks.workunit.client.1.smithi026.stdout:3/914: creat d4/dc/d73/f13c x:0 0 0 2024-03-16T10:45:32.952 INFO:tasks.workunit.client.1.smithi026.stdout:4/869: getdents d1/db/ddb/daa/dfb 0 2024-03-16T10:45:32.957 INFO:tasks.workunit.client.1.smithi026.stdout:7/940: link d6/de/d16/d44/c4d d6/d35/d3a/d4e/d6c/c134 0 2024-03-16T10:45:32.959 INFO:tasks.workunit.client.1.smithi026.stdout:7/941: chown d6/d2f/dad/l63 54805 1 2024-03-16T10:45:32.963 INFO:tasks.workunit.client.1.smithi026.stdout:7/942: fsync d6/de/d16/d44/f114 0 2024-03-16T10:45:32.963 INFO:tasks.workunit.client.1.smithi026.stdout:8/761: rmdir d5/d5d 39 2024-03-16T10:45:32.966 INFO:tasks.workunit.client.1.smithi026.stdout:1/714: dwrite d0/d17/d1f/d63/d6a/d7c/d3b/d64/f6f [0,4194304] 0 2024-03-16T10:45:32.966 INFO:tasks.workunit.client.1.smithi026.stdout:0/926: dwrite d1/d12/d2f/f7b [0,4194304] 0 2024-03-16T10:45:32.969 INFO:tasks.workunit.client.1.smithi026.stdout:9/904: creat d0/da/d4d/f137 x:0 0 0 2024-03-16T10:45:32.973 INFO:tasks.workunit.client.1.smithi026.stdout:7/943: dread d6/d35/d3a/d4e/dcc/d3f/d54/f8b [0,4194304] 0 2024-03-16T10:45:32.977 INFO:tasks.workunit.client.1.smithi026.stdout:5/783: fdatasync d1/db/f39 0 2024-03-16T10:45:32.980 INFO:tasks.workunit.client.1.smithi026.stdout:4/870: unlink d1/db/de/d4c/d6b/ff5 0 2024-03-16T10:45:32.989 INFO:tasks.workunit.client.1.smithi026.stdout:1/715: creat d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/fdf x:0 0 0 2024-03-16T10:45:32.990 INFO:tasks.workunit.client.1.smithi026.stdout:2/894: truncate d2/f116 8026357 0 2024-03-16T10:45:32.997 INFO:tasks.workunit.client.1.smithi026.stdout:7/944: creat d6/de/d16/d44/f135 x:0 0 0 2024-03-16T10:45:33.000 INFO:tasks.workunit.client.1.smithi026.stdout:6/655: dwrite d0/d41/fa7 [0,4194304] 0 2024-03-16T10:45:33.000 INFO:tasks.workunit.client.1.smithi026.stdout:8/762: write d5/dc/f38 [8599111,34183] 0 2024-03-16T10:45:33.003 INFO:tasks.workunit.client.1.smithi026.stdout:2/895: dwrite d2/d29/d3b/f9b [0,4194304] 0 2024-03-16T10:45:33.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.005+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd1c19c380 con 0x7efd0c06dd60 2024-03-16T10:45:33.006 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:32 smithi002 ceph-mon[29248]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:33.007 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:32 smithi002 ceph-mon[29248]: pgmap v11: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 116 MiB/s wr, 295 op/s 2024-03-16T10:45:33.007 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:32 smithi002 ceph-mon[29248]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:33.007 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:32 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3049296225' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:45:33.007 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:32 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/917773004' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:45:33.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.007+0000 7efd19ffb700 1 -- 172.21.15.2:0/4049616806 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7efd1c19c380 con 0x7efd0c06dd60 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr" 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "2/23 daemons upgraded", 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "", 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:45:33.008 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7efd0c06dd60 msgr2=0x7efd0c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7efd0c06dd60 0x7efd0c070220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd0800b800 tx=0x7efd08009f70 comp rx=0 tx=0).stop 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c0ffa10 msgr2=0x7efd1c1964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c0ffa10 0x7efd1c1964f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd1000e8f0 tx=0x7efd1000ec00 comp rx=0 tx=0).stop 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 shutdown_connections 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7efd0c06dd60 0x7efd0c070220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd1c0ffa10 0x7efd1c1964f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 --2- 172.21.15.2:0/4049616806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100c10 0x7efd1c196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 >> 172.21.15.2:0/4049616806 conn(0x7efd1c0faf70 msgr2=0x7efd1c069020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 shutdown_connections 2024-03-16T10:45:33.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.010+0000 7efd22a7c700 1 -- 172.21.15.2:0/4049616806 wait complete. 2024-03-16T10:45:33.012 INFO:tasks.workunit.client.1.smithi026.stdout:4/871: truncate d1/db/de/d4c/d6b/fae 18180 0 2024-03-16T10:45:33.012 INFO:tasks.workunit.client.1.smithi026.stdout:1/716: rmdir d0/d17/d1f/d63/d6a/d7c 39 2024-03-16T10:45:33.017 INFO:tasks.workunit.client.1.smithi026.stdout:6/656: creat d0/d28/dcb/fdd x:0 0 0 2024-03-16T10:45:33.019 INFO:tasks.workunit.client.1.smithi026.stdout:2/896: mknod d2/d29/d17/d1a/d2b/d2e/d60/d7f/c13f 0 2024-03-16T10:45:33.025 INFO:tasks.workunit.client.1.smithi026.stdout:9/905: link d0/dd7/cbf d0/dd7/c138 0 2024-03-16T10:45:33.027 INFO:tasks.workunit.client.1.smithi026.stdout:1/717: creat d0/d17/d1f/d63/d58/fe0 x:0 0 0 2024-03-16T10:45:33.027 INFO:tasks.workunit.client.1.smithi026.stdout:2/897: dwrite d2/d29/d17/d1a/d2b/d2e/d60/d7f/d131/d6d/f11c [0,4194304] 0 2024-03-16T10:45:33.029 INFO:tasks.workunit.client.1.smithi026.stdout:0/927: creat d1/d12/d2f/daa/dd1/d1a/d2b/d4f/f13d x:0 0 0 2024-03-16T10:45:33.034 INFO:tasks.workunit.client.1.smithi026.stdout:3/915: getdents d4/d89/d97 0 2024-03-16T10:45:33.037 INFO:tasks.workunit.client.1.smithi026.stdout:7/945: write d6/de/d16/d88/f93 [564679,29553] 0 2024-03-16T10:45:33.038 INFO:tasks.workunit.client.1.smithi026.stdout:6/657: mkdir d0/d10/d79/dca/dde 0 2024-03-16T10:45:33.038 INFO:tasks.workunit.client.1.smithi026.stdout:9/906: dwrite d0/dd7/f9b [0,4194304] 0 2024-03-16T10:45:33.040 INFO:tasks.workunit.client.1.smithi026.stdout:6/658: dread - d0/d41/f7f zero size 2024-03-16T10:45:33.044 INFO:tasks.workunit.client.1.smithi026.stdout:7/946: read - d6/d35/d3a/d4e/dcc/d3f/d70/f12a zero size 2024-03-16T10:45:33.046 INFO:tasks.workunit.client.1.smithi026.stdout:4/872: symlink d1/d85/da2/da8/df4/l10f 0 2024-03-16T10:45:33.049 INFO:tasks.workunit.client.1.smithi026.stdout:2/898: mkdir d2/d29/d85/d140 0 2024-03-16T10:45:33.050 INFO:tasks.workunit.client.1.smithi026.stdout:2/899: readlink d2/d29/d17/d1a/d2b/d2e/lc1 0 2024-03-16T10:45:33.051 INFO:tasks.workunit.client.1.smithi026.stdout:3/916: truncate d4/f77 101784 0 2024-03-16T10:45:33.052 INFO:tasks.workunit.client.1.smithi026.stdout:2/900: chown d2/d29/f56 50656 1 2024-03-16T10:45:33.052 INFO:tasks.workunit.client.1.smithi026.stdout:5/784: creat d1/d1c/d2a/dab/dc5/d69/f119 x:0 0 0 2024-03-16T10:45:33.053 INFO:tasks.workunit.client.1.smithi026.stdout:9/907: symlink d0/d63/d6b/ded/l139 0 2024-03-16T10:45:33.053 INFO:tasks.workunit.client.1.smithi026.stdout:3/917: chown d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/c112 274448 1 2024-03-16T10:45:33.056 INFO:tasks.workunit.client.1.smithi026.stdout:8/763: rmdir d5/d7/d2c/d36/d59/d93/dde/de0/de3 0 2024-03-16T10:45:33.056 INFO:tasks.workunit.client.1.smithi026.stdout:6/659: fdatasync d0/d41/d71/fb1 0 2024-03-16T10:45:33.057 INFO:tasks.workunit.client.1.smithi026.stdout:4/873: creat d1/db/de/d4c/d59/f110 x:0 0 0 2024-03-16T10:45:33.059 INFO:tasks.workunit.client.1.smithi026.stdout:4/874: chown d1/db/l39 5995241 1 2024-03-16T10:45:33.063 INFO:tasks.workunit.client.1.smithi026.stdout:4/875: dread d1/db/f7e [0,4194304] 0 2024-03-16T10:45:33.068 INFO:tasks.workunit.client.1.smithi026.stdout:1/718: dread d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fad [0,4194304] 0 2024-03-16T10:45:33.071 INFO:tasks.workunit.client.1.smithi026.stdout:0/928: write d1/d12/f4b [2604156,102656] 0 2024-03-16T10:45:33.074 INFO:tasks.workunit.client.1.smithi026.stdout:4/876: dread d1/db/de/d4c/d59/d6a/f82 [0,4194304] 0 2024-03-16T10:45:33.075 INFO:tasks.workunit.client.1.smithi026.stdout:8/764: dread d5/d7/d2c/f32 [0,4194304] 0 2024-03-16T10:45:33.076 INFO:tasks.workunit.client.1.smithi026.stdout:9/908: rename d0/da/d13/c32 to d0/da/d14/da7/c13a 0 2024-03-16T10:45:33.083 INFO:tasks.workunit.client.1.smithi026.stdout:5/785: write d1/db/f7d [775481,52918] 0 2024-03-16T10:45:33.090 INFO:tasks.workunit.client.1.smithi026.stdout:2/901: dwrite d2/f4d [0,4194304] 0 2024-03-16T10:45:33.091 INFO:tasks.workunit.client.1.smithi026.stdout:1/719: rmdir d0/d17/d1f/d63/d6a/d7c/d84/d9d 39 2024-03-16T10:45:33.096 INFO:tasks.workunit.client.1.smithi026.stdout:0/929: fdatasync d1/d12/d2f/d48/d55/d60/d9b/d51/f72 0 2024-03-16T10:45:33.097 INFO:tasks.workunit.client.1.smithi026.stdout:0/930: readlink d1/d12/d2f/daa/dd1/la4 0 2024-03-16T10:45:33.098 INFO:tasks.workunit.client.1.smithi026.stdout:7/947: dwrite d6/de/f74 [0,4194304] 0 2024-03-16T10:45:33.098 INFO:tasks.workunit.client.1.smithi026.stdout:4/877: chown d1/c9e 7 1 2024-03-16T10:45:33.102 INFO:tasks.workunit.client.1.smithi026.stdout:6/660: rename d0/d10/d79/d7b/l8f to d0/d10/d79/d91/ldf 0 2024-03-16T10:45:33.105 INFO:tasks.workunit.client.1.smithi026.stdout:3/918: dwrite d4/dc/d73/d83/d57/f9a [0,4194304] 0 2024-03-16T10:45:33.105 INFO:tasks.workunit.client.1.smithi026.stdout:4/878: chown d1/db/de/d4c/d6b/d7a/c86 405 1 2024-03-16T10:45:33.107 INFO:tasks.workunit.client.1.smithi026.stdout:1/720: mknod d0/d17/d1f/d63/dbc/ce1 0 2024-03-16T10:45:33.108 INFO:tasks.workunit.client.1.smithi026.stdout:3/919: chown d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dae/l12c 71734 1 2024-03-16T10:45:33.110 INFO:tasks.workunit.client.1.smithi026.stdout:8/765: symlink d5/d7/le5 0 2024-03-16T10:45:33.123 INFO:tasks.workunit.client.1.smithi026.stdout:2/902: mkdir d2/d29/d17/d1a/d2b/d141 0 2024-03-16T10:45:33.124 INFO:tasks.workunit.client.1.smithi026.stdout:5/786: dwrite d1/d1c/d2a/dab/dc5/f97 [4194304,4194304] 0 2024-03-16T10:45:33.130 INFO:tasks.workunit.client.1.smithi026.stdout:3/920: dwrite d4/dc/d31/dbe/f10a [0,4194304] 0 2024-03-16T10:45:33.133 INFO:tasks.workunit.client.1.smithi026.stdout:7/948: fsync d6/d35/d3a/d59/d9e/fc2 0 2024-03-16T10:45:33.138 INFO:tasks.workunit.client.1.smithi026.stdout:0/931: mkdir d1/d12/d2f/d48/d55/d60/d9b/d51/df6/d133/d13e 0 2024-03-16T10:45:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.136+0000 7f8bb0232700 1 -- 172.21.15.2:0/1398595659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba80ffa10 msgr2=0x7f8ba80ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.136+0000 7f8bb0232700 1 --2- 172.21.15.2:0/1398595659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba80ffa10 0x7f8ba80ffe30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009910 tx=0x7f8ba4009c20 comp rx=0 tx=0).stop 2024-03-16T10:45:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.137+0000 7f8bb0232700 1 -- 172.21.15.2:0/1398595659 shutdown_connections 2024-03-16T10:45:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.137+0000 7f8bb0232700 1 --2- 172.21.15.2:0/1398595659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8100c10 0x7f8ba8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.137+0000 7f8bb0232700 1 --2- 172.21.15.2:0/1398595659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba80ffa10 0x7f8ba80ffe30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.137+0000 7f8bb0232700 1 -- 172.21.15.2:0/1398595659 >> 172.21.15.2:0/1398595659 conn(0x7f8ba80fafb0 msgr2=0x7f8ba80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.137+0000 7f8bb0232700 1 -- 172.21.15.2:0/1398595659 shutdown_connections 2024-03-16T10:45:33.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.137+0000 7f8bb0232700 1 -- 172.21.15.2:0/1398595659 wait complete. 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.138+0000 7f8bb0232700 1 Processor -- start 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.138+0000 7f8bb0232700 1 -- start start 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.139+0000 7f8bb0232700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba80ffa10 0x7f8ba8069b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.139+0000 7f8bb0232700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8100c10 0x7f8ba8199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.139+0000 7f8bad7cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8100c10 0x7f8ba8199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.139+0000 7f8bad7cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8100c10 0x7f8ba8199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45276/0 (socket says 172.21.15.2:45276) 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.139+0000 7f8bad7cd700 1 -- 172.21.15.2:0/2207740276 learned_addr learned my addr 172.21.15.2:0/2207740276 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.139+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba81995f0 con 0x7f8ba8100c10 2024-03-16T10:45:33.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.139+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8199760 con 0x7f8ba80ffa10 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.140+0000 7f8bad7cd700 1 -- 172.21.15.2:0/2207740276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba80ffa10 msgr2=0x7f8ba8069b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.140+0000 7f8bad7cd700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba80ffa10 0x7f8ba8069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.140+0000 7f8bad7cd700 1 -- 172.21.15.2:0/2207740276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40095f0 con 0x7f8ba8100c10 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.140+0000 7f8bad7cd700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8100c10 0x7f8ba8199020 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c8a0 tx=0x7f8b9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.140+0000 7f8b9effd700 1 -- 172.21.15.2:0/2207740276 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b9800cea0 con 0x7f8ba8100c10 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.140+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba8199a40 con 0x7f8ba8100c10 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.140+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba819a010 con 0x7f8ba8100c10 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.141+0000 7f8b9effd700 1 -- 172.21.15.2:0/2207740276 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b98004830 con 0x7f8ba8100c10 2024-03-16T10:45:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.141+0000 7f8b9effd700 1 -- 172.21.15.2:0/2207740276 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b980055b0 con 0x7f8ba8100c10 2024-03-16T10:45:33.143 INFO:tasks.workunit.client.1.smithi026.stdout:6/661: creat d0/d28/d49/db0/fe0 x:0 0 0 2024-03-16T10:45:33.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.142+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c005320 con 0x7f8ba8100c10 2024-03-16T10:45:33.144 INFO:tasks.workunit.client.1.smithi026.stdout:6/662: stat d0/f51 0 2024-03-16T10:45:33.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.143+0000 7f8b9effd700 1 -- 172.21.15.2:0/2207740276 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 31) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f8b98004e10 con 0x7f8ba8100c10 2024-03-16T10:45:33.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.144+0000 7f8b9effd700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f8b9406dd60 0x7f8b94070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:45:33.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.144+0000 7f8b9effd700 1 -- 172.21.15.2:0/2207740276 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8b9808d640 con 0x7f8ba8100c10 2024-03-16T10:45:33.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.144+0000 7f8badfce700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f8b9406dd60 0x7f8b94070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:45:33.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.145+0000 7f8badfce700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f8b9406dd60 0x7f8b94070220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b5c0 tx=0x7f8ba4005160 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:45:33.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.149+0000 7f8b9effd700 1 -- 172.21.15.2:0/2207740276 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f8b9805a880 con 0x7f8ba8100c10 2024-03-16T10:45:33.152 INFO:tasks.workunit.client.1.smithi026.stdout:8/766: dread d5/d7/d2c/d36/d59/d93/dde/fe4 [0,4194304] 0 2024-03-16T10:45:33.156 INFO:tasks.workunit.client.1.smithi026.stdout:1/721: dwrite d0/fb3 [0,4194304] 0 2024-03-16T10:45:33.158 INFO:tasks.workunit.client.1.smithi026.stdout:9/909: dread d0/da/d13/f31 [0,4194304] 0 2024-03-16T10:45:33.158 INFO:tasks.workunit.client.1.smithi026.stdout:3/921: dread d4/dc/d31/d48/d69/f6f [0,4194304] 0 2024-03-16T10:45:33.162 INFO:tasks.workunit.client.1.smithi026.stdout:0/932: dread d1/d12/d2f/d48/d55/d60/d9b/d11/f8e [0,4194304] 0 2024-03-16T10:45:33.164 INFO:tasks.workunit.client.1.smithi026.stdout:0/933: write d1/d12/d2f/d48/d55/d60/de0/dea/f11b [2508728,93388] 0 2024-03-16T10:45:33.167 INFO:tasks.workunit.client.1.smithi026.stdout:0/934: truncate d1/d12/d2f/daa/dd1/d1a/d2b/d4f/fef 1372041 0 2024-03-16T10:45:33.169 INFO:tasks.workunit.client.1.smithi026.stdout:0/935: write d1/d12/d2f/daa/dd1/d1a/f2c [6252597,113807] 0 2024-03-16T10:45:33.308 INFO:tasks.workunit.client.1.smithi026.stdout:5/787: dread - d1/db/d1f/d61/d65/d82/dbb/dea/fd8 zero size 2024-03-16T10:45:33.333 INFO:tasks.workunit.client.1.smithi026.stdout:9/910: creat d0/da/d16/f13b x:0 0 0 2024-03-16T10:45:33.334 INFO:tasks.workunit.client.1.smithi026.stdout:9/911: write d0/d5f/da2/f136 [1044996,120639] 0 2024-03-16T10:45:33.339 INFO:tasks.workunit.client.1.smithi026.stdout:3/922: symlink d4/dc/d31/dbe/de0/l13d 0 2024-03-16T10:45:33.340 INFO:tasks.workunit.client.1.smithi026.stdout:3/923: chown d4/dc/dd/d47/d53/l12f 0 1 2024-03-16T10:45:33.354 INFO:tasks.workunit.client.1.smithi026.stdout:0/936: rmdir d1 39 2024-03-16T10:45:33.358 INFO:tasks.workunit.client.1.smithi026.stdout:2/903: mknod d2/d29/d17/d1a/d2b/c142 0 2024-03-16T10:45:33.365 INFO:tasks.workunit.client.1.smithi026.stdout:2/904: dread d2/d29/d51/ff6 [0,4194304] 0 2024-03-16T10:45:33.367 INFO:tasks.workunit.client.1.smithi026.stdout:2/905: readlink d2/d29/d36/l12d 0 2024-03-16T10:45:33.367 INFO:tasks.workunit.client.1.smithi026.stdout:6/663: link d0/d41/fcc d0/d10/d79/d91/fe1 0 2024-03-16T10:45:33.375 INFO:tasks.workunit.client.1.smithi026.stdout:3/924: dread d4/dc/f4e [0,4194304] 0 2024-03-16T10:45:33.378 INFO:tasks.workunit.client.1.smithi026.stdout:5/788: dread d1/d1c/d2a/dab/dc5/d69/f67 [0,4194304] 0 2024-03-16T10:45:33.378 INFO:tasks.workunit.client.1.smithi026.stdout:6/664: symlink d0/d28/d2d/le2 0 2024-03-16T10:45:33.381 INFO:tasks.workunit.client.1.smithi026.stdout:0/937: dread d1/d12/d2f/daa/dd1/d1a/d2b/d4f/fa8 [0,4194304] 0 2024-03-16T10:45:33.383 INFO:tasks.workunit.client.1.smithi026.stdout:8/767: rename d5/d23/d4e/d69 to d5/d7/de6 0 2024-03-16T10:45:33.384 INFO:tasks.workunit.client.1.smithi026.stdout:2/906: dwrite d2/d29/d17/d38/ff1 [0,4194304] 0 2024-03-16T10:45:33.391 INFO:tasks.workunit.client.1.smithi026.stdout:6/665: chown d0/d28/f9c 3408553 1 2024-03-16T10:45:33.392 INFO:tasks.workunit.client.1.smithi026.stdout:8/768: dread d5/d23/d4e/f5b [0,4194304] 0 2024-03-16T10:45:33.410 INFO:tasks.workunit.client.1.smithi026.stdout:6/666: rename d0/c20 to d0/d10/d79/d8e/ce3 0 2024-03-16T10:45:33.416 INFO:tasks.workunit.client.1.smithi026.stdout:6/667: chown d0/d10/c14 5 1 2024-03-16T10:45:33.422 INFO:tasks.workunit.client.1.smithi026.stdout:6/668: rename d0/d62/f85 to d0/fe4 0 2024-03-16T10:45:33.436 INFO:tasks.workunit.client.1.smithi026.stdout:6/669: unlink d0/d8a/c90 0 2024-03-16T10:45:33.436 INFO:tasks.workunit.client.1.smithi026.stdout:6/670: stat d0/d10/f38 0 2024-03-16T10:45:33.440 INFO:tasks.workunit.client.1.smithi026.stdout:6/671: symlink d0/d10/d79/d7b/le5 0 2024-03-16T10:45:33.444 INFO:tasks.workunit.client.1.smithi026.stdout:6/672: unlink d0/d8a/l8b 0 2024-03-16T10:45:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.446+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8b8c005190 con 0x7f8ba8100c10 2024-03-16T10:45:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.446+0000 7f8b9effd700 1 -- 172.21.15.2:0/2207740276 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8b9805a410 con 0x7f8ba8100c10 2024-03-16T10:45:33.447 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:45:33.447 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:45:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:45:33.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.448+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f8b9406dd60 msgr2=0x7f8b94070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f8b9406dd60 0x7f8b94070220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b5c0 tx=0x7f8ba4005160 comp rx=0 tx=0).stop 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8100c10 msgr2=0x7f8ba8199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8100c10 0x7f8ba8199020 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c8a0 tx=0x7f8b9800cc60 comp rx=0 tx=0).stop 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 shutdown_connections 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f8b9406dd60 0x7f8b94070220 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba80ffa10 0x7f8ba8069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 --2- 172.21.15.2:0/2207740276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8100c10 0x7f8ba8199020 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 >> 172.21.15.2:0/2207740276 conn(0x7f8ba80fafb0 msgr2=0x7f8ba8104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 shutdown_connections 2024-03-16T10:45:33.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:45:33.449+0000 7f8bb0232700 1 -- 172.21.15.2:0/2207740276 wait complete. 2024-03-16T10:45:33.484 INFO:tasks.workunit.client.1.smithi026.stdout:1/722: write d0/f1e [958102,36833] 0 2024-03-16T10:45:33.490 INFO:tasks.workunit.client.1.smithi026.stdout:1/723: mknod d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/ce2 0 2024-03-16T10:45:33.495 INFO:tasks.workunit.client.1.smithi026.stdout:2/907: truncate d2/d29/d17/d38/f81 4140837 0 2024-03-16T10:45:33.497 INFO:tasks.workunit.client.1.smithi026.stdout:1/724: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/ce3 0 2024-03-16T10:45:33.497 INFO:tasks.workunit.client.1.smithi026.stdout:5/789: dwrite d1/d1c/d4c/fe3 [0,4194304] 0 2024-03-16T10:45:33.498 INFO:tasks.workunit.client.1.smithi026.stdout:0/938: dwrite d1/d12/d2f/d48/d55/d60/d9b/f67 [0,4194304] 0 2024-03-16T10:45:33.501 INFO:tasks.workunit.client.1.smithi026.stdout:9/912: creat d0/da/d16/d5d/f13c x:0 0 0 2024-03-16T10:45:33.502 INFO:tasks.workunit.client.1.smithi026.stdout:5/790: fdatasync d1/f5 0 2024-03-16T10:45:33.502 INFO:tasks.workunit.client.1.smithi026.stdout:2/908: read d2/d29/d85/dc6/dd5/ff2 [224965,98298] 0 2024-03-16T10:45:33.504 INFO:tasks.workunit.client.1.smithi026.stdout:5/791: chown d1/d1c/d4c/d4d/l6b 12472 1 2024-03-16T10:45:33.505 INFO:tasks.workunit.client.1.smithi026.stdout:9/913: chown d0/f4 27 1 2024-03-16T10:45:33.507 INFO:tasks.workunit.client.1.smithi026.stdout:9/914: readlink d0/da/d16/l93 0 2024-03-16T10:45:33.513 INFO:tasks.workunit.client.1.smithi026.stdout:5/792: mkdir d1/d1c/d2a/dab/de8/d11a 0 2024-03-16T10:45:33.517 INFO:tasks.workunit.client.1.smithi026.stdout:0/939: getdents d1/d12/d2f/d48/d55/d60/d9b/d51/dde 0 2024-03-16T10:45:33.520 INFO:tasks.workunit.client.1.smithi026.stdout:7/949: unlink d6/f24 0 2024-03-16T10:45:33.524 INFO:tasks.workunit.client.1.smithi026.stdout:7/950: chown d6/d2f/dad/d7b/l9a 28821 1 2024-03-16T10:45:33.526 INFO:tasks.workunit.client.1.smithi026.stdout:7/951: fdatasync d6/d35/d3a/d59/fb9 0 2024-03-16T10:45:33.527 INFO:tasks.workunit.client.1.smithi026.stdout:9/915: dread d0/d63/d6b/fc2 [0,4194304] 0 2024-03-16T10:45:33.532 INFO:tasks.workunit.client.1.smithi026.stdout:9/916: mkdir d0/d63/d6b/ded/d13d 0 2024-03-16T10:45:33.533 INFO:tasks.workunit.client.1.smithi026.stdout:0/940: dwrite d1/d12/d2f/daa/dac/fd6 [0,4194304] 0 2024-03-16T10:45:33.534 INFO:tasks.workunit.client.1.smithi026.stdout:5/793: dread d1/d1c/d4c/d4d/f4e [0,4194304] 0 2024-03-16T10:45:33.535 INFO:tasks.workunit.client.1.smithi026.stdout:7/952: truncate d6/de/d16/f1f 4834843 0 2024-03-16T10:45:33.537 INFO:tasks.workunit.client.1.smithi026.stdout:9/917: mkdir d0/da/d14/da7/d126/d13e 0 2024-03-16T10:45:33.541 INFO:tasks.workunit.client.1.smithi026.stdout:0/941: mknod d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/df1/c13f 0 2024-03-16T10:45:33.545 INFO:tasks.workunit.client.1.smithi026.stdout:9/918: unlink d0/d63/d6b/ded/l139 0 2024-03-16T10:45:33.547 INFO:tasks.workunit.client.1.smithi026.stdout:7/953: symlink d6/de/d16/l136 0 2024-03-16T10:45:33.549 INFO:tasks.workunit.client.1.smithi026.stdout:0/942: rmdir d1/d12/d2f/d48/d55 39 2024-03-16T10:45:33.552 INFO:tasks.workunit.client.1.smithi026.stdout:5/794: rename d1/db/d1f/d61/d116/f41 to d1/d1c/d2a/d8e/f11b 0 2024-03-16T10:45:33.562 INFO:tasks.workunit.client.1.smithi026.stdout:0/943: mkdir d1/d12/d2f/d48/dfe/d140 0 2024-03-16T10:45:33.562 INFO:tasks.workunit.client.1.smithi026.stdout:4/879: sync 2024-03-16T10:45:33.564 INFO:tasks.workunit.client.1.smithi026.stdout:4/880: chown d1/d85/da2/dca 1 1 2024-03-16T10:45:33.564 INFO:tasks.workunit.client.1.smithi026.stdout:0/944: chown d1/d12/l11d 3367031 1 2024-03-16T10:45:33.567 INFO:tasks.workunit.client.1.smithi026.stdout:1/725: dwrite d0/d17/d1f/d63/d6a/d7c/d2e/d54/f5e [0,4194304] 0 2024-03-16T10:45:33.567 INFO:tasks.workunit.client.1.smithi026.stdout:4/881: chown d1/db/ddb/daa/fd0 92642361 1 2024-03-16T10:45:33.581 INFO:tasks.workunit.client.1.smithi026.stdout:9/919: creat d0/da/d14/da7/d126/d13e/f13f x:0 0 0 2024-03-16T10:45:33.582 INFO:tasks.workunit.client.1.smithi026.stdout:1/726: chown d0/d17/d1f/d63/d6a/d7c/d2e/c73 33 1 2024-03-16T10:45:33.583 INFO:tasks.workunit.client.1.smithi026.stdout:2/909: write d2/f35 [3560204,19304] 0 2024-03-16T10:45:33.589 INFO:tasks.workunit.client.1.smithi026.stdout:0/945: chown d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/d85/f111 0 1 2024-03-16T10:45:33.590 INFO:tasks.workunit.client.1.smithi026.stdout:5/795: dread d1/db/f28 [0,4194304] 0 2024-03-16T10:45:33.591 INFO:tasks.workunit.client.1.smithi026.stdout:9/920: dwrite d0/da/d16/d5d/f103 [0,4194304] 0 2024-03-16T10:45:33.591 INFO:tasks.workunit.client.1.smithi026.stdout:5/796: stat d1/d1c 0 2024-03-16T10:45:33.600 INFO:tasks.workunit.client.1.smithi026.stdout:2/910: mkdir d2/d29/d17/de8/d143 0 2024-03-16T10:45:33.606 INFO:tasks.workunit.client.1.smithi026.stdout:7/954: write d6/d35/d3a/d59/d9e/fc2 [2412890,33260] 0 2024-03-16T10:45:33.608 INFO:tasks.workunit.client.1.smithi026.stdout:7/955: truncate d6/d2f/dad/de9/f10a 1034986 0 2024-03-16T10:45:33.616 INFO:tasks.workunit.client.1.smithi026.stdout:9/921: unlink d0/d63/ffb 0 2024-03-16T10:45:33.621 INFO:tasks.workunit.client.1.smithi026.stdout:2/911: truncate d2/d29/fd9 65011 0 2024-03-16T10:45:33.623 INFO:tasks.workunit.client.1.smithi026.stdout:2/912: write d2/d29/d17/d1a/f9e [1565069,124630] 0 2024-03-16T10:45:33.628 INFO:tasks.workunit.client.1.smithi026.stdout:1/727: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/fb9 [0,4194304] 0 2024-03-16T10:45:33.640 INFO:tasks.workunit.client.1.smithi026.stdout:2/913: mkdir d2/d29/d85/dc6/d144 0 2024-03-16T10:45:33.645 INFO:tasks.workunit.client.1.smithi026.stdout:7/956: symlink d6/d35/d3a/d4e/dcc/d3f/d54/de1/l137 0 2024-03-16T10:45:33.656 INFO:tasks.workunit.client.1.smithi026.stdout:2/914: fsync d2/d29/d17/d1a/d2b/d2e/d60/d7f/d131/fc9 0 2024-03-16T10:45:33.658 INFO:tasks.workunit.client.1.smithi026.stdout:4/882: getdents d1/db/de/d4c/d6b/dbc 0 2024-03-16T10:45:33.662 INFO:tasks.workunit.client.1.smithi026.stdout:0/946: getdents d1/d12/d2f/d113/d131 0 2024-03-16T10:45:33.663 INFO:tasks.workunit.client.1.smithi026.stdout:1/728: creat d0/d17/d1f/d20/d65/dde/fe4 x:0 0 0 2024-03-16T10:45:33.665 INFO:tasks.workunit.client.1.smithi026.stdout:9/922: creat d0/d5f/f140 x:0 0 0 2024-03-16T10:45:33.666 INFO:tasks.workunit.client.1.smithi026.stdout:5/797: getdents d1/d6e/d9f 0 2024-03-16T10:45:33.669 INFO:tasks.workunit.client.1.smithi026.stdout:2/915: unlink d2/d29/d17/d1a/d2b/d4e/d67/fbe 0 2024-03-16T10:45:33.674 INFO:tasks.workunit.client.1.smithi026.stdout:9/923: dread - d0/da/d16/d17/db1/fcd zero size 2024-03-16T10:45:33.677 INFO:tasks.workunit.client.1.smithi026.stdout:2/916: fsync d2/dd1/fe1 0 2024-03-16T10:45:33.678 INFO:tasks.workunit.client.1.smithi026.stdout:1/729: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/dc2/ce5 0 2024-03-16T10:45:33.686 INFO:tasks.workunit.client.1.smithi026.stdout:1/730: rename d0/d17/d1f/dc7 to d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6 0 2024-03-16T10:45:33.687 INFO:tasks.workunit.client.1.smithi026.stdout:1/731: chown d0/d17/d1f/c67 593357 1 2024-03-16T10:45:33.691 INFO:tasks.workunit.client.1.smithi026.stdout:2/917: dread d2/d29/d17/d1a/d2b/d4e/d6f/f9c [0,4194304] 0 2024-03-16T10:45:33.692 INFO:tasks.workunit.client.1.smithi026.stdout:2/918: read - d2/d29/d85/dc6/f118 zero size 2024-03-16T10:45:33.694 INFO:tasks.workunit.client.1.smithi026.stdout:1/732: rename d0/d17/d1f/c15 to d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/ce7 0 2024-03-16T10:45:33.698 INFO:tasks.workunit.client.1.smithi026.stdout:2/919: dread d2/d29/d17/d1a/d2b/d4e/f71 [0,4194304] 0 2024-03-16T10:45:33.699 INFO:tasks.workunit.client.1.smithi026.stdout:1/733: creat d0/d8b/fe8 x:0 0 0 2024-03-16T10:45:33.705 INFO:tasks.workunit.client.1.smithi026.stdout:4/883: write d1/db/de/f55 [175473,77303] 0 2024-03-16T10:45:33.707 INFO:tasks.workunit.client.1.smithi026.stdout:4/884: dread - d1/db/ddb/daa/d44/fe4 zero size 2024-03-16T10:45:33.708 INFO:tasks.workunit.client.1.smithi026.stdout:7/957: truncate d6/d35/d3a/d4e/dcc/d3f/d65/fef 390221 0 2024-03-16T10:45:33.713 INFO:tasks.workunit.client.1.smithi026.stdout:7/958: mknod d6/d35/d3a/d4e/d6c/c138 0 2024-03-16T10:45:33.717 INFO:tasks.workunit.client.1.smithi026.stdout:7/959: mkdir d6/d35/d3a/d4e/dcc/d3f/d54/d139 0 2024-03-16T10:45:33.718 INFO:tasks.workunit.client.1.smithi026.stdout:5/798: truncate d1/d1c/f3c 1869074 0 2024-03-16T10:45:33.720 INFO:tasks.workunit.client.1.smithi026.stdout:5/799: write d1/db/d1f/d61/fa7 [1647025,13529] 0 2024-03-16T10:45:33.724 INFO:tasks.workunit.client.1.smithi026.stdout:5/800: mknod d1/db/d1f/c11c 0 2024-03-16T10:45:33.725 INFO:tasks.workunit.client.1.smithi026.stdout:7/960: dwrite d6/d1e/f108 [0,4194304] 0 2024-03-16T10:45:33.727 INFO:tasks.workunit.client.1.smithi026.stdout:7/961: write d6/de/d16/d44/f114 [1278512,88714] 0 2024-03-16T10:45:33.739 INFO:tasks.workunit.client.1.smithi026.stdout:7/962: dwrite d6/d35/d3a/d59/d9e/f113 [0,4194304] 0 2024-03-16T10:45:33.743 INFO:tasks.workunit.client.1.smithi026.stdout:7/963: creat d6/d35/d3a/d4e/dcc/d3f/d9d/f13a x:0 0 0 2024-03-16T10:45:33.746 INFO:tasks.workunit.client.1.smithi026.stdout:2/920: write d2/d29/d17/d1a/d2b/f73 [4545861,113686] 0 2024-03-16T10:45:33.752 INFO:tasks.workunit.client.1.smithi026.stdout:1/734: write d0/f40 [4689751,77463] 0 2024-03-16T10:45:33.764 INFO:tasks.workunit.client.1.smithi026.stdout:1/735: symlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/dc2/le9 0 2024-03-16T10:45:33.770 INFO:tasks.workunit.client.1.smithi026.stdout:4/885: dwrite d1/db/de/f1d [0,4194304] 0 2024-03-16T10:45:33.782 INFO:tasks.workunit.client.1.smithi026.stdout:5/801: write d1/d1c/d2a/f57 [8625527,2494] 0 2024-03-16T10:45:33.782 INFO:tasks.workunit.client.1.smithi026.stdout:5/802: readlink d1/l10d 0 2024-03-16T10:45:33.784 INFO:tasks.workunit.client.1.smithi026.stdout:5/803: chown d1/d1c/d2a/d55/d99/fb4 0 1 2024-03-16T10:45:33.793 INFO:tasks.workunit.client.1.smithi026.stdout:4/886: dread d1/db/f64 [0,4194304] 0 2024-03-16T10:45:33.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:33 smithi002 ceph-mon[29248]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:33.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:33 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/2207740276' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:45:33.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:33 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:33.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:33 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:33.816 INFO:tasks.workunit.client.1.smithi026.stdout:3/925: sync 2024-03-16T10:45:33.822 INFO:tasks.workunit.client.1.smithi026.stdout:2/921: dread d2/d29/d17/d1a/d2b/d4b/fb0 [0,4194304] 0 2024-03-16T10:45:33.824 INFO:tasks.workunit.client.1.smithi026.stdout:7/964: truncate d6/d35/f52 736753 0 2024-03-16T10:45:33.830 INFO:tasks.workunit.client.1.smithi026.stdout:5/804: dread d1/d1c/d2a/d55/d99/f96 [0,4194304] 0 2024-03-16T10:45:33.833 INFO:tasks.workunit.client.1.smithi026.stdout:4/887: creat d1/db/de/d4c/d6b/dbc/f111 x:0 0 0 2024-03-16T10:45:33.836 INFO:tasks.workunit.client.1.smithi026.stdout:4/888: write d1/db/de/d4c/d59/fc3 [8594393,57700] 0 2024-03-16T10:45:33.840 INFO:tasks.workunit.client.1.smithi026.stdout:3/926: creat d4/dc/dd/d80/dc2/f13e x:0 0 0 2024-03-16T10:45:33.845 INFO:tasks.workunit.client.1.smithi026.stdout:2/922: creat d2/d29/d3b/dbb/dfb/f145 x:0 0 0 2024-03-16T10:45:33.846 INFO:tasks.workunit.client.1.smithi026.stdout:7/965: mkdir d6/d35/d13b 0 2024-03-16T10:45:33.847 INFO:tasks.workunit.client.1.smithi026.stdout:1/736: truncate d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/fb9 1833350 0 2024-03-16T10:45:33.850 INFO:tasks.workunit.client.1.smithi026.stdout:1/737: dread d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fad [0,4194304] 0 2024-03-16T10:45:33.854 INFO:tasks.workunit.client.1.smithi026.stdout:3/927: symlink d4/dc/dd/d80/dc2/l13f 0 2024-03-16T10:45:33.859 INFO:tasks.workunit.client.1.smithi026.stdout:2/923: fsync d2/d29/d85/ff4 0 2024-03-16T10:45:33.862 INFO:tasks.workunit.client.1.smithi026.stdout:7/966: creat d6/de/d16/d44/f13c x:0 0 0 2024-03-16T10:45:33.863 INFO:tasks.workunit.client.1.smithi026.stdout:7/967: dread - d6/d35/f133 zero size 2024-03-16T10:45:33.868 INFO:tasks.workunit.client.1.smithi026.stdout:3/928: dread d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f3e [4194304,4194304] 0 2024-03-16T10:45:33.870 INFO:tasks.workunit.client.1.smithi026.stdout:5/805: write d1/f5 [67106,127551] 0 2024-03-16T10:45:33.877 INFO:tasks.workunit.client.1.smithi026.stdout:4/889: dwrite d1/db/de/f2b [4194304,4194304] 0 2024-03-16T10:45:33.879 INFO:tasks.workunit.client.1.smithi026.stdout:7/968: rmdir d6/d35/d3a/d59/df1 39 2024-03-16T10:45:33.886 INFO:tasks.workunit.client.1.smithi026.stdout:5/806: fdatasync d1/d1c/d2a/dab/dc5/d69/d51/d80/fa6 0 2024-03-16T10:45:33.891 INFO:tasks.workunit.client.1.smithi026.stdout:2/924: mkdir d2/d29/d17/d1a/d2b/d2e/d146 0 2024-03-16T10:45:33.906 INFO:tasks.workunit.client.1.smithi026.stdout:5/807: fsync d1/db/d1f/d61/d116/f60 0 2024-03-16T10:45:33.909 INFO:tasks.workunit.client.1.smithi026.stdout:2/925: mkdir d2/d29/d36/d98/d122/d147 0 2024-03-16T10:45:33.912 INFO:tasks.workunit.client.1.smithi026.stdout:7/969: getdents d6/d35/d3a/d4e/dcc/d3f/dc3/dcf 0 2024-03-16T10:45:33.913 INFO:tasks.workunit.client.1.smithi026.stdout:4/890: dwrite d1/db/de/d4c/d6b/fad [0,4194304] 0 2024-03-16T10:45:33.913 INFO:tasks.workunit.client.1.smithi026.stdout:1/738: getdents d0/d17/d1f/d63/d6a/d7c/d3b/d64/dc2 0 2024-03-16T10:45:33.916 INFO:tasks.workunit.client.1.smithi026.stdout:7/970: chown d6/d35/d3a/d4e/dcc/f40 663446167 1 2024-03-16T10:45:33.918 INFO:tasks.workunit.client.1.smithi026.stdout:4/891: write d1/db/de/d4c/d59/d6a/f9b [5391459,61072] 0 2024-03-16T10:45:33.921 INFO:tasks.workunit.client.1.smithi026.stdout:4/892: stat d1/db/de/l3f 0 2024-03-16T10:45:33.923 INFO:tasks.workunit.client.1.smithi026.stdout:3/929: link d4/d89/d97/l74 d4/l140 0 2024-03-16T10:45:33.924 INFO:tasks.workunit.client.1.smithi026.stdout:7/971: dread d6/d1e/da6/f95 [0,4194304] 0 2024-03-16T10:45:33.926 INFO:tasks.workunit.client.1.smithi026.stdout:1/739: symlink d0/d17/d1f/d63/d6a/d7c/d2e/d54/lea 0 2024-03-16T10:45:33.927 INFO:tasks.workunit.client.1.smithi026.stdout:7/972: chown d6/d35/d3a/d4e/db0/l12b 3068 1 2024-03-16T10:45:33.935 INFO:tasks.workunit.client.1.smithi026.stdout:3/930: readlink d4/dc/d31/l113 0 2024-03-16T10:45:33.937 INFO:tasks.workunit.client.1.smithi026.stdout:4/893: truncate d1/db/de/d4c/d59/d6a/db2/fb4 286130 0 2024-03-16T10:45:33.937 INFO:tasks.workunit.client.1.smithi026.stdout:5/808: dwrite d1/d6e/f92 [0,4194304] 0 2024-03-16T10:45:33.941 INFO:tasks.workunit.client.1.smithi026.stdout:8/769: sync 2024-03-16T10:45:33.942 INFO:tasks.workunit.client.1.smithi026.stdout:3/931: truncate d4/dc/d73/d83/d7c/f12d 2555962 0 2024-03-16T10:45:33.942 INFO:tasks.workunit.client.1.smithi026.stdout:2/926: truncate d2/d29/d17/fba 2389411 0 2024-03-16T10:45:33.946 INFO:tasks.workunit.client.1.smithi026.stdout:3/932: chown d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/cf5 26 1 2024-03-16T10:45:33.947 INFO:tasks.workunit.client.1.smithi026.stdout:5/809: dwrite d1/db/d1f/d61/fa7 [0,4194304] 0 2024-03-16T10:45:33.950 INFO:tasks.workunit.client.1.smithi026.stdout:1/740: mkdir d0/d17/d1f/deb 0 2024-03-16T10:45:33.958 INFO:tasks.workunit.client.1.smithi026.stdout:1/741: dread d0/d8b/fb6 [4194304,4194304] 0 2024-03-16T10:45:33.965 INFO:tasks.workunit.client.1.smithi026.stdout:8/770: fsync d5/d5d/d9a/fcf 0 2024-03-16T10:45:33.966 INFO:tasks.workunit.client.1.smithi026.stdout:4/894: write d1/d85/fc8 [331277,50950] 0 2024-03-16T10:45:33.974 INFO:tasks.workunit.client.1.smithi026.stdout:7/973: rename d6/d35/f66 to d6/f13d 0 2024-03-16T10:45:33.975 INFO:tasks.workunit.client.1.smithi026.stdout:0/947: sync 2024-03-16T10:45:33.975 INFO:tasks.workunit.client.1.smithi026.stdout:6/673: sync 2024-03-16T10:45:33.975 INFO:tasks.workunit.client.1.smithi026.stdout:9/924: sync 2024-03-16T10:45:33.977 INFO:tasks.workunit.client.1.smithi026.stdout:2/927: mknod d2/d29/d17/d1a/d2b/d141/c148 0 2024-03-16T10:45:33.978 INFO:tasks.workunit.client.1.smithi026.stdout:1/742: write d0/d17/d1f/fc9 [787209,95284] 0 2024-03-16T10:45:33.980 INFO:tasks.workunit.client.1.smithi026.stdout:6/674: write d0/d41/dbe/fd3 [798614,56965] 0 2024-03-16T10:45:33.980 INFO:tasks.workunit.client.1.smithi026.stdout:0/948: read d1/d12/d2f/daa/dd1/d1a/d2b/fe9 [282589,7010] 0 2024-03-16T10:45:33.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:33 smithi026 ceph-mon[37041]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:45:33.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:33 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/2207740276' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:45:33.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:33 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:33.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:33 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:33.984 INFO:tasks.workunit.client.1.smithi026.stdout:3/933: dwrite d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/f88 [4194304,4194304] 0 2024-03-16T10:45:33.987 INFO:tasks.workunit.client.1.smithi026.stdout:3/934: write d4/dc/d31/f3b [4870490,56734] 0 2024-03-16T10:45:33.991 INFO:tasks.workunit.client.1.smithi026.stdout:2/928: dwrite d2/d29/d17/d1a/d2b/d2e/f31 [0,4194304] 0 2024-03-16T10:45:33.991 INFO:tasks.workunit.client.1.smithi026.stdout:3/935: readlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/ld9 0 2024-03-16T10:45:33.993 INFO:tasks.workunit.client.1.smithi026.stdout:2/929: chown d2/d29/d36/d98/d122/d147 1 1 2024-03-16T10:45:33.993 INFO:tasks.workunit.client.1.smithi026.stdout:4/895: creat d1/d85/da2/dca/f112 x:0 0 0 2024-03-16T10:45:33.994 INFO:tasks.workunit.client.1.smithi026.stdout:3/936: fsync d4/dc/d31/d48/f9f 0 2024-03-16T10:45:33.996 INFO:tasks.workunit.client.1.smithi026.stdout:4/896: chown d1/db/ddb/daa/d61/f6c 48 1 2024-03-16T10:45:34.003 INFO:tasks.workunit.client.1.smithi026.stdout:5/810: dwrite d1/db/f38 [0,4194304] 0 2024-03-16T10:45:34.006 INFO:tasks.workunit.client.1.smithi026.stdout:3/937: dwrite d4/dc/d73/d83/d7c/dd7/f116 [0,4194304] 0 2024-03-16T10:45:34.008 INFO:tasks.workunit.client.1.smithi026.stdout:5/811: write d1/d1c/d2a/dab/dc5/d69/f119 [517680,75151] 0 2024-03-16T10:45:34.012 INFO:tasks.workunit.client.1.smithi026.stdout:5/812: read d1/d6e/f92 [3938653,101594] 0 2024-03-16T10:45:34.017 INFO:tasks.workunit.client.1.smithi026.stdout:1/743: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec 0 2024-03-16T10:45:34.018 INFO:tasks.workunit.client.1.smithi026.stdout:6/675: creat d0/d41/dbe/fe6 x:0 0 0 2024-03-16T10:45:34.019 INFO:tasks.workunit.client.1.smithi026.stdout:2/930: rename d2/d29/d36/d98/d117 to d2/d29/d17/d38/d149 0 2024-03-16T10:45:34.021 INFO:tasks.workunit.client.1.smithi026.stdout:7/974: mkdir d6/d35/d13b/d13e 0 2024-03-16T10:45:34.023 INFO:tasks.workunit.client.1.smithi026.stdout:4/897: mknod d1/db/de/d4c/d59/d6a/db2/db3/c113 0 2024-03-16T10:45:34.023 INFO:tasks.workunit.client.1.smithi026.stdout:2/931: write d2/d29/d36/da7/f102 [15159,75588] 0 2024-03-16T10:45:34.024 INFO:tasks.workunit.client.1.smithi026.stdout:6/676: chown d0/d10/d79/fa2 56808 1 2024-03-16T10:45:34.027 INFO:tasks.workunit.client.1.smithi026.stdout:0/949: dread d1/d12/d2f/d48/d55/d60/d9b/ff0 [0,4194304] 0 2024-03-16T10:45:34.033 INFO:tasks.workunit.client.1.smithi026.stdout:0/950: write d1/d12/d2f/daa/dd1/d1a/d2b/d4f/fef [1279218,9519] 0 2024-03-16T10:45:34.036 INFO:tasks.workunit.client.1.smithi026.stdout:5/813: mkdir d1/d1c/d2a/dab/d11d 0 2024-03-16T10:45:34.038 INFO:tasks.workunit.client.1.smithi026.stdout:6/677: dwrite d0/d28/dcb/fdd [0,4194304] 0 2024-03-16T10:45:34.039 INFO:tasks.workunit.client.1.smithi026.stdout:9/925: dwrite d0/d63/d6b/f70 [0,4194304] 0 2024-03-16T10:45:34.041 INFO:tasks.workunit.client.1.smithi026.stdout:9/926: chown d0/da 14136 1 2024-03-16T10:45:34.041 INFO:tasks.workunit.client.1.smithi026.stdout:4/898: dread d1/db/de/d4c/d6b/f98 [0,4194304] 0 2024-03-16T10:45:34.043 INFO:tasks.workunit.client.1.smithi026.stdout:4/899: chown d1/db/de/c9f 221 1 2024-03-16T10:45:34.044 INFO:tasks.workunit.client.1.smithi026.stdout:1/744: rename d0/f7f to d0/d17/d1f/d20/d65/fed 0 2024-03-16T10:45:34.048 INFO:tasks.workunit.client.1.smithi026.stdout:7/975: mkdir d6/d2f/dad/d13f 0 2024-03-16T10:45:34.055 INFO:tasks.workunit.client.1.smithi026.stdout:8/771: getdents d5/d7/d2c/d36/d59/d93/dde 0 2024-03-16T10:45:34.056 INFO:tasks.workunit.client.1.smithi026.stdout:8/772: dread - d5/d7/de6/dd5/fe2 zero size 2024-03-16T10:45:34.061 INFO:tasks.workunit.client.1.smithi026.stdout:5/814: mkdir d1/db/d1f/d61/d65/d11e 0 2024-03-16T10:45:34.065 INFO:tasks.workunit.client.1.smithi026.stdout:0/951: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/c141 0 2024-03-16T10:45:34.066 INFO:tasks.workunit.client.1.smithi026.stdout:1/745: dread d0/d17/d1f/d63/d58/f9a [0,4194304] 0 2024-03-16T10:45:34.077 INFO:tasks.workunit.client.1.smithi026.stdout:7/976: fsync d6/d35/d3a/d4e/dcc/d3f/d70/f9f 0 2024-03-16T10:45:34.078 INFO:tasks.workunit.client.1.smithi026.stdout:6/678: write d0/d41/d4f/fa9 [691229,51390] 0 2024-03-16T10:45:34.081 INFO:tasks.workunit.client.1.smithi026.stdout:2/932: symlink d2/d29/d17/d1a/d2b/d4e/l14a 0 2024-03-16T10:45:34.089 INFO:tasks.workunit.client.1.smithi026.stdout:0/952: symlink d1/d12/d2f/d48/d55/d60/d9b/d57/l142 0 2024-03-16T10:45:34.092 INFO:tasks.workunit.client.1.smithi026.stdout:9/927: rename d0/f3c to d0/dd7/f141 0 2024-03-16T10:45:34.096 INFO:tasks.workunit.client.1.smithi026.stdout:4/900: link d1/db/de/d4c/fa1 d1/db/ddb/daa/d61/f114 0 2024-03-16T10:45:34.102 INFO:tasks.workunit.client.1.smithi026.stdout:8/773: symlink d5/le7 0 2024-03-16T10:45:34.104 INFO:tasks.workunit.client.1.smithi026.stdout:3/938: link d4/dc/dd/d47/lfd d4/dc/d31/d48/l141 0 2024-03-16T10:45:34.106 INFO:tasks.workunit.client.1.smithi026.stdout:5/815: chown d1/db/d1f/d61/d65/d82/dbb 20 1 2024-03-16T10:45:34.108 INFO:tasks.workunit.client.1.smithi026.stdout:2/933: dwrite d2/d29/d17/d1a/d2b/d2e/fbf [0,4194304] 0 2024-03-16T10:45:34.109 INFO:tasks.workunit.client.1.smithi026.stdout:0/953: unlink d1/d12/d2f/d48/d55/d60/d9b/d57/c10a 0 2024-03-16T10:45:34.115 INFO:tasks.workunit.client.1.smithi026.stdout:7/977: mknod d6/d35/d3a/d59/c140 0 2024-03-16T10:45:34.118 INFO:tasks.workunit.client.1.smithi026.stdout:3/939: dwrite d4/dc/d31/d48/d69/d78/fea [0,4194304] 0 2024-03-16T10:45:34.127 INFO:tasks.workunit.client.1.smithi026.stdout:0/954: dwrite d1/d12/d2f/d48/d55/d60/d9b/d11/f138 [0,4194304] 0 2024-03-16T10:45:34.128 INFO:tasks.workunit.client.1.smithi026.stdout:5/816: mkdir d1/db/d1f/d61/d65/d82/dbb/dea/d11f 0 2024-03-16T10:45:34.129 INFO:tasks.workunit.client.1.smithi026.stdout:9/928: unlink d0/dd7/dff/d109/c11b 0 2024-03-16T10:45:34.132 INFO:tasks.workunit.client.1.smithi026.stdout:7/978: creat d6/d2f/dad/d7b/f141 x:0 0 0 2024-03-16T10:45:34.134 INFO:tasks.workunit.client.1.smithi026.stdout:6/679: link d0/d28/fb2 d0/d10/dd2/fe7 0 2024-03-16T10:45:34.134 INFO:tasks.workunit.client.1.smithi026.stdout:1/746: getdents d0/d17/d1f/d63/d6a/d7c/d3b 0 2024-03-16T10:45:34.137 INFO:tasks.workunit.client.1.smithi026.stdout:8/774: creat d5/da0/fe8 x:0 0 0 2024-03-16T10:45:34.138 INFO:tasks.workunit.client.1.smithi026.stdout:2/934: dread d2/d29/d85/dc6/dd5/f112 [0,4194304] 0 2024-03-16T10:45:34.141 INFO:tasks.workunit.client.1.smithi026.stdout:6/680: dread d0/d28/d2d/f4c [4194304,4194304] 0 2024-03-16T10:45:34.150 INFO:tasks.workunit.client.1.smithi026.stdout:4/901: write d1/f22 [1217632,111479] 0 2024-03-16T10:45:34.184 INFO:tasks.workunit.client.1.smithi026.stdout:9/929: symlink d0/d63/d6b/def/l142 0 2024-03-16T10:45:34.185 INFO:tasks.workunit.client.1.smithi026.stdout:5/817: creat d1/d1c/d2a/dab/de8/f120 x:0 0 0 2024-03-16T10:45:34.186 INFO:tasks.workunit.client.1.smithi026.stdout:5/818: readlink d1/d1c/lc1 0 2024-03-16T10:45:34.187 INFO:tasks.workunit.client.1.smithi026.stdout:5/819: chown d1/db/d1f/d61/d65/d82/dbb/dea/d11f 1033 1 2024-03-16T10:45:34.192 INFO:tasks.workunit.client.1.smithi026.stdout:3/940: rename d4/dc/dd/d47/d53/l72 to d4/d89/d118/l142 0 2024-03-16T10:45:34.196 INFO:tasks.workunit.client.1.smithi026.stdout:2/935: symlink d2/d29/d85/dc6/dd5/l14b 0 2024-03-16T10:45:34.197 INFO:tasks.workunit.client.1.smithi026.stdout:7/979: dread d6/fc [0,4194304] 0 2024-03-16T10:45:34.198 INFO:tasks.workunit.client.1.smithi026.stdout:0/955: truncate d1/d12/d2f/dba/fc7 419405 0 2024-03-16T10:45:34.200 INFO:tasks.workunit.client.1.smithi026.stdout:9/930: unlink d0/da/fe1 0 2024-03-16T10:45:34.206 INFO:tasks.workunit.client.1.smithi026.stdout:4/902: truncate d1/db/f42 2239656 0 2024-03-16T10:45:34.207 INFO:tasks.workunit.client.1.smithi026.stdout:2/936: mkdir d2/d29/d17/d1a/d2b/d2e/d60/d14c 0 2024-03-16T10:45:34.211 INFO:tasks.workunit.client.1.smithi026.stdout:5/820: mknod d1/db/c121 0 2024-03-16T10:45:34.213 INFO:tasks.workunit.client.1.smithi026.stdout:3/941: mknod d4/dc/dd/d135/c143 0 2024-03-16T10:45:34.216 INFO:tasks.workunit.client.1.smithi026.stdout:9/931: dread d0/dd7/f74 [0,4194304] 0 2024-03-16T10:45:34.219 INFO:tasks.workunit.client.1.smithi026.stdout:7/980: dread d6/de/fbe [0,4194304] 0 2024-03-16T10:45:34.222 INFO:tasks.workunit.client.1.smithi026.stdout:0/956: link d1/d12/f4b d1/d12/d2f/d48/d55/d60/d9b/d51/df6/f143 0 2024-03-16T10:45:34.223 INFO:tasks.workunit.client.1.smithi026.stdout:4/903: dread d1/db/de/d4c/f4d [0,4194304] 0 2024-03-16T10:45:34.224 INFO:tasks.workunit.client.1.smithi026.stdout:3/942: mknod d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/c144 0 2024-03-16T10:45:34.226 INFO:tasks.workunit.client.1.smithi026.stdout:0/957: write d1/d12/d2f/daa/dd1/d1a/d2b/fd5 [898942,6089] 0 2024-03-16T10:45:34.230 INFO:tasks.workunit.client.1.smithi026.stdout:5/821: truncate d1/d1c/d2a/dab/dc5/d69/d51/d80/fa6 1344661 0 2024-03-16T10:45:34.230 INFO:tasks.workunit.client.1.smithi026.stdout:7/981: dwrite d6/de/f64 [0,4194304] 0 2024-03-16T10:45:34.233 INFO:tasks.workunit.client.1.smithi026.stdout:3/943: creat d4/dc/dd/d47/d53/db1/d123/f145 x:0 0 0 2024-03-16T10:45:34.234 INFO:tasks.workunit.client.1.smithi026.stdout:0/958: creat d1/d12/d2f/dba/f144 x:0 0 0 2024-03-16T10:45:34.236 INFO:tasks.workunit.client.1.smithi026.stdout:3/944: chown d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d10b/d133 2 1 2024-03-16T10:45:34.236 INFO:tasks.workunit.client.1.smithi026.stdout:0/959: readlink d1/d12/d2f/daa/dd1/d1a/d2b/d4f/d96/l8d 0 2024-03-16T10:45:34.239 INFO:tasks.workunit.client.1.smithi026.stdout:0/960: read d1/d12/d2f/daa/dd1/d1a/d2b/fd5 [159758,91495] 0 2024-03-16T10:45:34.242 INFO:tasks.workunit.client.1.smithi026.stdout:4/904: rename d1/db/de/d4c/d6b/lcf to d1/db/de/d4c/d59/d6a/db2/l115 0 2024-03-16T10:45:34.242 INFO:tasks.workunit.client.1.smithi026.stdout:0/961: fdatasync d1/d12/d2f/daa/dd1/d1a/d2b/f4e 0 2024-03-16T10:45:34.249 INFO:tasks.workunit.client.1.smithi026.stdout:0/962: read d1/d12/d2f/d48/d55/d60/fa0 [3441325,81559] 0 2024-03-16T10:45:34.250 INFO:tasks.workunit.client.1.smithi026.stdout:3/945: link d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/l45 d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/l146 0 2024-03-16T10:45:34.252 INFO:tasks.workunit.client.1.smithi026.stdout:3/946: read d4/d89/f8f [2685672,21966] 0 2024-03-16T10:45:34.255 INFO:tasks.workunit.client.1.smithi026.stdout:0/963: mkdir d1/d12/d2f/daa/dd1/d1a/d145 0 2024-03-16T10:45:34.258 INFO:tasks.workunit.client.1.smithi026.stdout:4/905: dread d1/db/de/d4c/d59/f62 [0,4194304] 0 2024-03-16T10:45:34.258 INFO:tasks.workunit.client.1.smithi026.stdout:5/822: dread d1/f13 [0,4194304] 0 2024-03-16T10:45:34.259 INFO:tasks.workunit.client.1.smithi026.stdout:3/947: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/d104/l147 0 2024-03-16T10:45:34.260 INFO:tasks.workunit.client.1.smithi026.stdout:3/948: chown d4/dc/d31/dbe/de0 27 1 2024-03-16T10:45:34.261 INFO:tasks.workunit.client.1.smithi026.stdout:0/964: rmdir d1/d12/d2f/d48/d55/d60/d9b/d51/d122 39 2024-03-16T10:45:34.267 INFO:tasks.workunit.client.1.smithi026.stdout:5/823: dwrite d1/d1c/d2a/dab/dc5/d69/f119 [0,4194304] 0 2024-03-16T10:45:34.269 INFO:tasks.workunit.client.1.smithi026.stdout:3/949: read d4/dc/d31/f34 [1789959,122905] 0 2024-03-16T10:45:34.270 INFO:tasks.workunit.client.1.smithi026.stdout:0/965: fsync d1/d12/d2f/d48/d55/d60/d9b/d51/fcb 0 2024-03-16T10:45:34.271 INFO:tasks.workunit.client.1.smithi026.stdout:3/950: dread - d4/dc/dd/fd2 zero size 2024-03-16T10:45:34.272 INFO:tasks.workunit.client.1.smithi026.stdout:5/824: symlink d1/db/d1f/d61/d65/d82/dbb/l122 0 2024-03-16T10:45:34.273 INFO:tasks.workunit.client.1.smithi026.stdout:1/747: write d0/d17/d1f/d63/d6a/f94 [387854,39162] 0 2024-03-16T10:45:34.277 INFO:tasks.workunit.client.1.smithi026.stdout:1/748: chown d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa 74686 1 2024-03-16T10:45:34.279 INFO:tasks.workunit.client.1.smithi026.stdout:1/749: write d0/d17/d1f/d63/d6a/d7c/d3b/d64/f6f [5172253,89435] 0 2024-03-16T10:45:34.282 INFO:tasks.workunit.client.1.smithi026.stdout:3/951: dwrite d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/f117 [0,4194304] 0 2024-03-16T10:45:34.282 INFO:tasks.workunit.client.1.smithi026.stdout:1/750: chown d0/d17/d1f/d63/d6a/d7c/d2e/f7e 17 1 2024-03-16T10:45:34.284 INFO:tasks.workunit.client.1.smithi026.stdout:3/952: readlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/l68 0 2024-03-16T10:45:34.284 INFO:tasks.workunit.client.1.smithi026.stdout:1/751: chown d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6 198416773 1 2024-03-16T10:45:34.303 INFO:tasks.workunit.client.1.smithi026.stdout:5/825: mknod d1/d1c/d2a/dab/de8/c123 0 2024-03-16T10:45:34.308 INFO:tasks.workunit.client.1.smithi026.stdout:8/775: dwrite d5/d7/f6b [4194304,4194304] 0 2024-03-16T10:45:34.309 INFO:tasks.workunit.client.1.smithi026.stdout:1/752: read d0/d17/d1f/d63/d6a/d7c/d84/d9d/f4e [670911,17704] 0 2024-03-16T10:45:34.313 INFO:tasks.workunit.client.1.smithi026.stdout:3/953: symlink d4/dc/dd/d47/l148 0 2024-03-16T10:45:34.315 INFO:tasks.workunit.client.1.smithi026.stdout:0/966: link d1/d12/d2f/d48/d55/d60/de0/f102 d1/d12/d2f/d48/d55/d60/de0/dea/f146 0 2024-03-16T10:45:34.319 INFO:tasks.workunit.client.1.smithi026.stdout:1/753: truncate d0/d17/d1f/d20/d65/f8e 313206 0 2024-03-16T10:45:34.319 INFO:tasks.workunit.client.1.smithi026.stdout:9/932: write d0/da/d14/f6a [910307,112235] 0 2024-03-16T10:45:34.320 INFO:tasks.workunit.client.1.smithi026.stdout:2/937: write d2/d29/d36/f123 [1662793,38823] 0 2024-03-16T10:45:34.322 INFO:tasks.workunit.client.1.smithi026.stdout:9/933: stat d0/da/d16/d5d/d117 0 2024-03-16T10:45:34.324 INFO:tasks.workunit.client.1.smithi026.stdout:8/776: link d5/d23/f67 d5/d7/d2c/d36/d59/d93/dde/fe9 0 2024-03-16T10:45:34.326 INFO:tasks.workunit.client.1.smithi026.stdout:0/967: rename d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/df5/c117 to d1/d12/d2f/daa/dd1/d61/d126/c147 0 2024-03-16T10:45:34.330 INFO:tasks.workunit.client.1.smithi026.stdout:1/754: creat d0/d17/d1f/d20/dcc/fee x:0 0 0 2024-03-16T10:45:34.332 INFO:tasks.workunit.client.1.smithi026.stdout:2/938: mkdir d2/d29/d17/d1a/d104/d14d 0 2024-03-16T10:45:34.334 INFO:tasks.workunit.client.1.smithi026.stdout:8/777: mkdir d5/d7/d2c/d36/d59/d93/da3/dea 0 2024-03-16T10:45:34.335 INFO:tasks.workunit.client.1.smithi026.stdout:3/954: dread d4/d89/f8f [0,4194304] 0 2024-03-16T10:45:34.336 INFO:tasks.workunit.client.1.smithi026.stdout:2/939: write d2/d29/d17/d1a/d2b/d2e/d60/d7f/d131/d6d/f11c [5033282,115978] 0 2024-03-16T10:45:34.339 INFO:tasks.workunit.client.1.smithi026.stdout:9/934: symlink d0/da/d13/l143 0 2024-03-16T10:45:34.341 INFO:tasks.workunit.client.1.smithi026.stdout:8/778: fsync d5/d5d/d9a/ddd/f20 0 2024-03-16T10:45:34.343 INFO:tasks.workunit.client.1.smithi026.stdout:8/779: write d5/dc/faf [409782,8345] 0 2024-03-16T10:45:34.344 INFO:tasks.workunit.client.1.smithi026.stdout:1/755: dread d0/d17/d1f/d63/d6a/d7c/d84/fa5 [0,4194304] 0 2024-03-16T10:45:34.346 INFO:tasks.workunit.client.1.smithi026.stdout:9/935: creat d0/da/d14/da7/d126/dd5/f144 x:0 0 0 2024-03-16T10:45:34.347 INFO:tasks.workunit.client.1.smithi026.stdout:9/936: fsync d0/da/f11 0 2024-03-16T10:45:34.348 INFO:tasks.workunit.client.1.smithi026.stdout:9/937: chown d0/da/d16/f100 4440 1 2024-03-16T10:45:34.351 INFO:tasks.workunit.client.1.smithi026.stdout:8/780: truncate d5/d23/f61 715039 0 2024-03-16T10:45:34.353 INFO:tasks.workunit.client.1.smithi026.stdout:4/906: dwrite d1/db/de/d4c/d59/f80 [0,4194304] 0 2024-03-16T10:45:34.355 INFO:tasks.workunit.client.1.smithi026.stdout:1/756: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/fef x:0 0 0 2024-03-16T10:45:34.359 INFO:tasks.workunit.client.1.smithi026.stdout:3/955: rename d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d132 to d4/d149 0 2024-03-16T10:45:34.363 INFO:tasks.workunit.client.1.smithi026.stdout:8/781: creat d5/da0/feb x:0 0 0 2024-03-16T10:45:34.364 INFO:tasks.workunit.client.1.smithi026.stdout:8/782: chown d5/d5d/fd8 437 1 2024-03-16T10:45:34.365 INFO:tasks.workunit.client.1.smithi026.stdout:4/907: creat d1/db/de/d4c/f116 x:0 0 0 2024-03-16T10:45:34.367 INFO:tasks.workunit.client.1.smithi026.stdout:4/908: truncate d1/f104 851036 0 2024-03-16T10:45:34.373 INFO:tasks.workunit.client.1.smithi026.stdout:4/909: dread - d1/db/de/d4c/d59/d6a/df1/f109 zero size 2024-03-16T10:45:34.374 INFO:tasks.workunit.client.1.smithi026.stdout:8/783: dwrite d5/da0/feb [0,4194304] 0 2024-03-16T10:45:34.374 INFO:tasks.workunit.client.1.smithi026.stdout:9/938: creat d0/da/d14/da7/d126/d130/f145 x:0 0 0 2024-03-16T10:45:34.378 INFO:tasks.workunit.client.1.smithi026.stdout:5/826: dwrite d1/db/d1f/d61/d116/f60 [0,4194304] 0 2024-03-16T10:45:34.384 INFO:tasks.workunit.client.1.smithi026.stdout:4/910: dwrite d1/d85/da2/fb0 [0,4194304] 0 2024-03-16T10:45:34.389 INFO:tasks.workunit.client.1.smithi026.stdout:9/939: mknod d0/d63/d6b/ded/c146 0 2024-03-16T10:45:34.394 INFO:tasks.workunit.client.1.smithi026.stdout:8/784: symlink d5/d7/d2c/d36/d59/d93/db0/lec 0 2024-03-16T10:45:34.395 INFO:tasks.workunit.client.1.smithi026.stdout:0/968: dwrite d1/d12/d2f/d48/d55/d60/d9b/ff0 [0,4194304] 0 2024-03-16T10:45:34.403 INFO:tasks.workunit.client.1.smithi026.stdout:2/940: dwrite d2/d29/d17/d1a/d2b/f9a [0,4194304] 0 2024-03-16T10:45:34.404 INFO:tasks.workunit.client.1.smithi026.stdout:2/941: dread - d2/d29/d17/d1a/f133 zero size 2024-03-16T10:45:34.408 INFO:tasks.workunit.client.1.smithi026.stdout:4/911: creat d1/db/ddb/daa/d44/dd5/de9/f117 x:0 0 0 2024-03-16T10:45:34.413 INFO:tasks.workunit.client.1.smithi026.stdout:1/757: link d0/fab d0/d17/d1f/d63/d6a/d7c/d84/dbb/ff0 0 2024-03-16T10:45:34.420 INFO:tasks.workunit.client.1.smithi026.stdout:5/827: creat d1/d1c/d2a/dab/d11d/f124 x:0 0 0 2024-03-16T10:45:34.424 INFO:tasks.workunit.client.1.smithi026.stdout:2/942: read d2/d29/f56 [835661,82984] 0 2024-03-16T10:45:34.429 INFO:tasks.workunit.client.1.smithi026.stdout:3/956: write d4/d89/d97/fd6 [1074951,92057] 0 2024-03-16T10:45:34.431 INFO:tasks.workunit.client.1.smithi026.stdout:2/943: mkdir d2/d29/d110/d14e 0 2024-03-16T10:45:34.435 INFO:tasks.workunit.client.1.smithi026.stdout:1/758: mknod d0/d17/d1f/deb/cf1 0 2024-03-16T10:45:34.440 INFO:tasks.workunit.client.1.smithi026.stdout:4/912: dwrite d1/db/de/d6e/feb [4194304,4194304] 0 2024-03-16T10:45:34.441 INFO:tasks.workunit.client.1.smithi026.stdout:2/944: creat d2/d29/d17/de8/f14f x:0 0 0 2024-03-16T10:45:34.442 INFO:tasks.workunit.client.1.smithi026.stdout:4/913: chown d1/db/de/d4c/d6b/d7a/db1 1 1 2024-03-16T10:45:34.445 INFO:tasks.workunit.client.1.smithi026.stdout:1/759: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/ff2 x:0 0 0 2024-03-16T10:45:34.446 INFO:tasks.workunit.client.1.smithi026.stdout:2/945: rmdir d2/dd1 39 2024-03-16T10:45:34.447 INFO:tasks.workunit.client.1.smithi026.stdout:4/914: rmdir d1/db/ddb/daa/d44/d48/d8e 39 2024-03-16T10:45:34.453 INFO:tasks.workunit.client.1.smithi026.stdout:1/760: mknod d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/cf3 0 2024-03-16T10:45:34.455 INFO:tasks.workunit.client.1.smithi026.stdout:4/915: rename d1/db/ddb/f10d to d1/db/de/d4c/d6b/d89/d9c/f118 0 2024-03-16T10:45:34.456 INFO:tasks.workunit.client.1.smithi026.stdout:1/761: rename d0/d17/d1f/d63 to d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/df4 22 2024-03-16T10:45:34.457 INFO:tasks.workunit.client.1.smithi026.stdout:1/762: chown d0/d17/db7 9 1 2024-03-16T10:45:34.460 INFO:tasks.workunit.client.1.smithi026.stdout:2/946: rmdir d2/d29/d110/d14e 0 2024-03-16T10:45:34.461 INFO:tasks.workunit.client.1.smithi026.stdout:3/957: dread d4/dc/dd/d47/d53/f5c [0,4194304] 0 2024-03-16T10:45:34.463 INFO:tasks.workunit.client.1.smithi026.stdout:0/969: dwrite d1/d12/d2f/daa/dac/f100 [0,4194304] 0 2024-03-16T10:45:34.465 INFO:tasks.workunit.client.1.smithi026.stdout:4/916: truncate d1/db/de/d4c/d59/d6a/df1/f109 57965 0 2024-03-16T10:45:34.469 INFO:tasks.workunit.client.1.smithi026.stdout:1/763: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/df5 0 2024-03-16T10:45:34.473 INFO:tasks.workunit.client.1.smithi026.stdout:2/947: mknod d2/c150 0 2024-03-16T10:45:34.473 INFO:tasks.workunit.client.1.smithi026.stdout:5/828: dwrite d1/db/f44 [0,4194304] 0 2024-03-16T10:45:34.476 INFO:tasks.workunit.client.1.smithi026.stdout:4/917: mknod d1/db/de/d4c/d59/c119 0 2024-03-16T10:45:34.476 INFO:tasks.workunit.client.1.smithi026.stdout:4/918: stat d1/l5 0 2024-03-16T10:45:34.478 INFO:tasks.workunit.client.1.smithi026.stdout:4/919: chown d1/db/de/d4c/d59/c9d 67191 1 2024-03-16T10:45:34.484 INFO:tasks.workunit.client.1.smithi026.stdout:5/829: creat d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/dc6/f125 x:0 0 0 2024-03-16T10:45:34.488 INFO:tasks.workunit.client.1.smithi026.stdout:3/958: write d4/dc/dd/d47/d53/ff8 [737938,50562] 0 2024-03-16T10:45:34.489 INFO:tasks.workunit.client.1.smithi026.stdout:1/764: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6 0 2024-03-16T10:45:34.492 INFO:tasks.workunit.client.1.smithi026.stdout:5/830: fdatasync d1/d1c/d2a/d55/d99/f96 0 2024-03-16T10:45:34.493 INFO:tasks.workunit.client.1.smithi026.stdout:4/920: getdents d1/db/ddb/daa/dfb 0 2024-03-16T10:45:34.495 INFO:tasks.workunit.client.1.smithi026.stdout:4/921: read d1/db/de/d4c/d59/d6a/f9b [4661461,70233] 0 2024-03-16T10:45:34.496 INFO:tasks.workunit.client.1.smithi026.stdout:4/922: chown d1/db/de/c36 1741 1 2024-03-16T10:45:34.498 INFO:tasks.workunit.client.1.smithi026.stdout:2/948: rename d2/d29/da4 to d2/d29/d17/d1a/d2b/d69/d151 0 2024-03-16T10:45:34.502 INFO:tasks.workunit.client.1.smithi026.stdout:3/959: symlink d4/dc/d31/dbe/de0/l14a 0 2024-03-16T10:45:34.502 INFO:tasks.workunit.client.1.smithi026.stdout:4/923: dwrite d1/db/de/d4c/d59/d6a/f9b [0,4194304] 0 2024-03-16T10:45:34.504 INFO:tasks.workunit.client.1.smithi026.stdout:4/924: chown d1/db/de/d4c/d6b/c83 11952332 1 2024-03-16T10:45:34.510 INFO:tasks.workunit.client.1.smithi026.stdout:5/831: write d1/d6e/fd7 [1018615,113590] 0 2024-03-16T10:45:34.514 INFO:tasks.workunit.client.1.smithi026.stdout:3/960: fdatasync d4/f41 0 2024-03-16T10:45:34.514 INFO:tasks.workunit.client.1.smithi026.stdout:4/925: creat d1/db/ddb/daa/d44/dd5/de9/f11a x:0 0 0 2024-03-16T10:45:34.515 INFO:tasks.workunit.client.1.smithi026.stdout:2/949: mkdir d2/d29/d152 0 2024-03-16T10:45:34.517 INFO:tasks.workunit.client.1.smithi026.stdout:4/926: chown d1/db/ddb/lc4 97 1 2024-03-16T10:45:34.517 INFO:tasks.workunit.client.1.smithi026.stdout:5/832: unlink d1/db/d1f/d61/d116/f23 0 2024-03-16T10:45:34.521 INFO:tasks.workunit.client.1.smithi026.stdout:7/982: sync 2024-03-16T10:45:34.521 INFO:tasks.workunit.client.1.smithi026.stdout:6/681: sync 2024-03-16T10:45:34.522 INFO:tasks.workunit.client.1.smithi026.stdout:1/765: rename d0/d17/d1f/deb/cf1 to d0/d17/d1f/d63/d6a/d7c/d2e/cf7 0 2024-03-16T10:45:34.523 INFO:tasks.workunit.client.1.smithi026.stdout:3/961: creat d4/dc/d31/d48/d7f/f14b x:0 0 0 2024-03-16T10:45:34.525 INFO:tasks.workunit.client.1.smithi026.stdout:5/833: mkdir d1/db/d1f/d61/d65/d82/dbb/dea/d126 0 2024-03-16T10:45:34.527 INFO:tasks.workunit.client.1.smithi026.stdout:1/766: creat d0/d17/d1f/d63/dbc/ff8 x:0 0 0 2024-03-16T10:45:34.528 INFO:tasks.workunit.client.1.smithi026.stdout:2/950: mknod d2/d29/d152/c153 0 2024-03-16T10:45:34.528 INFO:tasks.workunit.client.1.smithi026.stdout:1/767: fdatasync d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/fdf 0 2024-03-16T10:45:34.542 INFO:tasks.workunit.client.1.smithi026.stdout:3/962: symlink d4/dc/dd/l14c 0 2024-03-16T10:45:34.549 INFO:tasks.workunit.client.1.smithi026.stdout:6/682: dread d0/d28/fb2 [0,4194304] 0 2024-03-16T10:45:34.550 INFO:tasks.workunit.client.1.smithi026.stdout:6/683: chown d0/d28/d49/f78 69723 1 2024-03-16T10:45:34.556 INFO:tasks.workunit.client.1.smithi026.stdout:5/834: write d1/db/d1f/d61/d65/d82/fcf [682131,22629] 0 2024-03-16T10:45:34.556 INFO:tasks.workunit.client.1.smithi026.stdout:6/684: link d0/f9a d0/d8a/fe8 0 2024-03-16T10:45:34.558 INFO:tasks.workunit.client.1.smithi026.stdout:6/685: read d0/f1d [266104,14559] 0 2024-03-16T10:45:34.559 INFO:tasks.workunit.client.1.smithi026.stdout:1/768: dread d0/d17/f48 [0,4194304] 0 2024-03-16T10:45:34.560 INFO:tasks.workunit.client.1.smithi026.stdout:5/835: chown d1/d1c/d2a/dab/dc5/d69/d51/f5d 140069324 1 2024-03-16T10:45:34.561 INFO:tasks.workunit.client.1.smithi026.stdout:1/769: readlink d0/d17/d1f/d63/d6a/d7c/d6b/l72 0 2024-03-16T10:45:34.561 INFO:tasks.workunit.client.1.smithi026.stdout:6/686: creat d0/d41/dbe/fe9 x:0 0 0 2024-03-16T10:45:34.563 INFO:tasks.workunit.client.1.smithi026.stdout:5/836: symlink d1/d1c/d68/l127 0 2024-03-16T10:45:34.563 INFO:tasks.workunit.client.1.smithi026.stdout:3/963: write d4/dc/d31/d48/fd1 [947254,55967] 0 2024-03-16T10:45:34.568 INFO:tasks.workunit.client.1.smithi026.stdout:6/687: rename d0/d28/d49/db0/fe0 to d0/d10/d79/dca/dde/fea 0 2024-03-16T10:45:34.569 INFO:tasks.workunit.client.1.smithi026.stdout:5/837: fsync d1/d1c/d2a/dab/dc5/d69/f72 0 2024-03-16T10:45:34.574 INFO:tasks.workunit.client.1.smithi026.stdout:3/964: rename d4/dc/d31/dbe/de0/l101 to d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/d10b/d133/dd5/l14d 0 2024-03-16T10:45:34.575 INFO:tasks.workunit.client.1.smithi026.stdout:5/838: dwrite d1/d1c/d2a/dab/dc5/d69/d51/f9b [0,4194304] 0 2024-03-16T10:45:34.577 INFO:tasks.workunit.client.1.smithi026.stdout:3/965: chown d4/dc/d31/d48/d69/d78/c8c 0 1 2024-03-16T10:45:34.582 INFO:tasks.workunit.client.1.smithi026.stdout:1/770: getdents d0/d17/d1f 0 2024-03-16T10:45:34.583 INFO:tasks.workunit.client.1.smithi026.stdout:1/771: chown d0/d17/d1f/deb 30977 1 2024-03-16T10:45:34.585 INFO:tasks.workunit.client.1.smithi026.stdout:6/688: fsync d0/d5b/f18 0 2024-03-16T10:45:34.587 INFO:tasks.workunit.client.1.smithi026.stdout:5/839: rename d1/d1c/d2a/dab/de8/f120 to d1/d1c/d4c/d4d/f128 0 2024-03-16T10:45:34.592 INFO:tasks.workunit.client.1.smithi026.stdout:1/772: rmdir d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6 39 2024-03-16T10:45:34.594 INFO:tasks.workunit.client.1.smithi026.stdout:6/689: mknod d0/d28/dcb/ceb 0 2024-03-16T10:45:34.594 INFO:tasks.workunit.client.1.smithi026.stdout:3/966: write d4/d89/d97/fb7 [653255,32608] 0 2024-03-16T10:45:34.601 INFO:tasks.workunit.client.1.smithi026.stdout:3/967: symlink d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/l14e 0 2024-03-16T10:45:34.605 INFO:tasks.workunit.client.1.smithi026.stdout:1/773: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/ff9 x:0 0 0 2024-03-16T10:45:34.606 INFO:tasks.workunit.client.1.smithi026.stdout:3/968: dread - d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/fec zero size 2024-03-16T10:45:34.607 INFO:tasks.workunit.client.1.smithi026.stdout:5/840: dwrite d1/d1c/d2a/dab/dc5/d69/d51/f5d [0,4194304] 0 2024-03-16T10:45:34.608 INFO:tasks.workunit.client.1.smithi026.stdout:6/690: link d0/f89 d0/d41/dc5/fec 0 2024-03-16T10:45:34.610 INFO:tasks.workunit.client.1.smithi026.stdout:6/691: write d0/d28/dcb/fdd [602256,123614] 0 2024-03-16T10:45:34.612 INFO:tasks.workunit.client.1.smithi026.stdout:1/774: unlink d0/d17/d1f/d63/d6a/d7c/d84/fa5 0 2024-03-16T10:45:34.616 INFO:tasks.workunit.client.1.smithi026.stdout:3/969: symlink d4/dc/d73/d83/d7c/dd7/l14f 0 2024-03-16T10:45:34.621 INFO:tasks.workunit.client.1.smithi026.stdout:5/841: dwrite d1/db/d1f/d61/fb9 [0,4194304] 0 2024-03-16T10:45:34.623 INFO:tasks.workunit.client.1.smithi026.stdout:6/692: dwrite d0/d10/fc3 [0,4194304] 0 2024-03-16T10:45:34.627 INFO:tasks.workunit.client.1.smithi026.stdout:1/775: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/ffa x:0 0 0 2024-03-16T10:45:34.628 INFO:tasks.workunit.client.1.smithi026.stdout:1/776: fsync d0/d17/d1f/d63/d6a/d7c/d84/d9d/fd5 0 2024-03-16T10:45:34.633 INFO:tasks.workunit.client.1.smithi026.stdout:3/970: rename d4/dc/dd/d47/d53/db1/ddd to d4/dc/dd/d47/d53/db1/d123/d150 0 2024-03-16T10:45:34.639 INFO:tasks.workunit.client.1.smithi026.stdout:5/842: mkdir d1/d1c/d2a/dab/dc5/d69/d51/d129 0 2024-03-16T10:45:34.646 INFO:tasks.workunit.client.1.smithi026.stdout:6/693: dread d0/d28/d2d/f55 [0,4194304] 0 2024-03-16T10:45:34.647 INFO:tasks.workunit.client.1.smithi026.stdout:3/971: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f151 x:0 0 0 2024-03-16T10:45:34.649 INFO:tasks.workunit.client.1.smithi026.stdout:5/843: symlink d1/l12a 0 2024-03-16T10:45:34.652 INFO:tasks.workunit.client.1.smithi026.stdout:1/777: symlink d0/d17/d1f/d63/d6a/d7c/lfb 0 2024-03-16T10:45:34.656 INFO:tasks.workunit.client.1.smithi026.stdout:3/972: truncate d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/f49 294884 0 2024-03-16T10:45:34.658 INFO:tasks.workunit.client.1.smithi026.stdout:6/694: dread d0/d10/d79/fb4 [0,4194304] 0 2024-03-16T10:45:34.659 INFO:tasks.workunit.client.1.smithi026.stdout:5/844: rename d1/d1c/d2a/d55/d99/fb4 to d1/d1c/d4c/f12b 0 2024-03-16T10:45:34.666 INFO:tasks.workunit.client.1.smithi026.stdout:1/778: dwrite d0/d17/d1f/d20/d65/f8c [0,4194304] 0 2024-03-16T10:45:34.669 INFO:tasks.workunit.client.1.smithi026.stdout:6/695: mknod d0/d62/ced 0 2024-03-16T10:45:34.670 INFO:tasks.workunit.client.1.smithi026.stdout:5/845: creat d1/d1c/d2a/dab/de8/f12c x:0 0 0 2024-03-16T10:45:34.672 INFO:tasks.workunit.client.1.smithi026.stdout:5/846: chown d1/d1c/d2a/dab/dc5/f76 2 1 2024-03-16T10:45:34.674 INFO:tasks.workunit.client.1.smithi026.stdout:3/973: getdents d4/dc/d73/d83/d57 0 2024-03-16T10:45:34.678 INFO:tasks.workunit.client.1.smithi026.stdout:5/847: mknod d1/db/d1f/d61/d116/c12d 0 2024-03-16T10:45:34.680 INFO:tasks.workunit.client.1.smithi026.stdout:5/848: dread - d1/db/d1f/d61/d65/d82/dbb/dea/fd8 zero size 2024-03-16T10:45:34.684 INFO:tasks.workunit.client.1.smithi026.stdout:6/696: creat d0/d10/d79/fee x:0 0 0 2024-03-16T10:45:34.685 INFO:tasks.workunit.client.1.smithi026.stdout:5/849: creat d1/db/d1f/d61/f12e x:0 0 0 2024-03-16T10:45:34.686 INFO:tasks.workunit.client.1.smithi026.stdout:5/850: chown d1/d1c/ff3 7 1 2024-03-16T10:45:34.690 INFO:tasks.workunit.client.1.smithi026.stdout:6/697: dwrite d0/d41/d4f/fa9 [0,4194304] 0 2024-03-16T10:45:34.691 INFO:tasks.workunit.client.1.smithi026.stdout:5/851: read d1/d1c/d2a/f3b [126157,94107] 0 2024-03-16T10:45:34.691 INFO:tasks.workunit.client.1.smithi026.stdout:3/974: dread d4/dc/f13 [0,4194304] 0 2024-03-16T10:45:34.693 INFO:tasks.workunit.client.1.smithi026.stdout:6/698: dread d0/d62/fbf [0,4194304] 0 2024-03-16T10:45:34.701 INFO:tasks.workunit.client.1.smithi026.stdout:1/779: write d0/d17/d1f/d63/d6a/d7c/fba [72602,33608] 0 2024-03-16T10:45:34.705 INFO:tasks.workunit.client.1.smithi026.stdout:3/975: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/f152 x:0 0 0 2024-03-16T10:45:34.713 INFO:tasks.workunit.client.1.smithi026.stdout:5/852: dread d1/d1c/d68/fa3 [0,4194304] 0 2024-03-16T10:45:34.714 INFO:tasks.workunit.client.1.smithi026.stdout:1/780: link d0/fb3 d0/d17/d1f/d63/d6a/ffc 0 2024-03-16T10:45:34.716 INFO:tasks.workunit.client.1.smithi026.stdout:3/976: creat d4/dc/f153 x:0 0 0 2024-03-16T10:45:34.717 INFO:tasks.workunit.client.1.smithi026.stdout:5/853: readlink d1/db/l84 0 2024-03-16T10:45:34.720 INFO:tasks.workunit.client.1.smithi026.stdout:3/977: symlink d4/dc/d31/d48/d69/l154 0 2024-03-16T10:45:34.722 INFO:tasks.workunit.client.1.smithi026.stdout:9/940: sync 2024-03-16T10:45:34.724 INFO:tasks.workunit.client.1.smithi026.stdout:1/781: link d0/d17/d1f/d20/fbd d0/d17/d1f/d63/d6a/d7c/d3b/ffd 0 2024-03-16T10:45:34.731 INFO:tasks.workunit.client.1.smithi026.stdout:5/854: dread d1/db/d1f/f26 [0,4194304] 0 2024-03-16T10:45:34.732 INFO:tasks.workunit.client.1.smithi026.stdout:1/782: rmdir d0/d17/d1f/d20/dcc 39 2024-03-16T10:45:34.733 INFO:tasks.workunit.client.1.smithi026.stdout:3/978: dread d4/dc/dd/d47/d53/db1/d123/d150/ffc [0,4194304] 0 2024-03-16T10:45:34.735 INFO:tasks.workunit.client.1.smithi026.stdout:9/941: mkdir d0/d5f/dfc/d147 0 2024-03-16T10:45:34.737 INFO:tasks.workunit.client.1.smithi026.stdout:5/855: mkdir d1/db/d1f/d61/d12f 0 2024-03-16T10:45:34.742 INFO:tasks.workunit.client.1.smithi026.stdout:6/699: write d0/d28/f37 [1858898,47573] 0 2024-03-16T10:45:34.745 INFO:tasks.workunit.client.1.smithi026.stdout:5/856: rmdir d1/d1c/d2a/dab/de8 39 2024-03-16T10:45:34.745 INFO:tasks.workunit.client.1.smithi026.stdout:1/783: dread d0/fb3 [0,4194304] 0 2024-03-16T10:45:34.751 INFO:tasks.workunit.client.1.smithi026.stdout:6/700: mkdir d0/d28/dcb/def 0 2024-03-16T10:45:34.752 INFO:tasks.workunit.client.1.smithi026.stdout:5/857: unlink d1/d1c/d2a/f3b 0 2024-03-16T10:45:34.756 INFO:tasks.workunit.client.1.smithi026.stdout:3/979: link d4/dc/l2d d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/d104/l155 0 2024-03-16T10:45:34.759 INFO:tasks.workunit.client.1.smithi026.stdout:6/701: truncate d0/d28/d2d/f55 4313365 0 2024-03-16T10:45:34.763 INFO:tasks.workunit.client.1.smithi026.stdout:1/784: creat d0/d17/d1f/d20/dcc/ffe x:0 0 0 2024-03-16T10:45:34.768 INFO:tasks.workunit.client.1.smithi026.stdout:5/858: symlink d1/d1c/d2a/dab/dc5/d69/d51/d129/l130 0 2024-03-16T10:45:34.773 INFO:tasks.workunit.client.1.smithi026.stdout:3/980: truncate d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/f131 832797 0 2024-03-16T10:45:34.774 INFO:tasks.workunit.client.1.smithi026.stdout:1/785: dwrite d0/d17/db7/fca [0,4194304] 0 2024-03-16T10:45:34.775 INFO:tasks.workunit.client.1.smithi026.stdout:3/981: readlink d4/d89/d97/l2a 0 2024-03-16T10:45:34.785 INFO:tasks.workunit.client.1.smithi026.stdout:5/859: truncate d1/db/d1f/d61/d116/f66 305934 0 2024-03-16T10:45:34.791 INFO:tasks.workunit.client.1.smithi026.stdout:3/982: creat d4/dc/d31/d48/d69/d78/db3/f156 x:0 0 0 2024-03-16T10:45:34.793 INFO:tasks.workunit.client.1.smithi026.stdout:1/786: mkdir d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dff 0 2024-03-16T10:45:34.801 INFO:tasks.workunit.client.1.smithi026.stdout:1/787: readlink d0/l1d 0 2024-03-16T10:45:34.802 INFO:tasks.workunit.client.1.smithi026.stdout:9/942: dwrite d0/da/f7a [4194304,4194304] 0 2024-03-16T10:45:34.803 INFO:tasks.workunit.client.1.smithi026.stdout:3/983: mkdir d4/dc/dd/d47/d53/db1/d123/d157 0 2024-03-16T10:45:34.815 INFO:tasks.workunit.client.1.smithi026.stdout:6/702: dread d0/d10/d79/d7b/f9d [0,4194304] 0 2024-03-16T10:45:34.819 INFO:tasks.workunit.client.1.smithi026.stdout:3/984: mkdir d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/dfa/d94/d158 0 2024-03-16T10:45:34.821 INFO:tasks.workunit.client.1.smithi026.stdout:9/943: fdatasync d0/da/d13/f49 0 2024-03-16T10:45:34.821 INFO:tasks.workunit.client.1.smithi026.stdout:3/985: write d4/dc/dd/d80/dc2/f13e [425694,48994] 0 2024-03-16T10:45:34.824 INFO:tasks.workunit.client.1.smithi026.stdout:1/788: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/d100 0 2024-03-16T10:45:34.825 INFO:tasks.workunit.client.1.smithi026.stdout:5/860: dread d1/d1c/d2a/dab/dc5/f52 [0,4194304] 0 2024-03-16T10:45:34.826 INFO:tasks.workunit.client.1.smithi026.stdout:6/703: mkdir d0/d10/d79/d8e/df0 0 2024-03-16T10:45:34.834 INFO:tasks.workunit.client.1.smithi026.stdout:8/785: sync 2024-03-16T10:45:34.844 INFO:tasks.workunit.client.1.smithi026.stdout:5/861: symlink d1/d1c/d2a/l131 0 2024-03-16T10:45:34.845 INFO:tasks.workunit.client.1.smithi026.stdout:8/786: dread - d5/d5d/d9a/fcf zero size 2024-03-16T10:45:34.845 INFO:tasks.workunit.client.1.smithi026.stdout:5/862: chown d1/d6e/f92 904756583 1 2024-03-16T10:45:34.847 INFO:tasks.workunit.client.1.smithi026.stdout:6/704: mknod d0/cf1 0 2024-03-16T10:45:34.849 INFO:tasks.workunit.client.1.smithi026.stdout:1/789: creat d0/d17/d1f/d63/f101 x:0 0 0 2024-03-16T10:45:34.853 INFO:tasks.workunit.client.1.smithi026.stdout:5/863: creat d1/d1c/d2a/dab/d11d/f132 x:0 0 0 2024-03-16T10:45:34.854 INFO:tasks.workunit.client.1.smithi026.stdout:3/986: dwrite d4/f9 [4194304,4194304] 0 2024-03-16T10:45:34.854 INFO:tasks.workunit.client.1.smithi026.stdout:6/705: mkdir d0/d28/d49/d7d/df2 0 2024-03-16T10:45:34.857 INFO:tasks.workunit.client.1.smithi026.stdout:8/787: symlink d5/d5d/d9a/dd6/led 0 2024-03-16T10:45:34.858 INFO:tasks.workunit.client.1.smithi026.stdout:6/706: write d0/d41/d4f/f96 [4959546,125341] 0 2024-03-16T10:45:34.859 INFO:tasks.workunit.client.1.smithi026.stdout:9/944: dwrite d0/dd7/fd1 [0,4194304] 0 2024-03-16T10:45:34.861 INFO:tasks.workunit.client.1.smithi026.stdout:8/788: dread - d5/dc/d40/d53/f63 zero size 2024-03-16T10:45:34.868 INFO:tasks.workunit.client.1.smithi026.stdout:3/987: rename d4/dc/dd/d47/d53/c55 to d4/dc/d31/dbe/de0/c159 0 2024-03-16T10:45:34.872 INFO:tasks.workunit.client.1.smithi026.stdout:9/945: truncate d0/da/d16/d17/d44/f1e 1932369 0 2024-03-16T10:45:34.877 INFO:tasks.workunit.client.1.smithi026.stdout:3/988: chown d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/d104/l155 12447638 1 2024-03-16T10:45:34.880 INFO:tasks.workunit.client.1.smithi026.stdout:8/789: rename d5/f7f to d5/d7/d7b/fee 0 2024-03-16T10:45:34.881 INFO:tasks.workunit.client.1.smithi026.stdout:3/989: symlink d4/dc/d73/d83/l15a 0 2024-03-16T10:45:34.885 INFO:tasks.workunit.client.1.smithi026.stdout:9/946: symlink d0/l148 0 2024-03-16T10:45:34.885 INFO:tasks.workunit.client.1.smithi026.stdout:5/864: write d1/d1c/d2a/dab/dc5/fba [443312,1042] 0 2024-03-16T10:45:34.888 INFO:tasks.workunit.client.1.smithi026.stdout:8/790: dwrite d5/d5d/f9d [0,4194304] 0 2024-03-16T10:45:34.888 INFO:tasks.workunit.client.1.smithi026.stdout:6/707: write d0/d28/f40 [4622911,63041] 0 2024-03-16T10:45:34.890 INFO:tasks.workunit.client.1.smithi026.stdout:3/990: rename d4/d89/d97/lba to d4/dc/dd/d47/d53/db1/d123/d157/l15b 0 2024-03-16T10:45:34.895 INFO:tasks.workunit.client.1.smithi026.stdout:9/947: mknod d0/d5f/d7f/d8c/c149 0 2024-03-16T10:45:34.903 INFO:tasks.workunit.client.1.smithi026.stdout:3/991: dread d4/dc/d73/d83/d7c/dd7/f116 [0,4194304] 0 2024-03-16T10:45:34.905 INFO:tasks.workunit.client.1.smithi026.stdout:6/708: truncate d0/d41/d4f/f69 545681 0 2024-03-16T10:45:34.912 INFO:tasks.workunit.client.1.smithi026.stdout:5/865: symlink d1/d1c/d2a/d55/l133 0 2024-03-16T10:45:34.917 INFO:tasks.workunit.client.1.smithi026.stdout:3/992: mknod d4/dc/dd/d47/d53/db1/d123/d150/c15c 0 2024-03-16T10:45:34.919 INFO:tasks.workunit.client.1.smithi026.stdout:6/709: write d0/d10/d79/fb4 [621797,64362] 0 2024-03-16T10:45:34.920 INFO:tasks.workunit.client.1.smithi026.stdout:6/710: fsync d0/f44 0 2024-03-16T10:45:34.924 INFO:tasks.workunit.client.1.smithi026.stdout:9/948: dwrite d0/da/f1b [8388608,4194304] 0 2024-03-16T10:45:34.925 INFO:tasks.workunit.client.1.smithi026.stdout:9/949: chown d0/da/d16/d5d/d72/l50 74894138 1 2024-03-16T10:45:34.926 INFO:tasks.workunit.client.1.smithi026.stdout:8/791: creat d5/d7/d2c/d36/fef x:0 0 0 2024-03-16T10:45:34.931 INFO:tasks.workunit.client.1.smithi026.stdout:3/993: creat d4/dc/d31/d48/d69/d78/db3/f15d x:0 0 0 2024-03-16T10:45:34.934 INFO:tasks.workunit.client.1.smithi026.stdout:8/792: write d5/d23/fac [3087074,104995] 0 2024-03-16T10:45:34.937 INFO:tasks.workunit.client.1.smithi026.stdout:9/950: truncate d0/da/f11 730530 0 2024-03-16T10:45:34.938 INFO:tasks.workunit.client.1.smithi026.stdout:5/866: dwrite d1/d1c/d2a/dab/dc5/f49 [0,4194304] 0 2024-03-16T10:45:34.939 INFO:tasks.workunit.client.1.smithi026.stdout:6/711: creat d0/d10/ff3 x:0 0 0 2024-03-16T10:45:34.941 INFO:tasks.workunit.client.1.smithi026.stdout:6/712: chown d0/d28/d49/f78 1050 1 2024-03-16T10:45:34.943 INFO:tasks.workunit.client.1.smithi026.stdout:6/713: chown d0/d62/fbf 59628 1 2024-03-16T10:45:34.945 INFO:tasks.workunit.client.1.smithi026.stdout:8/793: rmdir d5/d7/d2c 39 2024-03-16T10:45:34.946 INFO:tasks.workunit.client.1.smithi026.stdout:5/867: dwrite d1/f13 [0,4194304] 0 2024-03-16T10:45:34.947 INFO:tasks.workunit.client.1.smithi026.stdout:3/994: creat d4/dc/d73/d83/d7c/dd7/d62/d4f/de6/dff/d104/f15e x:0 0 0 2024-03-16T10:45:34.951 INFO:tasks.workunit.client.1.smithi026.stdout:9/951: symlink d0/da/d4d/dc5/l14a 0 2024-03-16T10:45:34.951 INFO:tasks.workunit.client.1.smithi026.stdout:9/952: stat d0/da/d16/c23 0 2024-03-16T10:45:34.953 INFO:tasks.workunit.client.1.smithi026.stdout:8/794: chown d5/d7/d2c/d36/d59/l6d 15 1 2024-03-16T10:45:34.954 INFO:tasks.workunit.client.1.smithi026.stdout:3/995: creat d4/dc/d73/d83/d7c/dd7/f15f x:0 0 0 2024-03-16T10:45:34.956 INFO:tasks.workunit.client.1.smithi026.stdout:9/953: creat d0/da/d16/d5d/d72/f14b x:0 0 0 2024-03-16T10:45:34.957 INFO:tasks.workunit.client.1.smithi026.stdout:6/714: mknod d0/d28/d49/d7d/df2/cf4 0 2024-03-16T10:45:34.957 INFO:tasks.workunit.client.1.smithi026.stdout:9/954: dread - d0/da/d16/d5d/d117/f12c zero size 2024-03-16T10:45:34.961 INFO:tasks.workunit.client.1.smithi026.stdout:3/996: rename d4/f9d to d4/dc/d73/d83/d7c/f160 0 2024-03-16T10:45:34.962 INFO:tasks.workunit.client.1.smithi026.stdout:6/715: symlink d0/d28/d49/lf5 0 2024-03-16T10:45:34.967 INFO:tasks.workunit.client.1.smithi026.stdout:3/997: mkdir d4/dc/d31/d48/d69/d78/db3/d161 0 2024-03-16T10:45:34.968 INFO:tasks.workunit.client.1.smithi026.stdout:6/716: dwrite d0/d41/fd8 [0,4194304] 0 2024-03-16T10:45:34.969 INFO:tasks.workunit.client.1.smithi026.stdout:3/998: readlink d4/dc/d73/d83/d7c/dd7/d62/d4f/d5a/da6/db9/l111 0 2024-03-16T10:45:34.975 INFO:tasks.workunit.client.1.smithi026.stdout:9/955: rename d0/d63/d6b/f86 to d0/da/d13/f14c 0 2024-03-16T10:45:34.976 INFO:tasks.workunit.client.1.smithi026.stdout:6/717: creat d0/d10/d79/d91/ff6 x:0 0 0 2024-03-16T10:45:34.979 INFO:tasks.workunit.client.1.smithi026.stdout:3/999: rmdir d4/dc/d31/d48/d69 39 2024-03-16T10:45:34.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:34 smithi026 ceph-mon[37041]: pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 61 MiB/s rd, 138 MiB/s wr, 356 op/s 2024-03-16T10:45:34.986 INFO:tasks.workunit.client.1.smithi026.stdout:8/795: dwrite d5/d23/f67 [0,4194304] 0 2024-03-16T10:45:34.999 INFO:tasks.workunit.client.1.smithi026.stdout:6/718: creat d0/d41/dc5/ff7 x:0 0 0 2024-03-16T10:45:35.000 INFO:tasks.workunit.client.1.smithi026.stdout:8/796: creat d5/d28/ff0 x:0 0 0 2024-03-16T10:45:35.000 INFO:tasks.workunit.client.1.smithi026.stdout:6/719: chown d0/d10/fad 1 1 2024-03-16T10:45:35.002 INFO:tasks.workunit.client.1.smithi026.stdout:9/956: rename d0/d63/c6f to d0/da/d13/c14d 0 2024-03-16T10:45:35.010 INFO:tasks.workunit.client.1.smithi026.stdout:9/957: rmdir d0/da/d14/da7/d126 39 2024-03-16T10:45:35.013 INFO:tasks.workunit.client.1.smithi026.stdout:8/797: dwrite d5/d7/d2c/d36/d59/d93/db0/fca [0,4194304] 0 2024-03-16T10:45:35.019 INFO:tasks.workunit.client.1.smithi026.stdout:6/720: dread d0/d10/d79/d7b/f82 [0,4194304] 0 2024-03-16T10:45:35.019 INFO:tasks.workunit.client.1.smithi026.stdout:8/798: truncate d5/d28/f39 612828 0 2024-03-16T10:45:35.020 INFO:tasks.workunit.client.1.smithi026.stdout:8/799: chown d5/d7/d7b 1186711 1 2024-03-16T10:45:35.027 INFO:tasks.workunit.client.1.smithi026.stdout:6/721: chown d0/d10/d79/d8e/ce3 67688222 1 2024-03-16T10:45:35.028 INFO:tasks.workunit.client.1.smithi026.stdout:6/722: stat d0/d8a/lcd 0 2024-03-16T10:45:35.028 INFO:tasks.workunit.client.1.smithi026.stdout:9/958: link d0/l20 d0/da/d14/da7/d126/dd5/l14e 0 2024-03-16T10:45:35.032 INFO:tasks.workunit.client.1.smithi026.stdout:8/800: rename d5/da0/fe8 to d5/dc/ff1 0 2024-03-16T10:45:35.035 INFO:tasks.workunit.client.1.smithi026.stdout:8/801: truncate d5/d7/d2c/d36/d59/d93/dde/fe4 5161640 0 2024-03-16T10:45:35.044 INFO:tasks.workunit.client.1.smithi026.stdout:6/723: dwrite d0/f89 [4194304,4194304] 0 2024-03-16T10:45:35.046 INFO:tasks.workunit.client.1.smithi026.stdout:8/802: unlink d5/d23/fac 0 2024-03-16T10:45:35.046 INFO:tasks.workunit.client.1.smithi026.stdout:9/959: write d0/da/d13/f26 [602658,89933] 0 2024-03-16T10:45:35.046 INFO:tasks.workunit.client.1.smithi026.stdout:8/803: chown d5/d5d/d9a/ddd/f20 218 1 2024-03-16T10:45:35.046 INFO:tasks.workunit.client.1.smithi026.stdout:8/804: chown d5/d7/d2c/d36/d59/d93 4 1 2024-03-16T10:45:35.047 INFO:tasks.workunit.client.1.smithi026.stdout:2/951: sync 2024-03-16T10:45:35.047 INFO:tasks.workunit.client.1.smithi026.stdout:4/927: sync 2024-03-16T10:45:35.047 INFO:tasks.workunit.client.1.smithi026.stdout:0/970: sync 2024-03-16T10:45:35.047 INFO:tasks.workunit.client.1.smithi026.stdout:7/983: sync 2024-03-16T10:45:35.048 INFO:tasks.workunit.client.1.smithi026.stdout:6/724: rename d0/d8a/lcd to d0/d41/d71/lf8 0 2024-03-16T10:45:35.050 INFO:tasks.workunit.client.1.smithi026.stdout:7/984: read d6/d35/d3a/d4e/db0/f107 [586979,92569] 0 2024-03-16T10:45:35.052 INFO:tasks.workunit.client.1.smithi026.stdout:6/725: chown d0/d5b/laf 730953968 1 2024-03-16T10:45:35.052 INFO:tasks.workunit.client.1.smithi026.stdout:8/805: unlink d5/d28/c65 0 2024-03-16T10:45:35.052 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:34 smithi002 ceph-mon[29248]: pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 61 MiB/s rd, 138 MiB/s wr, 356 op/s 2024-03-16T10:45:35.054 INFO:tasks.workunit.client.1.smithi026.stdout:2/952: dwrite d2/d29/d85/dc6/f118 [0,4194304] 0 2024-03-16T10:45:35.057 INFO:tasks.workunit.client.1.smithi026.stdout:7/985: mkdir d6/d1e/dbc/dc8/d142 0 2024-03-16T10:45:35.059 INFO:tasks.workunit.client.1.smithi026.stdout:8/806: creat d5/d7/d7b/db2/dbb/ff2 x:0 0 0 2024-03-16T10:45:35.061 INFO:tasks.workunit.client.1.smithi026.stdout:8/807: chown d5/dc/d89/f8a 5 1 2024-03-16T10:45:35.061 INFO:tasks.workunit.client.1.smithi026.stdout:6/726: fdatasync d0/d41/d4f/f87 0 2024-03-16T10:45:35.062 INFO:tasks.workunit.client.1.smithi026.stdout:8/808: fsync d5/da0/feb 0 2024-03-16T10:45:35.069 INFO:tasks.workunit.client.1.smithi026.stdout:7/986: dread d6/f15 [0,4194304] 0 2024-03-16T10:45:35.069 INFO:tasks.workunit.client.1.smithi026.stdout:7/987: fdatasync d6/d2f/ffc 0 2024-03-16T10:45:35.070 INFO:tasks.workunit.client.1.smithi026.stdout:6/727: mknod d0/d10/d79/dca/cf9 0 2024-03-16T10:45:35.073 INFO:tasks.workunit.client.1.smithi026.stdout:4/928: dread d1/db/ddb/daa/f5f [4194304,4194304] 0 2024-03-16T10:45:35.073 INFO:tasks.workunit.client.1.smithi026.stdout:9/960: rename d0/d63/le8 to d0/da/l14f 0 2024-03-16T10:45:35.075 INFO:tasks.workunit.client.1.smithi026.stdout:9/961: truncate d0/d5f/d7f/ff9 519787 0 2024-03-16T10:45:35.076 INFO:tasks.workunit.client.1.smithi026.stdout:7/988: fdatasync d6/d35/d3a/d4e/dcc/d3f/d70/fa3 0 2024-03-16T10:45:35.076 INFO:tasks.workunit.client.1.smithi026.stdout:4/929: readlink d1/db/de/d6e/lff 0 2024-03-16T10:45:35.079 INFO:tasks.workunit.client.1.smithi026.stdout:7/989: write d6/d35/d3a/d4e/fbb [3092988,113811] 0 2024-03-16T10:45:35.082 INFO:tasks.workunit.client.1.smithi026.stdout:9/962: rename d0/da/d16/d5d/d117/f12c to d0/da/d16/d17/db1/f150 0 2024-03-16T10:45:35.083 INFO:tasks.workunit.client.1.smithi026.stdout:7/990: creat d6/d35/d3a/d4e/dcc/d3f/dc3/dcf/f143 x:0 0 0 2024-03-16T10:45:35.085 INFO:tasks.workunit.client.1.smithi026.stdout:6/728: dread d0/d28/f37 [0,4194304] 0 2024-03-16T10:45:35.085 INFO:tasks.workunit.client.1.smithi026.stdout:9/963: read d0/f4 [2647882,11505] 0 2024-03-16T10:45:35.085 INFO:tasks.workunit.client.1.smithi026.stdout:4/930: dwrite d1/db/de/f2b [0,4194304] 0 2024-03-16T10:45:35.087 INFO:tasks.workunit.client.1.smithi026.stdout:9/964: dread - d0/da/d16/d5d/fd4 zero size 2024-03-16T10:45:35.088 INFO:tasks.workunit.client.1.smithi026.stdout:7/991: mknod d6/d35/d3a/d4e/dcc/d3f/d54/de1/c144 0 2024-03-16T10:45:35.089 INFO:tasks.workunit.client.1.smithi026.stdout:6/729: creat d0/d10/d79/d91/ffa x:0 0 0 2024-03-16T10:45:35.091 INFO:tasks.workunit.client.1.smithi026.stdout:7/992: read d6/d1e/f108 [3474810,35927] 0 2024-03-16T10:45:35.092 INFO:tasks.workunit.client.1.smithi026.stdout:6/730: dread d0/d41/d4f/f87 [0,4194304] 0 2024-03-16T10:45:35.094 INFO:tasks.workunit.client.1.smithi026.stdout:9/965: mkdir d0/da/d16/d17/db1/d151 0 2024-03-16T10:45:35.096 INFO:tasks.workunit.client.1.smithi026.stdout:6/731: read d0/f44 [3777945,119361] 0 2024-03-16T10:45:35.099 INFO:tasks.workunit.client.1.smithi026.stdout:6/732: chown d0/d41/d4f/f96 6007 1 2024-03-16T10:45:35.100 INFO:tasks.workunit.client.1.smithi026.stdout:6/733: dread - d0/d28/d49/fcf zero size 2024-03-16T10:45:35.101 INFO:tasks.workunit.client.1.smithi026.stdout:7/993: dwrite d6/d35/d3a/d4e/dcc/d73/dbf/feb [0,4194304] 0 2024-03-16T10:45:35.102 INFO:tasks.workunit.client.1.smithi026.stdout:7/994: chown d6/d35/d3a/d59/d9e/f113 97691823 1 2024-03-16T10:45:35.103 INFO:tasks.workunit.client.1.smithi026.stdout:4/931: link d1/l35 d1/d85/da2/l11b 0 2024-03-16T10:45:35.106 INFO:tasks.workunit.client.1.smithi026.stdout:9/966: rename d0/da/d16/d17/fc7 to d0/da/d16/d17/db1/dfa/d112/f152 0 2024-03-16T10:45:35.107 INFO:tasks.workunit.client.1.smithi026.stdout:6/734: fsync d0/f2c 0 2024-03-16T10:45:35.111 INFO:tasks.workunit.client.1.smithi026.stdout:7/995: rename d6/l69 to d6/d2f/dad/d13f/l145 0 2024-03-16T10:45:35.117 INFO:tasks.workunit.client.1.smithi026.stdout:4/932: rename d1/db/ddb/daa/d61/f114 to d1/db/ddb/daa/f11c 0 2024-03-16T10:45:35.118 INFO:tasks.workunit.client.1.smithi026.stdout:9/967: symlink d0/d63/d6b/ded/d13d/l153 0 2024-03-16T10:45:35.119 INFO:tasks.workunit.client.1.smithi026.stdout:4/933: write d1/db/ddb/daa/d44/fe4 [93181,81250] 0 2024-03-16T10:45:35.120 INFO:tasks.workunit.client.1.smithi026.stdout:9/968: write d0/d5f/fb6 [1006534,51795] 0 2024-03-16T10:45:35.121 INFO:tasks.workunit.client.1.smithi026.stdout:6/735: symlink d0/d10/d79/lfb 0 2024-03-16T10:45:35.125 INFO:tasks.workunit.client.1.smithi026.stdout:9/969: fsync d0/da/d14/f34 0 2024-03-16T10:45:35.126 INFO:tasks.workunit.client.1.smithi026.stdout:6/736: dread - d0/d41/d4f/f99 zero size 2024-03-16T10:45:35.127 INFO:tasks.workunit.client.1.smithi026.stdout:2/953: truncate d2/d29/d17/d1a/d2b/d2e/f31 163875 0 2024-03-16T10:45:35.134 INFO:tasks.workunit.client.1.smithi026.stdout:4/934: dwrite d1/db/ddb/daa/d61/f7c [4194304,4194304] 0 2024-03-16T10:45:35.159 INFO:tasks.workunit.client.1.smithi026.stdout:7/996: write d6/d35/d3a/d4e/dcc/d3f/f4b [2477762,129008] 0 2024-03-16T10:45:35.166 INFO:tasks.workunit.client.1.smithi026.stdout:7/997: dwrite d6/d35/d3a/d4e/dcc/d3f/d70/f12a [0,4194304] 0 2024-03-16T10:45:35.201 INFO:tasks.workunit.client.1.smithi026.stdout:6/737: mkdir d0/dfc 0 2024-03-16T10:45:35.201 INFO:tasks.workunit.client.1.smithi026.stdout:6/738: dread - d0/d10/fad zero size 2024-03-16T10:45:35.201 INFO:tasks.workunit.client.1.smithi026.stdout:4/935: mknod d1/db/de/c11d 0 2024-03-16T10:45:35.203 INFO:tasks.workunit.client.1.smithi026.stdout:6/739: write d0/fbd [145307,104103] 0 2024-03-16T10:45:35.203 INFO:tasks.workunit.client.1.smithi026.stdout:2/954: truncate d2/d29/d51/d72/fea 4617039 0 2024-03-16T10:45:35.213 INFO:tasks.workunit.client.1.smithi026.stdout:9/970: dread d0/da/d16/d17/f2e [0,4194304] 0 2024-03-16T10:45:35.216 INFO:tasks.workunit.client.1.smithi026.stdout:6/740: mknod d0/dfc/cfd 0 2024-03-16T10:45:35.216 INFO:tasks.workunit.client.1.smithi026.stdout:6/741: stat d0/d41/d71/fd0 0 2024-03-16T10:45:35.216 INFO:tasks.workunit.client.1.smithi026.stdout:2/955: creat d2/d29/d85/dc6/d144/f154 x:0 0 0 2024-03-16T10:45:35.224 INFO:tasks.workunit.client.1.smithi026.stdout:9/971: rename d0/da/d16/d17/d44/fdf to d0/da/d13/f154 0 2024-03-16T10:45:35.224 INFO:tasks.workunit.client.1.smithi026.stdout:2/956: symlink d2/d29/d17/d1a/d104/d105/l155 0 2024-03-16T10:45:35.228 INFO:tasks.workunit.client.1.smithi026.stdout:9/972: rename d0/da/d14/f34 to d0/d63/d6b/f155 0 2024-03-16T10:45:35.229 INFO:tasks.workunit.client.1.smithi026.stdout:9/973: dread - d0/da/d16/d5d/d72/d62/fe7 zero size 2024-03-16T10:45:35.233 INFO:tasks.workunit.client.1.smithi026.stdout:2/957: link d2/d29/d17/d1a/d2b/d2e/d60/d7f/d131/d6d/fb9 d2/d29/d17/d38/d64/dcb/f156 0 2024-03-16T10:45:35.238 INFO:tasks.workunit.client.1.smithi026.stdout:2/958: rename d2/d29/d85/dc6/dd5/f114 to d2/d29/d51/d72/f157 0 2024-03-16T10:45:35.240 INFO:tasks.workunit.client.1.smithi026.stdout:9/974: dread d0/d63/d6b/fc2 [0,4194304] 0 2024-03-16T10:45:35.241 INFO:tasks.workunit.client.1.smithi026.stdout:2/959: creat d2/d29/d17/d1a/d2b/d2e/d60/f158 x:0 0 0 2024-03-16T10:45:35.243 INFO:tasks.workunit.client.1.smithi026.stdout:1/790: sync 2024-03-16T10:45:35.244 INFO:tasks.workunit.client.1.smithi026.stdout:9/975: fdatasync d0/d63/d6b/ded/f10d 0 2024-03-16T10:45:35.251 INFO:tasks.workunit.client.1.smithi026.stdout:9/976: dwrite d0/d5f/d7f/ff9 [0,4194304] 0 2024-03-16T10:45:35.251 INFO:tasks.workunit.client.1.smithi026.stdout:2/960: creat d2/d29/d17/d38/d64/dcb/d138/f159 x:0 0 0 2024-03-16T10:45:35.253 INFO:tasks.workunit.client.1.smithi026.stdout:2/961: stat d2/d29/d17/d1a/d2b/d69/fdb 0 2024-03-16T10:45:35.254 INFO:tasks.workunit.client.1.smithi026.stdout:9/977: write d0/da/d13/f26 [1580610,2573] 0 2024-03-16T10:45:35.256 INFO:tasks.workunit.client.1.smithi026.stdout:9/978: chown d0/da/f12 4538 1 2024-03-16T10:45:35.256 INFO:tasks.workunit.client.1.smithi026.stdout:9/979: fsync d0/d5f/fb6 0 2024-03-16T10:45:35.258 INFO:tasks.workunit.client.1.smithi026.stdout:1/791: truncate d0/d17/d1f/d63/d6a/ffc 1152937 0 2024-03-16T10:45:35.260 INFO:tasks.workunit.client.1.smithi026.stdout:7/998: write d6/d35/f52 [1429755,4790] 0 2024-03-16T10:45:35.263 INFO:tasks.workunit.client.1.smithi026.stdout:6/742: write d0/d10/f1a [3644706,60406] 0 2024-03-16T10:45:35.265 INFO:tasks.workunit.client.1.smithi026.stdout:4/936: dwrite d1/db/de/d4c/fa1 [0,4194304] 0 2024-03-16T10:45:35.267 INFO:tasks.workunit.client.1.smithi026.stdout:9/980: symlink d0/dd7/dff/l156 0 2024-03-16T10:45:35.267 INFO:tasks.workunit.client.1.smithi026.stdout:7/999: dread d6/de/d16/d88/f93 [0,4194304] 0 2024-03-16T10:45:35.268 INFO:tasks.workunit.client.1.smithi026.stdout:4/937: fsync d1/db/de/d4c/d6f/df3/f107 0 2024-03-16T10:45:35.268 INFO:tasks.workunit.client.1.smithi026.stdout:1/792: symlink d0/d17/d1f/d20/dcc/l102 0 2024-03-16T10:45:35.275 INFO:tasks.workunit.client.1.smithi026.stdout:2/962: mkdir d2/d29/d17/d1a/d15a 0 2024-03-16T10:45:35.275 INFO:tasks.workunit.client.1.smithi026.stdout:4/938: dwrite d1/db/de/f2b [4194304,4194304] 0 2024-03-16T10:45:35.281 INFO:tasks.workunit.client.1.smithi026.stdout:9/981: dread - d0/da/d4d/f106 zero size 2024-03-16T10:45:35.282 INFO:tasks.workunit.client.1.smithi026.stdout:9/982: write d0/dd7/fd1 [1047992,17618] 0 2024-03-16T10:45:35.288 INFO:tasks.workunit.client.1.smithi026.stdout:2/963: stat d2/d29/d51/d72/fea 0 2024-03-16T10:45:35.289 INFO:tasks.workunit.client.1.smithi026.stdout:4/939: dread d1/db/de/d4c/f4d [0,4194304] 0 2024-03-16T10:45:35.290 INFO:tasks.workunit.client.1.smithi026.stdout:2/964: write d2/d29/d36/f123 [5163365,115288] 0 2024-03-16T10:45:35.293 INFO:tasks.workunit.client.1.smithi026.stdout:2/965: chown d2/d29/d17/d1a/d2b/d69/fdf 552 1 2024-03-16T10:45:35.298 INFO:tasks.workunit.client.1.smithi026.stdout:4/940: dwrite d1/db/de/f3d [0,4194304] 0 2024-03-16T10:45:35.303 INFO:tasks.workunit.client.1.smithi026.stdout:9/983: creat d0/da/d16/d17/d44/f157 x:0 0 0 2024-03-16T10:45:35.303 INFO:tasks.workunit.client.1.smithi026.stdout:6/743: write d0/d5b/f25 [6875749,96406] 0 2024-03-16T10:45:35.304 INFO:tasks.workunit.client.1.smithi026.stdout:9/984: readlink d0/d5f/d7f/l124 0 2024-03-16T10:45:35.310 INFO:tasks.workunit.client.1.smithi026.stdout:2/966: mknod d2/d29/d17/de8/c15b 0 2024-03-16T10:45:35.311 INFO:tasks.workunit.client.1.smithi026.stdout:2/967: dread - d2/d29/d3b/dbb/dfb/f145 zero size 2024-03-16T10:45:35.312 INFO:tasks.workunit.client.1.smithi026.stdout:2/968: chown d2/d29/d17/d1a/d15a 14343410 1 2024-03-16T10:45:35.312 INFO:tasks.workunit.client.1.smithi026.stdout:4/941: mkdir d1/d85/da2/da8/d11e 0 2024-03-16T10:45:35.313 INFO:tasks.workunit.client.1.smithi026.stdout:4/942: readlink d1/db/de/d4c/l69 0 2024-03-16T10:45:35.317 INFO:tasks.workunit.client.1.smithi026.stdout:6/744: rename d0/d41/d4f/c56 to d0/d8a/cfe 0 2024-03-16T10:45:35.319 INFO:tasks.workunit.client.1.smithi026.stdout:2/969: creat d2/d29/d3b/f15c x:0 0 0 2024-03-16T10:45:35.320 INFO:tasks.workunit.client.1.smithi026.stdout:1/793: getdents d0/d8b 0 2024-03-16T10:45:35.321 INFO:tasks.workunit.client.1.smithi026.stdout:2/970: chown d2/d29/d17/d1a/d2b/d69/d151/fee 306734 1 2024-03-16T10:45:35.322 INFO:tasks.workunit.client.1.smithi026.stdout:9/985: mknod d0/da/d4d/d135/c158 0 2024-03-16T10:45:35.326 INFO:tasks.workunit.client.1.smithi026.stdout:4/943: unlink d1/db/ddb/daa/d44/d48/l7b 0 2024-03-16T10:45:35.328 INFO:tasks.workunit.client.1.smithi026.stdout:4/944: readlink d1/db/ddb/lbf 0 2024-03-16T10:45:35.328 INFO:tasks.workunit.client.1.smithi026.stdout:6/745: fdatasync d0/d5b/f64 0 2024-03-16T10:45:35.328 INFO:tasks.workunit.client.1.smithi026.stdout:9/986: dwrite d0/da/d14/f6a [0,4194304] 0 2024-03-16T10:45:35.331 INFO:tasks.workunit.client.1.smithi026.stdout:9/987: readlink d0/da/d4d/dc5/l14a 0 2024-03-16T10:45:35.331 INFO:tasks.workunit.client.1.smithi026.stdout:2/971: symlink d2/d29/d152/l15d 0 2024-03-16T10:45:35.332 INFO:tasks.workunit.client.1.smithi026.stdout:9/988: chown d0/da/d4d/f106 486462084 1 2024-03-16T10:45:35.342 INFO:tasks.workunit.client.1.smithi026.stdout:4/945: readlink d1/db/ddb/daa/d44/d48/l101 0 2024-03-16T10:45:35.342 INFO:tasks.workunit.client.1.smithi026.stdout:1/794: dread d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/f89 [0,4194304] 0 2024-03-16T10:45:35.351 INFO:tasks.workunit.client.1.smithi026.stdout:6/746: write d0/d28/fce [802110,45546] 0 2024-03-16T10:45:35.351 INFO:tasks.workunit.client.1.smithi026.stdout:9/989: dread d0/da/f1b [4194304,4194304] 0 2024-03-16T10:45:35.352 INFO:tasks.workunit.client.1.smithi026.stdout:1/795: creat d0/d17/db7/f103 x:0 0 0 2024-03-16T10:45:35.354 INFO:tasks.workunit.client.1.smithi026.stdout:1/796: dread d0/d17/d1f/f90 [0,4194304] 0 2024-03-16T10:45:35.355 INFO:tasks.workunit.client.1.smithi026.stdout:2/972: rename d2/d29/d17/d1a/d2b/d69/c87 to d2/d29/d152/c15e 0 2024-03-16T10:45:35.356 INFO:tasks.workunit.client.1.smithi026.stdout:4/946: fdatasync d1/db/de/d4c/d6b/d7a/ffa 0 2024-03-16T10:45:35.361 INFO:tasks.workunit.client.1.smithi026.stdout:2/973: creat d2/d29/d17/d1a/d104/d105/f15f x:0 0 0 2024-03-16T10:45:35.362 INFO:tasks.workunit.client.1.smithi026.stdout:2/974: readlink d2/d29/d17/d1a/l12f 0 2024-03-16T10:45:35.367 INFO:tasks.workunit.client.1.smithi026.stdout:4/947: symlink d1/db/ddb/daa/d61/l11f 0 2024-03-16T10:45:35.368 INFO:tasks.workunit.client.1.smithi026.stdout:1/797: write d0/fab [1361842,91699] 0 2024-03-16T10:45:35.372 INFO:tasks.workunit.client.1.smithi026.stdout:9/990: creat d0/da/d13/f159 x:0 0 0 2024-03-16T10:45:35.372 INFO:tasks.workunit.client.1.smithi026.stdout:6/747: dwrite d0/d28/f5d [0,4194304] 0 2024-03-16T10:45:35.379 INFO:tasks.workunit.client.1.smithi026.stdout:1/798: fdatasync d0/fb3 0 2024-03-16T10:45:35.380 INFO:tasks.workunit.client.1.smithi026.stdout:6/748: mknod d0/d10/d79/d8e/cff 0 2024-03-16T10:45:35.382 INFO:tasks.workunit.client.1.smithi026.stdout:1/799: write d0/d17/d1f/d63/d6a/d7c/d2e/d54/f5e [1634594,130380] 0 2024-03-16T10:45:35.382 INFO:tasks.workunit.client.1.smithi026.stdout:9/991: truncate d0/d5f/f108 119868 0 2024-03-16T10:45:35.383 INFO:tasks.workunit.client.1.smithi026.stdout:2/975: rmdir d2/d29/d85/dc6/dd5/d137 0 2024-03-16T10:45:35.384 INFO:tasks.workunit.client.1.smithi026.stdout:4/948: getdents d1/db/ddb/ddc 0 2024-03-16T10:45:35.385 INFO:tasks.workunit.client.1.smithi026.stdout:2/976: chown d2/d29/d85/l88 826221 1 2024-03-16T10:45:35.387 INFO:tasks.workunit.client.1.smithi026.stdout:8/809: sync 2024-03-16T10:45:35.387 INFO:tasks.workunit.client.1.smithi026.stdout:5/868: sync 2024-03-16T10:45:35.388 INFO:tasks.workunit.client.1.smithi026.stdout:0/971: sync 2024-03-16T10:45:35.393 INFO:tasks.workunit.client.1.smithi026.stdout:5/869: fdatasync d1/d1c/d2a/dab/d11d/f132 0 2024-03-16T10:45:35.393 INFO:tasks.workunit.client.1.smithi026.stdout:9/992: fsync d0/f66 0 2024-03-16T10:45:35.394 INFO:tasks.workunit.client.1.smithi026.stdout:1/800: chown d0/fd 163490033 1 2024-03-16T10:45:35.396 INFO:tasks.workunit.client.1.smithi026.stdout:5/870: chown d1/f5 2378 1 2024-03-16T10:45:35.397 INFO:tasks.workunit.client.1.smithi026.stdout:0/972: chown d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/fd3 1740 1 2024-03-16T10:45:35.398 INFO:tasks.workunit.client.1.smithi026.stdout:2/977: dwrite d2/d29/d3b/f9b [0,4194304] 0 2024-03-16T10:45:35.398 INFO:tasks.workunit.client.1.smithi026.stdout:4/949: dwrite d1/db/de/d4c/f116 [0,4194304] 0 2024-03-16T10:45:35.403 INFO:tasks.workunit.client.1.smithi026.stdout:6/749: write d0/f5f [1653483,58027] 0 2024-03-16T10:45:35.405 INFO:tasks.workunit.client.1.smithi026.stdout:5/871: chown d1/d1c/d2a/dab/dc5/d69/l4f 16917994 1 2024-03-16T10:45:35.406 INFO:tasks.workunit.client.1.smithi026.stdout:5/872: write d1/f5 [2258985,36858] 0 2024-03-16T10:45:35.407 INFO:tasks.workunit.client.1.smithi026.stdout:6/750: chown d0/d41/dc5/fec 14882 1 2024-03-16T10:45:35.413 INFO:tasks.workunit.client.1.smithi026.stdout:4/950: write d1/db/de/d4c/d59/d6a/f9b [5043703,64882] 0 2024-03-16T10:45:35.417 INFO:tasks.workunit.client.1.smithi026.stdout:5/873: unlink d1/db/d1f/f26 0 2024-03-16T10:45:35.418 INFO:tasks.workunit.client.1.smithi026.stdout:8/810: link d5/dc/f1e d5/dc/d89/ff3 0 2024-03-16T10:45:35.419 INFO:tasks.workunit.client.1.smithi026.stdout:6/751: creat d0/d5b/f100 x:0 0 0 2024-03-16T10:45:35.421 INFO:tasks.workunit.client.1.smithi026.stdout:2/978: dread d2/d29/d17/d38/f59 [0,4194304] 0 2024-03-16T10:45:35.422 INFO:tasks.workunit.client.1.smithi026.stdout:9/993: truncate d0/da/d13/f69 1896337 0 2024-03-16T10:45:35.425 INFO:tasks.workunit.client.1.smithi026.stdout:4/951: read d1/f8 [3161508,5690] 0 2024-03-16T10:45:35.426 INFO:tasks.workunit.client.1.smithi026.stdout:8/811: creat d5/d23/ff4 x:0 0 0 2024-03-16T10:45:35.427 INFO:tasks.workunit.client.1.smithi026.stdout:8/812: fsync d5/d23/f67 0 2024-03-16T10:45:35.428 INFO:tasks.workunit.client.1.smithi026.stdout:5/874: creat d1/db/d1f/d61/d116/f134 x:0 0 0 2024-03-16T10:45:35.430 INFO:tasks.workunit.client.1.smithi026.stdout:1/801: creat d0/d17/d1f/d63/d6a/d7c/d2e/f104 x:0 0 0 2024-03-16T10:45:35.436 INFO:tasks.workunit.client.1.smithi026.stdout:9/994: fsync d0/da/d4d/f106 0 2024-03-16T10:45:35.438 INFO:tasks.workunit.client.1.smithi026.stdout:4/952: chown d1/db/de/d6e/c87 81304 1 2024-03-16T10:45:35.439 INFO:tasks.workunit.client.1.smithi026.stdout:4/953: readlink d1/db/de/d4c/d6b/d7a/db1/ldd 0 2024-03-16T10:45:35.440 INFO:tasks.workunit.client.1.smithi026.stdout:8/813: symlink d5/d5d/lf5 0 2024-03-16T10:45:35.442 INFO:tasks.workunit.client.1.smithi026.stdout:5/875: creat d1/d1c/d2a/d55/db1/f135 x:0 0 0 2024-03-16T10:45:35.442 INFO:tasks.workunit.client.1.smithi026.stdout:6/752: write d0/d10/d79/d91/f94 [4791986,85680] 0 2024-03-16T10:45:35.444 INFO:tasks.workunit.client.1.smithi026.stdout:1/802: creat d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/f105 x:0 0 0 2024-03-16T10:45:35.447 INFO:tasks.workunit.client.1.smithi026.stdout:2/979: rename d2/d29/d17/fba to d2/d29/d17/d38/d64/dcb/d138/f160 0 2024-03-16T10:45:35.449 INFO:tasks.workunit.client.1.smithi026.stdout:0/973: getdents d1/d12/d2f/d48/d55/d60/d9b/d51/df6/d133 0 2024-03-16T10:45:35.450 INFO:tasks.workunit.client.1.smithi026.stdout:4/954: symlink d1/db/l120 0 2024-03-16T10:45:35.451 INFO:tasks.workunit.client.1.smithi026.stdout:8/814: fsync d5/d7/f8d 0 2024-03-16T10:45:35.453 INFO:tasks.workunit.client.1.smithi026.stdout:5/876: creat d1/d1c/d4c/f136 x:0 0 0 2024-03-16T10:45:35.457 INFO:tasks.workunit.client.1.smithi026.stdout:6/753: rename d0/d28/d49/fcf to d0/d10/dd2/f101 0 2024-03-16T10:45:35.460 INFO:tasks.workunit.client.1.smithi026.stdout:5/877: dwrite d1/db/d1f/d61/fa7 [4194304,4194304] 0 2024-03-16T10:45:35.462 INFO:tasks.workunit.client.1.smithi026.stdout:5/878: chown d1/db/d1f/d61/d110 12 1 2024-03-16T10:45:35.462 INFO:tasks.workunit.client.1.smithi026.stdout:0/974: dread - d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/fd3 zero size 2024-03-16T10:45:35.466 INFO:tasks.workunit.client.1.smithi026.stdout:8/815: mknod d5/d7/d2c/d36/d59/d93/cf6 0 2024-03-16T10:45:35.468 INFO:tasks.workunit.client.1.smithi026.stdout:4/955: dread d1/db/ddb/daf/fb9 [0,4194304] 0 2024-03-16T10:45:35.470 INFO:tasks.workunit.client.1.smithi026.stdout:6/754: rename d0/d10/d79 to d0/d28/d49/d7d/d102 0 2024-03-16T10:45:35.471 INFO:tasks.workunit.client.1.smithi026.stdout:9/995: creat d0/da/f15a x:0 0 0 2024-03-16T10:45:35.472 INFO:tasks.workunit.client.1.smithi026.stdout:0/975: mknod d1/c148 0 2024-03-16T10:45:35.474 INFO:tasks.workunit.client.1.smithi026.stdout:5/879: symlink d1/d1c/d2a/d55/db1/l137 0 2024-03-16T10:45:35.478 INFO:tasks.workunit.client.1.smithi026.stdout:1/803: creat d0/d17/d1f/d63/f106 x:0 0 0 2024-03-16T10:45:35.480 INFO:tasks.workunit.client.1.smithi026.stdout:4/956: dread - d1/dcd/fd1 zero size 2024-03-16T10:45:35.607 INFO:tasks.workunit.client.1.smithi026.stdout:6/755: truncate d0/d41/d4f/f87 264257 0 2024-03-16T10:45:35.607 INFO:tasks.workunit.client.1.smithi026.stdout:0/976: unlink d1/d12/d2f/d48/d55/d60/d9b/d57/fbb 0 2024-03-16T10:45:35.608 INFO:tasks.workunit.client.1.smithi026.stdout:1/804: rmdir d0/d17/d1f/d20 39 2024-03-16T10:45:35.610 INFO:tasks.workunit.client.1.smithi026.stdout:9/996: mknod d0/da/d16/d17/db1/d151/c15b 0 2024-03-16T10:45:35.610 INFO:tasks.workunit.client.1.smithi026.stdout:0/977: write d1/d12/d2f/f77 [1249849,7191] 0 2024-03-16T10:45:35.611 INFO:tasks.workunit.client.1.smithi026.stdout:6/756: chown d0/d41/d4f/f69 47 1 2024-03-16T10:45:35.615 INFO:tasks.workunit.client.1.smithi026.stdout:4/957: mkdir d1/db/de/d4c/d59/d121 0 2024-03-16T10:45:35.616 INFO:tasks.workunit.client.1.smithi026.stdout:5/880: dwrite d1/d1c/d2a/dab/de8/f12c [0,4194304] 0 2024-03-16T10:45:35.625 INFO:tasks.workunit.client.1.smithi026.stdout:6/757: mknod d0/d41/dbe/c103 0 2024-03-16T10:45:35.626 INFO:tasks.workunit.client.1.smithi026.stdout:8/816: getdents d5/d7/d2c/d36/d59 0 2024-03-16T10:45:35.626 INFO:tasks.workunit.client.1.smithi026.stdout:8/817: stat d5/d7/d7b/d96/ca5 0 2024-03-16T10:45:35.630 INFO:tasks.workunit.client.1.smithi026.stdout:5/881: unlink d1/d1c/d4c/d4d/cc0 0 2024-03-16T10:45:35.633 INFO:tasks.workunit.client.1.smithi026.stdout:4/958: read d1/db/de/d4c/d59/d6a/f82 [398255,93187] 0 2024-03-16T10:45:35.634 INFO:tasks.workunit.client.1.smithi026.stdout:9/997: dwrite d0/da/d4d/dc5/f11f [0,4194304] 0 2024-03-16T10:45:35.644 INFO:tasks.workunit.client.1.smithi026.stdout:4/959: dwrite d1/db/ddb/daa/d44/dd5/de9/f117 [0,4194304] 0 2024-03-16T10:45:35.647 INFO:tasks.workunit.client.1.smithi026.stdout:9/998: truncate d0/da/f75 1261154 0 2024-03-16T10:45:35.652 INFO:tasks.workunit.client.1.smithi026.stdout:0/978: link d1/d12/d2f/d48/d55/d60/f19 d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/df1/f149 0 2024-03-16T10:45:35.660 INFO:tasks.workunit.client.1.smithi026.stdout:8/818: dwrite d5/d5d/d9a/fcf [0,4194304] 0 2024-03-16T10:45:35.662 INFO:tasks.workunit.client.1.smithi026.stdout:8/819: chown d5/dc/d40/d53/f63 6032607 1 2024-03-16T10:45:35.670 INFO:tasks.workunit.client.1.smithi026.stdout:1/805: dwrite d0/d17/d1f/d63/d6a/d7c/d6b/f87 [0,4194304] 0 2024-03-16T10:45:35.672 INFO:tasks.workunit.client.1.smithi026.stdout:1/806: fsync d0/d17/d1f/d63/d6a/d7c/d2e/f104 0 2024-03-16T10:45:35.673 INFO:tasks.workunit.client.1.smithi026.stdout:9/999: symlink d0/da/d114/l15c 0 2024-03-16T10:45:35.677 INFO:tasks.workunit.client.1.smithi026.stdout:5/882: creat d1/d1c/d2a/dab/f138 x:0 0 0 2024-03-16T10:45:35.680 INFO:tasks.workunit.client.1.smithi026.stdout:0/979: rename d1/d12/d2f/daa/dd1/la4 to d1/d12/d2f/daa/dd1/d61/d126/d12b/l14a 0 2024-03-16T10:45:35.683 INFO:tasks.workunit.client.1.smithi026.stdout:6/758: getdents d0/d28/d49/d7d 0 2024-03-16T10:45:35.683 INFO:tasks.workunit.client.1.smithi026.stdout:6/759: stat d0/d5b/f95 0 2024-03-16T10:45:35.684 INFO:tasks.workunit.client.1.smithi026.stdout:6/760: chown d0/d10/f27 1530045 1 2024-03-16T10:45:35.685 INFO:tasks.workunit.client.1.smithi026.stdout:6/761: chown d0/d5b/f30 5681 1 2024-03-16T10:45:35.685 INFO:tasks.workunit.client.1.smithi026.stdout:8/820: mkdir d5/d5d/d9a/df7 0 2024-03-16T10:45:35.691 INFO:tasks.workunit.client.1.smithi026.stdout:8/821: dwrite d5/d7/de6/d76/fcb [0,4194304] 0 2024-03-16T10:45:35.694 INFO:tasks.workunit.client.1.smithi026.stdout:1/807: truncate d0/d17/d1f/d63/d6a/d7c/d84/d9d/f4e 388171 0 2024-03-16T10:45:35.698 INFO:tasks.workunit.client.1.smithi026.stdout:4/960: dwrite d1/db/de/d4c/f8a [0,4194304] 0 2024-03-16T10:45:35.709 INFO:tasks.workunit.client.1.smithi026.stdout:8/822: creat d5/d7/d2c/d36/d59/d93/dde/de0/ff8 x:0 0 0 2024-03-16T10:45:35.714 INFO:tasks.workunit.client.1.smithi026.stdout:6/762: symlink d0/d28/d49/d7d/d102/dca/dd7/l104 0 2024-03-16T10:45:35.715 INFO:tasks.workunit.client.1.smithi026.stdout:4/961: rmdir d1/db/ddb/daa/d44/dd5 39 2024-03-16T10:45:35.716 INFO:tasks.workunit.client.1.smithi026.stdout:8/823: dwrite d5/d7/f11 [4194304,4194304] 0 2024-03-16T10:45:35.719 INFO:tasks.workunit.client.1.smithi026.stdout:0/980: link d1/d12/d2f/d48/d55/d60/f106 d1/d12/d2f/d48/d55/d60/d9b/d57/d11c/f14b 0 2024-03-16T10:45:35.722 INFO:tasks.workunit.client.1.smithi026.stdout:6/763: creat d0/d41/d4f/f105 x:0 0 0 2024-03-16T10:45:35.723 INFO:tasks.workunit.client.1.smithi026.stdout:2/980: sync 2024-03-16T10:45:35.724 INFO:tasks.workunit.client.1.smithi026.stdout:1/808: mknod d0/d17/d1f/d20/c107 0 2024-03-16T10:45:35.726 INFO:tasks.workunit.client.1.smithi026.stdout:4/962: creat d1/db/de/d4c/d59/d6a/df1/f122 x:0 0 0 2024-03-16T10:45:35.726 INFO:tasks.workunit.client.1.smithi026.stdout:8/824: dwrite d5/d7/d7b/d96/fdb [0,4194304] 0 2024-03-16T10:45:35.727 INFO:tasks.workunit.client.1.smithi026.stdout:2/981: chown d2/d29/d85/dc6/dd5/f112 25 1 2024-03-16T10:45:35.729 INFO:tasks.workunit.client.1.smithi026.stdout:6/764: rmdir d0/d28/d49/d7d/d102 39 2024-03-16T10:45:35.730 INFO:tasks.workunit.client.1.smithi026.stdout:8/825: chown d5/d7/d7b/f94 25266 1 2024-03-16T10:45:35.735 INFO:tasks.workunit.client.1.smithi026.stdout:0/981: read d1/d12/d2f/d48/d55/f7c [3175076,56368] 0 2024-03-16T10:45:35.740 INFO:tasks.workunit.client.1.smithi026.stdout:6/765: truncate d0/d28/d49/d7d/d102/d7b/f82 4792692 0 2024-03-16T10:45:35.741 INFO:tasks.workunit.client.1.smithi026.stdout:1/809: dread d0/d17/d1f/d20/f43 [0,4194304] 0 2024-03-16T10:45:35.742 INFO:tasks.workunit.client.1.smithi026.stdout:2/982: rename d2/d29/d17/d38/d64/dcb to d2/d29/d51/d161 0 2024-03-16T10:45:35.747 INFO:tasks.workunit.client.1.smithi026.stdout:8/826: mkdir d5/d7/d2c/d36/d59/d93/dde/de0/df9 0 2024-03-16T10:45:35.761 INFO:tasks.workunit.client.1.smithi026.stdout:5/883: truncate d1/db/d1f/d61/d116/f60 1452564 0 2024-03-16T10:45:35.761 INFO:tasks.workunit.client.1.smithi026.stdout:6/766: creat d0/d28/d2d/f106 x:0 0 0 2024-03-16T10:45:35.767 INFO:tasks.workunit.client.1.smithi026.stdout:0/982: getdents d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d89/d13a 0 2024-03-16T10:45:35.770 INFO:tasks.workunit.client.1.smithi026.stdout:8/827: rename d5/d7/d2c/d36/d59/d93/db0/lec to d5/d23/d4e/d60/dc8/dc4/lfa 0 2024-03-16T10:45:35.775 INFO:tasks.workunit.client.1.smithi026.stdout:6/767: rmdir d0/d5b 39 2024-03-16T10:45:35.782 INFO:tasks.workunit.client.1.smithi026.stdout:4/963: getdents d1/db 0 2024-03-16T10:45:35.786 INFO:tasks.workunit.client.1.smithi026.stdout:2/983: link d2/d29/d17/d1a/d2b/d2e/d60/fc2 d2/d29/d17/d1a/d104/f162 0 2024-03-16T10:45:35.790 INFO:tasks.workunit.client.1.smithi026.stdout:0/983: mkdir d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d14c 0 2024-03-16T10:45:35.790 INFO:tasks.workunit.client.1.smithi026.stdout:6/768: creat d0/d28/dcb/def/f107 x:0 0 0 2024-03-16T10:45:35.799 INFO:tasks.workunit.client.1.smithi026.stdout:6/769: mknod d0/d41/d71/c108 0 2024-03-16T10:45:35.800 INFO:tasks.workunit.client.1.smithi026.stdout:6/770: fsync d0/d28/d49/d7d/d102/d91/f94 0 2024-03-16T10:45:35.802 INFO:tasks.workunit.client.1.smithi026.stdout:4/964: creat d1/db/ddb/daa/d44/d48/f123 x:0 0 0 2024-03-16T10:45:35.803 INFO:tasks.workunit.client.1.smithi026.stdout:8/828: getdents d5/d7/d7b/d96 0 2024-03-16T10:45:35.806 INFO:tasks.workunit.client.1.smithi026.stdout:2/984: creat d2/d29/f163 x:0 0 0 2024-03-16T10:45:35.807 INFO:tasks.workunit.client.1.smithi026.stdout:2/985: readlink d2/d29/d17/d1a/l10c 0 2024-03-16T10:45:35.810 INFO:tasks.workunit.client.1.smithi026.stdout:1/810: write d0/d17/d1f/d63/fc3 [487434,21219] 0 2024-03-16T10:45:35.811 INFO:tasks.workunit.client.1.smithi026.stdout:6/771: symlink d0/d28/d49/db0/l109 0 2024-03-16T10:45:35.815 INFO:tasks.workunit.client.1.smithi026.stdout:5/884: write d1/d1c/d2a/dab/dc5/f52 [4512670,77321] 0 2024-03-16T10:45:35.818 INFO:tasks.workunit.client.1.smithi026.stdout:2/986: mknod d2/d29/d17/d1a/d2b/d4b/c164 0 2024-03-16T10:45:35.819 INFO:tasks.workunit.client.1.smithi026.stdout:4/965: dwrite d1/db/ddb/daa/d44/d48/f73 [4194304,4194304] 0 2024-03-16T10:45:35.821 INFO:tasks.workunit.client.1.smithi026.stdout:2/987: write d2/d29/d36/f123 [1447685,24661] 0 2024-03-16T10:45:35.826 INFO:tasks.workunit.client.1.smithi026.stdout:1/811: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/c108 0 2024-03-16T10:45:35.828 INFO:tasks.workunit.client.1.smithi026.stdout:2/988: dwrite d2/d29/d85/dc6/dd5/f126 [0,4194304] 0 2024-03-16T10:45:35.830 INFO:tasks.workunit.client.1.smithi026.stdout:8/829: unlink d5/d23/d4e/d60/dc8/dc4/lfa 0 2024-03-16T10:45:35.834 INFO:tasks.workunit.client.1.smithi026.stdout:8/830: stat d5/d7/d2c/d36/d59/l64 0 2024-03-16T10:45:35.836 INFO:tasks.workunit.client.1.smithi026.stdout:0/984: dwrite d1/d12/d2f/daa/dd1/fb3 [0,4194304] 0 2024-03-16T10:45:35.838 INFO:tasks.workunit.client.1.smithi026.stdout:0/985: chown d1/d12/d2f/d48/d55/d60/d9b/cc5 0 1 2024-03-16T10:45:35.844 INFO:tasks.workunit.client.1.smithi026.stdout:2/989: mknod d2/d29/d17/d1a/d2b/d4b/c165 0 2024-03-16T10:45:35.848 INFO:tasks.workunit.client.1.smithi026.stdout:8/831: chown d5/dc/f1e 32 1 2024-03-16T10:45:35.851 INFO:tasks.workunit.client.1.smithi026.stdout:6/772: dwrite d0/d10/f6c [0,4194304] 0 2024-03-16T10:45:35.852 INFO:tasks.workunit.client.1.smithi026.stdout:0/986: mknod d1/d12/d2f/d48/d55/d60/d9b/d57/d11c/c14d 0 2024-03-16T10:45:35.856 INFO:tasks.workunit.client.1.smithi026.stdout:0/987: readlink d1/d12/d2f/d48/dfe/l12c 0 2024-03-16T10:45:35.856 INFO:tasks.workunit.client.1.smithi026.stdout:4/966: write d1/db/ddb/daa/d44/f5a [2311498,4823] 0 2024-03-16T10:45:35.859 INFO:tasks.workunit.client.1.smithi026.stdout:5/885: write d1/d1c/d2a/ff1 [512599,107991] 0 2024-03-16T10:45:35.862 INFO:tasks.workunit.client.1.smithi026.stdout:8/832: mknod d5/d7/d2c/d36/d59/d93/dde/de0/cfb 0 2024-03-16T10:45:35.862 INFO:tasks.workunit.client.1.smithi026.stdout:1/812: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/fd4 [849455,45657] 0 2024-03-16T10:45:35.865 INFO:tasks.workunit.client.1.smithi026.stdout:1/813: chown d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/fcb 215451213 1 2024-03-16T10:45:35.865 INFO:tasks.workunit.client.1.smithi026.stdout:8/833: chown d5/d7/d2c/d36/d59/d93/dde/fe4 9 1 2024-03-16T10:45:35.866 INFO:tasks.workunit.client.1.smithi026.stdout:6/773: unlink d0/d28/d49/d7d/d102/d91/ff6 0 2024-03-16T10:45:35.869 INFO:tasks.workunit.client.1.smithi026.stdout:2/990: getdents d2/d29/d17/d1a/d2b/d4e/d67/def 0 2024-03-16T10:45:35.870 INFO:tasks.workunit.client.1.smithi026.stdout:4/967: symlink d1/db/de/d4c/d59/d6a/df1/l124 0 2024-03-16T10:45:35.870 INFO:tasks.workunit.client.1.smithi026.stdout:2/991: read d2/d29/f3a [2198536,3618] 0 2024-03-16T10:45:35.872 INFO:tasks.workunit.client.1.smithi026.stdout:5/886: write d1/d1c/d4c/d4d/f128 [147365,112178] 0 2024-03-16T10:45:35.873 INFO:tasks.workunit.client.1.smithi026.stdout:1/814: fdatasync d0/d17/f48 0 2024-03-16T10:45:35.874 INFO:tasks.workunit.client.1.smithi026.stdout:1/815: dread - d0/d17/d1f/d63/f101 zero size 2024-03-16T10:45:35.874 INFO:tasks.workunit.client.1.smithi026.stdout:5/887: chown d1/db/d1f/d61/d65/d82/dbb/dea/d126 6995130 1 2024-03-16T10:45:35.875 INFO:tasks.workunit.client.1.smithi026.stdout:1/816: chown d0/d17/db7/f103 102772146 1 2024-03-16T10:45:35.877 INFO:tasks.workunit.client.1.smithi026.stdout:8/834: read d5/d7/d7b/fee [1876938,24485] 0 2024-03-16T10:45:35.879 INFO:tasks.workunit.client.1.smithi026.stdout:6/774: truncate d0/d5b/f100 690466 0 2024-03-16T10:45:35.879 INFO:tasks.workunit.client.1.smithi026.stdout:1/817: truncate d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/fdf 410121 0 2024-03-16T10:45:35.880 INFO:tasks.workunit.client.1.smithi026.stdout:8/835: truncate d5/d7/d2c/d36/fef 939443 0 2024-03-16T10:45:35.881 INFO:tasks.workunit.client.1.smithi026.stdout:1/818: readlink d0/d17/d1f/d63/d6a/d7c/lfb 0 2024-03-16T10:45:35.883 INFO:tasks.workunit.client.1.smithi026.stdout:6/775: dread d0/d10/f6c [0,4194304] 0 2024-03-16T10:45:35.893 INFO:tasks.workunit.client.1.smithi026.stdout:0/988: dwrite d1/d12/d2f/dba/fc7 [0,4194304] 0 2024-03-16T10:45:35.896 INFO:tasks.workunit.client.1.smithi026.stdout:0/989: readlink d1/d12/d2f/daa/dd1/d1a/d2b/l30 0 2024-03-16T10:45:35.896 INFO:tasks.workunit.client.1.smithi026.stdout:5/888: write d1/d1c/d2a/dab/dc5/d69/d51/d80/de9/f115 [7342737,8214] 0 2024-03-16T10:45:35.899 INFO:tasks.workunit.client.1.smithi026.stdout:2/992: dwrite d2/d29/d36/d98/ffa [0,4194304] 0 2024-03-16T10:45:35.902 INFO:tasks.workunit.client.1.smithi026.stdout:2/993: read d2/d29/d17/d1a/f9e [1474731,11570] 0 2024-03-16T10:45:35.904 INFO:tasks.workunit.client.1.smithi026.stdout:6/776: mknod d0/d41/c10a 0 2024-03-16T10:45:35.907 INFO:tasks.workunit.client.1.smithi026.stdout:2/994: dread d2/d29/d36/da7/f102 [0,4194304] 0 2024-03-16T10:45:35.909 INFO:tasks.workunit.client.1.smithi026.stdout:4/968: symlink d1/db/ddb/l125 0 2024-03-16T10:45:35.917 INFO:tasks.workunit.client.1.smithi026.stdout:5/889: symlink d1/d1c/d2a/d55/d99/l139 0 2024-03-16T10:45:35.919 INFO:tasks.workunit.client.1.smithi026.stdout:6/777: dread d0/d28/d2d/f4c [0,4194304] 0 2024-03-16T10:45:35.924 INFO:tasks.workunit.client.1.smithi026.stdout:4/969: fsync d1/db/de/d4c/d59/d6a/f82 0 2024-03-16T10:45:35.945 INFO:tasks.workunit.client.1.smithi026.stdout:5/890: dwrite d1/d1c/d2a/dab/dc5/f52 [4194304,4194304] 0 2024-03-16T10:45:35.946 INFO:tasks.workunit.client.1.smithi026.stdout:2/995: mkdir d2/d29/d85/d140/d166 0 2024-03-16T10:45:35.947 INFO:tasks.workunit.client.1.smithi026.stdout:1/819: getdents d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6 0 2024-03-16T10:45:35.950 INFO:tasks.workunit.client.1.smithi026.stdout:5/891: mkdir d1/d1c/d4c/d13a 0 2024-03-16T10:45:35.952 INFO:tasks.workunit.client.1.smithi026.stdout:8/836: dwrite d5/d28/fdf [4194304,4194304] 0 2024-03-16T10:45:35.956 INFO:tasks.workunit.client.1.smithi026.stdout:8/837: rmdir d5/d7/d2c/d36/d8c/dd3 39 2024-03-16T10:45:35.958 INFO:tasks.workunit.client.1.smithi026.stdout:1/820: truncate d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/fdc 1035318 0 2024-03-16T10:45:35.960 INFO:tasks.workunit.client.1.smithi026.stdout:6/778: dread d0/f2c [0,4194304] 0 2024-03-16T10:45:35.965 INFO:tasks.workunit.client.1.smithi026.stdout:8/838: rename d5/d7/d7b/db2/dbb/ff2 to d5/d5d/d9a/dd6/ffc 0 2024-03-16T10:45:35.967 INFO:tasks.workunit.client.1.smithi026.stdout:6/779: unlink d0/d28/d49/lf5 0 2024-03-16T10:45:35.972 INFO:tasks.workunit.client.1.smithi026.stdout:4/970: write d1/db/de/d4c/d6b/d89/d9c/fa6 [341042,54241] 0 2024-03-16T10:45:35.976 INFO:tasks.workunit.client.1.smithi026.stdout:6/780: symlink d0/d28/d49/d7d/d102/dca/l10b 0 2024-03-16T10:45:35.977 INFO:tasks.workunit.client.1.smithi026.stdout:5/892: truncate d1/d1c/d2a/dab/dc5/d69/d51/f9b 222134 0 2024-03-16T10:45:35.980 INFO:tasks.workunit.client.1.smithi026.stdout:1/821: link d0/d17/d1f/d20/f43 d0/d17/d1f/d63/d58/f109 0 2024-03-16T10:45:35.982 INFO:tasks.workunit.client.1.smithi026.stdout:6/781: fdatasync d0/d10/dd2/fe7 0 2024-03-16T10:45:35.983 INFO:tasks.workunit.client.1.smithi026.stdout:8/839: dwrite d5/d23/d4e/d60/dc8/f91 [0,4194304] 0 2024-03-16T10:45:35.985 INFO:tasks.workunit.client.1.smithi026.stdout:6/782: write d0/d28/d2d/f68 [103097,29421] 0 2024-03-16T10:45:35.993 INFO:tasks.workunit.client.1.smithi026.stdout:4/971: write d1/db/de/f3c [1785112,65997] 0 2024-03-16T10:45:35.998 INFO:tasks.workunit.client.1.smithi026.stdout:6/783: dread d0/d28/d49/d7d/d102/fb4 [0,4194304] 0 2024-03-16T10:45:36.001 INFO:tasks.workunit.client.1.smithi026.stdout:1/822: mkdir d0/d17/d1f/d20/d10a 0 2024-03-16T10:45:36.008 INFO:tasks.workunit.client.1.smithi026.stdout:8/840: dread d5/d7/f1a [0,4194304] 0 2024-03-16T10:45:36.010 INFO:tasks.workunit.client.1.smithi026.stdout:6/784: creat d0/d41/dc5/f10c x:0 0 0 2024-03-16T10:45:36.010 INFO:tasks.workunit.client.1.smithi026.stdout:6/785: write d0/d10/fad [151879,85285] 0 2024-03-16T10:45:36.012 INFO:tasks.workunit.client.1.smithi026.stdout:1/823: mknod d0/d17/d1f/d63/d6a/d7c/d2e/d54/c10b 0 2024-03-16T10:45:36.014 INFO:tasks.workunit.client.1.smithi026.stdout:4/972: mkdir d1/db/de/d4c/d59/d121/d126 0 2024-03-16T10:45:36.015 INFO:tasks.workunit.client.1.smithi026.stdout:5/893: link d1/db/d1f/d61/d65/f7c d1/db/d1f/d61/d65/d82/dbb/dd2/f13b 0 2024-03-16T10:45:36.016 INFO:tasks.workunit.client.1.smithi026.stdout:8/841: dread - d5/d5d/d9a/fb8 zero size 2024-03-16T10:45:36.017 INFO:tasks.workunit.client.1.smithi026.stdout:8/842: chown d5/d7/f6b 107451017 1 2024-03-16T10:45:36.018 INFO:tasks.workunit.client.1.smithi026.stdout:6/786: truncate d0/d41/d4f/f77 8778561 0 2024-03-16T10:45:36.018 INFO:tasks.workunit.client.1.smithi026.stdout:6/787: dread - d0/d28/dcb/fda zero size 2024-03-16T10:45:36.019 INFO:tasks.workunit.client.1.smithi026.stdout:1/824: mknod d0/d17/d1f/d63/d6a/d7c/d84/d9d/c10c 0 2024-03-16T10:45:36.021 INFO:tasks.workunit.client.1.smithi026.stdout:4/973: creat d1/d85/da2/dca/f127 x:0 0 0 2024-03-16T10:45:36.023 INFO:tasks.workunit.client.1.smithi026.stdout:5/894: creat d1/db/d1f/d61/d65/df6/f13c x:0 0 0 2024-03-16T10:45:36.026 INFO:tasks.workunit.client.1.smithi026.stdout:8/843: creat d5/d7/de6/d76/ffd x:0 0 0 2024-03-16T10:45:36.026 INFO:tasks.workunit.client.1.smithi026.stdout:6/788: mknod d0/d28/d49/dc2/c10d 0 2024-03-16T10:45:36.030 INFO:tasks.workunit.client.1.smithi026.stdout:5/895: dwrite d1/d1c/d2a/dab/dc5/d69/f119 [0,4194304] 0 2024-03-16T10:45:36.031 INFO:tasks.workunit.client.1.smithi026.stdout:5/896: chown d1/d1c/d2a/dab/dc5/d69/d51/d129 44634 1 2024-03-16T10:45:36.032 INFO:tasks.workunit.client.1.smithi026.stdout:5/897: fdatasync d1/d1c/d4c/fe3 0 2024-03-16T10:45:36.035 INFO:tasks.workunit.client.1.smithi026.stdout:1/825: dread d0/d17/d1f/d63/d6a/d7c/d84/fa7 [0,4194304] 0 2024-03-16T10:45:36.038 INFO:tasks.workunit.client.1.smithi026.stdout:6/789: truncate d0/d28/d49/d7d/fbc 4315876 0 2024-03-16T10:45:36.102 INFO:tasks.workunit.client.1.smithi026.stdout:4/974: write d1/d85/da2/dca/fe5 [866000,16694] 0 2024-03-16T10:45:36.103 INFO:tasks.workunit.client.1.smithi026.stdout:1/826: mknod d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dff/c10d 0 2024-03-16T10:45:36.105 INFO:tasks.workunit.client.1.smithi026.stdout:8/844: dwrite d5/d28/f31 [4194304,4194304] 0 2024-03-16T10:45:36.105 INFO:tasks.workunit.client.1.smithi026.stdout:6/790: link d0/d41/dc5/fec d0/d28/d49/db0/f10e 0 2024-03-16T10:45:36.112 INFO:tasks.workunit.client.1.smithi026.stdout:1/827: write d0/d17/d1f/fc9 [482829,29419] 0 2024-03-16T10:45:36.115 INFO:tasks.workunit.client.1.smithi026.stdout:1/828: fsync d0/d17/d1f/d63/d6a/d7c/d6b/f7a 0 2024-03-16T10:45:36.119 INFO:tasks.workunit.client.1.smithi026.stdout:5/898: dwrite d1/d1c/d2a/f89 [0,4194304] 0 2024-03-16T10:45:36.128 INFO:tasks.workunit.client.1.smithi026.stdout:5/899: dread d1/d1c/d2a/d8e/f11b [0,4194304] 0 2024-03-16T10:45:36.129 INFO:tasks.workunit.client.1.smithi026.stdout:1/829: mknod d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/c10e 0 2024-03-16T10:45:36.131 INFO:tasks.workunit.client.1.smithi026.stdout:6/791: link d0/d10/fd d0/d28/d49/d7d/d102/d91/f10f 0 2024-03-16T10:45:36.131 INFO:tasks.workunit.client.1.smithi026.stdout:5/900: chown d1/db/d1f/d61/d12f 3555712 1 2024-03-16T10:45:36.141 INFO:tasks.workunit.client.1.smithi026.stdout:5/901: creat d1/d1c/d4c/f13d x:0 0 0 2024-03-16T10:45:36.141 INFO:tasks.workunit.client.1.smithi026.stdout:8/845: dwrite d5/d23/f82 [0,4194304] 0 2024-03-16T10:45:36.143 INFO:tasks.workunit.client.1.smithi026.stdout:5/902: fsync d1/d1c/d2a/d55/db1/f135 0 2024-03-16T10:45:36.198 INFO:tasks.workunit.client.1.smithi026.stdout:1/830: rename d0/d17/d1f/d63/d6a/d7c/d84/fc5 to d0/d17/db7/f10f 0 2024-03-16T10:45:36.201 INFO:tasks.workunit.client.1.smithi026.stdout:1/831: mknod d0/d17/d1f/d20/d65/dde/c110 0 2024-03-16T10:45:36.202 INFO:tasks.workunit.client.1.smithi026.stdout:8/846: mkdir d5/dc/d40/dfe 0 2024-03-16T10:45:36.205 INFO:tasks.workunit.client.1.smithi026.stdout:6/792: link d0/d28/d49/d7d/d102/fa2 d0/d41/dc5/f110 0 2024-03-16T10:45:36.206 INFO:tasks.workunit.client.1.smithi026.stdout:1/832: mknod d0/d17/d1f/d20/dcc/c111 0 2024-03-16T10:45:36.235 INFO:tasks.workunit.client.1.smithi026.stdout:1/833: rmdir d0/d17/d1f/d63/dbc 39 2024-03-16T10:45:36.237 INFO:tasks.workunit.client.1.smithi026.stdout:8/847: fdatasync d5/d23/f61 0 2024-03-16T10:45:36.237 INFO:tasks.workunit.client.1.smithi026.stdout:1/834: dread d0/d17/d1f/d63/d6a/d7c/fba [0,4194304] 0 2024-03-16T10:45:36.241 INFO:tasks.workunit.client.1.smithi026.stdout:8/848: dread d5/d7/f11 [4194304,4194304] 0 2024-03-16T10:45:36.250 INFO:tasks.workunit.client.1.smithi026.stdout:1/835: dwrite d0/d17/d1f/d63/d6a/d7c/fba [0,4194304] 0 2024-03-16T10:45:36.250 INFO:tasks.workunit.client.1.smithi026.stdout:6/793: link d0/d10/fd d0/f111 0 2024-03-16T10:45:36.251 INFO:tasks.workunit.client.1.smithi026.stdout:6/794: read - d0/d41/fcc zero size 2024-03-16T10:45:36.254 INFO:tasks.workunit.client.1.smithi026.stdout:6/795: mkdir d0/d28/d49/db0/d112 0 2024-03-16T10:45:36.257 INFO:tasks.workunit.client.1.smithi026.stdout:1/836: creat d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/f112 x:0 0 0 2024-03-16T10:45:36.261 INFO:tasks.workunit.client.1.smithi026.stdout:6/796: dwrite d0/d10/fc3 [0,4194304] 0 2024-03-16T10:45:36.263 INFO:tasks.workunit.client.1.smithi026.stdout:6/797: write d0/d41/dc5/ff7 [622854,123317] 0 2024-03-16T10:45:36.263 INFO:tasks.workunit.client.1.smithi026.stdout:1/837: symlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/l113 0 2024-03-16T10:45:36.267 INFO:tasks.workunit.client.1.smithi026.stdout:1/838: write d0/d17/d1f/d20/d65/dde/fe4 [616766,19393] 0 2024-03-16T10:45:36.272 INFO:tasks.workunit.client.1.smithi026.stdout:6/798: dwrite d0/d10/ff [0,4194304] 0 2024-03-16T10:45:36.274 INFO:tasks.workunit.client.1.smithi026.stdout:1/839: rename d0/d17/db7/f103 to d0/d17/d1f/d63/d6a/d7c/d3b/d64/dc2/f114 0 2024-03-16T10:45:36.279 INFO:tasks.workunit.client.1.smithi026.stdout:1/840: mkdir d0/d17/d1f/d20/d65/d115 0 2024-03-16T10:45:36.287 INFO:tasks.workunit.client.1.smithi026.stdout:1/841: rename d0/d17/fb2 to d0/d17/d1f/deb/f116 0 2024-03-16T10:45:36.292 INFO:tasks.workunit.client.1.smithi026.stdout:5/903: dwrite d1/f18 [0,4194304] 0 2024-03-16T10:45:36.294 INFO:tasks.workunit.client.1.smithi026.stdout:8/849: write d5/d23/d4e/f66 [4836549,66110] 0 2024-03-16T10:45:36.295 INFO:tasks.workunit.client.1.smithi026.stdout:5/904: read d1/d1c/d4c/d4d/f128 [85851,43828] 0 2024-03-16T10:45:36.297 INFO:tasks.workunit.client.1.smithi026.stdout:8/850: write d5/d23/d4e/d60/dc8/f91 [3331426,91826] 0 2024-03-16T10:45:36.297 INFO:tasks.workunit.client.1.smithi026.stdout:5/905: write d1/f18 [2099767,122412] 0 2024-03-16T10:45:36.306 INFO:tasks.workunit.client.1.smithi026.stdout:5/906: dwrite d1/d1c/d4c/f12b [0,4194304] 0 2024-03-16T10:45:36.314 INFO:tasks.workunit.client.1.smithi026.stdout:5/907: creat d1/d1c/d2a/dab/de8/d11a/f13e x:0 0 0 2024-03-16T10:45:36.315 INFO:tasks.workunit.client.1.smithi026.stdout:1/842: dwrite d0/d17/d1f/f3f [4194304,4194304] 0 2024-03-16T10:45:36.320 INFO:tasks.workunit.client.1.smithi026.stdout:5/908: fsync d1/d1c/d2a/dab/dc5/d69/d51/f5e 0 2024-03-16T10:45:36.326 INFO:tasks.workunit.client.1.smithi026.stdout:8/851: dwrite d5/d23/d4e/f7d [0,4194304] 0 2024-03-16T10:45:36.331 INFO:tasks.workunit.client.1.smithi026.stdout:8/852: chown d5/d7/c13 11469538 1 2024-03-16T10:45:36.336 INFO:tasks.workunit.client.1.smithi026.stdout:1/843: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/f78 [770344,58325] 0 2024-03-16T10:45:36.342 INFO:tasks.workunit.client.1.smithi026.stdout:1/844: mkdir d0/d17/d1f/d63/d6a/d7c/d2e/d117 0 2024-03-16T10:45:36.344 INFO:tasks.workunit.client.1.smithi026.stdout:1/845: read d0/d17/d1f/d63/d6a/ffc [980278,92666] 0 2024-03-16T10:45:36.347 INFO:tasks.workunit.client.1.smithi026.stdout:1/846: rmdir d0/d17/d1f/d63/d58 39 2024-03-16T10:45:36.348 INFO:tasks.workunit.client.1.smithi026.stdout:8/853: write d5/d23/d4e/d60/f84 [3416002,70876] 0 2024-03-16T10:45:36.351 INFO:tasks.workunit.client.1.smithi026.stdout:8/854: symlink d5/d5d/lff 0 2024-03-16T10:45:36.356 INFO:tasks.workunit.client.1.smithi026.stdout:8/855: fsync d5/d7/f4f 0 2024-03-16T10:45:36.362 INFO:tasks.workunit.client.1.smithi026.stdout:1/847: getdents d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dff 0 2024-03-16T10:45:36.362 INFO:tasks.workunit.client.1.smithi026.stdout:8/856: dwrite d5/d7/d7b/d96/fdb [0,4194304] 0 2024-03-16T10:45:36.367 INFO:tasks.workunit.client.1.smithi026.stdout:8/857: creat d5/d7/d2c/d36/d59/d93/db0/f100 x:0 0 0 2024-03-16T10:45:36.367 INFO:tasks.workunit.client.1.smithi026.stdout:8/858: stat d5/d23/d4e/f7d 0 2024-03-16T10:45:36.369 INFO:tasks.workunit.client.1.smithi026.stdout:8/859: read d5/d23/f82 [2110564,104061] 0 2024-03-16T10:45:36.372 INFO:tasks.workunit.client.1.smithi026.stdout:1/848: dwrite d0/d17/d1f/d20/d75/fcd [0,4194304] 0 2024-03-16T10:45:36.382 INFO:tasks.workunit.client.1.smithi026.stdout:8/860: getdents d5/d5d 0 2024-03-16T10:45:36.389 INFO:tasks.workunit.client.1.smithi026.stdout:1/849: write d0/d17/d1f/d63/d58/f109 [5461819,7479] 0 2024-03-16T10:45:36.389 INFO:tasks.workunit.client.1.smithi026.stdout:8/861: getdents d5/d5d/d9a/dd6 0 2024-03-16T10:45:36.392 INFO:tasks.workunit.client.1.smithi026.stdout:1/850: symlink d0/d17/l118 0 2024-03-16T10:45:36.393 INFO:tasks.workunit.client.1.smithi026.stdout:8/862: fdatasync d5/d5d/d9a/ddd/fbd 0 2024-03-16T10:45:36.395 INFO:tasks.workunit.client.1.smithi026.stdout:1/851: creat d0/d8b/f119 x:0 0 0 2024-03-16T10:45:36.402 INFO:tasks.workunit.client.1.smithi026.stdout:1/852: creat d0/d8b/f11a x:0 0 0 2024-03-16T10:45:36.406 INFO:tasks.workunit.client.1.smithi026.stdout:1/853: mknod d0/d17/d1f/d20/d75/c11b 0 2024-03-16T10:45:36.410 INFO:tasks.workunit.client.1.smithi026.stdout:1/854: mknod d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/c11c 0 2024-03-16T10:45:36.411 INFO:tasks.workunit.client.1.smithi026.stdout:8/863: write d5/d7/d2c/d36/d59/d93/da3/fa9 [1572861,95518] 0 2024-03-16T10:45:36.412 INFO:tasks.workunit.client.1.smithi026.stdout:1/855: creat d0/d8b/f11d x:0 0 0 2024-03-16T10:45:36.415 INFO:tasks.workunit.client.1.smithi026.stdout:1/856: unlink d0/d17/d1f/d20/c107 0 2024-03-16T10:45:36.417 INFO:tasks.workunit.client.1.smithi026.stdout:8/864: creat d5/d7/d2c/d36/f101 x:0 0 0 2024-03-16T10:45:36.420 INFO:tasks.workunit.client.1.smithi026.stdout:8/865: mkdir d5/d5d/d9a/ddd/d102 0 2024-03-16T10:45:36.425 INFO:tasks.workunit.client.1.smithi026.stdout:8/866: mkdir d5/d7/d2c/d36/d59/d93/da3/dea/d103 0 2024-03-16T10:45:36.427 INFO:tasks.workunit.client.1.smithi026.stdout:1/857: write d0/d17/d1f/d63/d58/f9a [5326186,13281] 0 2024-03-16T10:45:36.428 INFO:tasks.workunit.client.1.smithi026.stdout:8/867: creat d5/d7/d2c/d36/d59/f104 x:0 0 0 2024-03-16T10:45:36.429 INFO:tasks.workunit.client.1.smithi026.stdout:8/868: write d5/d5d/f9d [3145866,70832] 0 2024-03-16T10:45:36.430 INFO:tasks.workunit.client.1.smithi026.stdout:8/869: chown d5/d7/de6 64 1 2024-03-16T10:45:36.430 INFO:tasks.workunit.client.1.smithi026.stdout:8/870: fsync d5/d7/d2c/d36/f101 0 2024-03-16T10:45:36.436 INFO:tasks.workunit.client.1.smithi026.stdout:1/858: link d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/f105 d0/d17/d1f/d20/f11e 0 2024-03-16T10:45:36.438 INFO:tasks.workunit.client.1.smithi026.stdout:1/859: write d0/d17/d1f/d20/d75/fcd [2071047,104156] 0 2024-03-16T10:45:36.440 INFO:tasks.workunit.client.1.smithi026.stdout:1/860: read d0/d17/d1f/d63/d6a/d7c/d84/d9d/f6e [2962779,64151] 0 2024-03-16T10:45:36.443 INFO:tasks.workunit.client.1.smithi026.stdout:8/871: write d5/dc/d40/d53/fd4 [2355880,126496] 0 2024-03-16T10:45:36.449 INFO:tasks.workunit.client.1.smithi026.stdout:1/861: mknod d0/d17/d1f/d63/d6a/d7c/d84/c11f 0 2024-03-16T10:45:36.452 INFO:tasks.workunit.client.1.smithi026.stdout:1/862: dread - d0/d17/d1f/d63/dbc/ff8 zero size 2024-03-16T10:45:36.454 INFO:tasks.workunit.client.1.smithi026.stdout:1/863: dread - d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/f105 zero size 2024-03-16T10:45:36.459 INFO:tasks.workunit.client.1.smithi026.stdout:1/864: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/d100/d120 0 2024-03-16T10:45:36.460 INFO:tasks.workunit.client.1.smithi026.stdout:8/872: dwrite d5/d5d/d9a/ddd/f20 [0,4194304] 0 2024-03-16T10:45:36.468 INFO:tasks.workunit.client.1.smithi026.stdout:1/865: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/fd5 [4194304,4194304] 0 2024-03-16T10:45:36.470 INFO:tasks.workunit.client.1.smithi026.stdout:8/873: getdents d5/d7/d2c/d36/d59/d93/da3 0 2024-03-16T10:45:36.474 INFO:tasks.workunit.client.1.smithi026.stdout:1/866: write d0/d17/d1f/d20/dcc/fd9 [1876960,88732] 0 2024-03-16T10:45:36.474 INFO:tasks.workunit.client.1.smithi026.stdout:8/874: fdatasync d5/d23/d4e/f5b 0 2024-03-16T10:45:36.475 INFO:tasks.workunit.client.1.smithi026.stdout:2/996: sync 2024-03-16T10:45:36.476 INFO:tasks.workunit.client.1.smithi026.stdout:0/990: sync 2024-03-16T10:45:36.477 INFO:tasks.workunit.client.1.smithi026.stdout:8/875: truncate d5/d5d/d9a/f9b 5932805 0 2024-03-16T10:45:36.478 INFO:tasks.workunit.client.1.smithi026.stdout:1/867: unlink d0/d17/d1f/d63/dbc/fce 0 2024-03-16T10:45:36.479 INFO:tasks.workunit.client.1.smithi026.stdout:0/991: symlink d1/d12/l14e 0 2024-03-16T10:45:36.480 INFO:tasks.workunit.client.1.smithi026.stdout:1/868: stat d0/d17/d1f/d63/d6a/d7c/d84/d9d/c10c 0 2024-03-16T10:45:36.480 INFO:tasks.workunit.client.1.smithi026.stdout:8/876: creat d5/dc/d89/f105 x:0 0 0 2024-03-16T10:45:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:36 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:36 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:36 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:45:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:36 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:45:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:36 smithi026 ceph-mon[37041]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:45:36.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:36 smithi026 ceph-mon[37041]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:45:36.484 INFO:tasks.workunit.client.1.smithi026.stdout:0/992: unlink d1/d12/d2f/d48/d55/d60/f28 0 2024-03-16T10:45:36.487 INFO:tasks.workunit.client.1.smithi026.stdout:8/877: rmdir d5/d7/d2c/d36/d59 39 2024-03-16T10:45:36.490 INFO:tasks.workunit.client.1.smithi026.stdout:2/997: rename d2/d29/d17/d1a/d2b/d2e/c6e to d2/d29/d17/d1a/d2b/d69/d151/c167 0 2024-03-16T10:45:36.491 INFO:tasks.workunit.client.1.smithi026.stdout:2/998: write d2/d29/d110/f124 [426297,113428] 0 2024-03-16T10:45:36.492 INFO:tasks.workunit.client.1.smithi026.stdout:2/999: write d2/d29/d17/d1a/d2b/d4e/d6f/fcf [638603,73502] 0 2024-03-16T10:45:36.498 INFO:tasks.workunit.client.1.smithi026.stdout:1/869: write d0/d17/d1f/d63/d6a/d7c/f30 [93661,73562] 0 2024-03-16T10:45:36.501 INFO:tasks.workunit.client.1.smithi026.stdout:0/993: mknod d1/d12/d2f/d48/d55/d60/d9b/d51/d88/d14c/c14f 0 2024-03-16T10:45:36.504 INFO:tasks.workunit.client.1.smithi026.stdout:0/994: chown d1/d12/d2f/daa/dd1/d1a/d2b/c9f 1789402843 1 2024-03-16T10:45:36.504 INFO:tasks.workunit.client.1.smithi026.stdout:1/870: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dff/f121 x:0 0 0 2024-03-16T10:45:36.506 INFO:tasks.workunit.client.1.smithi026.stdout:8/878: dwrite d5/d7/f11 [8388608,4194304] 0 2024-03-16T10:45:36.509 INFO:tasks.workunit.client.1.smithi026.stdout:8/879: chown d5/d7/d7b/d96/ca5 4 1 2024-03-16T10:45:36.510 INFO:tasks.workunit.client.1.smithi026.stdout:0/995: creat d1/d12/d2f/daa/dd1/f150 x:0 0 0 2024-03-16T10:45:36.513 INFO:tasks.workunit.client.1.smithi026.stdout:1/871: dwrite d0/fab [0,4194304] 0 2024-03-16T10:45:36.513 INFO:tasks.workunit.client.1.smithi026.stdout:0/996: fdatasync d1/f40 0 2024-03-16T10:45:36.520 INFO:tasks.workunit.client.1.smithi026.stdout:1/872: read d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa/fb5 [591825,57777] 0 2024-03-16T10:45:36.524 INFO:tasks.workunit.client.1.smithi026.stdout:8/880: getdents d5/d7/d7b/db2/dbb 0 2024-03-16T10:45:36.525 INFO:tasks.workunit.client.1.smithi026.stdout:0/997: rmdir d1/d12/d2f/d48/d55/d60/d9b/d51/df6/d133/d13e 0 2024-03-16T10:45:36.529 INFO:tasks.workunit.client.1.smithi026.stdout:0/998: mkdir d1/d12/d2f/d48/d55/d60/d9b/d57/dd7/d10b/d151 0 2024-03-16T10:45:36.531 INFO:tasks.workunit.client.1.smithi026.stdout:0/999: fsync d1/d12/d2f/daa/dd1/d1a/f25 0 2024-03-16T10:45:36.533 INFO:tasks.workunit.client.1.smithi026.stdout:1/873: link d0/d17/d1f/d63/d6a/d7c/d2e/cf7 d0/d17/d1f/d63/d6a/c122 0 2024-03-16T10:45:36.535 INFO:tasks.workunit.client.1.smithi026.stdout:8/881: dread d5/d7/d7b/fee [0,4194304] 0 2024-03-16T10:45:36.545 INFO:tasks.workunit.client.1.smithi026.stdout:8/882: creat d5/d23/d4e/f106 x:0 0 0 2024-03-16T10:45:36.547 INFO:tasks.workunit.client.1.smithi026.stdout:1/874: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa/fb5 [0,4194304] 0 2024-03-16T10:45:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:36 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:36 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:36 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:45:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:36 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:45:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:36 smithi002 ceph-mon[29248]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:45:36.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:36 smithi002 ceph-mon[29248]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:45:36.549 INFO:tasks.workunit.client.1.smithi026.stdout:8/883: creat d5/d5d/d9a/f107 x:0 0 0 2024-03-16T10:45:36.554 INFO:tasks.workunit.client.1.smithi026.stdout:1/875: mkdir d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/d123 0 2024-03-16T10:45:36.555 INFO:tasks.workunit.client.1.smithi026.stdout:8/884: symlink d5/d23/d4e/d60/dc8/dc4/l108 0 2024-03-16T10:45:36.555 INFO:tasks.workunit.client.1.smithi026.stdout:8/885: readlink d5/d7/le5 0 2024-03-16T10:45:36.556 INFO:tasks.workunit.client.1.smithi026.stdout:8/886: stat d5/d23/c33 0 2024-03-16T10:45:36.556 INFO:tasks.workunit.client.1.smithi026.stdout:1/876: symlink d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/l124 0 2024-03-16T10:45:36.569 INFO:tasks.workunit.client.1.smithi026.stdout:1/877: write d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/f89 [3909428,127250] 0 2024-03-16T10:45:36.571 INFO:tasks.workunit.client.1.smithi026.stdout:8/887: truncate d5/d7/d2c/d36/d59/d93/db0/fca 3452844 0 2024-03-16T10:45:36.576 INFO:tasks.workunit.client.1.smithi026.stdout:1/878: dwrite d0/d17/d1f/d20/f43 [0,4194304] 0 2024-03-16T10:45:36.581 INFO:tasks.workunit.client.1.smithi026.stdout:6/799: sync 2024-03-16T10:45:36.581 INFO:tasks.workunit.client.1.smithi026.stdout:4/975: sync 2024-03-16T10:45:36.583 INFO:tasks.workunit.client.1.smithi026.stdout:1/879: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa/f125 x:0 0 0 2024-03-16T10:45:36.583 INFO:tasks.workunit.client.1.smithi026.stdout:8/888: link d5/d23/d4e/f5b d5/d7/d2c/f109 0 2024-03-16T10:45:36.586 INFO:tasks.workunit.client.1.smithi026.stdout:4/976: mknod d1/d85/da2/da8/df4/c128 0 2024-03-16T10:45:36.586 INFO:tasks.workunit.client.1.smithi026.stdout:1/880: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/f78 [221078,70053] 0 2024-03-16T10:45:36.590 INFO:tasks.workunit.client.1.smithi026.stdout:8/889: mkdir d5/d5d/d9a/ddd/d10a 0 2024-03-16T10:45:36.591 INFO:tasks.workunit.client.1.smithi026.stdout:1/881: dread d0/d17/d1f/f3f [4194304,4194304] 0 2024-03-16T10:45:36.592 INFO:tasks.workunit.client.1.smithi026.stdout:8/890: fsync d5/d7/d2c/d36/d59/d93/dde/de0/ff8 0 2024-03-16T10:45:36.593 INFO:tasks.workunit.client.1.smithi026.stdout:4/977: dread - d1/d85/fd6 zero size 2024-03-16T10:45:36.593 INFO:tasks.workunit.client.1.smithi026.stdout:8/891: chown d5/d7/d2c/d36/d59/d93/db0 776039035 1 2024-03-16T10:45:36.596 INFO:tasks.workunit.client.1.smithi026.stdout:1/882: fsync d0/d8b/fb6 0 2024-03-16T10:45:36.600 INFO:tasks.workunit.client.1.smithi026.stdout:4/978: dwrite d1/d85/da2/dca/f127 [0,4194304] 0 2024-03-16T10:45:36.609 INFO:tasks.workunit.client.1.smithi026.stdout:4/979: unlink d1/db/ddb/lbf 0 2024-03-16T10:45:36.611 INFO:tasks.workunit.client.1.smithi026.stdout:6/800: truncate d0/d28/d2d/f68 2790020 0 2024-03-16T10:45:36.619 INFO:tasks.workunit.client.1.smithi026.stdout:6/801: link d0/d28/lc6 d0/d28/d2d/l113 0 2024-03-16T10:45:36.620 INFO:tasks.workunit.client.1.smithi026.stdout:8/892: dwrite d5/dc/d40/d53/f58 [0,4194304] 0 2024-03-16T10:45:36.624 INFO:tasks.workunit.client.1.smithi026.stdout:8/893: symlink d5/da0/l10b 0 2024-03-16T10:45:36.624 INFO:tasks.workunit.client.1.smithi026.stdout:4/980: dread d1/db/f67 [8388608,4194304] 0 2024-03-16T10:45:36.625 INFO:tasks.workunit.client.1.smithi026.stdout:8/894: chown d5/d5d/lff 876358 1 2024-03-16T10:45:36.628 INFO:tasks.workunit.client.1.smithi026.stdout:6/802: rename d0/d28/d49/d7d/d102/d8e/ce3 to d0/d28/d49/d7d/d102/c114 0 2024-03-16T10:45:36.629 INFO:tasks.workunit.client.1.smithi026.stdout:1/883: dwrite d0/d17/d1f/d20/d65/f88 [0,4194304] 0 2024-03-16T10:45:36.630 INFO:tasks.workunit.client.1.smithi026.stdout:6/803: write d0/d10/fad [921474,126665] 0 2024-03-16T10:45:36.633 INFO:tasks.workunit.client.1.smithi026.stdout:8/895: creat d5/d23/d4e/d60/f10c x:0 0 0 2024-03-16T10:45:36.639 INFO:tasks.workunit.client.1.smithi026.stdout:6/804: creat d0/d28/d49/d7d/d102/dca/dde/f115 x:0 0 0 2024-03-16T10:45:36.641 INFO:tasks.workunit.client.1.smithi026.stdout:8/896: symlink d5/d7/d2c/d36/d59/l10d 0 2024-03-16T10:45:36.643 INFO:tasks.workunit.client.1.smithi026.stdout:4/981: creat d1/db/de/d4c/d6b/f129 x:0 0 0 2024-03-16T10:45:36.644 INFO:tasks.workunit.client.1.smithi026.stdout:4/982: chown d1/db/de/d4c/d59/d6a/f82 7 1 2024-03-16T10:45:36.645 INFO:tasks.workunit.client.1.smithi026.stdout:4/983: chown d1/db/de/d4c/d6b/f98 12965519 1 2024-03-16T10:45:36.650 INFO:tasks.workunit.client.1.smithi026.stdout:8/897: dread d5/d23/d4e/f5b [0,4194304] 0 2024-03-16T10:45:36.651 INFO:tasks.workunit.client.1.smithi026.stdout:8/898: chown d5/dc/d89/ff3 61 1 2024-03-16T10:45:36.655 INFO:tasks.workunit.client.1.smithi026.stdout:4/984: mkdir d1/db/de/d4c/d6b/d89/d105/d12a 0 2024-03-16T10:45:36.662 INFO:tasks.workunit.client.1.smithi026.stdout:1/884: link d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/fdd d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/f126 0 2024-03-16T10:45:36.664 INFO:tasks.workunit.client.1.smithi026.stdout:5/909: sync 2024-03-16T10:45:36.664 INFO:tasks.workunit.client.1.smithi026.stdout:6/805: truncate d0/d28/d49/f78 128519 0 2024-03-16T10:45:36.670 INFO:tasks.workunit.client.1.smithi026.stdout:4/985: dread d1/d85/fd8 [0,4194304] 0 2024-03-16T10:45:36.671 INFO:tasks.workunit.client.1.smithi026.stdout:1/885: mknod d0/d17/d1f/d20/d65/c127 0 2024-03-16T10:45:36.672 INFO:tasks.workunit.client.1.smithi026.stdout:4/986: fsync d1/db/de/f2b 0 2024-03-16T10:45:36.673 INFO:tasks.workunit.client.1.smithi026.stdout:5/910: dwrite d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/f10c [0,4194304] 0 2024-03-16T10:45:36.674 INFO:tasks.workunit.client.1.smithi026.stdout:1/886: write d0/d17/d1f/d20/dcc/ffe [692939,82910] 0 2024-03-16T10:45:36.675 INFO:tasks.workunit.client.1.smithi026.stdout:4/987: chown d1/db/de/d4c/d6b/fde 1489 1 2024-03-16T10:45:36.676 INFO:tasks.workunit.client.1.smithi026.stdout:4/988: fdatasync d1/d85/fc8 0 2024-03-16T10:45:36.679 INFO:tasks.workunit.client.1.smithi026.stdout:4/989: truncate d1/db/de/d4c/d6b/d89/d9c/fa6 802130 0 2024-03-16T10:45:36.681 INFO:tasks.workunit.client.1.smithi026.stdout:4/990: write d1/db/de/d4c/f116 [799330,55180] 0 2024-03-16T10:45:36.682 INFO:tasks.workunit.client.1.smithi026.stdout:8/899: truncate d5/d7/d2c/d36/d59/d93/da3/fa9 630403 0 2024-03-16T10:45:36.684 INFO:tasks.workunit.client.1.smithi026.stdout:8/900: dread d5/d7/d2c/f109 [0,4194304] 0 2024-03-16T10:45:36.689 INFO:tasks.workunit.client.1.smithi026.stdout:8/901: rmdir d5/d7/d7b/d96 39 2024-03-16T10:45:36.689 INFO:tasks.workunit.client.1.smithi026.stdout:4/991: dwrite d1/db/de/d4c/d6f/df3/f107 [0,4194304] 0 2024-03-16T10:45:36.691 INFO:tasks.workunit.client.1.smithi026.stdout:8/902: write d5/d7/d2c/d36/d59/d93/dde/de0/ff8 [756861,18768] 0 2024-03-16T10:45:36.694 INFO:tasks.workunit.client.1.smithi026.stdout:4/992: dread d1/db/ddb/daa/f5f [4194304,4194304] 0 2024-03-16T10:45:36.696 INFO:tasks.workunit.client.1.smithi026.stdout:6/806: write d0/d28/d49/d7d/d102/d91/fe1 [706093,89447] 0 2024-03-16T10:45:36.698 INFO:tasks.workunit.client.1.smithi026.stdout:4/993: symlink d1/db/ddb/daf/l12b 0 2024-03-16T10:45:36.703 INFO:tasks.workunit.client.1.smithi026.stdout:6/807: symlink d0/d28/d49/db0/l116 0 2024-03-16T10:45:36.707 INFO:tasks.workunit.client.1.smithi026.stdout:1/887: write d0/d17/d1f/d20/f85 [758179,117148] 0 2024-03-16T10:45:36.708 INFO:tasks.workunit.client.1.smithi026.stdout:6/808: dread d0/d10/fc3 [0,4194304] 0 2024-03-16T10:45:36.709 INFO:tasks.workunit.client.1.smithi026.stdout:1/888: truncate d0/d17/d1f/d63/f101 483046 0 2024-03-16T10:45:36.711 INFO:tasks.workunit.client.1.smithi026.stdout:8/903: link d5/d7/c14 d5/d5d/c10e 0 2024-03-16T10:45:36.715 INFO:tasks.workunit.client.1.smithi026.stdout:8/904: dread d5/d7/f4f [0,4194304] 0 2024-03-16T10:45:36.716 INFO:tasks.workunit.client.1.smithi026.stdout:5/911: truncate d1/f5 1497577 0 2024-03-16T10:45:36.718 INFO:tasks.workunit.client.1.smithi026.stdout:1/889: unlink d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/fdc 0 2024-03-16T10:45:36.719 INFO:tasks.workunit.client.1.smithi026.stdout:1/890: chown d0/d17/d1f/d20/d75/fcd 127516411 1 2024-03-16T10:45:36.721 INFO:tasks.workunit.client.1.smithi026.stdout:4/994: rename d1/db/de/d6e/feb to d1/db/ddb/daa/d44/d48/f12c 0 2024-03-16T10:45:36.726 INFO:tasks.workunit.client.1.smithi026.stdout:1/891: write d0/d17/f1a [2201437,64204] 0 2024-03-16T10:45:36.728 INFO:tasks.workunit.client.1.smithi026.stdout:1/892: write d0/d17/d1f/d63/d6a/d7c/d3b/d64/f6f [1859921,75131] 0 2024-03-16T10:45:36.730 INFO:tasks.workunit.client.1.smithi026.stdout:8/905: rename d5/dc/d89 to d5/dc/d40/dfe/d10f 0 2024-03-16T10:45:36.733 INFO:tasks.workunit.client.1.smithi026.stdout:4/995: fsync d1/db/ddb/daa/d44/dd5/de9/f11a 0 2024-03-16T10:45:36.736 INFO:tasks.workunit.client.1.smithi026.stdout:1/893: symlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/l128 0 2024-03-16T10:45:36.743 INFO:tasks.workunit.client.1.smithi026.stdout:1/894: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa/f125 [0,4194304] 0 2024-03-16T10:45:36.753 INFO:tasks.workunit.client.1.smithi026.stdout:4/996: creat d1/d85/f12d x:0 0 0 2024-03-16T10:45:36.754 INFO:tasks.workunit.client.1.smithi026.stdout:1/895: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/ff9 [0,4194304] 0 2024-03-16T10:45:36.755 INFO:tasks.workunit.client.1.smithi026.stdout:4/997: write d1/d85/da2/dca/f112 [36141,122928] 0 2024-03-16T10:45:36.758 INFO:tasks.workunit.client.1.smithi026.stdout:4/998: write d1/db/de/f55 [859347,41107] 0 2024-03-16T10:45:36.760 INFO:tasks.workunit.client.1.smithi026.stdout:4/999: dread - d1/dcd/fd1 zero size 2024-03-16T10:45:36.765 INFO:tasks.workunit.client.1.smithi026.stdout:1/896: dwrite d0/d17/d1f/d63/d58/f9a [0,4194304] 0 2024-03-16T10:45:36.772 INFO:tasks.workunit.client.1.smithi026.stdout:8/906: link d5/d7/l79 d5/d7/l110 0 2024-03-16T10:45:36.774 INFO:tasks.workunit.client.1.smithi026.stdout:8/907: write d5/da0/feb [2449297,103143] 0 2024-03-16T10:45:36.779 INFO:tasks.workunit.client.1.smithi026.stdout:8/908: truncate d5/d7/d2c/f109 436400 0 2024-03-16T10:45:36.781 INFO:tasks.workunit.client.1.smithi026.stdout:8/909: write d5/d7/d2c/d36/d59/f104 [605072,119244] 0 2024-03-16T10:45:36.782 INFO:tasks.workunit.client.1.smithi026.stdout:8/910: read - d5/d5d/d9a/f107 zero size 2024-03-16T10:45:36.790 INFO:tasks.workunit.client.1.smithi026.stdout:6/809: link d0/d41/d4f/f83 d0/d5b/f117 0 2024-03-16T10:45:36.793 INFO:tasks.workunit.client.1.smithi026.stdout:6/810: rmdir d0/d28/d2d 39 2024-03-16T10:45:36.803 INFO:tasks.workunit.client.1.smithi026.stdout:6/811: rename d0/d28/d49/f78 to d0/d28/d49/f118 0 2024-03-16T10:45:36.803 INFO:tasks.workunit.client.1.smithi026.stdout:6/812: readlink d0/d28/d49/d7d/d102/d91/lb5 0 2024-03-16T10:45:36.804 INFO:tasks.workunit.client.1.smithi026.stdout:6/813: read d0/d41/fd8 [729695,44795] 0 2024-03-16T10:45:36.806 INFO:tasks.workunit.client.1.smithi026.stdout:6/814: fsync d0/d5b/f64 0 2024-03-16T10:45:36.821 INFO:tasks.workunit.client.1.smithi026.stdout:8/911: dwrite d5/dc/d40/d53/fb7 [0,4194304] 0 2024-03-16T10:45:36.825 INFO:tasks.workunit.client.1.smithi026.stdout:8/912: fdatasync d5/dc/d40/d53/fcc 0 2024-03-16T10:45:36.826 INFO:tasks.workunit.client.1.smithi026.stdout:8/913: fsync d5/d5d/f97 0 2024-03-16T10:45:36.833 INFO:tasks.workunit.client.1.smithi026.stdout:8/914: creat d5/d7/d2c/d36/d59/d93/db0/f111 x:0 0 0 2024-03-16T10:45:36.834 INFO:tasks.workunit.client.1.smithi026.stdout:8/915: chown d5/dc/l19 30199950 1 2024-03-16T10:45:36.835 INFO:tasks.workunit.client.1.smithi026.stdout:6/815: dwrite d0/d10/fd [0,4194304] 0 2024-03-16T10:45:36.838 INFO:tasks.workunit.client.1.smithi026.stdout:8/916: dread d5/dc/f1e [0,4194304] 0 2024-03-16T10:45:36.839 INFO:tasks.workunit.client.1.smithi026.stdout:8/917: dread - d5/d5d/fc3 zero size 2024-03-16T10:45:36.846 INFO:tasks.workunit.client.1.smithi026.stdout:8/918: symlink d5/d7/de6/dd5/l112 0 2024-03-16T10:45:36.851 INFO:tasks.workunit.client.1.smithi026.stdout:8/919: creat d5/d5d/d9a/df7/f113 x:0 0 0 2024-03-16T10:45:36.854 INFO:tasks.workunit.client.1.smithi026.stdout:8/920: rmdir d5/d7 39 2024-03-16T10:45:36.856 INFO:tasks.workunit.client.1.smithi026.stdout:8/921: fdatasync d5/d28/fdf 0 2024-03-16T10:45:36.859 INFO:tasks.workunit.client.1.smithi026.stdout:6/816: dwrite d0/d28/d49/d7d/fc9 [0,4194304] 0 2024-03-16T10:45:36.866 INFO:tasks.workunit.client.1.smithi026.stdout:6/817: write d0/d28/d49/d7d/d102/d91/fa4 [344062,100247] 0 2024-03-16T10:45:36.870 INFO:tasks.workunit.client.1.smithi026.stdout:6/818: creat d0/d28/d49/db0/f119 x:0 0 0 2024-03-16T10:45:36.870 INFO:tasks.workunit.client.1.smithi026.stdout:8/922: dwrite d5/d7/d2c/f50 [4194304,4194304] 0 2024-03-16T10:45:36.871 INFO:tasks.workunit.client.1.smithi026.stdout:8/923: readlink d5/dc/l88 0 2024-03-16T10:45:36.872 INFO:tasks.workunit.client.1.smithi026.stdout:8/924: write d5/d7/de6/d76/fcb [391151,58033] 0 2024-03-16T10:45:36.874 INFO:tasks.workunit.client.1.smithi026.stdout:6/819: fdatasync d0/d62/fbf 0 2024-03-16T10:45:36.876 INFO:tasks.workunit.client.1.smithi026.stdout:8/925: mkdir d5/d5d/d9a/dd6/d114 0 2024-03-16T10:45:36.878 INFO:tasks.workunit.client.1.smithi026.stdout:6/820: symlink d0/d10/dd2/l11a 0 2024-03-16T10:45:36.881 INFO:tasks.workunit.client.1.smithi026.stdout:6/821: creat d0/d28/dcb/def/f11b x:0 0 0 2024-03-16T10:45:36.886 INFO:tasks.workunit.client.1.smithi026.stdout:6/822: link d0/d41/dc5/f10c d0/d28/dcb/f11c 0 2024-03-16T10:45:36.887 INFO:tasks.workunit.client.1.smithi026.stdout:6/823: readlink d0/d5b/laf 0 2024-03-16T10:45:36.889 INFO:tasks.workunit.client.1.smithi026.stdout:6/824: creat d0/d10/dd2/f11d x:0 0 0 2024-03-16T10:45:36.891 INFO:tasks.workunit.client.1.smithi026.stdout:6/825: write d0/d28/d49/d7d/d102/d7b/f82 [30088,74603] 0 2024-03-16T10:45:36.892 INFO:tasks.workunit.client.1.smithi026.stdout:6/826: readlink d0/d28/d49/d7d/d102/dca/l10b 0 2024-03-16T10:45:36.895 INFO:tasks.workunit.client.1.smithi026.stdout:8/926: dread d5/d5d/d9a/f9b [0,4194304] 0 2024-03-16T10:45:36.898 INFO:tasks.workunit.client.1.smithi026.stdout:8/927: unlink d5/d7/f6b 0 2024-03-16T10:45:36.899 INFO:tasks.workunit.client.1.smithi026.stdout:8/928: stat d5/d7/d2c/d36/d59/d93/dde/de0/cfb 0 2024-03-16T10:45:36.902 INFO:tasks.workunit.client.1.smithi026.stdout:8/929: dread d5/d7/d2c/f109 [0,4194304] 0 2024-03-16T10:45:36.906 INFO:tasks.workunit.client.1.smithi026.stdout:8/930: symlink d5/d7/d2c/d36/d8c/dd3/l115 0 2024-03-16T10:45:36.908 INFO:tasks.workunit.client.1.smithi026.stdout:6/827: write d0/d28/d49/d7d/d102/fa6 [42551,85228] 0 2024-03-16T10:45:36.909 INFO:tasks.workunit.client.1.smithi026.stdout:8/931: creat d5/d7/d2c/f116 x:0 0 0 2024-03-16T10:45:36.913 INFO:tasks.workunit.client.1.smithi026.stdout:6/828: creat d0/d28/d49/d7d/d102/dca/f11e x:0 0 0 2024-03-16T10:45:36.915 INFO:tasks.workunit.client.1.smithi026.stdout:8/932: dwrite d5/dc/f38 [0,4194304] 0 2024-03-16T10:45:36.920 INFO:tasks.workunit.client.1.smithi026.stdout:6/829: dwrite d0/d41/dc5/ff7 [0,4194304] 0 2024-03-16T10:45:36.950 INFO:tasks.workunit.client.1.smithi026.stdout:6/830: write d0/d10/f1a [1034947,104755] 0 2024-03-16T10:45:36.953 INFO:tasks.workunit.client.1.smithi026.stdout:6/831: readlink d0/d28/d49/d7d/d102/d91/lb5 0 2024-03-16T10:45:36.954 INFO:tasks.workunit.client.1.smithi026.stdout:8/933: dwrite d5/fbc [0,4194304] 0 2024-03-16T10:45:36.960 INFO:tasks.workunit.client.1.smithi026.stdout:8/934: rename d5/da0/feb to d5/dc/d40/d53/f117 0 2024-03-16T10:45:36.960 INFO:tasks.workunit.client.1.smithi026.stdout:8/935: chown d5/d7/d2c/d36/fef 619 1 2024-03-16T10:45:36.963 INFO:tasks.workunit.client.1.smithi026.stdout:6/832: truncate d0/f6 687869 0 2024-03-16T10:45:36.964 INFO:tasks.workunit.client.1.smithi026.stdout:6/833: read - d0/d41/dbe/fe9 zero size 2024-03-16T10:45:36.976 INFO:tasks.workunit.client.1.smithi026.stdout:8/936: write d5/dc/d40/f77 [964415,56979] 0 2024-03-16T10:45:36.977 INFO:tasks.workunit.client.1.smithi026.stdout:8/937: truncate d5/d5d/fc3 79363 0 2024-03-16T10:45:36.979 INFO:tasks.workunit.client.1.smithi026.stdout:6/834: write d0/f51 [142180,3696] 0 2024-03-16T10:45:36.983 INFO:tasks.workunit.client.1.smithi026.stdout:8/938: unlink d5/d7/d2c/d36/d59/d93/dde/de0/cfb 0 2024-03-16T10:45:36.985 INFO:tasks.workunit.client.1.smithi026.stdout:6/835: dwrite d0/d41/d4f/f105 [0,4194304] 0 2024-03-16T10:45:36.993 INFO:tasks.workunit.client.1.smithi026.stdout:6/836: creat d0/d28/d49/db0/d112/f11f x:0 0 0 2024-03-16T10:45:36.995 INFO:tasks.workunit.client.1.smithi026.stdout:6/837: rmdir d0/d41/d4f 39 2024-03-16T10:45:36.999 INFO:tasks.workunit.client.1.smithi026.stdout:8/939: rename d5/d5d/d9a/ddd/c44 to d5/d7/d2c/d36/c118 0 2024-03-16T10:45:37.003 INFO:tasks.workunit.client.1.smithi026.stdout:8/940: rmdir d5/d23/d4e/d60/dc8/dc4 39 2024-03-16T10:45:37.006 INFO:tasks.workunit.client.1.smithi026.stdout:8/941: rename d5/d23/d4e/f5b to d5/dc/d40/dfe/f119 0 2024-03-16T10:45:37.007 INFO:tasks.workunit.client.1.smithi026.stdout:8/942: chown d5/d23/ff4 52 1 2024-03-16T10:45:37.008 INFO:tasks.workunit.client.1.smithi026.stdout:8/943: stat d5/d7/d2c/d36/d59 0 2024-03-16T10:45:37.008 INFO:tasks.workunit.client.1.smithi026.stdout:6/838: write d0/d28/d2d/f76 [4082290,6537] 0 2024-03-16T10:45:37.009 INFO:tasks.workunit.client.1.smithi026.stdout:8/944: mknod d5/d28/c11a 0 2024-03-16T10:45:37.010 INFO:tasks.workunit.client.1.smithi026.stdout:6/839: symlink d0/dfc/l120 0 2024-03-16T10:45:37.011 INFO:tasks.workunit.client.1.smithi026.stdout:8/945: chown d5/d7/d2c/d36/d59/d93/dde/de0/df9 56521460 1 2024-03-16T10:45:37.011 INFO:tasks.workunit.client.1.smithi026.stdout:6/840: read d0/d10/ff [3217523,70796] 0 2024-03-16T10:45:37.014 INFO:tasks.workunit.client.1.smithi026.stdout:6/841: truncate d0/d5b/f95 1341414 0 2024-03-16T10:45:37.019 INFO:tasks.workunit.client.1.smithi026.stdout:8/946: rename d5/d7/d2c/d36/d59/d93/dde to d5/d23/d4e/d11b 0 2024-03-16T10:45:37.021 INFO:tasks.workunit.client.1.smithi026.stdout:8/947: chown d5/d7/d2c/f109 3550027 1 2024-03-16T10:45:37.031 INFO:tasks.workunit.client.1.smithi026.stdout:8/948: write d5/dc/d40/dfe/d10f/f8a [1373145,88026] 0 2024-03-16T10:45:37.032 INFO:tasks.workunit.client.1.smithi026.stdout:8/949: fsync d5/d23/d4e/d11b/fe9 0 2024-03-16T10:45:37.159 INFO:tasks.workunit.client.1.smithi026.stdout:5/912: sync 2024-03-16T10:45:37.211 INFO:tasks.workunit.client.1.smithi026.stdout:5/913: mkdir d1/d1c/d2a/dab/dc5/d69/d51/d129/d13f 0 2024-03-16T10:45:37.215 INFO:tasks.workunit.client.1.smithi026.stdout:5/914: getdents d1/d1c/d2a/db0 0 2024-03-16T10:45:37.218 INFO:tasks.workunit.client.1.smithi026.stdout:5/915: creat d1/d1c/d2a/d55/d91/f140 x:0 0 0 2024-03-16T10:45:37.235 INFO:tasks.workunit.client.1.smithi026.stdout:5/916: write d1/d1c/d2a/f32 [8426241,35125] 0 2024-03-16T10:45:37.239 INFO:tasks.workunit.client.1.smithi026.stdout:5/917: unlink d1/db/d1f/d61/d116/fd5 0 2024-03-16T10:45:37.240 INFO:tasks.workunit.client.1.smithi026.stdout:5/918: mknod d1/d1c/d2a/d55/d91/c141 0 2024-03-16T10:45:37.244 INFO:tasks.workunit.client.1.smithi026.stdout:5/919: creat d1/d1c/d4c/d13a/f142 x:0 0 0 2024-03-16T10:45:37.246 INFO:tasks.workunit.client.1.smithi026.stdout:5/920: symlink d1/d1c/d2a/dab/dc5/d69/d51/d80/l143 0 2024-03-16T10:45:37.249 INFO:tasks.workunit.client.1.smithi026.stdout:5/921: creat d1/d1c/d2a/dab/dc5/d69/d51/d129/d13f/f144 x:0 0 0 2024-03-16T10:45:37.251 INFO:tasks.workunit.client.1.smithi026.stdout:5/922: read d1/d1c/d2a/dab/dc5/f3a [2664869,93479] 0 2024-03-16T10:45:37.261 INFO:tasks.workunit.client.1.smithi026.stdout:5/923: dread d1/d1c/d2a/dab/dc5/f76 [0,4194304] 0 2024-03-16T10:45:37.261 INFO:tasks.workunit.client.1.smithi026.stdout:5/924: readlink d1/db/l100 0 2024-03-16T10:45:37.263 INFO:tasks.workunit.client.1.smithi026.stdout:5/925: write d1/d1c/d2a/dab/dc5/d69/d51/f5d [2579438,58588] 0 2024-03-16T10:45:37.273 INFO:tasks.workunit.client.1.smithi026.stdout:5/926: dread d1/d1c/d4c/d4d/fcc [0,4194304] 0 2024-03-16T10:45:37.274 INFO:tasks.workunit.client.1.smithi026.stdout:5/927: rename d1/d1c/d2a to d1/d1c/d2a/d8e/d145 22 2024-03-16T10:45:37.277 INFO:tasks.workunit.client.1.smithi026.stdout:5/928: rename d1/d1c/d4c/fe3 to d1/db/d1f/d61/f146 0 2024-03-16T10:45:37.280 INFO:tasks.workunit.client.1.smithi026.stdout:5/929: symlink d1/d1c/d2a/dab/dc5/d69/d51/d80/de9/l147 0 2024-03-16T10:45:37.281 INFO:tasks.workunit.client.1.smithi026.stdout:5/930: chown d1/d1c/d2a/dab 299536 1 2024-03-16T10:45:37.281 INFO:tasks.workunit.client.1.smithi026.stdout:1/897: sync 2024-03-16T10:45:37.282 INFO:tasks.workunit.client.1.smithi026.stdout:5/931: chown d1/d1c/d2a/d55/db1/ffb 4004071 1 2024-03-16T10:45:37.283 INFO:tasks.workunit.client.1.smithi026.stdout:5/932: write d1/d1c/d2a/dab/dc5/d69/f119 [702296,69496] 0 2024-03-16T10:45:37.287 INFO:tasks.workunit.client.1.smithi026.stdout:1/898: dread d0/fb3 [0,4194304] 0 2024-03-16T10:45:37.291 INFO:tasks.workunit.client.1.smithi026.stdout:5/933: dwrite d1/db/d1f/d61/d65/d82/fcf [4194304,4194304] 0 2024-03-16T10:45:37.299 INFO:tasks.workunit.client.1.smithi026.stdout:5/934: getdents d1/d1c/d4c 0 2024-03-16T10:45:37.302 INFO:tasks.workunit.client.1.smithi026.stdout:5/935: symlink d1/db/d1f/d61/d65/d82/dbb/l148 0 2024-03-16T10:45:37.303 INFO:tasks.workunit.client.1.smithi026.stdout:1/899: truncate d0/d17/d1f/d63/d58/f109 4289963 0 2024-03-16T10:45:37.308 INFO:tasks.workunit.client.1.smithi026.stdout:5/936: dwrite d1/d1c/d2a/dab/de8/f12c [0,4194304] 0 2024-03-16T10:45:37.323 INFO:tasks.workunit.client.1.smithi026.stdout:5/937: link d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/dc6/cd9 d1/d6e/d9f/c149 0 2024-03-16T10:45:37.323 INFO:tasks.workunit.client.1.smithi026.stdout:1/900: dwrite d0/d17/d1f/d63/d58/f109 [4194304,4194304] 0 2024-03-16T10:45:37.324 INFO:tasks.workunit.client.1.smithi026.stdout:5/938: dread - d1/d1c/d2a/d55/d91/f140 zero size 2024-03-16T10:45:37.327 INFO:tasks.workunit.client.1.smithi026.stdout:5/939: chown d1/d1c/d2a/dab/dc5/d69/d51/d80/de9/l147 28 1 2024-03-16T10:45:37.331 INFO:tasks.workunit.client.1.smithi026.stdout:1/901: rename d0/d17/d1f/l97 to d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/d100/l129 0 2024-03-16T10:45:37.331 INFO:tasks.workunit.client.1.smithi026.stdout:6/842: sync 2024-03-16T10:45:37.332 INFO:tasks.workunit.client.1.smithi026.stdout:8/950: sync 2024-03-16T10:45:37.332 INFO:tasks.workunit.client.1.smithi026.stdout:5/940: truncate d1/db/d1f/d61/fdc 252743 0 2024-03-16T10:45:37.334 INFO:tasks.workunit.client.1.smithi026.stdout:6/843: unlink d0/d10/dd2/l11a 0 2024-03-16T10:45:37.336 INFO:tasks.workunit.client.1.smithi026.stdout:5/941: mknod d1/d1c/d4c/d4d/c14a 0 2024-03-16T10:45:37.337 INFO:tasks.workunit.client.1.smithi026.stdout:5/942: readlink d1/d1c/d2a/d55/db1/l137 0 2024-03-16T10:45:37.341 INFO:tasks.workunit.client.1.smithi026.stdout:6/844: dread d0/d10/ff [0,4194304] 0 2024-03-16T10:45:37.342 INFO:tasks.workunit.client.1.smithi026.stdout:6/845: chown d0/d28/d2d/c5e 12014 1 2024-03-16T10:45:37.344 INFO:tasks.workunit.client.1.smithi026.stdout:6/846: write d0/d28/d49/d7d/d102/d91/fe1 [889689,76580] 0 2024-03-16T10:45:37.351 INFO:tasks.workunit.client.1.smithi026.stdout:6/847: dwrite d0/d10/fd [0,4194304] 0 2024-03-16T10:45:37.353 INFO:tasks.workunit.client.1.smithi026.stdout:6/848: chown d0/d28/dcb/f11c 14 1 2024-03-16T10:45:37.353 INFO:tasks.workunit.client.1.smithi026.stdout:1/902: dwrite d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fad [0,4194304] 0 2024-03-16T10:45:37.353 INFO:tasks.workunit.client.1.smithi026.stdout:5/943: link d1/d1c/d2a/dab/dc5/f76 d1/d1c/d4c/f14b 0 2024-03-16T10:45:37.359 INFO:tasks.workunit.client.1.smithi026.stdout:1/903: truncate d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fda 1002511 0 2024-03-16T10:45:37.359 INFO:tasks.workunit.client.1.smithi026.stdout:5/944: creat d1/f14c x:0 0 0 2024-03-16T10:45:37.360 INFO:tasks.workunit.client.1.smithi026.stdout:6/849: read d0/f6 [265686,14016] 0 2024-03-16T10:45:37.362 INFO:tasks.workunit.client.1.smithi026.stdout:5/945: chown d1/d1c/d2a/dab/dc5/d69/f8c 7 1 2024-03-16T10:45:37.366 INFO:tasks.workunit.client.1.smithi026.stdout:1/904: dwrite d0/d17/f1a [0,4194304] 0 2024-03-16T10:45:37.371 INFO:tasks.workunit.client.1.smithi026.stdout:1/905: truncate d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/ffa 213832 0 2024-03-16T10:45:37.374 INFO:tasks.workunit.client.1.smithi026.stdout:6/850: getdents d0/d28/dcb 0 2024-03-16T10:45:37.375 INFO:tasks.workunit.client.1.smithi026.stdout:1/906: readlink d0/d17/d1f/l9 0 2024-03-16T10:45:37.377 INFO:tasks.workunit.client.1.smithi026.stdout:6/851: unlink d0/cf1 0 2024-03-16T10:45:37.378 INFO:tasks.workunit.client.1.smithi026.stdout:5/946: dread d1/d1c/d2a/d55/d91/ffd [0,4194304] 0 2024-03-16T10:45:37.387 INFO:tasks.workunit.client.1.smithi026.stdout:5/947: mknod d1/d1c/d2a/dab/de8/d11a/c14d 0 2024-03-16T10:45:37.388 INFO:tasks.workunit.client.1.smithi026.stdout:1/907: fsync d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/f126 0 2024-03-16T10:45:37.389 INFO:tasks.workunit.client.1.smithi026.stdout:6/852: rmdir d0 39 2024-03-16T10:45:37.392 INFO:tasks.workunit.client.1.smithi026.stdout:5/948: stat d1/d1c/d4c/d4d/c7a 0 2024-03-16T10:45:37.399 INFO:tasks.workunit.client.1.smithi026.stdout:5/949: truncate d1/db/f39 4621074 0 2024-03-16T10:45:37.399 INFO:tasks.workunit.client.1.smithi026.stdout:6/853: rmdir d0/d28/dcb/def 39 2024-03-16T10:45:37.400 INFO:tasks.workunit.client.1.smithi026.stdout:5/950: readlink d1/db/l100 0 2024-03-16T10:45:37.400 INFO:tasks.workunit.client.1.smithi026.stdout:1/908: dread d0/f40 [0,4194304] 0 2024-03-16T10:45:37.403 INFO:tasks.workunit.client.1.smithi026.stdout:6/854: chown d0/d41/dc5/f110 283178514 1 2024-03-16T10:45:37.404 INFO:tasks.workunit.client.1.smithi026.stdout:5/951: mkdir d1/d6e/d9f/d14e 0 2024-03-16T10:45:37.410 INFO:tasks.workunit.client.1.smithi026.stdout:6/855: creat d0/d28/d49/db0/d112/f121 x:0 0 0 2024-03-16T10:45:37.412 INFO:tasks.workunit.client.1.smithi026.stdout:1/909: write d0/d17/d1f/d63/d6a/d7c/d2e/d54/fd1 [748685,112300] 0 2024-03-16T10:45:37.419 INFO:tasks.workunit.client.1.smithi026.stdout:5/952: link d1/db/l77 d1/l14f 0 2024-03-16T10:45:37.421 INFO:tasks.workunit.client.1.smithi026.stdout:6/856: dwrite d0/d10/f38 [0,4194304] 0 2024-03-16T10:45:37.423 INFO:tasks.workunit.client.1.smithi026.stdout:1/910: write d0/d17/f48 [2168467,72711] 0 2024-03-16T10:45:37.424 INFO:tasks.workunit.client.1.smithi026.stdout:5/953: creat d1/d6e/f150 x:0 0 0 2024-03-16T10:45:37.427 INFO:tasks.workunit.client.1.smithi026.stdout:6/857: creat d0/d28/d2d/f122 x:0 0 0 2024-03-16T10:45:37.433 INFO:tasks.workunit.client.1.smithi026.stdout:1/911: dwrite d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/fad [0,4194304] 0 2024-03-16T10:45:37.437 INFO:tasks.workunit.client.1.smithi026.stdout:6/858: dwrite d0/d28/d49/d7d/fc9 [4194304,4194304] 0 2024-03-16T10:45:37.437 INFO:tasks.workunit.client.1.smithi026.stdout:1/912: dread - d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dff/f121 zero size 2024-03-16T10:45:37.440 INFO:tasks.workunit.client.1.smithi026.stdout:6/859: creat d0/d28/d49/d7d/f123 x:0 0 0 2024-03-16T10:45:37.453 INFO:tasks.workunit.client.1.smithi026.stdout:6/860: dwrite d0/d10/dd2/fe7 [0,4194304] 0 2024-03-16T10:45:37.462 INFO:tasks.workunit.client.1.smithi026.stdout:6/861: write d0/d5b/f43 [449292,108885] 0 2024-03-16T10:45:37.474 INFO:tasks.workunit.client.1.smithi026.stdout:6/862: dwrite d0/d28/d49/d7d/f8c [0,4194304] 0 2024-03-16T10:45:37.481 INFO:tasks.workunit.client.1.smithi026.stdout:6/863: symlink d0/d28/l124 0 2024-03-16T10:45:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:45:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:45:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: pgmap v13: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 47 MiB/s rd, 103 MiB/s wr, 283 op/s 2024-03-16T10:45:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:45:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:45:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:45:37.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:45:37.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:37 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.484 INFO:tasks.workunit.client.1.smithi026.stdout:6/864: unlink d0/d28/f5d 0 2024-03-16T10:45:37.491 INFO:tasks.workunit.client.1.smithi026.stdout:6/865: dwrite d0/f35 [0,4194304] 0 2024-03-16T10:45:37.494 INFO:tasks.workunit.client.1.smithi026.stdout:6/866: creat d0/d28/d49/d7d/d102/d91/f125 x:0 0 0 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: pgmap v13: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 47 MiB/s rd, 103 MiB/s wr, 283 op/s 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.500 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:37 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:37.501 INFO:tasks.workunit.client.1.smithi026.stdout:6/867: truncate d0/d10/f27 2274066 0 2024-03-16T10:45:37.509 INFO:tasks.workunit.client.1.smithi026.stdout:6/868: dread d0/d10/dd2/fe7 [0,4194304] 0 2024-03-16T10:45:37.517 INFO:tasks.workunit.client.1.smithi026.stdout:6/869: write d0/f3 [1267835,130277] 0 2024-03-16T10:45:37.520 INFO:tasks.workunit.client.1.smithi026.stdout:6/870: dread d0/d10/fd [0,4194304] 0 2024-03-16T10:45:37.521 INFO:tasks.workunit.client.1.smithi026.stdout:6/871: read d0/d28/d49/d7d/d102/d91/f10f [342017,6721] 0 2024-03-16T10:45:37.523 INFO:tasks.workunit.client.1.smithi026.stdout:6/872: mknod d0/d41/d71/c126 0 2024-03-16T10:45:37.524 INFO:tasks.workunit.client.1.smithi026.stdout:6/873: readlink d0/d28/d49/dc2/lc8 0 2024-03-16T10:45:37.525 INFO:tasks.workunit.client.1.smithi026.stdout:6/874: write d0/d28/d2d/f4c [511808,12848] 0 2024-03-16T10:45:37.531 INFO:tasks.workunit.client.1.smithi026.stdout:6/875: dwrite d0/f35 [0,4194304] 0 2024-03-16T10:45:37.536 INFO:tasks.workunit.client.1.smithi026.stdout:6/876: rename d0/d28/d49/d7d/d102/d91/lb5 to d0/d8a/l127 0 2024-03-16T10:45:37.540 INFO:tasks.workunit.client.1.smithi026.stdout:6/877: read d0/d41/d4f/f77 [1536190,72557] 0 2024-03-16T10:45:37.543 INFO:tasks.workunit.client.1.smithi026.stdout:6/878: mkdir d0/d10/dd2/d128 0 2024-03-16T10:45:37.545 INFO:tasks.workunit.client.1.smithi026.stdout:6/879: stat d0/d28/d49/d7d/d102/d91/ldf 0 2024-03-16T10:45:37.547 INFO:tasks.workunit.client.1.smithi026.stdout:6/880: mkdir d0/d41/d4f/d129 0 2024-03-16T10:45:37.548 INFO:tasks.workunit.client.1.smithi026.stdout:6/881: chown d0/l3c 73 1 2024-03-16T10:45:37.554 INFO:tasks.workunit.client.1.smithi026.stdout:6/882: getdents d0/d28/d49/d7d/d102/d91 0 2024-03-16T10:45:37.558 INFO:tasks.workunit.client.1.smithi026.stdout:6/883: rename d0/d5b/f30 to d0/d41/f12a 0 2024-03-16T10:45:37.559 INFO:tasks.workunit.client.1.smithi026.stdout:6/884: fsync d0/d5b/f43 0 2024-03-16T10:45:37.559 INFO:tasks.workunit.client.1.smithi026.stdout:6/885: read - d0/d10/ff3 zero size 2024-03-16T10:45:37.560 INFO:tasks.workunit.client.1.smithi026.stdout:6/886: chown d0/d28/fb2 771920 1 2024-03-16T10:45:37.567 INFO:tasks.workunit.client.1.smithi026.stdout:6/887: write d0/d41/dc5/f10c [814895,31135] 0 2024-03-16T10:45:37.572 INFO:tasks.workunit.client.1.smithi026.stdout:6/888: link d0/d41/dc5/f110 d0/d28/d49/dc2/f12b 0 2024-03-16T10:45:37.573 INFO:tasks.workunit.client.1.smithi026.stdout:6/889: write d0/fbd [4982490,64334] 0 2024-03-16T10:45:37.579 INFO:tasks.workunit.client.1.smithi026.stdout:6/890: dread d0/d5b/f95 [0,4194304] 0 2024-03-16T10:45:37.580 INFO:tasks.workunit.client.1.smithi026.stdout:6/891: chown d0/d62/l73 945365322 1 2024-03-16T10:45:37.581 INFO:tasks.workunit.client.1.smithi026.stdout:6/892: readlink d0/d8a/l127 0 2024-03-16T10:45:37.587 INFO:tasks.workunit.client.1.smithi026.stdout:6/893: link d0/f1d d0/d28/d49/d7d/d102/d8e/df0/f12c 0 2024-03-16T10:45:37.590 INFO:tasks.workunit.client.1.smithi026.stdout:6/894: dread d0/d28/d49/f118 [0,4194304] 0 2024-03-16T10:45:37.591 INFO:tasks.workunit.client.1.smithi026.stdout:6/895: truncate d0/d28/d2d/f106 176843 0 2024-03-16T10:45:37.592 INFO:tasks.workunit.client.1.smithi026.stdout:6/896: creat d0/d41/d4f/f12d x:0 0 0 2024-03-16T10:45:37.596 INFO:tasks.workunit.client.1.smithi026.stdout:6/897: getdents d0/d41/dc5 0 2024-03-16T10:45:37.598 INFO:tasks.workunit.client.1.smithi026.stdout:6/898: unlink d0/d28/d49/d7d/d102/fd5 0 2024-03-16T10:45:37.605 INFO:tasks.workunit.client.1.smithi026.stdout:6/899: dwrite d0/d5b/f100 [0,4194304] 0 2024-03-16T10:45:37.605 INFO:tasks.workunit.client.1.smithi026.stdout:6/900: stat d0/d41/d4f/f69 0 2024-03-16T10:45:37.608 INFO:tasks.workunit.client.1.smithi026.stdout:6/901: symlink d0/d8a/db7/l12e 0 2024-03-16T10:45:37.609 INFO:tasks.workunit.client.1.smithi026.stdout:6/902: write d0/d28/d49/d7d/d102/d91/ffa [368791,48777] 0 2024-03-16T10:45:37.613 INFO:tasks.workunit.client.1.smithi026.stdout:6/903: mknod d0/d28/dcb/def/c12f 0 2024-03-16T10:45:37.615 INFO:tasks.workunit.client.1.smithi026.stdout:6/904: truncate d0/d10/f23 2818909 0 2024-03-16T10:45:37.618 INFO:tasks.workunit.client.1.smithi026.stdout:6/905: dread d0/d10/ff [0,4194304] 0 2024-03-16T10:45:37.620 INFO:tasks.workunit.client.1.smithi026.stdout:6/906: stat d0/d28/d49/d7d/d102/fa6 0 2024-03-16T10:45:37.628 INFO:tasks.workunit.client.1.smithi026.stdout:6/907: link d0/d5b/c74 d0/d28/c130 0 2024-03-16T10:45:37.630 INFO:tasks.workunit.client.1.smithi026.stdout:6/908: symlink d0/d5b/l131 0 2024-03-16T10:45:37.633 INFO:tasks.workunit.client.1.smithi026.stdout:6/909: dread d0/d28/d49/d7d/d102/d7b/f82 [4194304,4194304] 0 2024-03-16T10:45:37.641 INFO:tasks.workunit.client.1.smithi026.stdout:6/910: dread d0/d41/f12a [0,4194304] 0 2024-03-16T10:45:37.645 INFO:tasks.workunit.client.1.smithi026.stdout:6/911: mkdir d0/d8a/db7/d132 0 2024-03-16T10:45:37.650 INFO:tasks.workunit.client.1.smithi026.stdout:6/912: dwrite d0/d28/d2d/f106 [0,4194304] 0 2024-03-16T10:45:37.663 INFO:tasks.workunit.client.1.smithi026.stdout:6/913: getdents d0/d8a/db7/d132 0 2024-03-16T10:45:37.669 INFO:tasks.workunit.client.1.smithi026.stdout:6/914: link d0/d28/d49/db0/d112/f11f d0/d41/f133 0 2024-03-16T10:45:37.671 INFO:tasks.workunit.client.1.smithi026.stdout:8/951: sync 2024-03-16T10:45:37.673 INFO:tasks.workunit.client.1.smithi026.stdout:6/915: creat d0/d5b/f134 x:0 0 0 2024-03-16T10:45:37.674 INFO:tasks.workunit.client.1.smithi026.stdout:6/916: chown d0/d28/d49/d7d/d102/d91/fe1 707565 1 2024-03-16T10:45:37.679 INFO:tasks.workunit.client.1.smithi026.stdout:6/917: dwrite d0/d41/d4f/f96 [0,4194304] 0 2024-03-16T10:45:37.697 INFO:tasks.workunit.client.1.smithi026.stdout:6/918: dread d0/d41/dc5/fec [0,4194304] 0 2024-03-16T10:45:37.698 INFO:tasks.workunit.client.1.smithi026.stdout:6/919: write d0/d5b/f25 [7847424,122027] 0 2024-03-16T10:45:37.708 INFO:tasks.workunit.client.1.smithi026.stdout:6/920: mknod d0/d28/c135 0 2024-03-16T10:45:37.723 INFO:tasks.workunit.client.1.smithi026.stdout:6/921: getdents d0/d28/d49/d7d/d102 0 2024-03-16T10:45:37.723 INFO:tasks.workunit.client.1.smithi026.stdout:6/922: chown d0/d10/dd2 0 1 2024-03-16T10:45:37.727 INFO:tasks.workunit.client.1.smithi026.stdout:6/923: truncate d0/f2c 697940 0 2024-03-16T10:45:37.735 INFO:tasks.workunit.client.1.smithi026.stdout:6/924: link d0/d8a/cba d0/d8a/c136 0 2024-03-16T10:45:37.737 INFO:tasks.workunit.client.1.smithi026.stdout:6/925: write d0/d10/dd2/f101 [774636,121509] 0 2024-03-16T10:45:37.740 INFO:tasks.workunit.client.1.smithi026.stdout:6/926: getdents d0/d28/d49/dc2 0 2024-03-16T10:45:37.755 INFO:tasks.workunit.client.1.smithi026.stdout:6/927: dwrite d0/d41/d4f/f87 [0,4194304] 0 2024-03-16T10:45:37.761 INFO:tasks.workunit.client.1.smithi026.stdout:6/928: rename d0/d28/dcb/def/f11b to d0/d8a/db7/d132/f137 0 2024-03-16T10:45:37.762 INFO:tasks.workunit.client.1.smithi026.stdout:6/929: readlink d0/d28/d49/d7d/d102/dca/dd7/l104 0 2024-03-16T10:45:37.773 INFO:tasks.workunit.client.1.smithi026.stdout:6/930: dwrite d0/d5b/f117 [0,4194304] 0 2024-03-16T10:45:37.774 INFO:tasks.workunit.client.1.smithi026.stdout:6/931: chown d0/d28/d49/d7d/d102/dca 1322971 1 2024-03-16T10:45:37.779 INFO:tasks.workunit.client.1.smithi026.stdout:6/932: dread d0/d28/fce [0,4194304] 0 2024-03-16T10:45:37.780 INFO:tasks.workunit.client.1.smithi026.stdout:6/933: read d0/d28/f6f [388583,126060] 0 2024-03-16T10:45:37.781 INFO:tasks.workunit.client.1.smithi026.stdout:6/934: truncate d0/d5b/f43 1415955 0 2024-03-16T10:45:37.782 INFO:tasks.workunit.client.1.smithi026.stdout:6/935: stat d0/d28/d49/d7d/df2 0 2024-03-16T10:45:37.785 INFO:tasks.workunit.client.1.smithi026.stdout:6/936: mknod d0/d8a/db7/d132/c138 0 2024-03-16T10:45:37.789 INFO:tasks.workunit.client.1.smithi026.stdout:6/937: link d0/d41/dc5/ff7 d0/d28/d49/d7d/df2/f139 0 2024-03-16T10:45:37.792 INFO:tasks.workunit.client.1.smithi026.stdout:6/938: rename d0/d41/d71/fd0 to d0/d28/d49/d7d/d102/dca/f13a 0 2024-03-16T10:45:37.802 INFO:tasks.workunit.client.1.smithi026.stdout:6/939: dread d0/d41/fa7 [0,4194304] 0 2024-03-16T10:45:37.813 INFO:tasks.workunit.client.1.smithi026.stdout:6/940: truncate d0/d28/d49/d7d/df2/f139 3030837 0 2024-03-16T10:45:37.813 INFO:tasks.workunit.client.1.smithi026.stdout:6/941: chown d0/d28/dcb/f11c 541581478 1 2024-03-16T10:45:37.817 INFO:tasks.workunit.client.1.smithi026.stdout:6/942: symlink d0/d28/d49/d7d/d102/d8e/df0/l13b 0 2024-03-16T10:45:37.818 INFO:tasks.workunit.client.1.smithi026.stdout:6/943: rmdir d0/d28/d49/db0 39 2024-03-16T10:45:37.829 INFO:tasks.workunit.client.1.smithi026.stdout:6/944: dread d0/d28/d49/d7d/d102/d7b/f7e [0,4194304] 0 2024-03-16T10:45:37.835 INFO:tasks.workunit.client.1.smithi026.stdout:6/945: link d0/d28/d2d/c34 d0/d41/d4f/c13c 0 2024-03-16T10:45:37.840 INFO:tasks.workunit.client.1.smithi026.stdout:6/946: rename d0/d28/c135 to d0/d28/d49/db0/c13d 0 2024-03-16T10:45:37.842 INFO:tasks.workunit.client.1.smithi026.stdout:6/947: unlink d0/d10/f6c 0 2024-03-16T10:45:37.842 INFO:tasks.workunit.client.1.smithi026.stdout:6/948: chown d0/d41/d4f/fa9 1674926992 1 2024-03-16T10:45:37.844 INFO:tasks.workunit.client.1.smithi026.stdout:6/949: unlink d0/d5b/f100 0 2024-03-16T10:45:37.846 INFO:tasks.workunit.client.1.smithi026.stdout:6/950: creat d0/d41/dbe/f13e x:0 0 0 2024-03-16T10:45:37.848 INFO:tasks.workunit.client.1.smithi026.stdout:6/951: dread d0/d28/f6f [0,4194304] 0 2024-03-16T10:45:37.868 INFO:tasks.workunit.client.1.smithi026.stdout:6/952: write d0/d28/d49/d7d/d102/d8e/df0/f12c [3845541,5022] 0 2024-03-16T10:45:37.869 INFO:tasks.workunit.client.1.smithi026.stdout:6/953: chown d0/d5b/laf 44199885 1 2024-03-16T10:45:37.875 INFO:tasks.workunit.client.1.smithi026.stdout:5/954: sync 2024-03-16T10:45:37.880 INFO:tasks.workunit.client.1.smithi026.stdout:5/955: fsync d1/db/d1f/d61/f6d 0 2024-03-16T10:45:37.882 INFO:tasks.workunit.client.1.smithi026.stdout:5/956: chown d1/d1c/d2a/dab/dc5/d69/d51/f5e 38 1 2024-03-16T10:45:37.891 INFO:tasks.workunit.client.1.smithi026.stdout:6/954: dread d0/d28/d49/d7d/d102/d7b/f9e [0,4194304] 0 2024-03-16T10:45:37.892 INFO:tasks.workunit.client.1.smithi026.stdout:5/957: dread d1/d1c/d2a/dab/dc5/f3a [0,4194304] 0 2024-03-16T10:45:37.895 INFO:tasks.workunit.client.1.smithi026.stdout:6/955: dread d0/d28/dcb/f11c [0,4194304] 0 2024-03-16T10:45:37.895 INFO:tasks.workunit.client.1.smithi026.stdout:5/958: creat d1/f151 x:0 0 0 2024-03-16T10:45:37.896 INFO:tasks.workunit.client.1.smithi026.stdout:6/956: fdatasync d0/d28/d49/d7d/d102/dca/f11e 0 2024-03-16T10:45:37.898 INFO:tasks.workunit.client.1.smithi026.stdout:5/959: creat d1/db/d1f/d61/d65/d82/f152 x:0 0 0 2024-03-16T10:45:37.900 INFO:tasks.workunit.client.1.smithi026.stdout:5/960: creat d1/d6e/d9f/d14e/f153 x:0 0 0 2024-03-16T10:45:37.901 INFO:tasks.workunit.client.1.smithi026.stdout:5/961: fdatasync d1/f14c 0 2024-03-16T10:45:37.903 INFO:tasks.workunit.client.1.smithi026.stdout:6/957: dwrite d0/fe4 [0,4194304] 0 2024-03-16T10:45:37.905 INFO:tasks.workunit.client.1.smithi026.stdout:5/962: dread d1/d1c/d2a/d8e/f11b [0,4194304] 0 2024-03-16T10:45:37.907 INFO:tasks.workunit.client.1.smithi026.stdout:6/958: fsync d0/d28/d49/d7d/d102/d7b/fdc 0 2024-03-16T10:45:37.908 INFO:tasks.workunit.client.1.smithi026.stdout:6/959: fsync d0/f6 0 2024-03-16T10:45:37.912 INFO:tasks.workunit.client.1.smithi026.stdout:5/963: truncate d1/f5 1613911 0 2024-03-16T10:45:37.916 INFO:tasks.workunit.client.1.smithi026.stdout:6/960: creat d0/d28/d49/d7d/d102/dca/dde/f13f x:0 0 0 2024-03-16T10:45:37.922 INFO:tasks.workunit.client.1.smithi026.stdout:6/961: getdents d0/d28/d49/d7d/d102/dca 0 2024-03-16T10:45:37.925 INFO:tasks.workunit.client.1.smithi026.stdout:6/962: symlink d0/d28/d49/dc2/l140 0 2024-03-16T10:45:37.926 INFO:tasks.workunit.client.1.smithi026.stdout:5/964: dread d1/d1c/d4c/f12b [0,4194304] 0 2024-03-16T10:45:37.928 INFO:tasks.workunit.client.1.smithi026.stdout:6/963: symlink d0/d41/d4f/d129/l141 0 2024-03-16T10:45:37.928 INFO:tasks.workunit.client.1.smithi026.stdout:6/964: chown d0/dfc 1167062 1 2024-03-16T10:45:37.929 INFO:tasks.workunit.client.1.smithi026.stdout:6/965: read d0/d41/d4f/f96 [537811,54509] 0 2024-03-16T10:45:37.930 INFO:tasks.workunit.client.1.smithi026.stdout:5/965: rmdir d1/d1c/d2a/d55 39 2024-03-16T10:45:37.931 INFO:tasks.workunit.client.1.smithi026.stdout:6/966: mkdir d0/d8a/d142 0 2024-03-16T10:45:37.938 INFO:tasks.workunit.client.1.smithi026.stdout:6/967: dwrite d0/d28/d49/d7d/d102/fa6 [0,4194304] 0 2024-03-16T10:45:37.941 INFO:tasks.workunit.client.1.smithi026.stdout:5/966: dread d1/f71 [0,4194304] 0 2024-03-16T10:45:37.946 INFO:tasks.workunit.client.1.smithi026.stdout:1/913: sync 2024-03-16T10:45:37.947 INFO:tasks.workunit.client.1.smithi026.stdout:5/967: rename d1/db/d1f/d61/d65/d82/dbb/dea/d126 to d1/d1c/d2a/dab/dc5/d69/d51/d129/d154 0 2024-03-16T10:45:37.954 INFO:tasks.workunit.client.1.smithi026.stdout:1/914: symlink d0/d17/d1f/d63/d6a/d7c/d2e/l12a 0 2024-03-16T10:45:37.957 INFO:tasks.workunit.client.1.smithi026.stdout:5/968: dread d1/f13 [0,4194304] 0 2024-03-16T10:45:37.957 INFO:tasks.workunit.client.1.smithi026.stdout:6/968: truncate d0/d41/fa7 3984093 0 2024-03-16T10:45:37.959 INFO:tasks.workunit.client.1.smithi026.stdout:6/969: mknod d0/c143 0 2024-03-16T10:45:37.960 INFO:tasks.workunit.client.1.smithi026.stdout:1/915: creat d0/d17/d1f/d63/d6a/f12b x:0 0 0 2024-03-16T10:45:37.961 INFO:tasks.workunit.client.1.smithi026.stdout:1/916: chown d0/d17/d1f/d20/dcc 631380197 1 2024-03-16T10:45:37.962 INFO:tasks.workunit.client.1.smithi026.stdout:5/969: mknod d1/d1c/d2a/dab/dc5/d69/d51/d80/db6/dc6/c155 0 2024-03-16T10:45:37.965 INFO:tasks.workunit.client.1.smithi026.stdout:5/970: creat d1/d1c/d4c/d4d/f156 x:0 0 0 2024-03-16T10:45:37.967 INFO:tasks.workunit.client.1.smithi026.stdout:1/917: dread d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/ff9 [0,4194304] 0 2024-03-16T10:45:37.968 INFO:tasks.workunit.client.1.smithi026.stdout:1/918: fsync d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/daa/f125 0 2024-03-16T10:45:37.981 INFO:tasks.workunit.client.1.smithi026.stdout:1/919: dread d0/fd [0,4194304] 0 2024-03-16T10:45:37.984 INFO:tasks.workunit.client.1.smithi026.stdout:6/970: dwrite d0/d5b/f95 [0,4194304] 0 2024-03-16T10:45:37.986 INFO:tasks.workunit.client.1.smithi026.stdout:1/920: rmdir d0/d17/d1f/d20/d65/dde 39 2024-03-16T10:45:37.991 INFO:tasks.workunit.client.1.smithi026.stdout:8/952: sync 2024-03-16T10:45:37.994 INFO:tasks.workunit.client.1.smithi026.stdout:6/971: dwrite d0/d41/d4f/f105 [4194304,4194304] 0 2024-03-16T10:45:37.996 INFO:tasks.workunit.client.1.smithi026.stdout:5/971: dwrite d1/db/d1f/d61/f6d [0,4194304] 0 2024-03-16T10:45:37.999 INFO:tasks.workunit.client.1.smithi026.stdout:1/921: dwrite d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/fd4 [0,4194304] 0 2024-03-16T10:45:38.016 INFO:tasks.workunit.client.1.smithi026.stdout:5/972: creat d1/d1c/d2a/f157 x:0 0 0 2024-03-16T10:45:38.017 INFO:tasks.workunit.client.1.smithi026.stdout:5/973: stat d1/d1c/d2a/db0 0 2024-03-16T10:45:38.019 INFO:tasks.workunit.client.1.smithi026.stdout:8/953: truncate d5/dc/d40/d53/f58 848805 0 2024-03-16T10:45:38.023 INFO:tasks.workunit.client.1.smithi026.stdout:6/972: dwrite d0/d62/f63 [0,4194304] 0 2024-03-16T10:45:38.023 INFO:tasks.workunit.client.1.smithi026.stdout:5/974: unlink d1/db/f39 0 2024-03-16T10:45:38.024 INFO:tasks.workunit.client.1.smithi026.stdout:1/922: dwrite d0/d17/f81 [0,4194304] 0 2024-03-16T10:45:38.025 INFO:tasks.workunit.client.1.smithi026.stdout:8/954: symlink d5/d5d/l11c 0 2024-03-16T10:45:38.033 INFO:tasks.workunit.client.1.smithi026.stdout:6/973: chown d0/d28/d49/dc2/f12b 111 1 2024-03-16T10:45:38.035 INFO:tasks.workunit.client.1.smithi026.stdout:1/923: readlink d0/d17/d1f/d63/lb1 0 2024-03-16T10:45:38.037 INFO:tasks.workunit.client.1.smithi026.stdout:8/955: symlink d5/d7/de6/dd5/l11d 0 2024-03-16T10:45:38.041 INFO:tasks.workunit.client.1.smithi026.stdout:6/974: symlink d0/d41/d4f/d129/l144 0 2024-03-16T10:45:38.043 INFO:tasks.workunit.client.1.smithi026.stdout:1/924: creat d0/d17/d1f/d20/dcc/f12c x:0 0 0 2024-03-16T10:45:38.045 INFO:tasks.workunit.client.1.smithi026.stdout:6/975: fdatasync d0/d10/ff 0 2024-03-16T10:45:38.047 INFO:tasks.workunit.client.1.smithi026.stdout:1/925: mkdir d0/d17/d1f/d63/d6a/d7c/d2e/d54/d12d 0 2024-03-16T10:45:38.049 INFO:tasks.workunit.client.1.smithi026.stdout:5/975: dwrite d1/db/d1f/d61/fdc [0,4194304] 0 2024-03-16T10:45:38.053 INFO:tasks.workunit.client.1.smithi026.stdout:6/976: creat d0/d28/d49/d7d/d102/d8e/f145 x:0 0 0 2024-03-16T10:45:38.060 INFO:tasks.workunit.client.1.smithi026.stdout:5/976: rmdir d1/d1c 39 2024-03-16T10:45:38.066 INFO:tasks.workunit.client.1.smithi026.stdout:6/977: creat d0/d41/dc5/f146 x:0 0 0 2024-03-16T10:45:38.069 INFO:tasks.workunit.client.1.smithi026.stdout:1/926: creat d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/df5/f12e x:0 0 0 2024-03-16T10:45:38.070 INFO:tasks.workunit.client.1.smithi026.stdout:1/927: write d0/d17/d1f/d20/d65/f88 [652994,107401] 0 2024-03-16T10:45:38.071 INFO:tasks.workunit.client.1.smithi026.stdout:6/978: unlink d0/f89 0 2024-03-16T10:45:38.073 INFO:tasks.workunit.client.1.smithi026.stdout:5/977: creat d1/d1c/d2a/dab/dc5/d69/d51/d129/d13f/f158 x:0 0 0 2024-03-16T10:45:38.078 INFO:tasks.workunit.client.1.smithi026.stdout:5/978: chown d1/d1c/d2a/d55/d91 478318217 1 2024-03-16T10:45:38.080 INFO:tasks.workunit.client.1.smithi026.stdout:6/979: creat d0/d10/dd2/d128/f147 x:0 0 0 2024-03-16T10:45:38.081 INFO:tasks.workunit.client.1.smithi026.stdout:6/980: stat d0/d41 0 2024-03-16T10:45:38.082 INFO:tasks.workunit.client.1.smithi026.stdout:6/981: chown d0/d28/d49/dc2/f12b 860530 1 2024-03-16T10:45:38.083 INFO:tasks.workunit.client.1.smithi026.stdout:5/979: mknod d1/d1c/d2a/dab/d11d/c159 0 2024-03-16T10:45:38.087 INFO:tasks.workunit.client.1.smithi026.stdout:1/928: getdents d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4 0 2024-03-16T10:45:38.089 INFO:tasks.workunit.client.1.smithi026.stdout:6/982: creat d0/d28/d49/d7d/d102/dca/dde/f148 x:0 0 0 2024-03-16T10:45:38.091 INFO:tasks.workunit.client.1.smithi026.stdout:5/980: symlink d1/d1c/d4c/d13a/l15a 0 2024-03-16T10:45:38.098 INFO:tasks.workunit.client.1.smithi026.stdout:5/981: truncate d1/d1c/d2a/d55/f58 1723573 0 2024-03-16T10:45:38.099 INFO:tasks.workunit.client.1.smithi026.stdout:1/929: mknod d0/d17/d1f/d20/d65/dde/c12f 0 2024-03-16T10:45:38.100 INFO:tasks.workunit.client.1.smithi026.stdout:1/930: dread d0/d17/d1f/d63/f101 [0,4194304] 0 2024-03-16T10:45:38.103 INFO:tasks.workunit.client.1.smithi026.stdout:6/983: write d0/d8a/fd9 [467544,80455] 0 2024-03-16T10:45:38.104 INFO:tasks.workunit.client.1.smithi026.stdout:5/982: mkdir d1/db/d1f/d61/d12f/d15b 0 2024-03-16T10:45:38.107 INFO:tasks.workunit.client.1.smithi026.stdout:1/931: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/c130 0 2024-03-16T10:45:38.108 INFO:tasks.workunit.client.1.smithi026.stdout:1/932: readlink d0/d17/d1f/d63/d6a/d7c/d2e/d54/l76 0 2024-03-16T10:45:38.111 INFO:tasks.workunit.client.1.smithi026.stdout:6/984: mknod d0/d41/d4f/c149 0 2024-03-16T10:45:38.111 INFO:tasks.workunit.client.1.smithi026.stdout:6/985: readlink d0/d5b/laf 0 2024-03-16T10:45:38.113 INFO:tasks.workunit.client.1.smithi026.stdout:1/933: symlink d0/d17/d1f/d63/dbc/l131 0 2024-03-16T10:45:38.116 INFO:tasks.workunit.client.1.smithi026.stdout:1/934: dread d0/d17/d1f/d63/d6a/fa2 [0,4194304] 0 2024-03-16T10:45:38.117 INFO:tasks.workunit.client.1.smithi026.stdout:1/935: dread - d0/d17/d1f/d20/dcc/fee zero size 2024-03-16T10:45:38.447 INFO:tasks.workunit.client.1.smithi026.stdout:8/956: sync 2024-03-16T10:45:38.460 INFO:tasks.workunit.client.1.smithi026.stdout:8/957: dread d5/d23/d4e/d60/f84 [0,4194304] 0 2024-03-16T10:45:38.461 INFO:tasks.workunit.client.1.smithi026.stdout:8/958: chown d5/d5d/d9a/ddd/fbd 120525318 1 2024-03-16T10:45:38.464 INFO:tasks.workunit.client.1.smithi026.stdout:8/959: truncate d5/d5d/d9a/ddd/f20 4888649 0 2024-03-16T10:45:38.467 INFO:tasks.workunit.client.1.smithi026.stdout:8/960: rename d5/d23/ff4 to d5/d7/d2c/d36/d59/d93/da3/f11e 0 2024-03-16T10:45:38.472 INFO:tasks.workunit.client.1.smithi026.stdout:8/961: dwrite d5/d23/f67 [0,4194304] 0 2024-03-16T10:45:38.474 INFO:tasks.workunit.client.1.smithi026.stdout:8/962: fdatasync d5/d5d/d9a/dd6/ffc 0 2024-03-16T10:45:38.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:38 smithi026 ceph-mon[37041]: Reconfiguring prometheus.smithi002 (dependencies changed)... 2024-03-16T10:45:38.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:38 smithi026 ceph-mon[37041]: Reconfiguring daemon prometheus.smithi002 on smithi002 2024-03-16T10:45:38.485 INFO:tasks.workunit.client.1.smithi026.stdout:8/963: dread d5/dc/d40/f77 [0,4194304] 0 2024-03-16T10:45:38.486 INFO:tasks.workunit.client.1.smithi026.stdout:8/964: readlink d5/dc/d40/l42 0 2024-03-16T10:45:38.489 INFO:tasks.workunit.client.1.smithi026.stdout:8/965: creat d5/d5d/d9a/ddd/f11f x:0 0 0 2024-03-16T10:45:38.492 INFO:tasks.workunit.client.1.smithi026.stdout:8/966: rename d5/dc/d40/d53/f117 to d5/d5d/f120 0 2024-03-16T10:45:38.495 INFO:tasks.workunit.client.1.smithi026.stdout:8/967: write d5/d23/d4e/d11b/fe4 [6018234,67717] 0 2024-03-16T10:45:38.502 INFO:tasks.workunit.client.1.smithi026.stdout:5/983: sync 2024-03-16T10:45:38.515 INFO:tasks.workunit.client.1.smithi026.stdout:8/968: write d5/d7/f1a [604392,5684] 0 2024-03-16T10:45:38.521 INFO:tasks.workunit.client.1.smithi026.stdout:8/969: mknod d5/d5d/d9a/dd6/d114/c121 0 2024-03-16T10:45:38.523 INFO:tasks.workunit.client.1.smithi026.stdout:5/984: dwrite d1/db/d1f/d61/d65/f7c [0,4194304] 0 2024-03-16T10:45:38.523 INFO:tasks.workunit.client.1.smithi026.stdout:8/970: chown d5/d7/d2c/f50 2840 1 2024-03-16T10:45:38.524 INFO:tasks.workunit.client.1.smithi026.stdout:5/985: dread - d1/d6e/d9f/d14e/f153 zero size 2024-03-16T10:45:38.526 INFO:tasks.workunit.client.1.smithi026.stdout:8/971: creat d5/d23/d4e/d60/f122 x:0 0 0 2024-03-16T10:45:38.527 INFO:tasks.workunit.client.1.smithi026.stdout:8/972: chown d5/d5d/d9a/dd6/led 37724 1 2024-03-16T10:45:38.530 INFO:tasks.workunit.client.1.smithi026.stdout:5/986: mkdir d1/d1c/d2a/d55/d15c 0 2024-03-16T10:45:38.531 INFO:tasks.workunit.client.1.smithi026.stdout:8/973: truncate d5/d5d/d9a/ddd/fbd 251489 0 2024-03-16T10:45:38.532 INFO:tasks.workunit.client.1.smithi026.stdout:8/974: chown d5/dc/d40/d53/fd4 60413 1 2024-03-16T10:45:38.534 INFO:tasks.workunit.client.1.smithi026.stdout:5/987: rename d1/d1c/d2a/d55/d99/fa5 to d1/d1c/d2a/d55/d99/f15d 0 2024-03-16T10:45:38.536 INFO:tasks.workunit.client.1.smithi026.stdout:5/988: mkdir d1/db/d1f/d61/d65/d82/d15e 0 2024-03-16T10:45:38.538 INFO:tasks.workunit.client.1.smithi026.stdout:5/989: write d1/f151 [395666,62787] 0 2024-03-16T10:45:38.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:38 smithi002 ceph-mon[29248]: Reconfiguring prometheus.smithi002 (dependencies changed)... 2024-03-16T10:45:38.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:38 smithi002 ceph-mon[29248]: Reconfiguring daemon prometheus.smithi002 on smithi002 2024-03-16T10:45:38.550 INFO:tasks.workunit.client.1.smithi026.stdout:5/990: getdents d1/d1c/d2a/dab/d11d 0 2024-03-16T10:45:38.551 INFO:tasks.workunit.client.1.smithi026.stdout:5/991: chown d1/db/d1f/d61/d12f/d15b 397781378 1 2024-03-16T10:45:38.556 INFO:tasks.workunit.client.1.smithi026.stdout:8/975: dwrite d5/d7/d2c/d36/d59/d93/fab [0,4194304] 0 2024-03-16T10:45:38.564 INFO:tasks.workunit.client.1.smithi026.stdout:5/992: link d1/d1c/d2a/l56 d1/db/d1f/d61/d65/d82/dbb/dea/l15f 0 2024-03-16T10:45:38.567 INFO:tasks.workunit.client.1.smithi026.stdout:6/986: sync 2024-03-16T10:45:38.567 INFO:tasks.workunit.client.1.smithi026.stdout:1/936: sync 2024-03-16T10:45:38.570 INFO:tasks.workunit.client.1.smithi026.stdout:5/993: dread - d1/db/d1f/d61/fec zero size 2024-03-16T10:45:38.577 INFO:tasks.workunit.client.1.smithi026.stdout:6/987: rename d0/d8a/fd9 to d0/d28/d49/d7d/d102/dca/dd7/f14a 0 2024-03-16T10:45:38.579 INFO:tasks.workunit.client.1.smithi026.stdout:5/994: creat d1/d1c/d2a/dab/dc5/d69/d51/d129/f160 x:0 0 0 2024-03-16T10:45:38.579 INFO:tasks.workunit.client.1.smithi026.stdout:8/976: getdents d5/dc/d40/dfe 0 2024-03-16T10:45:38.581 INFO:tasks.workunit.client.1.smithi026.stdout:8/977: dread - d5/d7/d2c/d36/d59/d93/db0/f111 zero size 2024-03-16T10:45:38.582 INFO:tasks.workunit.client.1.smithi026.stdout:6/988: dread d0/d10/f27 [0,4194304] 0 2024-03-16T10:45:38.584 INFO:tasks.workunit.client.1.smithi026.stdout:5/995: rmdir d1/d1c/d2a/dab/dc5/d69/d51/d80/db6 39 2024-03-16T10:45:38.585 INFO:tasks.workunit.client.1.smithi026.stdout:8/978: dread d5/d7/d2c/d36/d59/d93/fab [0,4194304] 0 2024-03-16T10:45:38.590 INFO:tasks.workunit.client.1.smithi026.stdout:8/979: fsync d5/d7/d2c/d36/d59/d93/da3/fd9 0 2024-03-16T10:45:38.590 INFO:tasks.workunit.client.1.smithi026.stdout:6/989: dwrite d0/f44 [4194304,4194304] 0 2024-03-16T10:45:38.597 INFO:tasks.workunit.client.1.smithi026.stdout:6/990: mkdir d0/d28/d49/d7d/d102/dca/dd7/d14b 0 2024-03-16T10:45:38.598 INFO:tasks.workunit.client.1.smithi026.stdout:5/996: write d1/d1c/d2a/db0/fff [955051,11710] 0 2024-03-16T10:45:38.598 INFO:tasks.workunit.client.1.smithi026.stdout:6/991: fsync d0/d28/d49/d7d/d102/fee 0 2024-03-16T10:45:38.601 INFO:tasks.workunit.client.1.smithi026.stdout:8/980: mknod d5/d7/d2c/d36/d59/d93/da3/dea/d103/c123 0 2024-03-16T10:45:38.604 INFO:tasks.workunit.client.1.smithi026.stdout:8/981: readlink d5/dc/d40/la2 0 2024-03-16T10:45:38.609 INFO:tasks.workunit.client.1.smithi026.stdout:6/992: rename d0/d41/f70 to d0/d28/d49/d7d/f14c 0 2024-03-16T10:45:38.612 INFO:tasks.workunit.client.1.smithi026.stdout:8/982: dwrite d5/d7/d2c/d36/d59/d93/db0/f111 [0,4194304] 0 2024-03-16T10:45:38.614 INFO:tasks.workunit.client.1.smithi026.stdout:5/997: dread d1/db/f38 [0,4194304] 0 2024-03-16T10:45:38.614 INFO:tasks.workunit.client.1.smithi026.stdout:8/983: readlink d5/d7/d2c/d36/l47 0 2024-03-16T10:45:38.617 INFO:tasks.workunit.client.1.smithi026.stdout:8/984: fdatasync d5/d23/d4e/d60/dc8/f91 0 2024-03-16T10:45:38.618 INFO:tasks.workunit.client.1.smithi026.stdout:6/993: rename d0/d8a/db7/d132/f137 to d0/d28/d49/d7d/d102/dca/dd7/d14b/f14d 0 2024-03-16T10:45:38.623 INFO:tasks.workunit.client.1.smithi026.stdout:6/994: creat d0/dfc/f14e x:0 0 0 2024-03-16T10:45:38.626 INFO:tasks.workunit.client.1.smithi026.stdout:5/998: creat d1/db/d1f/d61/d65/f161 x:0 0 0 2024-03-16T10:45:38.629 INFO:tasks.workunit.client.1.smithi026.stdout:6/995: mknod d0/d28/d49/d7d/d102/d7b/c14f 0 2024-03-16T10:45:38.630 INFO:tasks.workunit.client.1.smithi026.stdout:8/985: getdents d5/d7/d2c/d36/d59/d93/da3 0 2024-03-16T10:45:38.632 INFO:tasks.workunit.client.1.smithi026.stdout:5/999: dwrite d1/f14c [0,4194304] 0 2024-03-16T10:45:38.638 INFO:tasks.workunit.client.1.smithi026.stdout:6/996: mkdir d0/d28/d49/db0/d112/d150 0 2024-03-16T10:45:38.638 INFO:tasks.workunit.client.1.smithi026.stdout:6/997: fsync d0/d5b/f134 0 2024-03-16T10:45:38.641 INFO:tasks.workunit.client.1.smithi026.stdout:6/998: mknod d0/d62/c151 0 2024-03-16T10:45:38.645 INFO:tasks.workunit.client.1.smithi026.stdout:6/999: unlink d0/d28/d49/d7d/d102/d7b/c86 0 2024-03-16T10:45:38.648 INFO:tasks.workunit.client.1.smithi026.stdout:8/986: dread d5/dc/f4d [4194304,4194304] 0 2024-03-16T10:45:38.652 INFO:tasks.workunit.client.1.smithi026.stdout:8/987: dread d5/d7/d2c/d36/d59/f104 [0,4194304] 0 2024-03-16T10:45:38.658 INFO:tasks.workunit.client.1.smithi026.stdout:8/988: rename d5/dc/d40/dfe/d10f/f6e to d5/d5d/d9a/f124 0 2024-03-16T10:45:38.670 INFO:tasks.workunit.client.1.smithi026.stdout:8/989: dwrite d5/d23/f61 [0,4194304] 0 2024-03-16T10:45:38.676 INFO:tasks.workunit.client.1.smithi026.stdout:8/990: dread d5/d23/d4e/d11b/fe9 [0,4194304] 0 2024-03-16T10:45:38.679 INFO:tasks.workunit.client.1.smithi026.stdout:8/991: truncate d5/dc/d40/dfe/d10f/ff3 1300900 0 2024-03-16T10:45:38.680 INFO:tasks.workunit.client.1.smithi026.stdout:8/992: chown d5/l41 113844437 1 2024-03-16T10:45:38.681 INFO:tasks.workunit.client.1.smithi026.stdout:8/993: write d5/d23/d4e/f7d [913559,100008] 0 2024-03-16T10:45:38.685 INFO:tasks.workunit.client.1.smithi026.stdout:8/994: rename d5/d7/d7b/l9f to d5/d7/d2c/d36/d59/d93/l125 0 2024-03-16T10:45:38.686 INFO:tasks.workunit.client.1.smithi026.stdout:8/995: chown d5/d7/de6 5 1 2024-03-16T10:45:38.688 INFO:tasks.workunit.client.1.smithi026.stdout:8/996: mkdir d5/d7/de6/dd5/d126 0 2024-03-16T10:45:38.693 INFO:tasks.workunit.client.1.smithi026.stdout:8/997: creat d5/dc/d40/f127 x:0 0 0 2024-03-16T10:45:38.696 INFO:tasks.workunit.client.1.smithi026.stdout:8/998: mkdir d5/d7/d2c/d36/d59/d93/da3/d128 0 2024-03-16T10:45:38.711 INFO:tasks.workunit.client.1.smithi026.stdout:8/999: dread d5/fbc [0,4194304] 0 2024-03-16T10:45:38.836 INFO:tasks.workunit.client.1.smithi026.stdout:1/937: sync 2024-03-16T10:45:38.839 INFO:tasks.workunit.client.1.smithi026.stdout:1/938: fsync d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/f126 0 2024-03-16T10:45:38.845 INFO:tasks.workunit.client.1.smithi026.stdout:1/939: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/f132 x:0 0 0 2024-03-16T10:45:38.851 INFO:tasks.workunit.client.1.smithi026.stdout:1/940: dwrite d0/d17/d1f/d63/d6a/d7c/d2e/f104 [0,4194304] 0 2024-03-16T10:45:38.854 INFO:tasks.workunit.client.1.smithi026.stdout:1/941: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/d100/d133 0 2024-03-16T10:45:38.860 INFO:tasks.workunit.client.1.smithi026.stdout:1/942: dread d0/d17/d1f/d63/d6a/d7c/d84/d9d/f4e [0,4194304] 0 2024-03-16T10:45:38.865 INFO:tasks.workunit.client.1.smithi026.stdout:1/943: link d0/d17/d1f/d20/f43 d0/d17/d1f/d63/d6a/d7c/d84/f134 0 2024-03-16T10:45:38.871 INFO:tasks.workunit.client.1.smithi026.stdout:1/944: dwrite d0/d8b/f11a [0,4194304] 0 2024-03-16T10:45:38.876 INFO:tasks.workunit.client.1.smithi026.stdout:1/945: mknod d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/df5/c135 0 2024-03-16T10:45:38.876 INFO:tasks.workunit.client.1.smithi026.stdout:1/946: fdatasync d0/d17/f48 0 2024-03-16T10:45:38.882 INFO:tasks.workunit.client.1.smithi026.stdout:1/947: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/d136 0 2024-03-16T10:45:38.885 INFO:tasks.workunit.client.1.smithi026.stdout:1/948: mknod d0/d17/d1f/d20/d10a/c137 0 2024-03-16T10:45:38.888 INFO:tasks.workunit.client.1.smithi026.stdout:1/949: symlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/l138 0 2024-03-16T10:45:38.889 INFO:tasks.workunit.client.1.smithi026.stdout:1/950: chown d0/d17/d1f/d63/fc3 1043597 1 2024-03-16T10:45:38.890 INFO:tasks.workunit.client.1.smithi026.stdout:1/951: fsync d0/d17/d1f/d20/d75/fcd 0 2024-03-16T10:45:38.904 INFO:tasks.workunit.client.1.smithi026.stdout:1/952: write d0/d17/d1f/d20/f11e [676631,109887] 0 2024-03-16T10:45:38.907 INFO:tasks.workunit.client.1.smithi026.stdout:1/953: rmdir d0/d17/db7 39 2024-03-16T10:45:38.908 INFO:tasks.workunit.client.1.smithi026.stdout:1/954: stat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/ff9 0 2024-03-16T10:45:38.910 INFO:tasks.workunit.client.1.smithi026.stdout:1/955: fsync d0/d17/d1f/d63/f101 0 2024-03-16T10:45:38.917 INFO:tasks.workunit.client.1.smithi026.stdout:1/956: creat d0/d17/d1f/d63/d6a/d7c/d84/dbb/f139 x:0 0 0 2024-03-16T10:45:38.922 INFO:tasks.workunit.client.1.smithi026.stdout:1/957: write d0/d17/d1f/d63/d6a/d7c/d3b/ffd [774201,116727] 0 2024-03-16T10:45:38.936 INFO:tasks.workunit.client.1.smithi026.stdout:1/958: dwrite d0/d17/d1f/d63/d6a/d7c/d2e/f7e [0,4194304] 0 2024-03-16T10:45:38.949 INFO:tasks.workunit.client.1.smithi026.stdout:1/959: truncate d0/d17/d1f/d63/d6a/fa2 3061576 0 2024-03-16T10:45:38.954 INFO:tasks.workunit.client.1.smithi026.stdout:1/960: link d0/d17/d1f/d20/dcc/c111 d0/c13a 0 2024-03-16T10:45:38.956 INFO:tasks.workunit.client.1.smithi026.stdout:1/961: dread - d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/f132 zero size 2024-03-16T10:45:38.960 INFO:tasks.workunit.client.1.smithi026.stdout:1/962: unlink d0/d17/d1f/d63/dbc/ff8 0 2024-03-16T10:45:38.966 INFO:tasks.workunit.client.1.smithi026.stdout:1/963: mkdir d0/d17/d1f/d20/d65/d115/d13b 0 2024-03-16T10:45:38.969 INFO:tasks.workunit.client.1.smithi026.stdout:1/964: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/f13c x:0 0 0 2024-03-16T10:45:38.973 INFO:tasks.workunit.client.1.smithi026.stdout:1/965: creat d0/d17/d1f/d20/d75/f13d x:0 0 0 2024-03-16T10:45:38.986 INFO:tasks.workunit.client.1.smithi026.stdout:1/966: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/f6e [604366,58069] 0 2024-03-16T10:45:38.991 INFO:tasks.workunit.client.1.smithi026.stdout:1/967: mkdir d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/d100/d120/d13e 0 2024-03-16T10:45:38.992 INFO:tasks.workunit.client.1.smithi026.stdout:1/968: chown d0/d17 0 1 2024-03-16T10:45:38.992 INFO:tasks.workunit.client.1.smithi026.stdout:1/969: fsync d0/fab 0 2024-03-16T10:45:38.992 INFO:tasks.workunit.client.1.smithi026.stdout:1/970: stat d0/d8b/f11a 0 2024-03-16T10:45:38.996 INFO:tasks.workunit.client.1.smithi026.stdout:1/971: creat d0/d17/d1f/d20/d10a/f13f x:0 0 0 2024-03-16T10:45:38.997 INFO:tasks.workunit.client.1.smithi026.stdout:1/972: read - d0/d17/d1f/d63/d6a/d7c/d2e/d54/d62/fdd zero size 2024-03-16T10:45:39.006 INFO:tasks.workunit.client.1.smithi026.stdout:1/973: write d0/d17/d1f/d63/d6a/d7c/d84/f134 [8103327,40361] 0 2024-03-16T10:45:39.012 INFO:tasks.workunit.client.1.smithi026.stdout:1/974: rename d0/d17/d1f/d63/d58/dc1 to d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dc8/d70/d140 0 2024-03-16T10:45:39.014 INFO:tasks.workunit.client.1.smithi026.stdout:1/975: symlink d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/l141 0 2024-03-16T10:45:39.014 INFO:tasks.workunit.client.1.smithi026.stdout:1/976: fdatasync d0/d8b/fb6 0 2024-03-16T10:45:39.017 INFO:tasks.workunit.client.1.smithi026.stdout:1/977: mkdir d0/d17/d1f/d63/d6a/d7c/d2e/d142 0 2024-03-16T10:45:39.021 INFO:tasks.workunit.client.1.smithi026.stdout:1/978: rename d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/l124 to d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6/de6/l143 0 2024-03-16T10:45:39.023 INFO:tasks.workunit.client.1.smithi026.stdout:1/979: creat d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/da0/f144 x:0 0 0 2024-03-16T10:45:39.024 INFO:tasks.workunit.client.1.smithi026.stdout:1/980: stat d0/d17/d1f/d63/d6a/c93 0 2024-03-16T10:45:39.024 INFO:tasks.workunit.client.1.smithi026.stdout:1/981: stat d0/d17/d1f/d63/d6a/d7c/f59 0 2024-03-16T10:45:39.287 INFO:tasks.workunit.client.1.smithi026.stdout:1/982: sync 2024-03-16T10:45:39.290 INFO:tasks.workunit.client.1.smithi026.stdout:1/983: creat d0/d17/d1f/d63/d6a/d7c/d2e/d117/f145 x:0 0 0 2024-03-16T10:45:39.296 INFO:tasks.workunit.client.1.smithi026.stdout:1/984: rename d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/l141 to d0/d17/d1f/d63/d6a/d7c/d2e/l146 0 2024-03-16T10:45:39.302 INFO:tasks.workunit.client.1.smithi026.stdout:1/985: dwrite d0/d17/d1f/d63/d6a/d7c/d3b/d64/d8a/dec/df6/f105 [0,4194304] 0 2024-03-16T10:45:39.310 INFO:tasks.workunit.client.1.smithi026.stdout:1/986: link d0/d17/d1f/f3f d0/d17/d1f/d63/d6a/d7c/d2e/f147 0 2024-03-16T10:45:39.312 INFO:tasks.workunit.client.1.smithi026.stdout:1/987: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/ff2 [502223,18439] 0 2024-03-16T10:45:39.313 INFO:tasks.workunit.client.1.smithi026.stdout:1/988: fsync d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dff/f121 0 2024-03-16T10:45:39.315 INFO:tasks.workunit.client.1.smithi026.stdout:1/989: chown d0/d17/d1f/d63/d6a/d7c/d84/d9d/dc6 5 1 2024-03-16T10:45:39.315 INFO:tasks.workunit.client.1.smithi026.stdout:1/990: chown d0/d17/d1f/deb/f116 60 1 2024-03-16T10:45:39.318 INFO:tasks.workunit.client.1.smithi026.stdout:1/991: mknod d0/d17/d1f/d20/dcc/c148 0 2024-03-16T10:45:39.320 INFO:tasks.workunit.client.1.smithi026.stdout:1/992: write d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/f13c [639310,57220] 0 2024-03-16T10:45:39.321 INFO:tasks.workunit.client.1.smithi026.stdout:1/993: read d0/d17/d1f/d63/fc3 [29048,103579] 0 2024-03-16T10:45:39.323 INFO:tasks.workunit.client.1.smithi026.stdout:1/994: write d0/d17/d1f/d63/d6a/d7c/d2e/d54/fd1 [544737,75654] 0 2024-03-16T10:45:39.323 INFO:tasks.workunit.client.1.smithi026.stdout:1/995: chown d0/d17/c4c 20 1 2024-03-16T10:45:39.328 INFO:tasks.workunit.client.1.smithi026.stdout:1/996: dread d0/d17/d1f/d63/d6a/d7c/d2e/faf [0,4194304] 0 2024-03-16T10:45:39.349 INFO:tasks.workunit.client.1.smithi026.stdout:1/997: dwrite d0/d17/d1f/d63/d6a/fa2 [0,4194304] 0 2024-03-16T10:45:39.354 INFO:tasks.workunit.client.1.smithi026.stdout:1/998: creat d0/d17/d1f/d63/d6a/d7c/d84/d9d/da4/dff/f149 x:0 0 0 2024-03-16T10:45:39.358 INFO:tasks.workunit.client.1.smithi026.stdout:1/999: creat d0/d17/d1f/d63/d6a/d7c/d2e/d54/f14a x:0 0 0 2024-03-16T10:45:39.362 INFO:tasks.workunit.client.1.smithi026.stderr:+ rm -rf -- ./tmp.cGmekZRYkU 2024-03-16T10:45:39.484 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:39 smithi002 ceph-mon[29248]: pgmap v14: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 69 MiB/s rd, 155 MiB/s wr, 415 op/s 2024-03-16T10:45:39.581 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:39 smithi026 ceph-mon[37041]: pgmap v14: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 69 MiB/s rd, 155 MiB/s wr, 415 op/s 2024-03-16T10:45:40.333 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-16T10:45:40.333 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T10:45:41.533 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:41 smithi002 ceph-mon[29248]: pgmap v15: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 52 MiB/s rd, 104 MiB/s wr, 287 op/s 2024-03-16T10:45:41.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:41 smithi026 ceph-mon[37041]: pgmap v15: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 52 MiB/s rd, 104 MiB/s wr, 287 op/s 2024-03-16T10:45:42.499 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:42 smithi002 ceph-mon[29248]: pgmap v16: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 109 MiB/s wr, 306 op/s 2024-03-16T10:45:42.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:42 smithi026 ceph-mon[37041]: pgmap v16: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 109 MiB/s wr, 306 op/s 2024-03-16T10:45:44.640 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:44 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:44.640 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:44 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:44.640 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:44 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:45:44.640 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:44 smithi026 ceph-mon[37041]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:45:44.640 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:44 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:44.640 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:44 smithi026 ceph-mon[37041]: pgmap v17: 65 pgs: 65 active+clean; 2.4 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 52 MiB/s rd, 108 MiB/s wr, 351 op/s 2024-03-16T10:45:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:44 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:44 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:44 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:45:44.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:44 smithi002 ceph-mon[29248]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:45:44.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:44 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:44.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:44 smithi002 ceph-mon[29248]: pgmap v17: 65 pgs: 65 active+clean; 2.4 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 52 MiB/s rd, 108 MiB/s wr, 351 op/s 2024-03-16T10:45:46.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:46 smithi026 ceph-mon[37041]: pgmap v18: 65 pgs: 65 active+clean; 2.4 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 31 MiB/s rd, 72 MiB/s wr, 227 op/s 2024-03-16T10:45:46.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:46 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:47.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:46 smithi002 ceph-mon[29248]: pgmap v18: 65 pgs: 65 active+clean; 2.4 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 31 MiB/s rd, 72 MiB/s wr, 227 op/s 2024-03-16T10:45:47.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:46 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:45:48.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:48 smithi026 ceph-mon[37041]: pgmap v19: 65 pgs: 65 active+clean; 1.7 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 31 MiB/s rd, 72 MiB/s wr, 300 op/s 2024-03-16T10:45:48.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:48 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:45:48.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:48 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:45:48.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:48 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:45:49.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:48 smithi002 ceph-mon[29248]: pgmap v19: 65 pgs: 65 active+clean; 1.7 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 31 MiB/s rd, 72 MiB/s wr, 300 op/s 2024-03-16T10:45:49.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:48 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:45:49.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:48 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:45:49.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:48 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:45:49.703 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:49 smithi026 ceph-mon[37041]: Upgrade: Updating mgr.smithi026.omjkpo 2024-03-16T10:45:49.703 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:49 smithi026 ceph-mon[37041]: Deploying daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:45:50.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:49 smithi002 ceph-mon[29248]: Upgrade: Updating mgr.smithi026.omjkpo 2024-03-16T10:45:50.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:49 smithi002 ceph-mon[29248]: Deploying daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:45:50.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:50 smithi026 ceph-mon[37041]: pgmap v20: 65 pgs: 65 active+clean; 1.7 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 167 op/s 2024-03-16T10:45:51.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:50 smithi002 ceph-mon[29248]: pgmap v20: 65 pgs: 65 active+clean; 1.7 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 167 op/s 2024-03-16T10:45:52.649 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:52 smithi026 ceph-mon[37041]: pgmap v21: 65 pgs: 65 active+clean; 1.5 GiB data, 4.9 GiB used, 531 GiB / 536 GiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 176 op/s 2024-03-16T10:45:52.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:52 smithi002 ceph-mon[29248]: pgmap v21: 65 pgs: 65 active+clean; 1.5 GiB data, 4.9 GiB used, 531 GiB / 536 GiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 176 op/s 2024-03-16T10:45:54.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:54 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:54.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:54 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:54.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:54 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:54.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:54 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:54.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:54 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:54.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:54 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:54.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:54 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:54.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:54 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:45:54.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:54 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:54.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:54 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:55.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:55 smithi026 ceph-mon[37041]: pgmap v22: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 204 op/s 2024-03-16T10:45:55.538 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:55 smithi002 ceph-mon[29248]: pgmap v22: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 204 op/s 2024-03-16T10:45:57.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:57 smithi002 ceph-mon[29248]: pgmap v23: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-16T10:45:57.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:57 smithi026 ceph-mon[37041]: pgmap v23: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-16T10:45:58.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:58 smithi002 ceph-mon[29248]: pgmap v24: 65 pgs: 65 active+clean; 741 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-03-16T10:45:58.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:58 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:58.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:45:58 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:58.927 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:58 smithi026 ceph-mon[37041]: pgmap v24: 65 pgs: 65 active+clean; 741 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 38 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-03-16T10:45:58.927 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:58 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:58.927 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:45:58 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:45:59.569 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-16T10:45:59.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-16T10:46:00.201 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:00 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:00.201 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:00 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:00.201 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:00 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:00.201 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:00 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:00.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:00 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:00.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:00 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:00.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:00 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:00.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:00 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:01 smithi002 ceph-mon[29248]: pgmap v25: 65 pgs: 65 active+clean; 741 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-03-16T10:46:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:01 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:01 smithi026 ceph-mon[37041]: pgmap v25: 65 pgs: 65 active+clean; 741 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-03-16T10:46:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:01 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:02.678 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:02 smithi002 ceph-mon[29248]: pgmap v26: 65 pgs: 65 active+clean; 609 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-03-16T10:46:02.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:02 smithi026 ceph-mon[37041]: pgmap v26: 65 pgs: 65 active+clean; 609 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-03-16T10:46:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:03 smithi026 ceph-mon[37041]: Standby manager daemon smithi026.omjkpo restarted 2024-03-16T10:46:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:03 smithi026 ceph-mon[37041]: Standby manager daemon smithi026.omjkpo started 2024-03-16T10:46:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:03 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:46:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:03 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:46:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:03 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:46:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:03 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 -- 172.21.15.2:0/3086904948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 msgr2=0x7fe734106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/3086904948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe734106280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe728009910 tx=0x7fe728009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 -- 172.21.15.2:0/3086904948 shutdown_connections 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/3086904948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe734106280 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/3086904948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe734101560 0x7fe734103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 -- 172.21.15.2:0/3086904948 >> 172.21.15.2:0/3086904948 conn(0x7fe7340faf70 msgr2=0x7fe7340fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 -- 172.21.15.2:0/3086904948 shutdown_connections 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 -- 172.21.15.2:0/3086904948 wait complete. 2024-03-16T10:46:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.561+0000 7fe73a2ea700 1 Processor -- start 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe73a2ea700 1 -- start start 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe73a2ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe734101560 0x7fe734196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe73a2ea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe7341969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe73a2ea700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe734196ff0 con 0x7fe734103e90 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe73a2ea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe734197130 con 0x7fe734101560 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe7337fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe7341969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe7337fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe7341969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41152/0 (socket says 172.21.15.2:41152) 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe7337fe700 1 -- 172.21.15.2:0/2369345475 learned_addr learned my addr 172.21.15.2:0/2369345475 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe733fff700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe734101560 0x7fe734196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe7337fe700 1 -- 172.21.15.2:0/2369345475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe734101560 msgr2=0x7fe734196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe7337fe700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe734101560 0x7fe734196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.562+0000 7fe7337fe700 1 -- 172.21.15.2:0/2369345475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7280095f0 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.563+0000 7fe7337fe700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe7341969d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe72800b800 tx=0x7fe728005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.563+0000 7fe7317fa700 1 -- 172.21.15.2:0/2369345475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72801d070 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.563+0000 7fe7317fa700 1 -- 172.21.15.2:0/2369345475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7280052a0 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.563+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73419bb30 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.563+0000 7fe7317fa700 1 -- 172.21.15.2:0/2369345475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe728017400 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.563+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73419c0a0 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.564+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73404ea80 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.564+0000 7fe7317fa700 1 -- 172.21.15.2:0/2369345475 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 32) v1 ==== 91794+0+0 (secure 0 0 0) 0x7fe728003780 con 0x7fe734103e90 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.565+0000 7fe7317fa700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe71c06dd60 0x7fe71c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.565+0000 7fe7317fa700 1 -- 172.21.15.2:0/2369345475 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe72808edb0 con 0x7fe734103e90 2024-03-16T10:46:03.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.565+0000 7fe733fff700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe71c06dd60 0x7fe71c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:03.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.565+0000 7fe733fff700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe71c06dd60 0x7fe71c070220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe724009c60 tx=0x7fe724009260 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:03.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.570+0000 7fe7317fa700 1 -- 172.21.15.2:0/2369345475 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fe72805c090 con 0x7fe734103e90 2024-03-16T10:46:03.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:03 smithi002 ceph-mon[29248]: Standby manager daemon smithi026.omjkpo restarted 2024-03-16T10:46:03.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:03 smithi002 ceph-mon[29248]: Standby manager daemon smithi026.omjkpo started 2024-03-16T10:46:03.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:03 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:46:03.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:03 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:46:03.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:03 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:46:03.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:03 smithi002 ceph-mon[29248]: from='mgr.? 172.21.15.26:0/2928623939' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:46:03.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.808+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe734105d70 con 0x7fe71c06dd60 2024-03-16T10:46:03.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.810+0000 7fe7317fa700 1 -- 172.21.15.2:0/2369345475 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe734105d70 con 0x7fe71c06dd60 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe71c06dd60 msgr2=0x7fe71c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe71c06dd60 0x7fe71c070220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe724009c60 tx=0x7fe724009260 comp rx=0 tx=0).stop 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 msgr2=0x7fe7341969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe7341969d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe72800b800 tx=0x7fe728005f00 comp rx=0 tx=0).stop 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 shutdown_connections 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fe71c06dd60 0x7fe71c070220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe734101560 0x7fe734196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 --2- 172.21.15.2:0/2369345475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe734103e90 0x7fe7341969d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 >> 172.21.15.2:0/2369345475 conn(0x7fe7340faf70 msgr2=0x7fe7340fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 shutdown_connections 2024-03-16T10:46:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.812+0000 7fe73a2ea700 1 -- 172.21.15.2:0/2369345475 wait complete. 2024-03-16T10:46:03.823 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:46:03.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/1510812055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 msgr2=0x7f6e44102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:03.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/1510812055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44102130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009910 tx=0x7f6e34009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:03.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/1510812055 shutdown_connections 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/1510812055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44102130 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/1510812055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e44100ad0 0x7f6e44100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/1510812055 >> 172.21.15.2:0/1510812055 conn(0x7f6e440fc090 msgr2=0x7f6e440fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/1510812055 shutdown_connections 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/1510812055 wait complete. 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 Processor -- start 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.923+0000 7f6e4b8ed700 1 -- start start 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e4b8ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e44100ad0 0x7f6e44196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e4b8ed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e4b8ed700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44197080 con 0x7f6e44101cd0 2024-03-16T10:46:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e4b8ed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e441971c0 con 0x7f6e44100ad0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e48e88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e48e88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41178/0 (socket says 172.21.15.2:41178) 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e48e88700 1 -- 172.21.15.2:0/2754699076 learned_addr learned my addr 172.21.15.2:0/2754699076 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e49689700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e44100ad0 0x7f6e44196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e48e88700 1 -- 172.21.15.2:0/2754699076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e44100ad0 msgr2=0x7f6e44196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e48e88700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e44100ad0 0x7f6e44196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e48e88700 1 -- 172.21.15.2:0/2754699076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340095f0 con 0x7f6e44101cd0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e48e88700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44196a60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400b650 tx=0x7f6e34005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/2754699076 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3401d070 con 0x7f6e44101cd0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/2754699076 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e340052a0 con 0x7f6e44101cd0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4419bbc0 con 0x7f6e44101cd0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/2754699076 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e34017400 con 0x7f6e44101cd0 2024-03-16T10:46:03.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.924+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4419c130 con 0x7f6e44101cd0 2024-03-16T10:46:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.925+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e44190650 con 0x7f6e44101cd0 2024-03-16T10:46:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.926+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/2754699076 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 32) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f6e34003810 con 0x7f6e44101cd0 2024-03-16T10:46:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.926+0000 7f6e3a7fc700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f6e3006dd50 0x7f6e30070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.927+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/2754699076 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6e3408ee50 con 0x7f6e44101cd0 2024-03-16T10:46:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.927+0000 7f6e49689700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f6e3006dd50 0x7f6e30070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.927+0000 7f6e49689700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f6e3006dd50 0x7f6e30070210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e44101b30 tx=0x7f6e40009260 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:03.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:03.931+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/2754699076 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f6e3405c010 con 0x7f6e44101cd0 2024-03-16T10:46:04.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.187+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e440611f0 con 0x7f6e3006dd50 2024-03-16T10:46:04.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.190+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/2754699076 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f6e440611f0 con 0x7f6e3006dd50 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.194+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f6e3006dd50 msgr2=0x7f6e30070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.194+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f6e3006dd50 0x7f6e30070210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e44101b30 tx=0x7f6e40009260 comp rx=0 tx=0).stop 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.194+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 msgr2=0x7f6e44196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.194+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44196a60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400b650 tx=0x7f6e34005f00 comp rx=0 tx=0).stop 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.195+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 shutdown_connections 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.195+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f6e3006dd50 0x7f6e30070210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.195+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e44100ad0 0x7f6e44196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.195+0000 7f6e4b8ed700 1 --2- 172.21.15.2:0/2754699076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e44101cd0 0x7f6e44196a60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.195+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 >> 172.21.15.2:0/2754699076 conn(0x7f6e440fc090 msgr2=0x7f6e44105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.195+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 shutdown_connections 2024-03-16T10:46:04.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.195+0000 7f6e4b8ed700 1 -- 172.21.15.2:0/2754699076 wait complete. 2024-03-16T10:46:04.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 -- 172.21.15.2:0/2757371053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 msgr2=0x7f08680ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2757371053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 0x7f08680ffdf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0858009910 tx=0x7f0858009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:04.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 -- 172.21.15.2:0/2757371053 shutdown_connections 2024-03-16T10:46:04.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2757371053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 0x7f0868101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2757371053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 0x7f08680ffdf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 -- 172.21.15.2:0/2757371053 >> 172.21.15.2:0/2757371053 conn(0x7f08680faf70 msgr2=0x7f08680fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:04.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 -- 172.21.15.2:0/2757371053 shutdown_connections 2024-03-16T10:46:04.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.353+0000 7f086d6d0700 1 -- 172.21.15.2:0/2757371053 wait complete. 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.354+0000 7f086d6d0700 1 Processor -- start 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.354+0000 7f086d6d0700 1 -- start start 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.354+0000 7f086d6d0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 0x7f0868198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.354+0000 7f086d6d0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 0x7f0868198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.354+0000 7f086d6d0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0868199290 con 0x7f08680ff9d0 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.354+0000 7f086d6d0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681993d0 con 0x7f0868100bd0 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f08667fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 0x7f0868198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f08667fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 0x7f0868198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49746/0 (socket says 172.21.15.2:49746) 2024-03-16T10:46:04.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f08667fc700 1 -- 172.21.15.2:0/2866565404 learned_addr learned my addr 172.21.15.2:0/2866565404 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:04.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f0866ffd700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 0x7f0868198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:04.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f08667fc700 1 -- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 msgr2=0x7f0868198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f08667fc700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 0x7f0868198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f08667fc700 1 -- 172.21.15.2:0/2866565404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580095f0 con 0x7f0868100bd0 2024-03-16T10:46:04.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f0866ffd700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 0x7f0868198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:46:04.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.355+0000 7f08667fc700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 0x7f0868198c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e920 tx=0x7f085c00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:04.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.357+0000 7f084ffff700 1 -- 172.21.15.2:0/2866565404 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085c00c9d0 con 0x7f0868100bd0 2024-03-16T10:46:04.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.357+0000 7f084ffff700 1 -- 172.21.15.2:0/2866565404 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f085c010430 con 0x7f0868100bd0 2024-03-16T10:46:04.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.357+0000 7f084ffff700 1 -- 172.21.15.2:0/2866565404 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085c018630 con 0x7f0868100bd0 2024-03-16T10:46:04.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.357+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086819de30 con 0x7f0868100bd0 2024-03-16T10:46:04.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.357+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086819e310 con 0x7f0868100bd0 2024-03-16T10:46:04.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.359+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086804ea80 con 0x7f0868100bd0 2024-03-16T10:46:04.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.359+0000 7f084ffff700 1 -- 172.21.15.2:0/2866565404 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f085c0188a0 con 0x7f0868100bd0 2024-03-16T10:46:04.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.360+0000 7f084ffff700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f085006dd50 0x7f0850070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:04.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.360+0000 7f0866ffd700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f085006dd50 0x7f0850070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:04.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.360+0000 7f0866ffd700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f085006dd50 0x7f0850070210 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f085800b6f0 tx=0x7f0858005fb0 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:04.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.361+0000 7f084ffff700 1 -- 172.21.15.2:0/2866565404 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f085c014070 con 0x7f0868100bd0 2024-03-16T10:46:04.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.367+0000 7f084ffff700 1 -- 172.21.15.2:0/2866565404 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f085c0534c0 con 0x7f0868100bd0 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: mgrmap e32: smithi002.zevrtm(active, since 47s), standbys: smithi026.omjkpo 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: pgmap v27: 65 pgs: 65 active+clean; 351 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:04.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: Upgrade: Setting container_image for all mgr 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi002.zevrtm"}]: dispatch 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi002.zevrtm"}]': finished 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi026.omjkpo"}]: dispatch 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi026.omjkpo"}]': finished 2024-03-16T10:46:04.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:04 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T10:46:04.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.600+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0868068be0 con 0x7f085006dd50 2024-03-16T10:46:04.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.613+0000 7f084ffff700 1 -- 172.21.15.2:0/2866565404 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0868068be0 con 0x7f085006dd50 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (8m) 5s ago 9m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (9m) 5s ago 9m 7726k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (8m) 7s ago 8m 7776k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (9m) 5s ago 9m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (8m) 7s ago 8m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (8m) 5s ago 9m 87.4M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:46:04.614 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (4m) 5s ago 4m 16.8M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:46:04.617 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (4m) 5s ago 4m 13.9M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (4m) 7s ago 4m 247M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (3m) 7s ago 3m 15.7M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (77s) 5s ago 11m 495M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (12s) 7s ago 8m 62.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (11m) 5s ago 11m 53.8M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 3e28a22ab682 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (8m) 7s ago 8m 40.4M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (9m) 5s ago 9m 17.2M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (8m) 7s ago 8m 15.5M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (7m) 5s ago 7m 175M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (7m) 5s ago 7m 174M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (6m) 5s ago 6m 150M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (6m) 7s ago 6m 211M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (6m) 7s ago 6m 178M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:46:04.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (5m) 7s ago 5m 172M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:46:04.619 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (21s) 5s ago 9m 49.9M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:46:04.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f085006dd50 msgr2=0x7f0850070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f085006dd50 0x7f0850070210 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f085800b6f0 tx=0x7f0858005fb0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 msgr2=0x7f0868198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 0x7f0868198c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e920 tx=0x7f085c00ec30 comp rx=0 tx=0).stop 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 shutdown_connections 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f085006dd50 0x7f0850070210 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ff9d0 0x7f0868198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 --2- 172.21.15.2:0/2866565404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0868100bd0 0x7f0868198c70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 >> 172.21.15.2:0/2866565404 conn(0x7f08680faf70 msgr2=0x7f08680684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 shutdown_connections 2024-03-16T10:46:04.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.615+0000 7f086d6d0700 1 -- 172.21.15.2:0/2866565404 wait complete. 2024-03-16T10:46:04.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: mgrmap e32: smithi002.zevrtm(active, since 47s), standbys: smithi026.omjkpo 2024-03-16T10:46:04.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: pgmap v27: 65 pgs: 65 active+clean; 351 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: Upgrade: Setting container_image for all mgr 2024-03-16T10:46:04.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:04.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi002.zevrtm"}]: dispatch 2024-03-16T10:46:04.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi002.zevrtm"}]': finished 2024-03-16T10:46:04.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi026.omjkpo"}]: dispatch 2024-03-16T10:46:04.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi026.omjkpo"}]': finished 2024-03-16T10:46:04.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:04 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 -- 172.21.15.2:0/265998261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 msgr2=0x7f3c14106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 --2- 172.21.15.2:0/265998261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c14106280 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009910 tx=0x7f3c04009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 -- 172.21.15.2:0/265998261 shutdown_connections 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 --2- 172.21.15.2:0/265998261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c14106280 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 --2- 172.21.15.2:0/265998261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c14101560 0x7f3c14103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 -- 172.21.15.2:0/265998261 >> 172.21.15.2:0/265998261 conn(0x7f3c140faf70 msgr2=0x7f3c140fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 -- 172.21.15.2:0/265998261 shutdown_connections 2024-03-16T10:46:04.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.733+0000 7f3c1c653700 1 -- 172.21.15.2:0/265998261 wait complete. 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c1c653700 1 Processor -- start 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c1c653700 1 -- start start 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c1c653700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c14101560 0x7f3c14196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c1c653700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c141969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c1c653700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14196ff0 con 0x7f3c14103e90 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c1c653700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14197130 con 0x7f3c14101560 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c19bee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c141969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c19bee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c141969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41228/0 (socket says 172.21.15.2:41228) 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c19bee700 1 -- 172.21.15.2:0/4092622037 learned_addr learned my addr 172.21.15.2:0/4092622037 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.734+0000 7f3c1a3ef700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c14101560 0x7f3c14196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c19bee700 1 -- 172.21.15.2:0/4092622037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c14101560 msgr2=0x7f3c14196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c19bee700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c14101560 0x7f3c14196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c19bee700 1 -- 172.21.15.2:0/4092622037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c040095f0 con 0x7f3c14103e90 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c1a3ef700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c14101560 0x7f3c14196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c19bee700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c141969d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3c04005f30 tx=0x7f3c04005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c0b7fe700 1 -- 172.21.15.2:0/4092622037 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c0401d070 con 0x7f3c14103e90 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1419bb30 con 0x7f3c14103e90 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c0b7fe700 1 -- 172.21.15.2:0/4092622037 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c040052a0 con 0x7f3c14103e90 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c0b7fe700 1 -- 172.21.15.2:0/4092622037 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c04017400 con 0x7f3c14103e90 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.735+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1419c0a0 con 0x7f3c14103e90 2024-03-16T10:46:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.736+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1404ea80 con 0x7f3c14103e90 2024-03-16T10:46:04.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.737+0000 7f3c0b7fe700 1 -- 172.21.15.2:0/4092622037 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 32) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f3c040176c0 con 0x7f3c14103e90 2024-03-16T10:46:04.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.737+0000 7f3c0b7fe700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f3c0006dd60 0x7f3c00070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:04.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.738+0000 7f3c0b7fe700 1 -- 172.21.15.2:0/4092622037 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3c0408ee40 con 0x7f3c14103e90 2024-03-16T10:46:04.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.738+0000 7f3c1a3ef700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f3c0006dd60 0x7f3c00070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:04.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.738+0000 7f3c1a3ef700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f3c0006dd60 0x7f3c00070220 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3c10011490 tx=0x7f3c10009260 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:04.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:04.742+0000 7f3c0b7fe700 1 -- 172.21.15.2:0/4092622037 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f3c0405c0a0 con 0x7f3c14103e90 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.068+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3c1419c380 con 0x7f3c14103e90 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.069+0000 7f3c0b7fe700 1 -- 172.21.15.2:0/4092622037 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f3c0405bc30 con 0x7f3c14103e90 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:46:05.070 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 12, 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:46:05.071 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:46:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f3c0006dd60 msgr2=0x7f3c00070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f3c0006dd60 0x7f3c00070220 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3c10011490 tx=0x7f3c10009260 comp rx=0 tx=0).stop 2024-03-16T10:46:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 msgr2=0x7f3c141969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c141969d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3c04005f30 tx=0x7f3c04005960 comp rx=0 tx=0).stop 2024-03-16T10:46:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 shutdown_connections 2024-03-16T10:46:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f3c0006dd60 0x7f3c00070220 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c14101560 0x7f3c14196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 --2- 172.21.15.2:0/4092622037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c14103e90 0x7f3c141969d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 >> 172.21.15.2:0/4092622037 conn(0x7f3c140faf70 msgr2=0x7f3c140fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 shutdown_connections 2024-03-16T10:46:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.071+0000 7f3c1c653700 1 -- 172.21.15.2:0/4092622037 wait complete. 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 -- 172.21.15.2:0/3415439546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff9d0 msgr2=0x7fcffc0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 --2- 172.21.15.2:0/3415439546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff9d0 0x7fcffc0ffdf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009910 tx=0x7fcff8009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 -- 172.21.15.2:0/3415439546 shutdown_connections 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 --2- 172.21.15.2:0/3415439546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc100bd0 0x7fcffc101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 --2- 172.21.15.2:0/3415439546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff9d0 0x7fcffc0ffdf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 -- 172.21.15.2:0/3415439546 >> 172.21.15.2:0/3415439546 conn(0x7fcffc0faf70 msgr2=0x7fcffc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 -- 172.21.15.2:0/3415439546 shutdown_connections 2024-03-16T10:46:05.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.188+0000 7fd003b49700 1 -- 172.21.15.2:0/3415439546 wait complete. 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd003b49700 1 Processor -- start 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd003b49700 1 -- start start 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd003b49700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc0ff9d0 0x7fcffc1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd003b49700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc100bd0 0x7fcffc196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd003b49700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc197040 con 0x7fcffc100bd0 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd003b49700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc197180 con 0x7fcffc0ff9d0 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd0010e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc100bd0 0x7fcffc196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd0010e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc100bd0 0x7fcffc196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41242/0 (socket says 172.21.15.2:41242) 2024-03-16T10:46:05.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd0010e4700 1 -- 172.21.15.2:0/1034998038 learned_addr learned my addr 172.21.15.2:0/1034998038 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd0018e5700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc0ff9d0 0x7fcffc1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd0010e4700 1 -- 172.21.15.2:0/1034998038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc0ff9d0 msgr2=0x7fcffc1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd0010e4700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc0ff9d0 0x7fcffc1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.189+0000 7fd0010e4700 1 -- 172.21.15.2:0/1034998038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff80095f0 con 0x7fcffc100bd0 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.190+0000 7fd0010e4700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc100bd0 0x7fcffc196a20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00e920 tx=0x7fcfec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.190+0000 7fcff2ffd700 1 -- 172.21.15.2:0/1034998038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfec00c9d0 con 0x7fcffc100bd0 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.190+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc19bbe0 con 0x7fcffc100bd0 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.190+0000 7fcff2ffd700 1 -- 172.21.15.2:0/1034998038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfec010430 con 0x7fcffc100bd0 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.190+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc19c1b0 con 0x7fcffc100bd0 2024-03-16T10:46:05.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.190+0000 7fcff2ffd700 1 -- 172.21.15.2:0/1034998038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfec018630 con 0x7fcffc100bd0 2024-03-16T10:46:05.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.191+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe00052f0 con 0x7fcffc100bd0 2024-03-16T10:46:05.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.191+0000 7fcff2ffd700 1 -- 172.21.15.2:0/1034998038 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 32) v1 ==== 91794+0+0 (secure 0 0 0) 0x7fcfec0188a0 con 0x7fcffc100bd0 2024-03-16T10:46:05.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.192+0000 7fcff2ffd700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fcfe806dd60 0x7fcfe8070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:05.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.192+0000 7fcff2ffd700 1 -- 172.21.15.2:0/1034998038 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcfec014070 con 0x7fcffc100bd0 2024-03-16T10:46:05.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.192+0000 7fd0018e5700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fcfe806dd60 0x7fcfe8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:05.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.193+0000 7fd0018e5700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fcfe806dd60 0x7fcfe8070220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcff800b800 tx=0x7fcff8009f70 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:05.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.200+0000 7fcff2ffd700 1 -- 172.21.15.2:0/1034998038 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7fcfec0534c0 con 0x7fcffc100bd0 2024-03-16T10:46:05.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.520+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcfe0005c90 con 0x7fcffc100bd0 2024-03-16T10:46:05.521 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:05 smithi002 ceph-mon[29248]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:05.521 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:05 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/4092622037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.520+0000 7fcff2ffd700 1 -- 172.21.15.2:0/1034998038 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fcfec05ae80 con 0x7fcffc100bd0 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:46:05.522 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:46:05.523 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:05.524 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:05.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fcfe806dd60 msgr2=0x7fcfe8070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fcfe806dd60 0x7fcfe8070220 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcff800b800 tx=0x7fcff8009f70 comp rx=0 tx=0).stop 2024-03-16T10:46:05.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc100bd0 msgr2=0x7fcffc196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc100bd0 0x7fcffc196a20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00e920 tx=0x7fcfec00ec30 comp rx=0 tx=0).stop 2024-03-16T10:46:05.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 shutdown_connections 2024-03-16T10:46:05.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7fcfe806dd60 0x7fcfe8070220 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc0ff9d0 0x7fcffc1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 --2- 172.21.15.2:0/1034998038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc100bd0 0x7fcffc196a20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.523+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 >> 172.21.15.2:0/1034998038 conn(0x7fcffc0faf70 msgr2=0x7fcffc068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:05.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.524+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 shutdown_connections 2024-03-16T10:46:05.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.524+0000 7fd003b49700 1 -- 172.21.15.2:0/1034998038 wait complete. 2024-03-16T10:46:05.526 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:46:05.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 -- 172.21.15.2:0/4007952579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac101560 msgr2=0x7f83ac103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 --2- 172.21.15.2:0/4007952579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac101560 0x7f83ac103950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f839c009910 tx=0x7f839c009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:05.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 -- 172.21.15.2:0/4007952579 shutdown_connections 2024-03-16T10:46:05.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 --2- 172.21.15.2:0/4007952579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac103e90 0x7f83ac106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 --2- 172.21.15.2:0/4007952579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac101560 0x7f83ac103950 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 -- 172.21.15.2:0/4007952579 >> 172.21.15.2:0/4007952579 conn(0x7f83ac0faf70 msgr2=0x7f83ac0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:05.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 -- 172.21.15.2:0/4007952579 shutdown_connections 2024-03-16T10:46:05.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.635+0000 7f83b28c3700 1 -- 172.21.15.2:0/4007952579 wait complete. 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.636+0000 7f83b28c3700 1 Processor -- start 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.636+0000 7f83b28c3700 1 -- start start 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.636+0000 7f83b28c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac101560 0x7f83ac196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.636+0000 7f83b28c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103e90 0x7f83ac1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.636+0000 7f83b28c3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac196ff0 con 0x7f83ac103e90 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.636+0000 7f83b28c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ac197130 con 0x7f83ac101560 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac101560 0x7f83ac196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83ab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103e90 0x7f83ac1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac101560 0x7f83ac196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49806/0 (socket says 172.21.15.2:49806) 2024-03-16T10:46:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83abfff700 1 -- 172.21.15.2:0/3667954068 learned_addr learned my addr 172.21.15.2:0/3667954068 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83abfff700 1 -- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103e90 msgr2=0x7f83ac1969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83abfff700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103e90 0x7f83ac1969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83abfff700 1 -- 172.21.15.2:0/3667954068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0095f0 con 0x7f83ac101560 2024-03-16T10:46:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83abfff700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac101560 0x7f83ac196490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f839c00b800 tx=0x7f839c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83a97fa700 1 -- 172.21.15.2:0/3667954068 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f839c01d070 con 0x7f83ac101560 2024-03-16T10:46:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac19bb30 con 0x7f83ac101560 2024-03-16T10:46:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83a97fa700 1 -- 172.21.15.2:0/3667954068 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f839c0052a0 con 0x7f83ac101560 2024-03-16T10:46:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83a97fa700 1 -- 172.21.15.2:0/3667954068 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f839c017400 con 0x7f83ac101560 2024-03-16T10:46:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.637+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ac19c020 con 0x7f83ac101560 2024-03-16T10:46:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.639+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ac04ea80 con 0x7f83ac101560 2024-03-16T10:46:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.639+0000 7f83a97fa700 1 -- 172.21.15.2:0/3667954068 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f839c003780 con 0x7f83ac101560 2024-03-16T10:46:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.640+0000 7f83a97fa700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f839406dd60 0x7f8394070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.640+0000 7f83ab7fe700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f839406dd60 0x7f8394070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.640+0000 7f83a97fa700 1 -- 172.21.15.2:0/3667954068 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f839c08ef50 con 0x7f83ac101560 2024-03-16T10:46:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.640+0000 7f83ab7fe700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f839406dd60 0x7f8394070220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009c60 tx=0x7f83a0009260 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:05.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.648+0000 7f83a97fa700 1 -- 172.21.15.2:0/3667954068 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f839c05c110 con 0x7f83ac101560 2024-03-16T10:46:05.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:05 smithi026 ceph-mon[37041]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:05.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:05 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/4092622037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:46:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.890+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 --> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f83ac105d70 con 0x7f839406dd60 2024-03-16T10:46:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.892+0000 7f83a97fa700 1 -- 172.21.15.2:0/3667954068 <== mgr.14712 v2:172.21.15.2:6800/730693462 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f83ac105d70 con 0x7f839406dd60 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr" 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "2/23 daemons upgraded", 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading mon daemons", 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:46:05.893 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f839406dd60 msgr2=0x7f8394070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f839406dd60 0x7f8394070220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009c60 tx=0x7f83a0009260 comp rx=0 tx=0).stop 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac101560 msgr2=0x7f83ac196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac101560 0x7f83ac196490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f839c00b800 tx=0x7f839c005f00 comp rx=0 tx=0).stop 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 shutdown_connections 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f839406dd60 0x7f8394070220 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83ac101560 0x7f83ac196490 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 --2- 172.21.15.2:0/3667954068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83ac103e90 0x7f83ac1969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 >> 172.21.15.2:0/3667954068 conn(0x7f83ac0faf70 msgr2=0x7f83ac0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 shutdown_connections 2024-03-16T10:46:05.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:05.894+0000 7f83b28c3700 1 -- 172.21.15.2:0/3667954068 wait complete. 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.002+0000 7f55275cf700 1 -- 172.21.15.2:0/3996134519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55200feda0 msgr2=0x7f55200ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.002+0000 7f55275cf700 1 --2- 172.21.15.2:0/3996134519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55200feda0 0x7f55200ff1c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f551c009910 tx=0x7f551c009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 -- 172.21.15.2:0/3996134519 shutdown_connections 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 --2- 172.21.15.2:0/3996134519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5520101d80 0x7f55201021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 --2- 172.21.15.2:0/3996134519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55200feda0 0x7f55200ff1c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 -- 172.21.15.2:0/3996134519 >> 172.21.15.2:0/3996134519 conn(0x7f55200fa700 msgr2=0x7f55200fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 -- 172.21.15.2:0/3996134519 shutdown_connections 2024-03-16T10:46:06.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 -- 172.21.15.2:0/3996134519 wait complete. 2024-03-16T10:46:06.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 Processor -- start 2024-03-16T10:46:06.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 -- start start 2024-03-16T10:46:06.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55200feda0 0x7f5520101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:06.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5520101d80 0x7f55200ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:06.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5520100270 con 0x7f5520101d80 2024-03-16T10:46:06.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.003+0000 7f55275cf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55201003b0 con 0x7f55200feda0 2024-03-16T10:46:06.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f5524b6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5520101d80 0x7f55200ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:06.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f5524b6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5520101d80 0x7f55200ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41284/0 (socket says 172.21.15.2:41284) 2024-03-16T10:46:06.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f5524b6a700 1 -- 172.21.15.2:0/3118211948 learned_addr learned my addr 172.21.15.2:0/3118211948 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:06.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f552536b700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55200feda0 0x7f5520101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:06.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f5524b6a700 1 -- 172.21.15.2:0/3118211948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55200feda0 msgr2=0x7f5520101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f5524b6a700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55200feda0 0x7f5520101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f5524b6a700 1 -- 172.21.15.2:0/3118211948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c0095f0 con 0x7f5520101d80 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f5524b6a700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5520101d80 0x7f55200ffd30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f551000e920 tx=0x7f551000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f55167fc700 1 -- 172.21.15.2:0/3118211948 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551000c9d0 con 0x7f5520101d80 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f55167fc700 1 -- 172.21.15.2:0/3118211948 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5510010430 con 0x7f5520101d80 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5520100690 con 0x7f5520101d80 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f55167fc700 1 -- 172.21.15.2:0/3118211948 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5510018630 con 0x7f5520101d80 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.004+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552019e5f0 con 0x7f5520101d80 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.005+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5520190660 con 0x7f5520101d80 2024-03-16T10:46:06.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.006+0000 7f55167fc700 1 -- 172.21.15.2:0/3118211948 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 32) v1 ==== 91794+0+0 (secure 0 0 0) 0x7f55100105a0 con 0x7f5520101d80 2024-03-16T10:46:06.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.006+0000 7f55167fc700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f550c06dd60 0x7f550c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:06.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.006+0000 7f55167fc700 1 -- 172.21.15.2:0/3118211948 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5510014070 con 0x7f5520101d80 2024-03-16T10:46:06.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.006+0000 7f552536b700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f550c06dd60 0x7f550c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:06.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.007+0000 7f552536b700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f550c06dd60 0x7f550c070220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f551c00b800 tx=0x7f551c009f70 comp rx=0 tx=0).ready entity=mgr.14712 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:06.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.012+0000 7f55167fc700 1 -- 172.21.15.2:0/3118211948 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185516 (secure 0 0 0) 0x7f55100534c0 con 0x7f5520101d80 2024-03-16T10:46:06.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.400+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f552004ea80 con 0x7f5520101d80 2024-03-16T10:46:06.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.400+0000 7f55167fc700 1 -- 172.21.15.2:0/3118211948 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f551005af40 con 0x7f5520101d80 2024-03-16T10:46:06.401 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:06.401 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:06.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f550c06dd60 msgr2=0x7f550c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:06.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f550c06dd60 0x7f550c070220 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f551c00b800 tx=0x7f551c009f70 comp rx=0 tx=0).stop 2024-03-16T10:46:06.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5520101d80 msgr2=0x7f55200ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5520101d80 0x7f55200ffd30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f551000e920 tx=0x7f551000ec30 comp rx=0 tx=0).stop 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 shutdown_connections 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:6800/730693462,v1:172.21.15.2:6801/730693462] conn(0x7f550c06dd60 0x7f550c070220 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55200feda0 0x7f5520101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 --2- 172.21.15.2:0/3118211948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5520101d80 0x7f55200ffd30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.403+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 >> 172.21.15.2:0/3118211948 conn(0x7f55200fa700 msgr2=0x7f55201053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.404+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 shutdown_connections 2024-03-16T10:46:06.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:06.404+0000 7f55275cf700 1 -- 172.21.15.2:0/3118211948 wait complete. 2024-03-16T10:46:06.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/1034998038' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:46:06.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: pgmap v28: 65 pgs: 65 active+clean; 351 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-03-16T10:46:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: Upgrade: Updating mon.smithi002 2024-03-16T10:46:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:46:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:46:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:06.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:06 smithi002 ceph-mon[29248]: Deploying daemon mon.smithi002 on smithi002 2024-03-16T10:46:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/1034998038' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:46:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: pgmap v28: 65 pgs: 65 active+clean; 351 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-03-16T10:46:06.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:46:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: Upgrade: Updating mon.smithi002 2024-03-16T10:46:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:46:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:46:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:06.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:06 smithi026 ceph-mon[37041]: Deploying daemon mon.smithi002 on smithi002 2024-03-16T10:46:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:07 smithi002 ceph-mon[29248]: from='client.? 172.21.15.2:0/3118211948' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:46:07.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:07 smithi026 ceph-mon[37041]: from='client.? 172.21.15.2:0/3118211948' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:46:08.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:08 smithi026 ceph-mon[37041]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 146 op/s 2024-03-16T10:46:08.767 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:08 smithi002 ceph-mon[29248]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.0 MiB/s wr, 146 op/s 2024-03-16T10:46:08.767 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:08 smithi002 systemd[1]: Stopping Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:46:09.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:08 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[29244]: 2024-03-16T10:46:08.766+0000 7ff8b4a60700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi002 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:46:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:08 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[29244]: 2024-03-16T10:46:08.766+0000 7ff8b4a60700 -1 mon.smithi002@0(leader) e2 *** Got Signal Terminated *** 2024-03-16T10:46:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:08 smithi002 podman[94061]: 2024-03-16 10:46:08.817764064 +0000 UTC m=+0.164171940 container died 3e28a22ab682e8d4a3b13e1d1f76800a8e811c140725e40a15d1aea73f6e1183 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=reef-a752c2f, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-03-16T10:46:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:09 smithi002 podman[94061]: 2024-03-16 10:46:09.317713211 +0000 UTC m=+0.664121078 container cleanup 3e28a22ab682e8d4a3b13e1d1f76800a8e811c140725e40a15d1aea73f6e1183 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=reef-a752c2f, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-16T10:46:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:09 smithi002 bash[94061]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002 2024-03-16T10:46:10.012 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:09 smithi002 podman[94086]: 2024-03-16 10:46:09.642116897 +0000 UTC m=+0.825646644 container remove 3e28a22ab682e8d4a3b13e1d1f76800a8e811c140725e40a15d1aea73f6e1183 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=reef-a752c2f, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-16T10:46:10.012 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:09 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service: Deactivated successfully. 2024-03-16T10:46:10.012 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:09 smithi002 systemd[1]: Stopped Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:46:10.012 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:09 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service: Consumed 11.882s CPU time. 2024-03-16T10:46:10.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 systemd[1]: Starting Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:46:10.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 podman[94220]: 2024-03-16 10:46:10.134787881 +0000 UTC m=+0.019183317 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:46:10.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 podman[94220]: 2024-03-16 10:46:10.253832041 +0000 UTC m=+0.138227455 container create 89f53efb5b4422894b2c7326c68154d53a7e8b0fe2b50e7744a83f8f6746ff5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-16T10:46:10.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 podman[94220]: 2024-03-16 10:46:10.490354834 +0000 UTC m=+0.374750265 container init 89f53efb5b4422894b2c7326c68154d53a7e8b0fe2b50e7744a83f8f6746ff5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-16T10:46:10.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 podman[94220]: 2024-03-16 10:46:10.496591789 +0000 UTC m=+0.380987209 container start 89f53efb5b4422894b2c7326c68154d53a7e8b0fe2b50e7744a83f8f6746ff5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-16T10:46:10.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T10:46:10.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: pidfile_write: ignore empty --pid-file 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: load: jerasure load: lrc 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: RocksDB version: 7.9.2 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Git sha 0 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: DB SUMMARY 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: DB Session ID: H51RMB2MBHGZTZJ8K7FX 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: CURRENT file: CURRENT 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1501 Bytes 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi002/store.db dir, Total Num: 1, files: 000032.sst 2024-03-16T10:46:10.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi002/store.db: 000030.log size: 954861 ; 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.error_if_exists: 0 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.create_if_missing: 0 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.paranoid_checks: 1 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.env: 0x559fcec91c60 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.info_log: 0x559fcff9d980 2024-03-16T10:46:10.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T10:46:10.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.statistics: (nil) 2024-03-16T10:46:10.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.use_fsync: 0 2024-03-16T10:46:10.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_log_file_size: 0 2024-03-16T10:46:10.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.allow_fallocate: 1 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.use_direct_reads: 0 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.db_log_dir: 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.wal_dir: 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T10:46:10.803 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.write_buffer_manager: 0x559fcff701e0 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T10:46:10.804 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.unordered_write: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.row_cache: None 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.wal_filter: None 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.two_write_queues: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.wal_compression: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.atomic_flush: 0 2024-03-16T10:46:10.805 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.log_readahead_size: 0 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_background_jobs: 2 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_background_compactions: -1 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_subcompactions: 1 2024-03-16T10:46:10.806 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_open_files: -1 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_background_flushes: -1 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Compression algorithms supported: 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T10:46:10.807 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kZSTD supported: 0 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kXpressCompression supported: 0 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kZlibCompression supported: 1 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kSnappyCompression supported: 1 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kLZ4Compression supported: 1 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: kBZip2Compression supported: 0 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi002/store.db/MANIFEST-000015 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.merge_operator: 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_filter: None 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T10:46:10.808 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559fcff9da80) 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: cache_index_and_filter_blocks: 1 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: pin_top_level_index_and_filter: 1 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: index_type: 0 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: data_block_index_type: 0 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: index_shortening: 1 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: checksum: 4 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: no_block_cache: 0 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache: 0x559fcffa3090 2024-03-16T10:46:10.809 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache_name: BinnedLRUCache 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache_options: 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: capacity : 536870912 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: num_shard_bits : 4 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: strict_capacity_limit : 0 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: high_pri_pool_ratio: 0.000 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_cache_compressed: (nil) 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: persistent_cache: (nil) 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_size: 4096 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_size_deviation: 10 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_restart_interval: 16 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: index_block_restart_interval: 1 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: metadata_block_size: 4096 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: partition_filters: 0 2024-03-16T10:46:10.810 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: use_delta_encoding: 1 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: filter_policy: bloomfilter 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: whole_key_filtering: 1 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: verify_compression: 0 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: read_amp_bytes_per_bit: 0 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: format_version: 5 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: enable_index_compression: 1 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: block_align: 0 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: max_auto_readahead_size: 262144 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: prepopulate_block_cache: 0 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: initial_auto_readahead_size: 8192 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression: NoCompression 2024-03-16T10:46:10.811 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.num_levels: 7 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T10:46:10.812 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T10:46:10.813 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T10:46:10.814 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T10:46:10.815 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.table_properties_collectors: 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.inplace_update_support: 0 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T10:46:10.816 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.bloom_locality: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.max_successive_merges: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.ttl: 2592000 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.enable_blob_files: false 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.min_blob_size: 0 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T10:46:10.817 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi002/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12119, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4c5de884-776c-4b23-b701-ba39ad44b2cb 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585970526987, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585970530984, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 852470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12120, "largest_seqno": 12726, "table_properties": {"data_size": 848987, "index_size": 1765, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 6257, "raw_average_key_size": 23, "raw_value_size": 843127, "raw_average_value_size": 3205, "num_data_blocks": 80, "num_entries": 263, "num_filter_entries": 263, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710585970, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4c5de884-776c-4b23-b701-ba39ad44b2cb", "db_session_id": "H51RMB2MBHGZTZJ8K7FX", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585970531114, "job": 1, "event": "recovery_finished"} 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-03-16T10:46:10.818 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi002/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559fd0096000 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: DB pointer 0x559fd0080000 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** DB Stats ** 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T10:46:10.819 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** Compaction Stats [default] ** 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: L0 1/0 832.49 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 332.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: L6 1/0 5.82 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Sum 2/0 6.63 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 332.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 332.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** Compaction Stats [default] ** 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 332.0 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T10:46:10.820 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Cumulative compaction: 0.00 GB write, 118.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Interval compaction: 0.00 GB write, 118.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Block cache BinnedLRUCache@0x559fcffa3090#2 capacity: 512.00 MB usage: 2.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.78 KB,0.000149012%) IndexBlock(1,1.83 KB,0.000348687%) Misc(1,0.00 KB,0%) 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: starting mon.smithi002 rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi002 fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???) e2 preinit fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:10.821 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???).mds e17 new map 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???).mds e17 print_map 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: e17 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: legacy client fscid: 1 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Filesystem 'cephfs' (1) 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: fs_name cephfs 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: epoch 16 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:46:10.822 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: tableserver 0 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: root 0 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: session_timeout 60 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: session_autoclose 300 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: max_file_size 1099511627776 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: max_xattr_size 65536 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: required_client_features {} 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: last_failure 0 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: last_failure_osd_epoch 47 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: max_mds 1 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: in 0 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: up {0=24255} 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: failed 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: damaged 2024-03-16T10:46:10.823 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: stopped 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: data_pools [3] 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: metadata_pool 2 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: inline_data enabled 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: balancer 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: bal_rank_mask -1 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: standby_count_wanted 1 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: qdb_cluster leader: 0 members: 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: [mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: Standby daemons: 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: [mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: [mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout: [mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:10.824 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-16T10:46:10.825 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T10:46:10.825 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T10:46:10.825 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T10:46:10.825 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 ceph-mon[94239]: mon.smithi002@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-16T10:46:10.825 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 bash[94220]: 89f53efb5b4422894b2c7326c68154d53a7e8b0fe2b50e7744a83f8f6746ff5f 2024-03-16T10:46:10.825 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:10 smithi002 systemd[1]: Started Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: mon.smithi002 calling monitor election 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: mon.smithi002 is new leader, mons smithi002,smithi026 in quorum (ranks 0,1) 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: monmap epoch 2 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: last_changed 2024-03-16T10:37:28.545499+0000 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: created 2024-03-16T10:34:38.356713+0000 2024-03-16T10:46:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: min_mon_release 18 (reef) 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: election_strategy: 1 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.smithi002 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: 1: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.smithi026 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: osdmap e50: 6 total, 6 up, 6 in 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: mgrmap e32: smithi002.zevrtm(active, since 56s), standbys: smithi026.omjkpo 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: from='mgr.14712 ' entity='' 2024-03-16T10:46:12.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:12 smithi002 ceph-mon[94239]: mgrmap e33: smithi002.zevrtm(active, since 56s), standbys: smithi026.omjkpo 2024-03-16T10:46:12.307 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:46:12.307 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: from='mgr.14712 172.21.15.2:0/2879203377' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: mon.smithi002 calling monitor election 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: mon.smithi002 is new leader, mons smithi002,smithi026 in quorum (ranks 0,1) 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: monmap epoch 2 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: last_changed 2024-03-16T10:37:28.545499+0000 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: created 2024-03-16T10:34:38.356713+0000 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: min_mon_release 18 (reef) 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: election_strategy: 1 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.smithi002 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: 1: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.smithi026 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: osdmap e50: 6 total, 6 up, 6 in 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: mgrmap e32: smithi002.zevrtm(active, since 56s), standbys: smithi026.omjkpo 2024-03-16T10:46:12.308 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:12.309 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:12.309 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:12.309 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: from='mgr.14712 ' entity='' 2024-03-16T10:46:12.309 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:12 smithi026 ceph-mon[37041]: mgrmap e33: smithi002.zevrtm(active, since 56s), standbys: smithi026.omjkpo 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: Active manager daemon smithi002.zevrtm restarted 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: Activating manager daemon smithi002.zevrtm 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: osdmap e51: 6 total, 6 up, 6 in 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: mgrmap e34: smithi002.zevrtm(active, starting, since 0.00649911s), standbys: smithi026.omjkpo 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:46:20.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:46:20.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:19 smithi002 ceph-mon[94239]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: Active manager daemon smithi002.zevrtm restarted 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: Activating manager daemon smithi002.zevrtm 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: osdmap e51: 6 total, 6 up, 6 in 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: mgrmap e34: smithi002.zevrtm(active, starting, since 0.00649911s), standbys: smithi026.omjkpo 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi002.zevrtm", "id": "smithi002.zevrtm"}]: dispatch 2024-03-16T10:46:20.085 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr metadata", "who": "smithi026.omjkpo", "id": "smithi026.omjkpo"}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T10:46:20.086 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:19 smithi026 ceph-mon[37041]: Manager daemon smithi002.zevrtm is now available 2024-03-16T10:46:21.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:21.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:21.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:21.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:46:21.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:46:21.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:20 smithi002 ceph-mon[94239]: mgrmap e35: smithi002.zevrtm(active, since 1.01076s), standbys: smithi026.omjkpo 2024-03-16T10:46:21.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:20 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:21.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:20 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:21.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:20 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:21.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:20 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/mirror_snapshot_schedule"}]: dispatch 2024-03-16T10:46:21.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:20 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi002.zevrtm/trash_purge_schedule"}]: dispatch 2024-03-16T10:46:21.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:20 smithi026 ceph-mon[37041]: mgrmap e35: smithi002.zevrtm(active, since 1.01076s), standbys: smithi026.omjkpo 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: [16/Mar/2024:10:46:20] ENGINE Bus STARTING 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: [16/Mar/2024:10:46:20] ENGINE Serving on http://172.21.15.2:8765 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: [16/Mar/2024:10:46:20] ENGINE Serving on https://172.21.15.2:7150 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: [16/Mar/2024:10:46:20] ENGINE Bus STARTED 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: pgmap v3: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: Standby manager daemon smithi026.omjkpo restarted 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: Standby manager daemon smithi026.omjkpo started 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:46:22.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:21 smithi002 ceph-mon[94239]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:46:22.183 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: [16/Mar/2024:10:46:20] ENGINE Bus STARTING 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: [16/Mar/2024:10:46:20] ENGINE Serving on http://172.21.15.2:8765 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: [16/Mar/2024:10:46:20] ENGINE Serving on https://172.21.15.2:7150 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: [16/Mar/2024:10:46:20] ENGINE Bus STARTED 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: pgmap v3: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: Standby manager daemon smithi026.omjkpo restarted 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: Standby manager daemon smithi026.omjkpo started 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/crt"}]: dispatch 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi026.omjkpo/key"}]: dispatch 2024-03-16T10:46:22.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:21 smithi026 ceph-mon[37041]: from='mgr.? 172.21.15.26:0/3825855585' entity='mgr.smithi026.omjkpo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T10:46:23.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:22 smithi002 ceph-mon[94239]: pgmap v4: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:23.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:22 smithi002 ceph-mon[94239]: mgrmap e36: smithi002.zevrtm(active, since 2s), standbys: smithi026.omjkpo 2024-03-16T10:46:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:22 smithi026 ceph-mon[37041]: pgmap v4: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:22 smithi026 ceph-mon[37041]: mgrmap e36: smithi002.zevrtm(active, since 2s), standbys: smithi026.omjkpo 2024-03-16T10:46:25.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:24 smithi002 ceph-mon[94239]: pgmap v5: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:25.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:24 smithi002 ceph-mon[94239]: mgrmap e37: smithi002.zevrtm(active, since 4s), standbys: smithi026.omjkpo 2024-03-16T10:46:25.157 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:24 smithi026 ceph-mon[37041]: pgmap v5: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:25.157 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:24 smithi026 ceph-mon[37041]: mgrmap e37: smithi002.zevrtm(active, since 4s), standbys: smithi026.omjkpo 2024-03-16T10:46:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:26 smithi002 ceph-mon[94239]: pgmap v6: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:26.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:26.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:26 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:26 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:26 smithi026 ceph-mon[37041]: pgmap v6: 65 pgs: 65 active+clean; 306 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:46:26.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:26 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:26.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:26 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.703 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:27 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.703 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:27 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.704 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:27 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.704 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:27 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.910 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:27 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.910 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:27 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.910 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:27 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:27.910 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:27 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:28.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:28 smithi026 ceph-mon[37041]: pgmap v7: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-03-16T10:46:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:28 smithi002 ceph-mon[94239]: pgmap v7: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-03-16T10:46:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:30 smithi002 ceph-mon[94239]: pgmap v8: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 885 KiB/s wr, 69 op/s 2024-03-16T10:46:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:31.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:46:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:30 smithi026 ceph-mon[37041]: pgmap v8: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 885 KiB/s wr, 69 op/s 2024-03-16T10:46:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:30 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:30 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:31.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:30 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:31.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:30 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:46:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:31 smithi026 ceph-mon[37041]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:46:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:31 smithi026 ceph-mon[37041]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:46:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:31 smithi026 ceph-mon[37041]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:46:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:31 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:31 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:32.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:31 smithi002 ceph-mon[94239]: Updating smithi002:/etc/ceph/ceph.conf 2024-03-16T10:46:32.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:31 smithi002 ceph-mon[94239]: Updating smithi026:/etc/ceph/ceph.conf 2024-03-16T10:46:32.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:31 smithi002 ceph-mon[94239]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:46:32.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:31 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:32.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:31 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:46:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: pgmap v9: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:46:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:32 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T10:46:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T10:46:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:46:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: Updating smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: Updating smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: pgmap v9: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:46:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-16T10:46:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:34 smithi002 ceph-mon[94239]: pgmap v10: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-16T10:46:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:34 smithi002 ceph-mon[94239]: Upgrade: Updating mon.smithi026 2024-03-16T10:46:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:46:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:46:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:34 smithi002 ceph-mon[94239]: Deploying daemon mon.smithi026 on smithi026 2024-03-16T10:46:35.214 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:34 smithi026 ceph-mon[37041]: pgmap v10: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-16T10:46:35.214 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:34 smithi026 ceph-mon[37041]: Upgrade: Updating mon.smithi026 2024-03-16T10:46:35.214 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:34 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:35.214 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:34 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:46:35.214 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:34 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:46:35.214 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:34 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:46:35.214 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:34 smithi026 ceph-mon[37041]: Deploying daemon mon.smithi026 on smithi026 2024-03-16T10:46:35.918 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:35 smithi026 ceph-mon[37041]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:35.918 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:35 smithi026 systemd[1]: Stopping Ceph mon.smithi026 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:46:36.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:36 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026[37023]: 2024-03-16T10:46:36.202+0000 7fecbf60d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi026 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:46:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:36 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026[37023]: 2024-03-16T10:46:36.202+0000 7fecbf60d700 -1 mon.smithi026@1(peon) e2 *** Got Signal Terminated *** 2024-03-16T10:46:36.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:36 smithi026 podman[82450]: 2024-03-16 10:46:36.314823615 +0000 UTC m=+0.240683377 container died 8f45632739a8611b528e8c17ffe71640de287303e3aae7b6537105199a8cc546 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=reef-a752c2f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-16T10:46:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3853046662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18100b00 msgr2=0x7efc18100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3853046662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18100b00 0x7efc18100f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efc0800b0a0 tx=0x7efc0800b3b0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3853046662 shutdown_connections 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3853046662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc18101d00 0x7efc18102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3853046662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18100b00 0x7efc18100f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3853046662 >> 172.21.15.2:0/3853046662 conn(0x7efc180fc0a0 msgr2=0x7efc180fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3853046662 shutdown_connections 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.523+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3853046662 wait complete. 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.524+0000 7efc1dc6d700 1 Processor -- start 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.524+0000 7efc1dc6d700 1 -- start start 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc1dc6d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc18100b00 0x7efc181965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc1dc6d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18101d00 0x7efc18196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc1dc6d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc18197150 con 0x7efc18101d00 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc1dc6d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc18197290 con 0x7efc18100b00 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc177fe700 1 -- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc18100b00 msgr2=0x7efc181965f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:3300/0 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc18100b00 0x7efc181965f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:46:36.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18101d00 0x7efc18196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18101d00 0x7efc18196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47698/0 (socket says 172.21.15.2:47698) 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc16ffd700 1 -- 172.21.15.2:0/3066960374 learned_addr learned my addr 172.21.15.2:0/3066960374 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc16ffd700 1 -- 172.21.15.2:0/3066960374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc18100b00 msgr2=0x7efc181965f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc16ffd700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc18100b00 0x7efc181965f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc16ffd700 1 -- 172.21.15.2:0/3066960374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08009d00 con 0x7efc18101d00 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc16ffd700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18101d00 0x7efc18196b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc0000b700 tx=0x7efc0000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc14ff9700 1 -- 172.21.15.2:0/3066960374 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc00011800 con 0x7efc18101d00 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1819bcf0 con 0x7efc18101d00 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.525+0000 7efc14ff9700 1 -- 172.21.15.2:0/3066960374 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc00011e20 con 0x7efc18101d00 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.526+0000 7efc14ff9700 1 -- 172.21.15.2:0/3066960374 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0000f2e0 con 0x7efc18101d00 2024-03-16T10:46:36.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.526+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1819c2c0 con 0x7efc18101d00 2024-03-16T10:46:36.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.526+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc18190720 con 0x7efc18101d00 2024-03-16T10:46:36.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.527+0000 7efc14ff9700 1 -- 172.21.15.2:0/3066960374 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7efc00010380 con 0x7efc18101d00 2024-03-16T10:46:36.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.528+0000 7efc14ff9700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efc0406e020 0x7efc040704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:36.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.528+0000 7efc14ff9700 1 -- 172.21.15.2:0/3066960374 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efc0008db70 con 0x7efc18101d00 2024-03-16T10:46:36.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.528+0000 7efc177fe700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efc0406e020 0x7efc040704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:36.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.528+0000 7efc177fe700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efc0406e020 0x7efc040704e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc08007820 tx=0x7efc08009440 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:36.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.533+0000 7efc14ff9700 1 -- 172.21.15.2:0/3066960374 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efc0005aa00 con 0x7efc18101d00 2024-03-16T10:46:36.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.826+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc180611f0 con 0x7efc0406e020 2024-03-16T10:46:36.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.827+0000 7efc14ff9700 1 -- 172.21.15.2:0/3066960374 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7efc180611f0 con 0x7efc0406e020 2024-03-16T10:46:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efc0406e020 msgr2=0x7efc040704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efc0406e020 0x7efc040704e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc08007820 tx=0x7efc08009440 comp rx=0 tx=0).stop 2024-03-16T10:46:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18101d00 msgr2=0x7efc18196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18101d00 0x7efc18196b30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc0000b700 tx=0x7efc0000bac0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 shutdown_connections 2024-03-16T10:46:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efc0406e020 0x7efc040704e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc18100b00 0x7efc181965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 --2- 172.21.15.2:0/3066960374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efc18101d00 0x7efc18196b30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.829+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 >> 172.21.15.2:0/3066960374 conn(0x7efc180fc0a0 msgr2=0x7efc18105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:36.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.830+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 shutdown_connections 2024-03-16T10:46:36.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.830+0000 7efc1dc6d700 1 -- 172.21.15.2:0/3066960374 wait complete. 2024-03-16T10:46:36.841 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:46:36.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 -- 172.21.15.2:0/1959678693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 msgr2=0x7f900c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:36.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 --2- 172.21.15.2:0/1959678693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c100ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009910 tx=0x7f8ffc009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:36.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 -- 172.21.15.2:0/1959678693 shutdown_connections 2024-03-16T10:46:36.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 --2- 172.21.15.2:0/1959678693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c101cd0 0x7f900c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 --2- 172.21.15.2:0/1959678693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c100ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 -- 172.21.15.2:0/1959678693 >> 172.21.15.2:0/1959678693 conn(0x7f900c0fc090 msgr2=0x7f900c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:36.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 -- 172.21.15.2:0/1959678693 shutdown_connections 2024-03-16T10:46:36.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.942+0000 7f90129e0700 1 -- 172.21.15.2:0/1959678693 wait complete. 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f90129e0700 1 Processor -- start 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f90129e0700 1 -- start start 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f90129e0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f90129e0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c101cd0 0x7f900c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f90129e0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1971e0 con 0x7f900c100ad0 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f90129e0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c197320 con 0x7f900c101cd0 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f900b7fe700 1 -- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c101cd0 msgr2=0x7f900c196bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:3300/0 2024-03-16T10:46:36.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f900b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c101cd0 0x7f900c196bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f900bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f900bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47726/0 (socket says 172.21.15.2:47726) 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.943+0000 7f900bfff700 1 -- 172.21.15.2:0/4056439362 learned_addr learned my addr 172.21.15.2:0/4056439362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f900bfff700 1 -- 172.21.15.2:0/4056439362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c101cd0 msgr2=0x7f900c196bc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f900bfff700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c101cd0 0x7f900c196bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f900bfff700 1 -- 172.21.15.2:0/4056439362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0095f0 con 0x7f900c100ad0 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f900bfff700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c196680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f90097fa700 1 -- 172.21.15.2:0/4056439362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ffc01d070 con 0x7f900c100ad0 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c19bd20 con 0x7f900c100ad0 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f90097fa700 1 -- 172.21.15.2:0/4056439362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ffc005dc0 con 0x7f900c100ad0 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f90097fa700 1 -- 172.21.15.2:0/4056439362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ffc00f460 con 0x7f900c100ad0 2024-03-16T10:46:36.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.944+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c19c240 con 0x7f900c100ad0 2024-03-16T10:46:36.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.945+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c1906f0 con 0x7f900c100ad0 2024-03-16T10:46:36.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.946+0000 7f90097fa700 1 -- 172.21.15.2:0/4056439362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f8ffc022470 con 0x7f900c100ad0 2024-03-16T10:46:36.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.946+0000 7f90097fa700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8ff406e020 0x7f8ff40704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:36.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.946+0000 7f90097fa700 1 -- 172.21.15.2:0/4056439362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8ffc08f530 con 0x7f900c100ad0 2024-03-16T10:46:36.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.946+0000 7f900b7fe700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8ff406e020 0x7f8ff40704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:36.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.947+0000 7f900b7fe700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8ff406e020 0x7f8ff40704e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9000001050 tx=0x7f9000006df0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:36.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:36.951+0000 7f90097fa700 1 -- 172.21.15.2:0/4056439362 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8ffc05c360 con 0x7f900c100ad0 2024-03-16T10:46:37.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.191+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f900c0611f0 con 0x7f8ff406e020 2024-03-16T10:46:37.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.192+0000 7f90097fa700 1 -- 172.21.15.2:0/4056439362 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f900c0611f0 con 0x7f8ff406e020 2024-03-16T10:46:37.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.195+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8ff406e020 msgr2=0x7f8ff40704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:37.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.195+0000 7f90129e0700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8ff406e020 0x7f8ff40704e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9000001050 tx=0x7f9000006df0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.195+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 msgr2=0x7f900c196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:37.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.195+0000 7f90129e0700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c196680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc00be20 comp rx=0 tx=0).stop 2024-03-16T10:46:37.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.196+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 shutdown_connections 2024-03-16T10:46:37.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.196+0000 7f90129e0700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8ff406e020 0x7f8ff40704e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.196+0000 7f90129e0700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f900c100ad0 0x7f900c196680 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.196+0000 7f90129e0700 1 --2- 172.21.15.2:0/4056439362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c101cd0 0x7f900c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.196+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 >> 172.21.15.2:0/4056439362 conn(0x7f900c0fc090 msgr2=0x7f900c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:37.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.196+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 shutdown_connections 2024-03-16T10:46:37.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.196+0000 7f90129e0700 1 -- 172.21.15.2:0/4056439362 wait complete. 2024-03-16T10:46:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:36 smithi026 podman[82450]: 2024-03-16 10:46:36.970073593 +0000 UTC m=+0.895933363 container cleanup 8f45632739a8611b528e8c17ffe71640de287303e3aae7b6537105199a8cc546 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=reef-a752c2f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-03-16T10:46:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:36 smithi026 bash[82450]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 -- 172.21.15.2:0/506183518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70101cd0 msgr2=0x7fec70102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 --2- 172.21.15.2:0/506183518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70101cd0 0x7fec70102130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fec60009910 tx=0x7fec60009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 -- 172.21.15.2:0/506183518 shutdown_connections 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 --2- 172.21.15.2:0/506183518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70101cd0 0x7fec70102130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 --2- 172.21.15.2:0/506183518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec70100ad0 0x7fec70100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 -- 172.21.15.2:0/506183518 >> 172.21.15.2:0/506183518 conn(0x7fec700fc090 msgr2=0x7fec700fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 -- 172.21.15.2:0/506183518 shutdown_connections 2024-03-16T10:46:37.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.303+0000 7fec783c4700 1 -- 172.21.15.2:0/506183518 wait complete. 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec783c4700 1 Processor -- start 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec783c4700 1 -- start start 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec783c4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70100ad0 0x7fec70196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec783c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec70101cd0 0x7fec70196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec783c4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701971d0 con 0x7fec70100ad0 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec783c4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70197310 con 0x7fec70101cd0 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec7595f700 1 -- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec70101cd0 msgr2=0x7fec70196bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:3300/0 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec7595f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec70101cd0 0x7fec70196bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec76160700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70100ad0 0x7fec70196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec76160700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70100ad0 0x7fec70196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47734/0 (socket says 172.21.15.2:47734) 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec76160700 1 -- 172.21.15.2:0/3257570506 learned_addr learned my addr 172.21.15.2:0/3257570506 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec76160700 1 -- 172.21.15.2:0/3257570506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec70101cd0 msgr2=0x7fec70196bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec76160700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec70101cd0 0x7fec70196bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec76160700 1 -- 172.21.15.2:0/3257570506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600095f0 con 0x7fec70100ad0 2024-03-16T10:46:37.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.304+0000 7fec76160700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70100ad0 0x7fec70196670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00b4d0 tx=0x7fec6c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:37.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.305+0000 7fec677fe700 1 -- 172.21.15.2:0/3257570506 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6c011800 con 0x7fec70100ad0 2024-03-16T10:46:37.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.305+0000 7fec677fe700 1 -- 172.21.15.2:0/3257570506 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec6c011e20 con 0x7fec70100ad0 2024-03-16T10:46:37.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.305+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7019bcb0 con 0x7fec70100ad0 2024-03-16T10:46:37.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.305+0000 7fec677fe700 1 -- 172.21.15.2:0/3257570506 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6c010e90 con 0x7fec70100ad0 2024-03-16T10:46:37.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.305+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7019c230 con 0x7fec70100ad0 2024-03-16T10:46:37.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.306+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec701906f0 con 0x7fec70100ad0 2024-03-16T10:46:37.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.306+0000 7fec677fe700 1 -- 172.21.15.2:0/3257570506 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fec6c010380 con 0x7fec70100ad0 2024-03-16T10:46:37.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.307+0000 7fec677fe700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fec5c06e020 0x7fec5c0704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:37.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.307+0000 7fec677fe700 1 -- 172.21.15.2:0/3257570506 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fec6c08da80 con 0x7fec70100ad0 2024-03-16T10:46:37.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.307+0000 7fec7595f700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fec5c06e020 0x7fec5c0704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:37.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.307+0000 7fec7595f700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fec5c06e020 0x7fec5c0704e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fec60005090 tx=0x7fec60009fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:37.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.311+0000 7fec677fe700 1 -- 172.21.15.2:0/3257570506 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec6c05a990 con 0x7fec70100ad0 2024-03-16T10:46:37.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.540+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fec700611f0 con 0x7fec5c06e020 2024-03-16T10:46:37.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.551+0000 7fec677fe700 1 -- 172.21.15.2:0/3257570506 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fec700611f0 con 0x7fec5c06e020 2024-03-16T10:46:37.552 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:46:37.552 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (8m) 11s ago 10m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (10m) 11s ago 10m 7759k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (9m) 12s ago 9m 7814k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (10m) 11s ago 10m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (9m) 12s ago 9m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (8m) 11s ago 9m 85.7M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (4m) 11s ago 4m 17.4M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (4m) 11s ago 4m 14.3M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (4m) 12s ago 4m 238M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (4m) 12s ago 4m 16.0M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (110s) 11s ago 11m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (44s) 12s ago 9m 425M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (27s) 11s ago 11m 27.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (9m) 12s ago 9m 47.5M 2048M 18.2.2-878-ga752c2f7 f692adb9a265 8f45632739a8 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (10m) 11s ago 10m 16.7M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:46:37.553 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (9m) 12s ago 9m 15.8M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:46:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (8m) 11s ago 8m 181M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:46:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (7m) 11s ago 7m 178M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:46:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (7m) 11s ago 7m 159M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:46:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (7m) 12s ago 7m 227M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:46:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (6m) 12s ago 6m 187M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:46:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (6m) 12s ago 6m 177M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:46:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (54s) 11s ago 9m 57.8M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:46:37.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fec5c06e020 msgr2=0x7fec5c0704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:37.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fec5c06e020 0x7fec5c0704e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fec60005090 tx=0x7fec60009fd0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70100ad0 msgr2=0x7fec70196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:37.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70100ad0 0x7fec70196670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00b4d0 tx=0x7fec6c00b890 comp rx=0 tx=0).stop 2024-03-16T10:46:37.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 shutdown_connections 2024-03-16T10:46:37.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fec5c06e020 0x7fec5c0704e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec70100ad0 0x7fec70196670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 --2- 172.21.15.2:0/3257570506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec70101cd0 0x7fec70196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 >> 172.21.15.2:0/3257570506 conn(0x7fec700fc090 msgr2=0x7fec70105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:37.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 shutdown_connections 2024-03-16T10:46:37.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.554+0000 7fec783c4700 1 -- 172.21.15.2:0/3257570506 wait complete. 2024-03-16T10:46:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 -- 172.21.15.2:0/2282237685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 msgr2=0x7fbfa4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2282237685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4100ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009910 tx=0x7fbf94009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 -- 172.21.15.2:0/2282237685 shutdown_connections 2024-03-16T10:46:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2282237685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfa4101cd0 0x7fbfa4102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2282237685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4100ef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 -- 172.21.15.2:0/2282237685 >> 172.21.15.2:0/2282237685 conn(0x7fbfa40fc090 msgr2=0x7fbfa40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:37.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 -- 172.21.15.2:0/2282237685 shutdown_connections 2024-03-16T10:46:37.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.663+0000 7fbfa914d700 1 -- 172.21.15.2:0/2282237685 wait complete. 2024-03-16T10:46:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa914d700 1 Processor -- start 2024-03-16T10:46:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa914d700 1 -- start start 2024-03-16T10:46:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa914d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa914d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfa4101cd0 0x7fbfa40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa914d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40736e0 con 0x7fbfa4100ad0 2024-03-16T10:46:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa914d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4073820 con 0x7fbfa4101cd0 2024-03-16T10:46:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa259c700 1 -- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfa4101cd0 msgr2=0x7fbfa40731a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:3300/0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfa4101cd0 0x7fbfa40731a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47738/0 (socket says 172.21.15.2:47738) 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.664+0000 7fbfa2d9d700 1 -- 172.21.15.2:0/2122601201 learned_addr learned my addr 172.21.15.2:0/2122601201 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbfa2d9d700 1 -- 172.21.15.2:0/2122601201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfa4101cd0 msgr2=0x7fbfa40731a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbfa2d9d700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfa4101cd0 0x7fbfa40731a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbfa2d9d700 1 -- 172.21.15.2:0/2122601201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940095f0 con 0x7fbfa4100ad0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbfa2d9d700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4074b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbf8bfff700 1 -- 172.21.15.2:0/2122601201 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf9401d070 con 0x7fbfa4100ad0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbf8bfff700 1 -- 172.21.15.2:0/2122601201 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf940048e0 con 0x7fbfa4100ad0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbf8bfff700 1 -- 172.21.15.2:0/2122601201 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf94003c10 con 0x7fbfa4100ad0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa4073aa0 con 0x7fbfa4100ad0 2024-03-16T10:46:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.665+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa4073fb0 con 0x7fbfa4100ad0 2024-03-16T10:46:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.666+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa41906f0 con 0x7fbfa4100ad0 2024-03-16T10:46:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.667+0000 7fbf8bfff700 1 -- 172.21.15.2:0/2122601201 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fbf94004a50 con 0x7fbfa4100ad0 2024-03-16T10:46:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.667+0000 7fbf8bfff700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fbf8c06e080 0x7fbf8c070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.667+0000 7fbf8bfff700 1 -- 172.21.15.2:0/2122601201 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbf9408f510 con 0x7fbfa4100ad0 2024-03-16T10:46:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.667+0000 7fbfa259c700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fbf8c06e080 0x7fbf8c070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.668+0000 7fbfa259c700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fbf8c06e080 0x7fbf8c070540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf98001050 tx=0x7fbf98006df0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:37.673+0000 7fbf8bfff700 1 -- 172.21.15.2:0/2122601201 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf9405c270 con 0x7fbfa4100ad0 2024-03-16T10:46:37.711 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:37 smithi026 podman[82463]: 2024-03-16 10:46:37.305604198 +0000 UTC m=+0.990744972 container remove 8f45632739a8611b528e8c17ffe71640de287303e3aae7b6537105199a8cc546 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-a752c2f, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-16T10:46:37.711 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:37 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi026.service: Deactivated successfully. 2024-03-16T10:46:37.711 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:37 smithi026 systemd[1]: Stopped Ceph mon.smithi026 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:46:37.711 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:37 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi026.service: Consumed 6.561s CPU time. 2024-03-16T10:46:37.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:37 smithi026 systemd[1]: Starting Ceph mon.smithi026 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:46:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.050+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbfa402ce40 con 0x7fbfa4100ad0 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.051+0000 7fbf8bfff700 1 -- 172.21.15.2:0/2122601201 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7fbf9405be00 con 0x7fbfa4100ad0 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 1, 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:46:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 11, 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 3 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:46:38.053 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:46:38.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.053+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fbf8c06e080 msgr2=0x7fbf8c070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.053+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fbf8c06e080 0x7fbf8c070540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf98001050 tx=0x7fbf98006df0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.053+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 msgr2=0x7fbfa4074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.053+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4074b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400be20 comp rx=0 tx=0).stop 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.053+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 shutdown_connections 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.053+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fbf8c06e080 0x7fbf8c070540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.054+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfa4100ad0 0x7fbfa4074b00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.054+0000 7fbfa914d700 1 --2- 172.21.15.2:0/2122601201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfa4101cd0 0x7fbfa40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.054+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 >> 172.21.15.2:0/2122601201 conn(0x7fbfa40fc090 msgr2=0x7fbfa4105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.054+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 shutdown_connections 2024-03-16T10:46:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.054+0000 7fbfa914d700 1 -- 172.21.15.2:0/2122601201 wait complete. 2024-03-16T10:46:38.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.163+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/432749657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44101ce0 msgr2=0x7f8f44102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.163+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/432749657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44101ce0 0x7f8f44102140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009910 tx=0x7f8f38009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:38.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/432749657 shutdown_connections 2024-03-16T10:46:38.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/432749657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44101ce0 0x7f8f44102140 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/432749657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f44100ae0 0x7f8f44100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/432749657 >> 172.21.15.2:0/432749657 conn(0x7f8f440fc0a0 msgr2=0x7f8f440fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/432749657 shutdown_connections 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/432749657 wait complete. 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 Processor -- start 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.164+0000 7f8f4a0a6700 1 -- start start 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f4a0a6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44100ae0 0x7f8f441987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f4a0a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f44101ce0 0x7f8f44198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f4a0a6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44199300 con 0x7f8f44100ae0 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f4a0a6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f44199440 con 0x7f8f44101ce0 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f42ffd700 1 -- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f44101ce0 msgr2=0x7f8f44198ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:3300/0 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f42ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f44101ce0 0x7f8f44198ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:46:38.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f437fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44100ae0 0x7f8f441987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:38.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f437fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44100ae0 0x7f8f441987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47744/0 (socket says 172.21.15.2:47744) 2024-03-16T10:46:38.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f437fe700 1 -- 172.21.15.2:0/2986106491 learned_addr learned my addr 172.21.15.2:0/2986106491 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f437fe700 1 -- 172.21.15.2:0/2986106491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f44101ce0 msgr2=0x7f8f44198ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f437fe700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f44101ce0 0x7f8f44198ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f437fe700 1 -- 172.21.15.2:0/2986106491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380095f0 con 0x7f8f44100ae0 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f437fe700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44100ae0 0x7f8f441987a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b4d0 tx=0x7f8f3400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f40ff9700 1 -- 172.21.15.2:0/2986106491 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f34011800 con 0x7f8f44100ae0 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f40ff9700 1 -- 172.21.15.2:0/2986106491 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f34011e20 con 0x7f8f44100ae0 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f40ff9700 1 -- 172.21.15.2:0/2986106491 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f34010e90 con 0x7f8f44100ae0 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4419dea0 con 0x7f8f44100ae0 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.165+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4419e410 con 0x7f8f44100ae0 2024-03-16T10:46:38.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.167+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f441066b0 con 0x7f8f44100ae0 2024-03-16T10:46:38.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.167+0000 7f8f40ff9700 1 -- 172.21.15.2:0/2986106491 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f8f34010380 con 0x7f8f44100ae0 2024-03-16T10:46:38.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.167+0000 7f8f40ff9700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8f2c06e020 0x7f8f2c0704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:38.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.168+0000 7f8f40ff9700 1 -- 172.21.15.2:0/2986106491 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8f3408db30 con 0x7f8f44100ae0 2024-03-16T10:46:38.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.168+0000 7f8f42ffd700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8f2c06e020 0x7f8f2c0704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:38.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.168+0000 7f8f42ffd700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8f2c06e020 0x7f8f2c0704e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005090 tx=0x7f8f38009fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:38.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.173+0000 7f8f40ff9700 1 -- 172.21.15.2:0/2986106491 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f3405a9a0 con 0x7f8f44100ae0 2024-03-16T10:46:38.333 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:37 smithi026 podman[82582]: 2024-03-16 10:46:37.902392076 +0000 UTC m=+0.019158203 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:46:38.334 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 podman[82582]: 2024-03-16 10:46:38.123408491 +0000 UTC m=+0.240174619 container create c4945cc8635f643a5e818e1cd87884b04558ff16aebe346affb21c008694918a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:46:38.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.499+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8f4404ea80 con 0x7f8f44100ae0 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.500+0000 7f8f40ff9700 1 -- 172.21.15.2:0/2986106491 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f8f3405a530 con 0x7f8f44100ae0 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:46:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:46:38.502 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.503 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.504 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.502+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8f2c06e020 msgr2=0x7f8f2c0704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.502+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8f2c06e020 0x7f8f2c0704e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005090 tx=0x7f8f38009fd0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.502+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44100ae0 msgr2=0x7f8f441987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.502+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44100ae0 0x7f8f441987a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b4d0 tx=0x7f8f3400b890 comp rx=0 tx=0).stop 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.503+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 shutdown_connections 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.503+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8f2c06e020 0x7f8f2c0704e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.503+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f44100ae0 0x7f8f441987a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.503+0000 7f8f4a0a6700 1 --2- 172.21.15.2:0/2986106491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f44101ce0 0x7f8f44198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.503+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 >> 172.21.15.2:0/2986106491 conn(0x7f8f440fc0a0 msgr2=0x7f8f440733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.503+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 shutdown_connections 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.503+0000 7f8f4a0a6700 1 -- 172.21.15.2:0/2986106491 wait complete. 2024-03-16T10:46:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 -- 172.21.15.2:0/4218450732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101570 msgr2=0x7f2160103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 --2- 172.21.15.2:0/4218450732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101570 0x7f2160103960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f215c009910 tx=0x7f215c009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 -- 172.21.15.2:0/4218450732 shutdown_connections 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 --2- 172.21.15.2:0/4218450732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160103ea0 0x7f2160106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 --2- 172.21.15.2:0/4218450732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160101570 0x7f2160103960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 -- 172.21.15.2:0/4218450732 >> 172.21.15.2:0/4218450732 conn(0x7f21600faf80 msgr2=0x7f21600fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 -- 172.21.15.2:0/4218450732 shutdown_connections 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.613+0000 7f21678c2700 1 -- 172.21.15.2:0/4218450732 wait complete. 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f21678c2700 1 Processor -- start 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f21678c2700 1 -- start start 2024-03-16T10:46:38.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f21678c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 0x7f2160100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f21678c2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160103ea0 0x7f21600ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f21678c2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600ffa60 con 0x7f2160103ea0 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f21678c2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600ffba0 con 0x7f2160101570 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f2164e5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160103ea0 0x7f21600ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f2164e5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160103ea0 0x7f21600ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47764/0 (socket says 172.21.15.2:47764) 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f2164e5d700 1 -- 172.21.15.2:0/2587879330 learned_addr learned my addr 172.21.15.2:0/2587879330 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f216565e700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 0x7f2160100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f216565e700 1 -- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 msgr2=0x7f2160100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f216565e700 1 -- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 msgr2=0x7f2160100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f216565e700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 0x7f2160100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f216565e700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 0x7f2160100e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f2164e5d700 1 -- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 msgr2=0x7f2160100e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f2164e5d700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 0x7f2160100e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.614+0000 7f2164e5d700 1 -- 172.21.15.2:0/2587879330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0095f0 con 0x7f2160103ea0 2024-03-16T10:46:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.615+0000 7f2164e5d700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160103ea0 0x7f21600ff520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f215000d8d0 tx=0x7f215000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:38.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.615+0000 7f21567fc700 1 -- 172.21.15.2:0/2587879330 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2150009940 con 0x7f2160103ea0 2024-03-16T10:46:38.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.615+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21600ffe80 con 0x7f2160103ea0 2024-03-16T10:46:38.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.615+0000 7f21567fc700 1 -- 172.21.15.2:0/2587879330 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2150010460 con 0x7f2160103ea0 2024-03-16T10:46:38.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.615+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216019e5c0 con 0x7f2160103ea0 2024-03-16T10:46:38.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.615+0000 7f21567fc700 1 -- 172.21.15.2:0/2587879330 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215000f5f0 con 0x7f2160103ea0 2024-03-16T10:46:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.616+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216004ea80 con 0x7f2160103ea0 2024-03-16T10:46:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.616+0000 7f21567fc700 1 -- 172.21.15.2:0/2587879330 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f21500105d0 con 0x7f2160103ea0 2024-03-16T10:46:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.617+0000 7f21567fc700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f214c06e0d0 0x7f214c070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.617+0000 7f21567fc700 1 -- 172.21.15.2:0/2587879330 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f215008dee0 con 0x7f2160103ea0 2024-03-16T10:46:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.617+0000 7f216565e700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f214c06e0d0 0x7f214c070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:38.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.617+0000 7f216565e700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f214c06e0d0 0x7f214c070590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f215c0051c0 tx=0x7f215c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:38.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.622+0000 7f21567fc700 1 -- 172.21.15.2:0/2587879330 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f215005ace0 con 0x7f2160103ea0 2024-03-16T10:46:38.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 podman[82582]: 2024-03-16 10:46:38.460158435 +0000 UTC m=+0.576924570 container init c4945cc8635f643a5e818e1cd87884b04558ff16aebe346affb21c008694918a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-16T10:46:38.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 podman[82582]: 2024-03-16 10:46:38.464188285 +0000 UTC m=+0.580954408 container start c4945cc8635f643a5e818e1cd87884b04558ff16aebe346affb21c008694918a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T10:46:38.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T10:46:38.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T10:46:38.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: pidfile_write: ignore empty --pid-file 2024-03-16T10:46:38.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: load: jerasure load: lrc 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: RocksDB version: 7.9.2 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Git sha 0 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: DB SUMMARY 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: DB Session ID: BN9B398H8JF2SXNDM1K9 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: CURRENT file: CURRENT 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1395 Bytes 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi026/store.db dir, Total Num: 1, files: 000027.sst 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi026/store.db: 000025.log size: 4044196 ; 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.error_if_exists: 0 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.create_if_missing: 0 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.paranoid_checks: 1 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T10:46:38.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T10:46:38.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T10:46:38.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.env: 0x55bcbfbcac60 2024-03-16T10:46:38.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T10:46:38.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.info_log: 0x55bcc147d980 2024-03-16T10:46:38.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.statistics: (nil) 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.use_fsync: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_log_file_size: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.allow_fallocate: 1 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.use_direct_reads: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T10:46:38.735 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.db_log_dir: 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.wal_dir: 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.write_buffer_manager: 0x55bcc14501e0 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T10:46:38.736 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.unordered_write: 0 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.row_cache: None 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.wal_filter: None 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.two_write_queues: 0 2024-03-16T10:46:38.737 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.wal_compression: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.atomic_flush: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.log_readahead_size: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T10:46:38.738 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_background_jobs: 2 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_background_compactions: -1 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_subcompactions: 1 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_open_files: -1 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T10:46:38.739 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_background_flushes: -1 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Compression algorithms supported: 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kZSTD supported: 0 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kXpressCompression supported: 0 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kZlibCompression supported: 1 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kSnappyCompression supported: 1 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kLZ4Compression supported: 1 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: kBZip2Compression supported: 0 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T10:46:38.740 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi026/store.db/MANIFEST-000010 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.merge_operator: 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_filter: None 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bcc147da80) 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: cache_index_and_filter_blocks: 1 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: pin_top_level_index_and_filter: 1 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: index_type: 0 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: data_block_index_type: 0 2024-03-16T10:46:38.741 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: index_shortening: 1 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: checksum: 4 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: no_block_cache: 0 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_cache: 0x55bcc1483090 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_cache_name: BinnedLRUCache 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_cache_options: 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: capacity : 536870912 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: num_shard_bits : 4 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: strict_capacity_limit : 0 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: high_pri_pool_ratio: 0.000 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_cache_compressed: (nil) 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: persistent_cache: (nil) 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_size: 4096 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_size_deviation: 10 2024-03-16T10:46:38.742 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_restart_interval: 16 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: index_block_restart_interval: 1 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: metadata_block_size: 4096 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: partition_filters: 0 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: use_delta_encoding: 1 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: filter_policy: bloomfilter 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: whole_key_filtering: 1 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: verify_compression: 0 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: read_amp_bytes_per_bit: 0 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: format_version: 5 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: enable_index_compression: 1 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: block_align: 0 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: max_auto_readahead_size: 262144 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: prepopulate_block_cache: 0 2024-03-16T10:46:38.743 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: initial_auto_readahead_size: 8192 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression: NoCompression 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.num_levels: 7 2024-03-16T10:46:38.744 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T10:46:38.745 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T10:46:38.746 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T10:46:38.747 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.table_properties_collectors: 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.inplace_update_support: 0 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T10:46:38.748 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.bloom_locality: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.max_successive_merges: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.ttl: 2592000 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.enable_blob_files: false 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.min_blob_size: 0 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T10:46:38.749 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi026/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12149, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7a923ad8-d07c-4f71-8cf3-68095bb31d4d 2024-03-16T10:46:38.750 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585998500417, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-16T10:46:38.753 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-16T10:46:38.753 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585998514256, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2489162, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12154, "largest_seqno": 13316, "table_properties": {"data_size": 2483635, "index_size": 3292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1221, "raw_key_size": 11274, "raw_average_key_size": 23, "raw_value_size": 2473124, "raw_average_value_size": 5184, "num_data_blocks": 151, "num_entries": 477, "num_filter_entries": 477, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710585998, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7a923ad8-d07c-4f71-8cf3-68095bb31d4d", "db_session_id": "BN9B398H8JF2SXNDM1K9", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-16T10:46:38.753 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710585998514397, "job": 1, "event": "recovery_finished"} 2024-03-16T10:46:38.753 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi026/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bcc1576000 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: DB pointer 0x55bcc1560000 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: ** DB Stats ** 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T10:46:38.754 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: ** Compaction Stats [default] ** 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: L0 1/0 2.37 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 414.6 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: L6 1/0 5.82 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Sum 2/0 8.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 414.6 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 414.6 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: ** Compaction Stats [default] ** 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 414.6 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.755 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Cumulative compaction: 0.00 GB write, 137.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Interval compaction: 0.00 GB write, 137.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Block cache BinnedLRUCache@0x55bcc1483090#2 capacity: 512.00 MB usage: 4.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.28 KB,0.000244379%) IndexBlock(1,3.33 KB,0.000634789%) Misc(1,0.00 KB,0%) 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T10:46:38.756 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: starting mon.smithi026 rank 1 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi026 fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???) e2 preinit fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???).mds e17 new map 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???).mds e17 print_map 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: e17 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: legacy client fscid: 1 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Filesystem 'cephfs' (1) 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: fs_name cephfs 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: epoch 16 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: tableserver 0 2024-03-16T10:46:38.757 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: root 0 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: session_timeout 60 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: session_autoclose 300 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: max_file_size 1099511627776 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: max_xattr_size 65536 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: required_client_features {} 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: last_failure 0 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: last_failure_osd_epoch 47 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: max_mds 1 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: in 0 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: up {0=24255} 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: failed 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: damaged 2024-03-16T10:46:38.758 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: stopped 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: data_pools [3] 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: metadata_pool 2 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: inline_data enabled 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: balancer 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: bal_rank_mask -1 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: standby_count_wanted 1 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: qdb_cluster leader: 0 members: 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: [mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: Standby daemons: 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: [mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: [mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.759 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout: [mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:46:38.760 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???).osd e51 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-16T10:46:38.760 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T10:46:38.760 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T10:46:38.760 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2024-03-16T10:46:38.760 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 ceph-mon[82600]: mon.smithi026@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-03-16T10:46:38.760 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 bash[82582]: c4945cc8635f643a5e818e1cd87884b04558ff16aebe346affb21c008694918a 2024-03-16T10:46:38.760 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:38 smithi026 systemd[1]: Started Ceph mon.smithi026 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:46:38.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.915+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2160105e30 con 0x7f214c06e0d0 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.916+0000 7f21567fc700 1 -- 172.21.15.2:0/2587879330 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f2160105e30 con 0x7f214c06e0d0 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr" 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "3/23 daemons upgraded", 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading mon daemons", 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:46:38.917 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:46:38.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f214c06e0d0 msgr2=0x7f214c070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f214c06e0d0 0x7f214c070590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f215c0051c0 tx=0x7f215c00b540 comp rx=0 tx=0).stop 2024-03-16T10:46:38.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160103ea0 msgr2=0x7f21600ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:38.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160103ea0 0x7f21600ff520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f215000d8d0 tx=0x7f215000dc90 comp rx=0 tx=0).stop 2024-03-16T10:46:38.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 shutdown_connections 2024-03-16T10:46:38.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f214c06e0d0 0x7f214c070590 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2160101570 0x7f2160100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 --2- 172.21.15.2:0/2587879330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2160103ea0 0x7f21600ff520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:38.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 >> 172.21.15.2:0/2587879330 conn(0x7f21600faf80 msgr2=0x7f21600fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:38.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 shutdown_connections 2024-03-16T10:46:38.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:38.918+0000 7f21678c2700 1 -- 172.21.15.2:0/2587879330 wait complete. 2024-03-16T10:46:39.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 -- 172.21.15.2:0/2232205635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e80ff9e0 msgr2=0x7f06e80ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:39.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 --2- 172.21.15.2:0/2232205635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e80ff9e0 0x7f06e80ffe00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009910 tx=0x7f06d8009c20 comp rx=0 tx=0).stop 2024-03-16T10:46:39.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 -- 172.21.15.2:0/2232205635 shutdown_connections 2024-03-16T10:46:39.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 --2- 172.21.15.2:0/2232205635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e8100be0 0x7f06e8101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:39.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 --2- 172.21.15.2:0/2232205635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e80ff9e0 0x7f06e80ffe00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:39.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 -- 172.21.15.2:0/2232205635 >> 172.21.15.2:0/2232205635 conn(0x7f06e80faf80 msgr2=0x7f06e80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:39.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 -- 172.21.15.2:0/2232205635 shutdown_connections 2024-03-16T10:46:39.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.030+0000 7f06ecb75700 1 -- 172.21.15.2:0/2232205635 wait complete. 2024-03-16T10:46:39.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.031+0000 7f06ecb75700 1 Processor -- start 2024-03-16T10:46:39.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.031+0000 7f06ecb75700 1 -- start start 2024-03-16T10:46:39.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.031+0000 7f06ecb75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 0x7f06e8196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:39.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.031+0000 7f06ecb75700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e8100be0 0x7f06e8196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.031+0000 7f06ecb75700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81970d0 con 0x7f06e8100be0 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.031+0000 7f06ecb75700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e8197210 con 0x7f06e80ff9e0 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.031+0000 7f06e5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e8100be0 0x7f06e8196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e8100be0 0x7f06e8196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47784/0 (socket says 172.21.15.2:47784) 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e5d9b700 1 -- 172.21.15.2:0/3282499831 learned_addr learned my addr 172.21.15.2:0/3282499831 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 0x7f06e8196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e659c700 1 -- 172.21.15.2:0/3282499831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 msgr2=0x7f06e8196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e659c700 1 -- 172.21.15.2:0/3282499831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 msgr2=0x7f06e8196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e659c700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 0x7f06e8196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e659c700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 0x7f06e8196570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e5d9b700 1 -- 172.21.15.2:0/3282499831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 msgr2=0x7f06e8196570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e5d9b700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 0x7f06e8196570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e5d9b700 1 -- 172.21.15.2:0/3282499831 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80095f0 con 0x7f06e8100be0 2024-03-16T10:46:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06e5d9b700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e8100be0 0x7f06e8196ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00d8d0 tx=0x7f06dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:39.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06d77fe700 1 -- 172.21.15.2:0/3282499831 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06dc009940 con 0x7f06e8100be0 2024-03-16T10:46:39.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e819bbd0 con 0x7f06e8100be0 2024-03-16T10:46:39.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06d77fe700 1 -- 172.21.15.2:0/3282499831 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06dc010460 con 0x7f06e8100be0 2024-03-16T10:46:39.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06d77fe700 1 -- 172.21.15.2:0/3282499831 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06dc00f5f0 con 0x7f06e8100be0 2024-03-16T10:46:39.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.032+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e819c1a0 con 0x7f06e8100be0 2024-03-16T10:46:39.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.033+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e804ea80 con 0x7f06e8100be0 2024-03-16T10:46:39.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.034+0000 7f06d77fe700 1 -- 172.21.15.2:0/3282499831 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f06dc0105d0 con 0x7f06e8100be0 2024-03-16T10:46:39.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.034+0000 7f06d77fe700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f06d006e0e0 0x7f06d00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:46:39.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.034+0000 7f06d77fe700 1 -- 172.21.15.2:0/3282499831 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f06dc08dff0 con 0x7f06e8100be0 2024-03-16T10:46:39.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.034+0000 7f06e659c700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f06d006e0e0 0x7f06d00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:46:39.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.035+0000 7f06e659c700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f06d006e0e0 0x7f06d00705a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f06d8006010 tx=0x7f06d800b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:46:39.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.041+0000 7f06d77fe700 1 -- 172.21.15.2:0/3282499831 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06dc05ad70 con 0x7f06e8100be0 2024-03-16T10:46:39.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.354+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f06e819c390 con 0x7f06e8100be0 2024-03-16T10:46:39.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.355+0000 7f06d77fe700 1 -- 172.21.15.2:0/3282499831 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f06dc05a900 con 0x7f06e8100be0 2024-03-16T10:46:39.356 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:39.356 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:39.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f06d006e0e0 msgr2=0x7f06d00705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:39.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f06d006e0e0 0x7f06d00705a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f06d8006010 tx=0x7f06d800b580 comp rx=0 tx=0).stop 2024-03-16T10:46:39.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e8100be0 msgr2=0x7f06e8196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e8100be0 0x7f06e8196ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00d8d0 tx=0x7f06dc00dc90 comp rx=0 tx=0).stop 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 shutdown_connections 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f06d006e0e0 0x7f06d00705a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80ff9e0 0x7f06e8196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 --2- 172.21.15.2:0/3282499831 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06e8100be0 0x7f06e8196ab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 >> 172.21.15.2:0/3282499831 conn(0x7f06e80faf80 msgr2=0x7f06e8068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 shutdown_connections 2024-03-16T10:46:39.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:46:39.357+0000 7f06ecb75700 1 -- 172.21.15.2:0/3282499831 wait complete. 2024-03-16T10:46:40.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:39 smithi026 ceph-mon[82600]: pgmap v11: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-16T10:46:40.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:39 smithi002 ceph-mon[94239]: pgmap v11: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-16T10:46:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: pgmap v13: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-03-16T10:46:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.smithi002 2024-03-16T10:46:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: 1: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.smithi026 2024-03-16T10:46:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:46:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: osdmap e51: 6 total, 6 up, 6 in 2024-03-16T10:46:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: mgrmap e37: smithi002.zevrtm(active, since 20s), standbys: smithi026.omjkpo 2024-03-16T10:46:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: mon.smithi002 calling monitor election 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: mon.smithi026 calling monitor election 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: mon.smithi002 is new leader, mons smithi002,smithi026 in quorum (ranks 0,1) 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: monmap epoch 3 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: last_changed 2024-03-16T10:46:39.813731+0000 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: created 2024-03-16T10:34:38.356713+0000 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: min_mon_release 19 (squid) 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: election_strategy: 1 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.smithi002 2024-03-16T10:46:41.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: 1: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.smithi026 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: osdmap e51: 6 total, 6 up, 6 in 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: mgrmap e37: smithi002.zevrtm(active, since 20s), standbys: smithi026.omjkpo 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:41.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: pgmap v13: 65 pgs: 65 active+clean; 304 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-03-16T10:46:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.smithi002 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: 1: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.smithi026 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: osdmap e51: 6 total, 6 up, 6 in 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: mgrmap e37: smithi002.zevrtm(active, since 20s), standbys: smithi026.omjkpo 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: mon.smithi002 calling monitor election 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: mon.smithi026 calling monitor election 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: mon.smithi002 is new leader, mons smithi002,smithi026 in quorum (ranks 0,1) 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi002"}]: dispatch 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mon metadata", "id": "smithi026"}]: dispatch 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: monmap epoch 3 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: fsid b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T10:46:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: last_changed 2024-03-16T10:46:39.813731+0000 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: created 2024-03-16T10:34:38.356713+0000 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: min_mon_release 19 (squid) 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: election_strategy: 1 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.smithi002 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: 1: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.smithi026 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: osdmap e51: 6 total, 6 up, 6 in 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: mgrmap e37: smithi002.zevrtm(active, since 20s), standbys: smithi026.omjkpo 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:41.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:46:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:42 smithi026 ceph-mon[82600]: pgmap v14: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-03-16T10:46:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:42 smithi002 ceph-mon[94239]: pgmap v14: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-03-16T10:46:44.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:44.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:44.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:44 smithi026 ceph-mon[82600]: pgmap v15: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-03-16T10:46:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:44 smithi002 ceph-mon[94239]: pgmap v15: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-03-16T10:46:45.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:45 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:45.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:45 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:46.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:45 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:46.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:45 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:46:46.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:46 smithi026 ceph-mon[82600]: pgmap v16: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-03-16T10:46:47.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:46 smithi002 ceph-mon[94239]: pgmap v16: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 219 op/s 2024-03-16T10:46:48.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:48 smithi026 ceph-mon[82600]: pgmap v17: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-03-16T10:46:49.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:48 smithi002 ceph-mon[94239]: pgmap v17: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-03-16T10:46:51.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:50 smithi002 ceph-mon[94239]: pgmap v18: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 279 op/s 2024-03-16T10:46:51.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:50 smithi026 ceph-mon[82600]: pgmap v18: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 1.3 MiB/s wr, 279 op/s 2024-03-16T10:46:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:46:53.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:52 smithi002 ceph-mon[94239]: pgmap v19: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 356 op/s 2024-03-16T10:46:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:52 smithi026 ceph-mon[82600]: pgmap v19: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 356 op/s 2024-03-16T10:46:54.116 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-16T10:46:54.116 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-16T10:46:54.444 DEBUG:teuthology.parallel:result is None 2024-03-16T10:46:55.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:54 smithi026 ceph-mon[82600]: pgmap v20: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-03-16T10:46:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:54 smithi002 ceph-mon[94239]: pgmap v20: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-03-16T10:46:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:56 smithi026 ceph-mon[82600]: pgmap v21: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 198 op/s 2024-03-16T10:46:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:56 smithi002 ceph-mon[94239]: pgmap v21: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 198 op/s 2024-03-16T10:46:59.132 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:46:58 smithi026 ceph-mon[82600]: pgmap v22: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-03-16T10:46:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:46:58 smithi002 ceph-mon[94239]: pgmap v22: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-03-16T10:47:00.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:00 smithi026 ceph-mon[82600]: pgmap v23: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-03-16T10:47:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:00 smithi002 ceph-mon[94239]: pgmap v23: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-03-16T10:47:02.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: Reconfiguring mon.smithi002 (monmap changed)... 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:02.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:01 smithi026 ceph-mon[82600]: Reconfiguring daemon mon.smithi002 on smithi002 2024-03-16T10:47:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: Reconfiguring mon.smithi002 (monmap changed)... 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:01 smithi002 ceph-mon[94239]: Reconfiguring daemon mon.smithi002 on smithi002 2024-03-16T10:47:03.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:02 smithi026 ceph-mon[82600]: pgmap v24: 65 pgs: 65 active+clean; 271 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-03-16T10:47:03.525 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:03.525 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:03.525 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:47:03.525 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:47:03.526 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:03.526 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:02 smithi002 ceph-mon[94239]: pgmap v24: 65 pgs: 65 active+clean; 271 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.5 MiB/s wr, 228 op/s 2024-03-16T10:47:03.527 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:03.527 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:03.527 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi002.zevrtm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:47:03.527 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:47:03.527 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:03.528 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-16T10:47:03.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-16T10:47:03.809 DEBUG:teuthology.parallel:result is None 2024-03-16T10:47:03.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T10:47:03.845 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T10:47:03.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-16T10:47:03.882 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-16T10:47:03.882 DEBUG:teuthology.parallel:result is None 2024-03-16T10:47:04.103 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:03 smithi002 ceph-mon[94239]: Reconfiguring mgr.smithi002.zevrtm (monmap changed)... 2024-03-16T10:47:04.103 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:03 smithi002 ceph-mon[94239]: Reconfiguring daemon mgr.smithi002.zevrtm on smithi002 2024-03-16T10:47:04.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:03 smithi026 ceph-mon[82600]: Reconfiguring mgr.smithi002.zevrtm (monmap changed)... 2024-03-16T10:47:04.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:03 smithi026 ceph-mon[82600]: Reconfiguring daemon mgr.smithi002.zevrtm on smithi002 2024-03-16T10:47:05.161 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: pgmap v25: 65 pgs: 65 active+clean; 271 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 939 KiB/s wr, 151 op/s 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi002"}]: dispatch 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:05.162 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: pgmap v25: 65 pgs: 65 active+clean; 271 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 939 KiB/s wr, 151 op/s 2024-03-16T10:47:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-16T10:47:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi002"}]: dispatch 2024-03-16T10:47:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:05.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:05 smithi026 ceph-mon[82600]: Reconfiguring ceph-exporter.smithi002 (monmap changed)... 2024-03-16T10:47:05.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:05 smithi026 ceph-mon[82600]: Unable to update caps for client.ceph-exporter.smithi002 2024-03-16T10:47:05.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:05 smithi026 ceph-mon[82600]: Reconfiguring daemon ceph-exporter.smithi002 on smithi002 2024-03-16T10:47:06.202 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:05 smithi002 ceph-mon[94239]: Reconfiguring ceph-exporter.smithi002 (monmap changed)... 2024-03-16T10:47:06.202 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:05 smithi002 ceph-mon[94239]: Unable to update caps for client.ceph-exporter.smithi002 2024-03-16T10:47:06.202 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:05 smithi002 ceph-mon[94239]: Reconfiguring daemon ceph-exporter.smithi002 on smithi002 2024-03-16T10:47:07.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:06 smithi002 ceph-mon[94239]: pgmap v26: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 981 KiB/s wr, 175 op/s 2024-03-16T10:47:07.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:07.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:07.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:07.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:06 smithi026 ceph-mon[82600]: pgmap v26: 65 pgs: 65 active+clean; 265 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 981 KiB/s wr, 175 op/s 2024-03-16T10:47:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:08.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:07 smithi026 ceph-mon[82600]: Reconfiguring crash.smithi002 (monmap changed)... 2024-03-16T10:47:08.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:07 smithi026 ceph-mon[82600]: Reconfiguring daemon crash.smithi002 on smithi002 2024-03-16T10:47:08.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:07 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:08.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:07 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:08.285 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:07 smithi002 ceph-mon[94239]: Reconfiguring crash.smithi002 (monmap changed)... 2024-03-16T10:47:08.285 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:07 smithi002 ceph-mon[94239]: Reconfiguring daemon crash.smithi002 on smithi002 2024-03-16T10:47:08.285 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:07 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:08.285 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:07 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:09.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:08 smithi026 ceph-mon[82600]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1022 KiB/s wr, 202 op/s 2024-03-16T10:47:09.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:08 smithi026 ceph-mon[82600]: Reconfiguring osd.0 (monmap changed)... 2024-03-16T10:47:09.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T10:47:09.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:09.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:08 smithi026 ceph-mon[82600]: Reconfiguring daemon osd.0 on smithi002 2024-03-16T10:47:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:08 smithi002 ceph-mon[94239]: pgmap v27: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1022 KiB/s wr, 202 op/s 2024-03-16T10:47:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:08 smithi002 ceph-mon[94239]: Reconfiguring osd.0 (monmap changed)... 2024-03-16T10:47:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T10:47:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:08 smithi002 ceph-mon[94239]: Reconfiguring daemon osd.0 on smithi002 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 -- 172.21.15.2:0/3973709553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77181013c0 msgr2=0x7f77181037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 --2- 172.21.15.2:0/3973709553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77181013c0 0x7f77181037b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7714009910 tx=0x7f7714009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 -- 172.21.15.2:0/3973709553 shutdown_connections 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 --2- 172.21.15.2:0/3973709553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7718103cf0 0x7f77181060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 --2- 172.21.15.2:0/3973709553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77181013c0 0x7f77181037b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 -- 172.21.15.2:0/3973709553 >> 172.21.15.2:0/3973709553 conn(0x7f77180fadf0 msgr2=0x7f77180fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 -- 172.21.15.2:0/3973709553 shutdown_connections 2024-03-16T10:47:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 -- 172.21.15.2:0/3973709553 wait complete. 2024-03-16T10:47:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.489+0000 7f771eb9e700 1 Processor -- start 2024-03-16T10:47:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771eb9e700 1 -- start start 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771eb9e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77181013c0 0x7f7718196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771eb9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7718103cf0 0x7f7718196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771eb9e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718196e80 con 0x7f7718103cf0 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771eb9e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718196fc0 con 0x7f77181013c0 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771db9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77181013c0 0x7f7718196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771db9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77181013c0 0x7f7718196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:57746/0 (socket says 172.21.15.2:57746) 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771db9c700 1 -- 172.21.15.2:0/2322772382 learned_addr learned my addr 172.21.15.2:0/2322772382 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771d39b700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7718103cf0 0x7f7718196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771d39b700 1 -- 172.21.15.2:0/2322772382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77181013c0 msgr2=0x7f7718196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771d39b700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77181013c0 0x7f7718196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771d39b700 1 -- 172.21.15.2:0/2322772382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140095f0 con 0x7f7718103cf0 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771db9c700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77181013c0 0x7f7718196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:47:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.490+0000 7f771d39b700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7718103cf0 0x7f7718196860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f770800d8d0 tx=0x7f770800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.491+0000 7f770effd700 1 -- 172.21.15.2:0/2322772382 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f770800f860 con 0x7f7718103cf0 2024-03-16T10:47:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.491+0000 7f770effd700 1 -- 172.21.15.2:0/2322772382 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f770800fe80 con 0x7f7718103cf0 2024-03-16T10:47:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.491+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771819ba20 con 0x7f7718103cf0 2024-03-16T10:47:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.491+0000 7f770effd700 1 -- 172.21.15.2:0/2322772382 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7708010ea0 con 0x7f7718103cf0 2024-03-16T10:47:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.491+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771819bff0 con 0x7f7718103cf0 2024-03-16T10:47:09.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.492+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771804f2d0 con 0x7f7718103cf0 2024-03-16T10:47:09.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.492+0000 7f770effd700 1 -- 172.21.15.2:0/2322772382 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f770800e680 con 0x7f7718103cf0 2024-03-16T10:47:09.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.493+0000 7f770effd700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f770406e0d0 0x7f7704070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:09.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.493+0000 7f770effd700 1 -- 172.21.15.2:0/2322772382 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f770808dc00 con 0x7f7718103cf0 2024-03-16T10:47:09.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.493+0000 7f771db9c700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f770406e0d0 0x7f7704070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:09.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.493+0000 7f771db9c700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f770406e0d0 0x7f7704070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f77140051c0 tx=0x7f771400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:09.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.498+0000 7f770effd700 1 -- 172.21.15.2:0/2322772382 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7708093020 con 0x7f7718103cf0 2024-03-16T10:47:09.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.725+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77181046d0 con 0x7f770406e0d0 2024-03-16T10:47:09.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.726+0000 7f770effd700 1 -- 172.21.15.2:0/2322772382 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f77181046d0 con 0x7f770406e0d0 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f770406e0d0 msgr2=0x7f7704070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f770406e0d0 0x7f7704070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f77140051c0 tx=0x7f771400b540 comp rx=0 tx=0).stop 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7718103cf0 msgr2=0x7f7718196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7718103cf0 0x7f7718196860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f770800d8d0 tx=0x7f770800dbe0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 shutdown_connections 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f770406e0d0 0x7f7704070590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77181013c0 0x7f7718196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 --2- 172.21.15.2:0/2322772382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7718103cf0 0x7f7718196860 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 >> 172.21.15.2:0/2322772382 conn(0x7f77180fadf0 msgr2=0x7f77180fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 shutdown_connections 2024-03-16T10:47:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.729+0000 7f771eb9e700 1 -- 172.21.15.2:0/2322772382 wait complete. 2024-03-16T10:47:09.741 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 -- 172.21.15.2:0/1522172217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 msgr2=0x7f2afc0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 --2- 172.21.15.2:0/1522172217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc0ffdf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009910 tx=0x7f2af0009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 -- 172.21.15.2:0/1522172217 shutdown_connections 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 --2- 172.21.15.2:0/1522172217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2afc100bd0 0x7f2afc101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 --2- 172.21.15.2:0/1522172217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc0ffdf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 -- 172.21.15.2:0/1522172217 >> 172.21.15.2:0/1522172217 conn(0x7f2afc0faf70 msgr2=0x7f2afc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 -- 172.21.15.2:0/1522172217 shutdown_connections 2024-03-16T10:47:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.837+0000 7f2b04083700 1 -- 172.21.15.2:0/1522172217 wait complete. 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b04083700 1 Processor -- start 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b04083700 1 -- start start 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b04083700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b04083700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2afc100bd0 0x7f2afc196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b04083700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1971e0 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b04083700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc197320 con 0x7f2afc100bd0 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b01e1f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:09.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b01e1f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39022/0 (socket says 172.21.15.2:39022) 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.838+0000 7f2b01e1f700 1 -- 172.21.15.2:0/1546281736 learned_addr learned my addr 172.21.15.2:0/1546281736 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2b0161e700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2afc100bd0 0x7f2afc196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2b01e1f700 1 -- 172.21.15.2:0/1546281736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2afc100bd0 msgr2=0x7f2afc196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2b01e1f700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2afc100bd0 0x7f2afc196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2b01e1f700 1 -- 172.21.15.2:0/1546281736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00095f0 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2b01e1f700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc196680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2af00059d0 tx=0x7f2af000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2aeeffd700 1 -- 172.21.15.2:0/1546281736 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af001d070 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc19bcd0 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc19c1c0 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2aeeffd700 1 -- 172.21.15.2:0/1546281736 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2af0005dc0 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.839+0000 7f2aeeffd700 1 -- 172.21.15.2:0/1546281736 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af0021410 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.841+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc04ea80 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.841+0000 7f2aeeffd700 1 -- 172.21.15.2:0/1546281736 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f2af000f820 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.842+0000 7f2aeeffd700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2ae806e0e0 0x7f2ae80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:09.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.842+0000 7f2aeeffd700 1 -- 172.21.15.2:0/1546281736 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2af008f270 con 0x7f2afc0ff9d0 2024-03-16T10:47:09.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.842+0000 7f2b0161e700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2ae806e0e0 0x7f2ae80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:09.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.842+0000 7f2b0161e700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2ae806e0e0 0x7f2ae80705a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009a10 tx=0x7f2af8009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:09.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:09.847+0000 7f2aeeffd700 1 -- 172.21.15.2:0/1546281736 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2af005bff0 con 0x7f2afc0ff9d0 2024-03-16T10:47:10.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.084+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2afc068c00 con 0x7f2ae806e0e0 2024-03-16T10:47:10.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.085+0000 7f2aeeffd700 1 -- 172.21.15.2:0/1546281736 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f2afc068c00 con 0x7f2ae806e0e0 2024-03-16T10:47:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2ae806e0e0 msgr2=0x7f2ae80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2ae806e0e0 0x7f2ae80705a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009a10 tx=0x7f2af8009190 comp rx=0 tx=0).stop 2024-03-16T10:47:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 msgr2=0x7f2afc196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc196680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2af00059d0 tx=0x7f2af000bd70 comp rx=0 tx=0).stop 2024-03-16T10:47:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 shutdown_connections 2024-03-16T10:47:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2ae806e0e0 0x7f2ae80705a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2afc0ff9d0 0x7f2afc196680 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 --2- 172.21.15.2:0/1546281736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2afc100bd0 0x7f2afc196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 >> 172.21.15.2:0/1546281736 conn(0x7f2afc0faf70 msgr2=0x7f2afc0684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 shutdown_connections 2024-03-16T10:47:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.087+0000 7f2b04083700 1 -- 172.21.15.2:0/1546281736 wait complete. 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.192+0000 7fc785482700 1 -- 172.21.15.2:0/1721295485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780101ce0 msgr2=0x7fc780102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.192+0000 7fc785482700 1 --2- 172.21.15.2:0/1721295485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780101ce0 0x7fc780102140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc774009910 tx=0x7fc774009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 -- 172.21.15.2:0/1721295485 shutdown_connections 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 --2- 172.21.15.2:0/1721295485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780101ce0 0x7fc780102140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 --2- 172.21.15.2:0/1721295485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780100ae0 0x7fc780100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 -- 172.21.15.2:0/1721295485 >> 172.21.15.2:0/1721295485 conn(0x7fc7800fc0a0 msgr2=0x7fc7800fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 -- 172.21.15.2:0/1721295485 shutdown_connections 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 -- 172.21.15.2:0/1721295485 wait complete. 2024-03-16T10:47:10.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 Processor -- start 2024-03-16T10:47:10.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.193+0000 7fc785482700 1 -- start start 2024-03-16T10:47:10.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc785482700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780100ae0 0x7fc7801964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc785482700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101ce0 0x7fc780196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc785482700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc780197020 con 0x7fc780100ae0 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc785482700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc780197160 con 0x7fc780101ce0 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780100ae0 0x7fc7801964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780100ae0 0x7fc7801964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39050/0 (socket says 172.21.15.2:39050) 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77effd700 1 -- 172.21.15.2:0/4204357392 learned_addr learned my addr 172.21.15.2:0/4204357392 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77e7fc700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101ce0 0x7fc780196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77effd700 1 -- 172.21.15.2:0/4204357392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101ce0 msgr2=0x7fc780196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77effd700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101ce0 0x7fc780196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77effd700 1 -- 172.21.15.2:0/4204357392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7740095f0 con 0x7fc780100ae0 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc77effd700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780100ae0 0x7fc7801964c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc77000e8f0 tx=0x7fc77000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc767fff700 1 -- 172.21.15.2:0/4204357392 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc770009790 con 0x7fc780100ae0 2024-03-16T10:47:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc767fff700 1 -- 172.21.15.2:0/4204357392 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc77000c810 con 0x7fc780100ae0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78019bbc0 con 0x7fc780100ae0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc767fff700 1 -- 172.21.15.2:0/4204357392 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc770003d20 con 0x7fc780100ae0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.194+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78019c110 con 0x7fc780100ae0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.195+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc780190660 con 0x7fc780100ae0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.196+0000 7fc767fff700 1 -- 172.21.15.2:0/4204357392 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fc7700041f0 con 0x7fc780100ae0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.196+0000 7fc767fff700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc76806e0e0 0x7fc7680705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.196+0000 7fc767fff700 1 -- 172.21.15.2:0/4204357392 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc770014070 con 0x7fc780100ae0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.196+0000 7fc77e7fc700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc76806e0e0 0x7fc7680705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.197+0000 7fc77e7fc700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc76806e0e0 0x7fc7680705a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc77400b800 tx=0x7fc774009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:10.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.202+0000 7fc767fff700 1 -- 172.21.15.2:0/4204357392 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc77005b080 con 0x7fc780100ae0 2024-03-16T10:47:10.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.427+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7800611f0 con 0x7fc76806e0e0 2024-03-16T10:47:10.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.438+0000 7fc767fff700 1 -- 172.21.15.2:0/4204357392 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc7800611f0 con 0x7fc76806e0e0 2024-03-16T10:47:10.439 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (9m) 44s ago 10m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (11m) 44s ago 11m 7759k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (10m) 27s ago 9m 7843k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (10m) 44s ago 10m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 da0ec8dc7732 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (9m) 27s ago 9m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (9m) 44s ago 10m 85.7M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (5m) 44s ago 5m 17.4M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (5m) 44s ago 5m 14.3M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (5m) 27s ago 5m 212M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (5m) 27s ago 5m 16.2M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (2m) 44s ago 12m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (77s) 27s ago 9m 448M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (60s) 44s ago 12m 27.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:47:10.440 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (32s) 27s ago 9m 21.0M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (10m) 44s ago 10m 16.7M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (9m) 27s ago 9m 16.0M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (8m) 44s ago 8m 181M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (8m) 44s ago 8m 178M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (8m) 44s ago 7m 159M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (7m) 27s ago 7m 241M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (7m) 27s ago 7m 201M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (6m) 27s ago 6m 188M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:47:10.441 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (87s) 44s ago 10m 57.8M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc76806e0e0 msgr2=0x7fc7680705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc76806e0e0 0x7fc7680705a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc77400b800 tx=0x7fc774009f70 comp rx=0 tx=0).stop 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780100ae0 msgr2=0x7fc7801964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780100ae0 0x7fc7801964c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc77000e8f0 tx=0x7fc77000ec00 comp rx=0 tx=0).stop 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 shutdown_connections 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc76806e0e0 0x7fc7680705a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc780100ae0 0x7fc7801964c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 --2- 172.21.15.2:0/4204357392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc780101ce0 0x7fc780196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 >> 172.21.15.2:0/4204357392 conn(0x7fc7800fc0a0 msgr2=0x7fc7801052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:10.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 shutdown_connections 2024-03-16T10:47:10.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.440+0000 7fc785482700 1 -- 172.21.15.2:0/4204357392 wait complete. 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 -- 172.21.15.2:0/3568288632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 msgr2=0x7f9954102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 --2- 172.21.15.2:0/3568288632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954102530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9948009910 tx=0x7f9948009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 -- 172.21.15.2:0/3568288632 shutdown_connections 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 --2- 172.21.15.2:0/3568288632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99540feb90 0x7f99540feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 --2- 172.21.15.2:0/3568288632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954102530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 -- 172.21.15.2:0/3568288632 >> 172.21.15.2:0/3568288632 conn(0x7f99540fa730 msgr2=0x7f99540fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 -- 172.21.15.2:0/3568288632 shutdown_connections 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.547+0000 7f995aef6700 1 -- 172.21.15.2:0/3568288632 wait complete. 2024-03-16T10:47:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f995aef6700 1 Processor -- start 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f995aef6700 1 -- start start 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f995aef6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99540feb90 0x7f99541965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f995aef6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f995aef6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954197150 con 0x7f9954102110 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f995aef6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954197290 con 0x7f99540feb90 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f9953fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f9953fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39072/0 (socket says 172.21.15.2:39072) 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f9953fff700 1 -- 172.21.15.2:0/4056080879 learned_addr learned my addr 172.21.15.2:0/4056080879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f9958c92700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99540feb90 0x7f99541965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f9953fff700 1 -- 172.21.15.2:0/4056080879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99540feb90 msgr2=0x7f99541965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f9953fff700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99540feb90 0x7f99541965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.548+0000 7f9953fff700 1 -- 172.21.15.2:0/4056080879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99480095f0 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.549+0000 7f9953fff700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954196b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f994000b4b0 tx=0x7f994000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.549+0000 7f9951ffb700 1 -- 172.21.15.2:0/4056080879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940011800 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.549+0000 7f9951ffb700 1 -- 172.21.15.2:0/4056080879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9940011e20 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.549+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995419bcf0 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.549+0000 7f9951ffb700 1 -- 172.21.15.2:0/4056080879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994000f2e0 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.549+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995419c2c0 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.550+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99541906f0 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.550+0000 7f9951ffb700 1 -- 172.21.15.2:0/4056080879 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f9940010380 con 0x7f9954102110 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.551+0000 7f9951ffb700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f994406e0e0 0x7f99440705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.551+0000 7f9951ffb700 1 -- 172.21.15.2:0/4056080879 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f994008d960 con 0x7f9954102110 2024-03-16T10:47:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.551+0000 7f9958c92700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f994406e0e0 0x7f99440705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.551+0000 7f9958c92700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f994406e0e0 0x7f99440705a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f994800b5c0 tx=0x7f9948005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:10.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.556+0000 7f9951ffb700 1 -- 172.21.15.2:0/4056080879 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f994005a870 con 0x7f9954102110 2024-03-16T10:47:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: Reconfiguring osd.1 (monmap changed)... 2024-03-16T10:47:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:47:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:10.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: Reconfiguring daemon osd.1 on smithi002 2024-03-16T10:47:10.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:10.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:10 smithi002 ceph-mon[94239]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 744 KiB/s wr, 152 op/s 2024-03-16T10:47:10.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.858+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f995404ea80 con 0x7f9954102110 2024-03-16T10:47:10.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.858+0000 7f9951ffb700 1 -- 172.21.15.2:0/4056080879 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7f994005a400 con 0x7f9954102110 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:47:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 10, 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:47:10.862 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:47:10.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f994406e0e0 msgr2=0x7f99440705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f994406e0e0 0x7f99440705a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f994800b5c0 tx=0x7f9948005960 comp rx=0 tx=0).stop 2024-03-16T10:47:10.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 msgr2=0x7f9954196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954196b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f994000b4b0 tx=0x7f994000b7c0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 shutdown_connections 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f994406e0e0 0x7f99440705a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99540feb90 0x7f99541965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 --2- 172.21.15.2:0/4056080879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9954102110 0x7f9954196b30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 >> 172.21.15.2:0/4056080879 conn(0x7f99540fa730 msgr2=0x7f9954105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.860+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 shutdown_connections 2024-03-16T10:47:10.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.861+0000 7f995aef6700 1 -- 172.21.15.2:0/4056080879 wait complete. 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 -- 172.21.15.2:0/3686403986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 msgr2=0x7f1a9c0ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/3686403986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c0ff1c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009910 tx=0x7f1a8c009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 -- 172.21.15.2:0/3686403986 shutdown_connections 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/3686403986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9c101d80 0x7f1a9c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/3686403986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c0ff1c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 -- 172.21.15.2:0/3686403986 >> 172.21.15.2:0/3686403986 conn(0x7f1a9c0fa700 msgr2=0x7f1a9c0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 -- 172.21.15.2:0/3686403986 shutdown_connections 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 -- 172.21.15.2:0/3686403986 wait complete. 2024-03-16T10:47:10.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.966+0000 7f1aa1d18700 1 Processor -- start 2024-03-16T10:47:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1aa1d18700 1 -- start start 2024-03-16T10:47:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1aa1d18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1aa1d18700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9c101d80 0x7f1a9c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1aa1d18700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c199470 con 0x7f1a9c0feda0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1aa1d18700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c1995b0 con 0x7f1a9c101d80 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39090/0 (socket says 172.21.15.2:39090) 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9b7fe700 1 -- 172.21.15.2:0/1161562028 learned_addr learned my addr 172.21.15.2:0/1161562028 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9affd700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9c101d80 0x7f1a9c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9b7fe700 1 -- 172.21.15.2:0/1161562028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9c101d80 msgr2=0x7f1a9c198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9b7fe700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9c101d80 0x7f1a9c198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9b7fe700 1 -- 172.21.15.2:0/1161562028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c0095f0 con 0x7f1a9c0feda0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9affd700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9c101d80 0x7f1a9c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.967+0000 7f1a9b7fe700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c198910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c005850 tx=0x7f1a8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.968+0000 7f1a98ff9700 1 -- 172.21.15.2:0/1161562028 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a8c01d070 con 0x7f1a9c0feda0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.968+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c19dfb0 con 0x7f1a9c0feda0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.968+0000 7f1a98ff9700 1 -- 172.21.15.2:0/1161562028 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a8c00f460 con 0x7f1a9c0feda0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.968+0000 7f1a98ff9700 1 -- 172.21.15.2:0/1161562028 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a8c017610 con 0x7f1a9c0feda0 2024-03-16T10:47:10.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.968+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c19e4a0 con 0x7f1a9c0feda0 2024-03-16T10:47:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.969+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c106750 con 0x7f1a9c0feda0 2024-03-16T10:47:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.969+0000 7f1a98ff9700 1 -- 172.21.15.2:0/1161562028 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f1a8c00fab0 con 0x7f1a9c0feda0 2024-03-16T10:47:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.969+0000 7f1a98ff9700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a8406e0e0 0x7f1a840705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:10.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.970+0000 7f1a98ff9700 1 -- 172.21.15.2:0/1161562028 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1a8c08f4b0 con 0x7f1a9c0feda0 2024-03-16T10:47:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.970+0000 7f1a9affd700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a8406e0e0 0x7f1a840705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.970+0000 7f1a9affd700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a8406e0e0 0x7f1a840705a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1a90007900 tx=0x7f1a90008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:10.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:10.974+0000 7f1a98ff9700 1 -- 172.21.15.2:0/1161562028 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a8c05c230 con 0x7f1a9c0feda0 2024-03-16T10:47:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: Reconfiguring osd.1 (monmap changed)... 2024-03-16T10:47:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:47:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:10.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: Reconfiguring daemon osd.1 on smithi002 2024-03-16T10:47:10.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:10.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:10 smithi026 ceph-mon[82600]: pgmap v28: 65 pgs: 65 active+clean; 258 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 744 KiB/s wr, 152 op/s 2024-03-16T10:47:11.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.227+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a9c02ccb0 con 0x7f1a9c0feda0 2024-03-16T10:47:11.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.227+0000 7f1a98ff9700 1 -- 172.21.15.2:0/1161562028 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f1a8c05bdc0 con 0x7f1a9c0feda0 2024-03-16T10:47:11.229 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:47:11.229 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:47:11.230 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:47:11.231 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:11.232 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a8406e0e0 msgr2=0x7f1a840705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a8406e0e0 0x7f1a840705a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1a90007900 tx=0x7f1a90008040 comp rx=0 tx=0).stop 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 msgr2=0x7f1a9c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c198910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c005850 tx=0x7f1a8c00be20 comp rx=0 tx=0).stop 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 shutdown_connections 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a8406e0e0 0x7f1a840705a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a9c0feda0 0x7f1a9c198910 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 --2- 172.21.15.2:0/1161562028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a9c101d80 0x7f1a9c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 >> 172.21.15.2:0/1161562028 conn(0x7f1a9c0fa700 msgr2=0x7f1a9c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:11.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.230+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 shutdown_connections 2024-03-16T10:47:11.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.231+0000 7f1aa1d18700 1 -- 172.21.15.2:0/1161562028 wait complete. 2024-03-16T10:47:11.234 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:47:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 -- 172.21.15.2:0/2584875013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0101ce0 msgr2=0x7f70c0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 --2- 172.21.15.2:0/2584875013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0101ce0 0x7f70c0102140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009910 tx=0x7f70b4009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 -- 172.21.15.2:0/2584875013 shutdown_connections 2024-03-16T10:47:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 --2- 172.21.15.2:0/2584875013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0101ce0 0x7f70c0102140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 --2- 172.21.15.2:0/2584875013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70c0100ae0 0x7f70c0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 -- 172.21.15.2:0/2584875013 >> 172.21.15.2:0/2584875013 conn(0x7f70c00fc0a0 msgr2=0x7f70c00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 -- 172.21.15.2:0/2584875013 shutdown_connections 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.349+0000 7f70c53f4700 1 -- 172.21.15.2:0/2584875013 wait complete. 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70c53f4700 1 Processor -- start 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70c53f4700 1 -- start start 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70c53f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0100ae0 0x7f70c01964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70c53f4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70c0101ce0 0x7f70c0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70c53f4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c0197030 con 0x7f70c0100ae0 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70c53f4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c0197170 con 0x7f70c0101ce0 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0100ae0 0x7f70c01964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0100ae0 0x7f70c01964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39106/0 (socket says 172.21.15.2:39106) 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70beffd700 1 -- 172.21.15.2:0/1546396885 learned_addr learned my addr 172.21.15.2:0/1546396885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70be7fc700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70c0101ce0 0x7f70c0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70beffd700 1 -- 172.21.15.2:0/1546396885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70c0101ce0 msgr2=0x7f70c0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70beffd700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70c0101ce0 0x7f70c0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.350+0000 7f70beffd700 1 -- 172.21.15.2:0/1546396885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40095f0 con 0x7f70c0100ae0 2024-03-16T10:47:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.351+0000 7f70beffd700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0100ae0 0x7f70c01964d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f70b000d900 tx=0x7f70b000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.351+0000 7f70a7fff700 1 -- 172.21.15.2:0/1546396885 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b000b920 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.351+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c019bb80 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.351+0000 7f70a7fff700 1 -- 172.21.15.2:0/1546396885 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70b0010460 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.351+0000 7f70a7fff700 1 -- 172.21.15.2:0/1546396885 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b000f590 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.351+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c019c0d0 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.352+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c0190660 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.353+0000 7f70a7fff700 1 -- 172.21.15.2:0/1546396885 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f70b00105d0 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.353+0000 7f70a7fff700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f70a806e0e0 0x7f70a80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.353+0000 7f70a7fff700 1 -- 172.21.15.2:0/1546396885 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f70b008dee0 con 0x7f70c0100ae0 2024-03-16T10:47:11.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.353+0000 7f70be7fc700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f70a806e0e0 0x7f70a80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:11.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.354+0000 7f70be7fc700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f70a806e0e0 0x7f70a80705a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70b4005090 tx=0x7f70b4005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:11.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.358+0000 7f70a7fff700 1 -- 172.21.15.2:0/1546396885 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f70b005ace0 con 0x7f70c0100ae0 2024-03-16T10:47:11.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.587+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70c00611f0 con 0x7f70a806e0e0 2024-03-16T10:47:11.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.589+0000 7f70a7fff700 1 -- 172.21.15.2:0/1546396885 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f70c00611f0 con 0x7f70a806e0e0 2024-03-16T10:47:11.590 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:47:11.590 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "4/23 daemons upgraded", 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading mon daemons", 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:47:11.591 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:47:11.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f70a806e0e0 msgr2=0x7f70a80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f70a806e0e0 0x7f70a80705a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f70b4005090 tx=0x7f70b4005960 comp rx=0 tx=0).stop 2024-03-16T10:47:11.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0100ae0 msgr2=0x7f70c01964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0100ae0 0x7f70c01964d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f70b000d900 tx=0x7f70b000dc10 comp rx=0 tx=0).stop 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 shutdown_connections 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f70a806e0e0 0x7f70a80705a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70c0100ae0 0x7f70c01964d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 --2- 172.21.15.2:0/1546396885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70c0101ce0 0x7f70c0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 >> 172.21.15.2:0/1546396885 conn(0x7f70c00fc0a0 msgr2=0x7f70c01052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 shutdown_connections 2024-03-16T10:47:11.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.591+0000 7f70c53f4700 1 -- 172.21.15.2:0/1546396885 wait complete. 2024-03-16T10:47:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 -- 172.21.15.2:0/1612882947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 msgr2=0x7ff79c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 --2- 172.21.15.2:0/1612882947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c100ef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009910 tx=0x7ff78c009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 -- 172.21.15.2:0/1612882947 shutdown_connections 2024-03-16T10:47:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 --2- 172.21.15.2:0/1612882947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79c101cd0 0x7ff79c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 --2- 172.21.15.2:0/1612882947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c100ef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 -- 172.21.15.2:0/1612882947 >> 172.21.15.2:0/1612882947 conn(0x7ff79c0fc090 msgr2=0x7ff79c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 -- 172.21.15.2:0/1612882947 shutdown_connections 2024-03-16T10:47:11.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.699+0000 7ff7a2093700 1 -- 172.21.15.2:0/1612882947 wait complete. 2024-03-16T10:47:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.700+0000 7ff7a2093700 1 Processor -- start 2024-03-16T10:47:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.700+0000 7ff7a2093700 1 -- start start 2024-03-16T10:47:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.700+0000 7ff7a2093700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.700+0000 7ff7a2093700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79c101cd0 0x7ff79c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.700+0000 7ff7a2093700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0736e0 con 0x7ff79c100ad0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.700+0000 7ff7a2093700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c073820 con 0x7ff79c101cd0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.700+0000 7ff79b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff79b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39130/0 (socket says 172.21.15.2:39130) 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff79b7fe700 1 -- 172.21.15.2:0/2785668048 learned_addr learned my addr 172.21.15.2:0/2785668048 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff79affd700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79c101cd0 0x7ff79c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff79b7fe700 1 -- 172.21.15.2:0/2785668048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79c101cd0 msgr2=0x7ff79c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff79b7fe700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79c101cd0 0x7ff79c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff79b7fe700 1 -- 172.21.15.2:0/2785668048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0095f0 con 0x7ff79c100ad0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff79b7fe700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c074ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0059d0 tx=0x7ff78c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff798ff9700 1 -- 172.21.15.2:0/2785668048 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff78c01d070 con 0x7ff79c100ad0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c073aa0 con 0x7ff79c100ad0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c074010 con 0x7ff79c100ad0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff798ff9700 1 -- 172.21.15.2:0/2785668048 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff78c00f460 con 0x7ff79c100ad0 2024-03-16T10:47:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.701+0000 7ff798ff9700 1 -- 172.21.15.2:0/2785668048 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff78c021620 con 0x7ff79c100ad0 2024-03-16T10:47:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.702+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c1906d0 con 0x7ff79c100ad0 2024-03-16T10:47:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.703+0000 7ff798ff9700 1 -- 172.21.15.2:0/2785668048 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7ff78c00fa70 con 0x7ff79c100ad0 2024-03-16T10:47:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.703+0000 7ff798ff9700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff78406e0e0 0x7ff7840705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.703+0000 7ff798ff9700 1 -- 172.21.15.2:0/2785668048 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff78c08f130 con 0x7ff79c100ad0 2024-03-16T10:47:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.703+0000 7ff79affd700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff78406e0e0 0x7ff7840705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:11.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.704+0000 7ff79affd700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff78406e0e0 0x7ff7840705a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff790006fd0 tx=0x7ff790008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:11.708+0000 7ff798ff9700 1 -- 172.21.15.2:0/2785668048 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff78c05bf30 con 0x7ff79c100ad0 2024-03-16T10:47:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/4056080879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1161562028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:47:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:47:11.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:11.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:11.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:11.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/4056080879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1161562028' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:47:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:47:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:12.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.006+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff79c074010 con 0x7ff79c100ad0 2024-03-16T10:47:12.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.006+0000 7ff798ff9700 1 -- 172.21.15.2:0/2785668048 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff79c074010 con 0x7ff79c100ad0 2024-03-16T10:47:12.008 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:47:12.008 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:47:12.008 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:47:12.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.008+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff78406e0e0 msgr2=0x7ff7840705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.008+0000 7ff7a2093700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff78406e0e0 0x7ff7840705a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff790006fd0 tx=0x7ff790008040 comp rx=0 tx=0).stop 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 msgr2=0x7ff79c074ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c074ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0059d0 tx=0x7ff78c00be20 comp rx=0 tx=0).stop 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 shutdown_connections 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff78406e0e0 0x7ff7840705a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c100ad0 0x7ff79c074ab0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 --2- 172.21.15.2:0/2785668048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79c101cd0 0x7ff79c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 >> 172.21.15.2:0/2785668048 conn(0x7ff79c0fc090 msgr2=0x7ff79c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 shutdown_connections 2024-03-16T10:47:12.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:12.009+0000 7ff7a2093700 1 -- 172.21.15.2:0/2785668048 wait complete. 2024-03-16T10:47:12.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:12 smithi002 ceph-mon[94239]: Reconfiguring osd.2 (monmap changed)... 2024-03-16T10:47:12.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:12 smithi002 ceph-mon[94239]: Reconfiguring daemon osd.2 on smithi002 2024-03-16T10:47:12.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:12 smithi002 ceph-mon[94239]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:12.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:12 smithi002 ceph-mon[94239]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 776 KiB/s wr, 153 op/s 2024-03-16T10:47:12.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:12 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2785668048' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:47:12.816 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:12 smithi026 ceph-mon[82600]: Reconfiguring osd.2 (monmap changed)... 2024-03-16T10:47:12.816 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:12 smithi026 ceph-mon[82600]: Reconfiguring daemon osd.2 on smithi002 2024-03-16T10:47:12.816 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:12 smithi026 ceph-mon[82600]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:12.816 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:12 smithi026 ceph-mon[82600]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 776 KiB/s wr, 153 op/s 2024-03-16T10:47:12.817 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:12 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2785668048' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:47:14.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:14.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:14.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:14.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:14.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:14.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:14.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:14.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:15.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:15 smithi002 ceph-mon[94239]: Reconfiguring mds.cephfs.smithi002.qvbseu (monmap changed)... 2024-03-16T10:47:15.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:15 smithi002 ceph-mon[94239]: Reconfiguring daemon mds.cephfs.smithi002.qvbseu on smithi002 2024-03-16T10:47:15.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:15 smithi002 ceph-mon[94239]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 342 KiB/s wr, 71 op/s 2024-03-16T10:47:15.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:15 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:15.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:15 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:15.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:15 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:15.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:15 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:15 smithi026 ceph-mon[82600]: Reconfiguring mds.cephfs.smithi002.qvbseu (monmap changed)... 2024-03-16T10:47:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:15 smithi026 ceph-mon[82600]: Reconfiguring daemon mds.cephfs.smithi002.qvbseu on smithi002 2024-03-16T10:47:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:15 smithi026 ceph-mon[82600]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 342 KiB/s wr, 71 op/s 2024-03-16T10:47:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:15 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:15 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:15 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:15.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:15 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:16.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:16 smithi002 ceph-mon[94239]: Reconfiguring mds.cephfs.smithi002.ijcmqs (monmap changed)... 2024-03-16T10:47:16.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:16 smithi002 ceph-mon[94239]: Reconfiguring daemon mds.cephfs.smithi002.ijcmqs on smithi002 2024-03-16T10:47:16.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:16 smithi026 ceph-mon[82600]: Reconfiguring mds.cephfs.smithi002.ijcmqs (monmap changed)... 2024-03-16T10:47:16.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:16 smithi026 ceph-mon[82600]: Reconfiguring daemon mds.cephfs.smithi002.ijcmqs on smithi002 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 342 KiB/s wr, 71 op/s 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi026"}]: dispatch 2024-03-16T10:47:17.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:17.350 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 342 KiB/s wr, 71 op/s 2024-03-16T10:47:17.350 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:17.350 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:17.350 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:17.350 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:47:17.351 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-16T10:47:17.351 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi026"}]: dispatch 2024-03-16T10:47:17.351 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:18.216 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:18 smithi026 ceph-mon[82600]: Reconfiguring ceph-exporter.smithi026 (monmap changed)... 2024-03-16T10:47:18.217 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:18 smithi026 ceph-mon[82600]: Unable to update caps for client.ceph-exporter.smithi026 2024-03-16T10:47:18.217 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:18 smithi026 ceph-mon[82600]: Reconfiguring daemon ceph-exporter.smithi026 on smithi026 2024-03-16T10:47:18.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:18 smithi002 ceph-mon[94239]: Reconfiguring ceph-exporter.smithi026 (monmap changed)... 2024-03-16T10:47:18.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:18 smithi002 ceph-mon[94239]: Unable to update caps for client.ceph-exporter.smithi026 2024-03-16T10:47:18.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:18 smithi002 ceph-mon[94239]: Reconfiguring daemon ceph-exporter.smithi026 on smithi026 2024-03-16T10:47:19.046 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:19 smithi026 ceph-mon[82600]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 300 KiB/s wr, 48 op/s 2024-03-16T10:47:19.046 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:19.046 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:19.046 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:19.047 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:19 smithi002 ceph-mon[94239]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 300 KiB/s wr, 48 op/s 2024-03-16T10:47:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:19.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:19.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:20.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: Reconfiguring crash.smithi026 (monmap changed)... 2024-03-16T10:47:20.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: Reconfiguring daemon crash.smithi026 on smithi026 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 32 KiB/s wr, 1 op/s 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: Reconfiguring mgr.smithi026.omjkpo (monmap changed)... 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:20.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:20 smithi026 ceph-mon[82600]: Reconfiguring daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:47:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: Reconfiguring crash.smithi026 (monmap changed)... 2024-03-16T10:47:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: Reconfiguring daemon crash.smithi026 on smithi026 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 32 KiB/s wr, 1 op/s 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: Reconfiguring mgr.smithi026.omjkpo (monmap changed)... 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi026.omjkpo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:20.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:20 smithi002 ceph-mon[94239]: Reconfiguring daemon mgr.smithi026.omjkpo on smithi026 2024-03-16T10:47:22.900 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:22.900 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:22.900 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: Reconfiguring mon.smithi026 (monmap changed)... 2024-03-16T10:47:22.900 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:47:22.900 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:47:22.900 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:22.900 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: Reconfiguring daemon mon.smithi026 on smithi026 2024-03-16T10:47:22.901 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:22 smithi026 ceph-mon[82600]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 32 KiB/s wr, 1 op/s 2024-03-16T10:47:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: Reconfiguring mon.smithi026 (monmap changed)... 2024-03-16T10:47:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T10:47:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T10:47:23.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:23.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: Reconfiguring daemon mon.smithi026 on smithi026 2024-03-16T10:47:23.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:22 smithi002 ceph-mon[94239]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 32 KiB/s wr, 1 op/s 2024-03-16T10:47:24.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:24 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:24.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:24 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:24.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:24 smithi026 ceph-mon[82600]: Reconfiguring osd.3 (monmap changed)... 2024-03-16T10:47:24.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:24 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:47:24.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:24 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:24.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:24 smithi026 ceph-mon[82600]: Reconfiguring daemon osd.3 on smithi026 2024-03-16T10:47:24.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:24 smithi026 ceph-mon[82600]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:47:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:24 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:24 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:24.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:24 smithi002 ceph-mon[94239]: Reconfiguring osd.3 (monmap changed)... 2024-03-16T10:47:24.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:24 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:47:24.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:24 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:24.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:24 smithi002 ceph-mon[94239]: Reconfiguring daemon osd.3 on smithi026 2024-03-16T10:47:24.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:24 smithi002 ceph-mon[94239]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:47:26.172 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:25 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:26.173 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:25 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:26.173 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:25 smithi026 ceph-mon[82600]: Reconfiguring osd.4 (monmap changed)... 2024-03-16T10:47:26.173 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:25 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:47:26.173 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:25 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:26.173 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:25 smithi026 ceph-mon[82600]: Reconfiguring daemon osd.4 on smithi026 2024-03-16T10:47:26.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:25 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:26.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:25 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:26.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:25 smithi002 ceph-mon[94239]: Reconfiguring osd.4 (monmap changed)... 2024-03-16T10:47:26.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:25 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:47:26.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:25 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:26.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:25 smithi002 ceph-mon[94239]: Reconfiguring daemon osd.4 on smithi026 2024-03-16T10:47:27.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:26 smithi026 ceph-mon[82600]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:47:27.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:47:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:26 smithi002 ceph-mon[94239]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:47:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:27.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:47:27.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:28.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:27 smithi026 ceph-mon[82600]: Reconfiguring osd.5 (monmap changed)... 2024-03-16T10:47:28.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:27 smithi026 ceph-mon[82600]: Reconfiguring daemon osd.5 on smithi026 2024-03-16T10:47:28.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:27 smithi002 ceph-mon[94239]: Reconfiguring osd.5 (monmap changed)... 2024-03-16T10:47:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:27 smithi002 ceph-mon[94239]: Reconfiguring daemon osd.5 on smithi026 2024-03-16T10:47:29.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:28 smithi026 ceph-mon[82600]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:47:29.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:29.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:29.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:29.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:28 smithi002 ceph-mon[94239]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-16T10:47:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:29 smithi026 ceph-mon[82600]: Reconfiguring mds.cephfs.smithi026.dzzagf (monmap changed)... 2024-03-16T10:47:30.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:29 smithi026 ceph-mon[82600]: Reconfiguring daemon mds.cephfs.smithi026.dzzagf on smithi026 2024-03-16T10:47:30.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:29 smithi002 ceph-mon[94239]: Reconfiguring mds.cephfs.smithi026.dzzagf (monmap changed)... 2024-03-16T10:47:30.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:29 smithi002 ceph-mon[94239]: Reconfiguring daemon mds.cephfs.smithi026.dzzagf on smithi026 2024-03-16T10:47:31.151 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:30 smithi026 ceph-mon[82600]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:30 smithi002 ceph-mon[94239]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:32 smithi026 ceph-mon[82600]: Reconfiguring mds.cephfs.smithi026.ojzcsw (monmap changed)... 2024-03-16T10:47:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:32 smithi026 ceph-mon[82600]: Reconfiguring daemon mds.cephfs.smithi026.ojzcsw on smithi026 2024-03-16T10:47:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:32 smithi026 ceph-mon[82600]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:33.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:33.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:33.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:32 smithi002 ceph-mon[94239]: Reconfiguring mds.cephfs.smithi026.ojzcsw (monmap changed)... 2024-03-16T10:47:33.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:47:33.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:33.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:32 smithi002 ceph-mon[94239]: Reconfiguring daemon mds.cephfs.smithi026.ojzcsw on smithi026 2024-03-16T10:47:33.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:32 smithi002 ceph-mon[94239]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:34.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:34.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:34.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:47:34.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:34.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:34.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all mon 2024-03-16T10:47:34.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:34.733 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi002"}]: dispatch 2024-03-16T10:47:34.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi002"}]': finished 2024-03-16T10:47:34.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi026"}]: dispatch 2024-03-16T10:47:34.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi026"}]': finished 2024-03-16T10:47:34.734 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:34 smithi026 ceph-mon[82600]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all mon 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi002"}]: dispatch 2024-03-16T10:47:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi002"}]': finished 2024-03-16T10:47:34.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi026"}]: dispatch 2024-03-16T10:47:34.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi026"}]': finished 2024-03-16T10:47:34.800 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:34 smithi002 ceph-mon[94239]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:35.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:35.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:35.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:35.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:35.951 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:35.952 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:35.952 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi002", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:35.952 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:36.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:36 smithi002 ceph-mon[94239]: Upgrade: Updating crash.smithi002 (1/2) 2024-03-16T10:47:36.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:36 smithi002 ceph-mon[94239]: Deploying daemon crash.smithi002 on smithi002 2024-03-16T10:47:36.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:36 smithi002 ceph-mon[94239]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:36.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:36 smithi026 ceph-mon[82600]: Upgrade: Updating crash.smithi002 (1/2) 2024-03-16T10:47:36.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:36 smithi026 ceph-mon[82600]: Deploying daemon crash.smithi002 on smithi002 2024-03-16T10:47:36.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:36 smithi026 ceph-mon[82600]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:39.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:38 smithi002 ceph-mon[94239]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:38 smithi026 ceph-mon[82600]: pgmap v42: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:40.451 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:40.451 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:40.452 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:40 smithi026 ceph-mon[82600]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:40.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:40.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:40.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:40 smithi002 ceph-mon[94239]: pgmap v43: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:41.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:41 smithi026 ceph-mon[82600]: Upgrade: Updating crash.smithi026 (2/2) 2024-03-16T10:47:41.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:41 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:41.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:41 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:41.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:41 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:41.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:41 smithi026 ceph-mon[82600]: Deploying daemon crash.smithi026 on smithi026 2024-03-16T10:47:42.133 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:41 smithi002 ceph-mon[94239]: Upgrade: Updating crash.smithi026 (2/2) 2024-03-16T10:47:42.133 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:41 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:42.133 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:41 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi026", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-16T10:47:42.133 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:41 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:42.133 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:41 smithi002 ceph-mon[94239]: Deploying daemon crash.smithi026 on smithi026 2024-03-16T10:47:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2202211947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 msgr2=0x7f0aac1062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2202211947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1062b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009910 tx=0x7f0aa8009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2202211947 shutdown_connections 2024-03-16T10:47:42.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2202211947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1062b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2202211947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aac101590 0x7f0aac103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2202211947 >> 172.21.15.2:0/2202211947 conn(0x7f0aac0fafc0 msgr2=0x7f0aac0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2202211947 shutdown_connections 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.131+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2202211947 wait complete. 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab37f9700 1 Processor -- start 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab37f9700 1 -- start start 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab37f9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aac101590 0x7f0aac1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab37f9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab37f9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac197000 con 0x7f0aac103ec0 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab37f9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac197140 con 0x7f0aac101590 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab0d94700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab0d94700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47122/0 (socket says 172.21.15.2:47122) 2024-03-16T10:47:42.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab0d94700 1 -- 172.21.15.2:0/2574045230 learned_addr learned my addr 172.21.15.2:0/2574045230 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.132+0000 7f0ab1595700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aac101590 0x7f0aac1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0ab0d94700 1 -- 172.21.15.2:0/2574045230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aac101590 msgr2=0x7f0aac1964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0ab0d94700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aac101590 0x7f0aac1964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0ab0d94700 1 -- 172.21.15.2:0/2574045230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80095f0 con 0x7f0aac103ec0 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0ab0d94700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1969e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8006010 tx=0x7f0aa800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0aa27fc700 1 -- 172.21.15.2:0/2574045230 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aa801d070 con 0x7f0aac103ec0 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0aa27fc700 1 -- 172.21.15.2:0/2574045230 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aa800f460 con 0x7f0aac103ec0 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0aa27fc700 1 -- 172.21.15.2:0/2574045230 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aa8021640 con 0x7f0aac103ec0 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac19bb40 con 0x7f0aac103ec0 2024-03-16T10:47:42.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.133+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac19c0b0 con 0x7f0aac103ec0 2024-03-16T10:47:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.134+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac04ea80 con 0x7f0aac103ec0 2024-03-16T10:47:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.135+0000 7f0aa27fc700 1 -- 172.21.15.2:0/2574045230 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f0aa800fab0 con 0x7f0aac103ec0 2024-03-16T10:47:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.135+0000 7f0aa27fc700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0a9806e0e0 0x7f0a980705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.135+0000 7f0aa27fc700 1 -- 172.21.15.2:0/2574045230 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0aa808f300 con 0x7f0aac103ec0 2024-03-16T10:47:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.135+0000 7f0ab1595700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0a9806e0e0 0x7f0a980705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.136+0000 7f0ab1595700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0a9806e0e0 0x7f0a980705a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005fd0 tx=0x7f0a9c00a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:42.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.140+0000 7f0aa27fc700 1 -- 172.21.15.2:0/2574045230 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0aa805c100 con 0x7f0aac103ec0 2024-03-16T10:47:42.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.366+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0aac105da0 con 0x7f0a9806e0e0 2024-03-16T10:47:42.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.368+0000 7f0aa27fc700 1 -- 172.21.15.2:0/2574045230 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f0aac105da0 con 0x7f0a9806e0e0 2024-03-16T10:47:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0a9806e0e0 msgr2=0x7f0a980705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0a9806e0e0 0x7f0a980705a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005fd0 tx=0x7f0a9c00a560 comp rx=0 tx=0).stop 2024-03-16T10:47:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 msgr2=0x7f0aac1969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1969e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8006010 tx=0x7f0aa800be20 comp rx=0 tx=0).stop 2024-03-16T10:47:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 shutdown_connections 2024-03-16T10:47:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0a9806e0e0 0x7f0a980705a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aac101590 0x7f0aac1964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 --2- 172.21.15.2:0/2574045230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0aac103ec0 0x7f0aac1969e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 >> 172.21.15.2:0/2574045230 conn(0x7f0aac0fafc0 msgr2=0x7f0aac0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 shutdown_connections 2024-03-16T10:47:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.370+0000 7f0ab37f9700 1 -- 172.21.15.2:0/2574045230 wait complete. 2024-03-16T10:47:42.382 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 -- 172.21.15.2:0/1668613315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 msgr2=0x7ff7d80ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/1668613315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d80ff1f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800b0a0 tx=0x7ff7c800b3b0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 -- 172.21.15.2:0/1668613315 shutdown_connections 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/1668613315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d8101e50 0x7ff7d81022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/1668613315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d80ff1f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 -- 172.21.15.2:0/1668613315 >> 172.21.15.2:0/1668613315 conn(0x7ff7d80fa730 msgr2=0x7ff7d80fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 -- 172.21.15.2:0/1668613315 shutdown_connections 2024-03-16T10:47:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 -- 172.21.15.2:0/1668613315 wait complete. 2024-03-16T10:47:42.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 Processor -- start 2024-03-16T10:47:42.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.477+0000 7ff7df2f1700 1 -- start start 2024-03-16T10:47:42.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7df2f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d8196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7df2f1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d8101e50 0x7ff7d8196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7df2f1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d81970f0 con 0x7ff7d80fedd0 2024-03-16T10:47:42.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7df2f1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8197230 con 0x7ff7d8101e50 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dd08d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d8196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dd08d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d8196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59070/0 (socket says 172.21.15.2:59070) 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dd08d700 1 -- 172.21.15.2:0/163589361 learned_addr learned my addr 172.21.15.2:0/163589361 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dc88c700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d8101e50 0x7ff7d8196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dd08d700 1 -- 172.21.15.2:0/163589361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d8101e50 msgr2=0x7ff7d8196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dd08d700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d8101e50 0x7ff7d8196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dd08d700 1 -- 172.21.15.2:0/163589361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c8009d00 con 0x7ff7d80fedd0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7dd08d700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d8196590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800b070 tx=0x7ff7c8003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.478+0000 7ff7d67fc700 1 -- 172.21.15.2:0/163589361 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c800e040 con 0x7ff7d80fedd0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.479+0000 7ff7d67fc700 1 -- 172.21.15.2:0/163589361 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7c80096e0 con 0x7ff7d80fedd0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.479+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d819bbe0 con 0x7ff7d80fedd0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.479+0000 7ff7d67fc700 1 -- 172.21.15.2:0/163589361 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c801d850 con 0x7ff7d80fedd0 2024-03-16T10:47:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.479+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d819c010 con 0x7ff7d80fedd0 2024-03-16T10:47:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.480+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d8190670 con 0x7ff7d80fedd0 2024-03-16T10:47:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.481+0000 7ff7d67fc700 1 -- 172.21.15.2:0/163589361 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7ff7c8004250 con 0x7ff7d80fedd0 2024-03-16T10:47:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.481+0000 7ff7d67fc700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff7c006e0d0 0x7ff7c0070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.481+0000 7ff7d67fc700 1 -- 172.21.15.2:0/163589361 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff7c808f7a0 con 0x7ff7d80fedd0 2024-03-16T10:47:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.481+0000 7ff7dc88c700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff7c006e0d0 0x7ff7c0070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.481+0000 7ff7dc88c700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff7c006e0d0 0x7ff7c0070590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005950 tx=0x7ff7cc00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:42.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.486+0000 7ff7d67fc700 1 -- 172.21.15.2:0/163589361 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff7c805c5a0 con 0x7ff7d80fedd0 2024-03-16T10:47:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.715+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7d80611f0 con 0x7ff7c006e0d0 2024-03-16T10:47:42.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.716+0000 7ff7d67fc700 1 -- 172.21.15.2:0/163589361 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7ff7d80611f0 con 0x7ff7c006e0d0 2024-03-16T10:47:42.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff7c006e0d0 msgr2=0x7ff7c0070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff7c006e0d0 0x7ff7c0070590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc005950 tx=0x7ff7cc00b410 comp rx=0 tx=0).stop 2024-03-16T10:47:42.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 msgr2=0x7ff7d8196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d8196590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800b070 tx=0x7ff7c8003c30 comp rx=0 tx=0).stop 2024-03-16T10:47:42.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 shutdown_connections 2024-03-16T10:47:42.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff7c006e0d0 0x7ff7c0070590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7d80fedd0 0x7ff7d8196590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 --2- 172.21.15.2:0/163589361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d8101e50 0x7ff7d8196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 >> 172.21.15.2:0/163589361 conn(0x7ff7d80fa730 msgr2=0x7ff7d81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:42.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 shutdown_connections 2024-03-16T10:47:42.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.718+0000 7ff7df2f1700 1 -- 172.21.15.2:0/163589361 wait complete. 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 -- 172.21.15.2:0/1219783068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590100ae0 msgr2=0x7f3590100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 --2- 172.21.15.2:0/1219783068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590100ae0 0x7f3590100f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3580009910 tx=0x7f3580009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 -- 172.21.15.2:0/1219783068 shutdown_connections 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 --2- 172.21.15.2:0/1219783068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3590101ce0 0x7f3590102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 --2- 172.21.15.2:0/1219783068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590100ae0 0x7f3590100f00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 -- 172.21.15.2:0/1219783068 >> 172.21.15.2:0/1219783068 conn(0x7f35900fc0a0 msgr2=0x7f35900fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 -- 172.21.15.2:0/1219783068 shutdown_connections 2024-03-16T10:47:42.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 -- 172.21.15.2:0/1219783068 wait complete. 2024-03-16T10:47:42.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 Processor -- start 2024-03-16T10:47:42.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.824+0000 7f3596839700 1 -- start start 2024-03-16T10:47:42.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f3596839700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3590100ae0 0x7f35901964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f3596839700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590101ce0 0x7f3590196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f3596839700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3590197020 con 0x7f3590101ce0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f3596839700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3590197160 con 0x7f3590100ae0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590101ce0 0x7f3590196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590101ce0 0x7f3590196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59076/0 (socket says 172.21.15.2:59076) 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3590100ae0 0x7f35901964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358f7fe700 1 -- 172.21.15.2:0/3827242012 learned_addr learned my addr 172.21.15.2:0/3827242012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358f7fe700 1 -- 172.21.15.2:0/3827242012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3590100ae0 msgr2=0x7f35901964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358f7fe700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3590100ae0 0x7f35901964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358f7fe700 1 -- 172.21.15.2:0/3827242012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35800095f0 con 0x7f3590101ce0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.825+0000 7f358f7fe700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590101ce0 0x7f3590196a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f358400e920 tx=0x7f358400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.826+0000 7f358d7fa700 1 -- 172.21.15.2:0/3827242012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358400c9d0 con 0x7f3590101ce0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.826+0000 7f358d7fa700 1 -- 172.21.15.2:0/3827242012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3584010430 con 0x7f3590101ce0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.826+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359019bbc0 con 0x7f3590101ce0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.826+0000 7f358d7fa700 1 -- 172.21.15.2:0/3827242012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3584018630 con 0x7f3590101ce0 2024-03-16T10:47:42.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.826+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359019c190 con 0x7f3590101ce0 2024-03-16T10:47:42.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.827+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3590190660 con 0x7f3590101ce0 2024-03-16T10:47:42.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.827+0000 7f358d7fa700 1 -- 172.21.15.2:0/3827242012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f35840105a0 con 0x7f3590101ce0 2024-03-16T10:47:42.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.828+0000 7f358d7fa700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f357806e0e0 0x7f35780705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:42.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.828+0000 7f358d7fa700 1 -- 172.21.15.2:0/3827242012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3584014070 con 0x7f3590101ce0 2024-03-16T10:47:42.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.828+0000 7f358ffff700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f357806e0e0 0x7f35780705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:42.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.828+0000 7f358ffff700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f357806e0e0 0x7f35780705a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f358000b800 tx=0x7f3580009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:42.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:42.833+0000 7f358d7fa700 1 -- 172.21.15.2:0/3827242012 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3584053740 con 0x7f3590101ce0 2024-03-16T10:47:43.005 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:42 smithi026 ceph-mon[82600]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:43.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.046+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f35900611f0 con 0x7f357806e0e0 2024-03-16T10:47:43.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.057+0000 7f358d7fa700 1 -- 172.21.15.2:0/3827242012 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f35900611f0 con 0x7f357806e0e0 2024-03-16T10:47:43.058 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:47:43.058 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (9m) 76s ago 11m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:47:43.058 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (11m) 76s ago 11m 7759k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:47:43.058 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (10m) 59s ago 10m 7843k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 starting - - - - 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (10m) 59s ago 10m 6660k - 18.2.2-878-ga752c2f7 f692adb9a265 c0b05b3b15d6 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (9m) 76s ago 11m 85.7M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (5m) 76s ago 5m 17.4M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (5m) 76s ago 5m 14.3M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (5m) 59s ago 5m 212M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (5m) 59s ago 5m 16.2M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (2m) 76s ago 12m 473M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (110s) 59s ago 10m 448M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (92s) 76s ago 13m 27.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (65s) 59s ago 10m 21.0M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (11m) 76s ago 11m 16.7M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (10m) 59s ago 10m 16.0M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (9m) 76s ago 9m 181M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 5c3ea0b8d47f 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (9m) 76s ago 9m 178M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:47:43.059 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (8m) 76s ago 8m 159M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:47:43.060 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (8m) 59s ago 8m 241M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:47:43.060 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (7m) 59s ago 7m 201M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:47:43.060 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (7m) 59s ago 7m 188M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:47:43.060 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (2m) 76s ago 10m 57.8M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f357806e0e0 msgr2=0x7f35780705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f357806e0e0 0x7f35780705a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f358000b800 tx=0x7f3580009f70 comp rx=0 tx=0).stop 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590101ce0 msgr2=0x7f3590196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590101ce0 0x7f3590196a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f358400e920 tx=0x7f358400ec30 comp rx=0 tx=0).stop 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 shutdown_connections 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f357806e0e0 0x7f35780705a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3590100ae0 0x7f35901964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 --2- 172.21.15.2:0/3827242012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3590101ce0 0x7f3590196a00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 >> 172.21.15.2:0/3827242012 conn(0x7f35900fc0a0 msgr2=0x7f35901052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 shutdown_connections 2024-03-16T10:47:43.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.060+0000 7f3596839700 1 -- 172.21.15.2:0/3827242012 wait complete. 2024-03-16T10:47:43.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.163+0000 7fcda6583700 1 -- 172.21.15.2:0/1234787337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0101600 msgr2=0x7fcda01039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.163+0000 7fcda6583700 1 --2- 172.21.15.2:0/1234787337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0101600 0x7fcda01039f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009910 tx=0x7fcd90009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.164+0000 7fcda6583700 1 -- 172.21.15.2:0/1234787337 shutdown_connections 2024-03-16T10:47:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.164+0000 7fcda6583700 1 --2- 172.21.15.2:0/1234787337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcda0103f30 0x7fcda0106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.164+0000 7fcda6583700 1 --2- 172.21.15.2:0/1234787337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0101600 0x7fcda01039f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.164+0000 7fcda6583700 1 -- 172.21.15.2:0/1234787337 >> 172.21.15.2:0/1234787337 conn(0x7fcda00fafb0 msgr2=0x7fcda00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:43.165 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:42 smithi002 ceph-mon[94239]: pgmap v44: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.164+0000 7fcda6583700 1 -- 172.21.15.2:0/1234787337 shutdown_connections 2024-03-16T10:47:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.164+0000 7fcda6583700 1 -- 172.21.15.2:0/1234787337 wait complete. 2024-03-16T10:47:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcda6583700 1 Processor -- start 2024-03-16T10:47:43.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcda6583700 1 -- start start 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcda6583700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcda0101600 0x7fcda0100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcda6583700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0103f30 0x7fcda00ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcda6583700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00ffaf0 con 0x7fcda0103f30 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcda6583700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00ffc30 con 0x7fcda0101600 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcd9f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0103f30 0x7fcda00ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcd9f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0103f30 0x7fcda00ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59090/0 (socket says 172.21.15.2:59090) 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcd9f7fe700 1 -- 172.21.15.2:0/3285894940 learned_addr learned my addr 172.21.15.2:0/3285894940 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcd9ffff700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcda0101600 0x7fcda0100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcd9f7fe700 1 -- 172.21.15.2:0/3285894940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcda0101600 msgr2=0x7fcda0100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.165+0000 7fcd9f7fe700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcda0101600 0x7fcda0100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.166+0000 7fcd9f7fe700 1 -- 172.21.15.2:0/3285894940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd900095f0 con 0x7fcda0103f30 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.166+0000 7fcd9f7fe700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0103f30 0x7fcda00ff5b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400b4b0 tx=0x7fcd9400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.166+0000 7fcd9d7fa700 1 -- 172.21.15.2:0/3285894940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd94011800 con 0x7fcda0103f30 2024-03-16T10:47:43.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.166+0000 7fcd9d7fa700 1 -- 172.21.15.2:0/3285894940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd94011e20 con 0x7fcda0103f30 2024-03-16T10:47:43.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.166+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda00fff10 con 0x7fcda0103f30 2024-03-16T10:47:43.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.166+0000 7fcd9d7fa700 1 -- 172.21.15.2:0/3285894940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd9400f2e0 con 0x7fcda0103f30 2024-03-16T10:47:43.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.166+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda019e6e0 con 0x7fcda0103f30 2024-03-16T10:47:43.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.167+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda004ea80 con 0x7fcda0103f30 2024-03-16T10:47:43.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.167+0000 7fcd9d7fa700 1 -- 172.21.15.2:0/3285894940 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fcd94011960 con 0x7fcda0103f30 2024-03-16T10:47:43.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.168+0000 7fcd9d7fa700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcd8806e0e0 0x7fcd880705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.168+0000 7fcd9d7fa700 1 -- 172.21.15.2:0/3285894940 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcd9408d9e0 con 0x7fcda0103f30 2024-03-16T10:47:43.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.168+0000 7fcd9ffff700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcd8806e0e0 0x7fcd880705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.168+0000 7fcd9ffff700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcd8806e0e0 0x7fcd880705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcd90005090 tx=0x7fcd9001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:43.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.173+0000 7fcd9d7fa700 1 -- 172.21.15.2:0/3285894940 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcd9405a8f0 con 0x7fcda0103f30 2024-03-16T10:47:43.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.470+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcda019e870 con 0x7fcda0103f30 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.471+0000 7fcd9d7fa700 1 -- 172.21.15.2:0/3285894940 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7fcd9405a480 con 0x7fcda0103f30 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6 2024-03-16T10:47:43.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 10, 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:47:43.473 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcd8806e0e0 msgr2=0x7fcd880705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcd8806e0e0 0x7fcd880705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcd90005090 tx=0x7fcd9001a040 comp rx=0 tx=0).stop 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0103f30 msgr2=0x7fcda00ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0103f30 0x7fcda00ff5b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400b4b0 tx=0x7fcd9400b870 comp rx=0 tx=0).stop 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 shutdown_connections 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcd8806e0e0 0x7fcd880705a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcda0101600 0x7fcda0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 --2- 172.21.15.2:0/3285894940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcda0103f30 0x7fcda00ff5b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 >> 172.21.15.2:0/3285894940 conn(0x7fcda00fafb0 msgr2=0x7fcda00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:43.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 shutdown_connections 2024-03-16T10:47:43.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.473+0000 7fcda6583700 1 -- 172.21.15.2:0/3285894940 wait complete. 2024-03-16T10:47:43.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.578+0000 7fd0741f0700 1 -- 172.21.15.2:0/3847101921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c101d90 msgr2=0x7fd06c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.578+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3847101921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c101d90 0x7fd06c1021f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009910 tx=0x7fd05c009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:43.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 -- 172.21.15.2:0/3847101921 shutdown_connections 2024-03-16T10:47:43.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3847101921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c101d90 0x7fd06c1021f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3847101921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c0fedb0 0x7fd06c0ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 -- 172.21.15.2:0/3847101921 >> 172.21.15.2:0/3847101921 conn(0x7fd06c0fa710 msgr2=0x7fd06c0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 -- 172.21.15.2:0/3847101921 shutdown_connections 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 -- 172.21.15.2:0/3847101921 wait complete. 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 Processor -- start 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 -- start start 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c0fedb0 0x7fd06c101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c101d90 0x7fd06c0ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c100280 con 0x7fd06c0fedb0 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.579+0000 7fd0741f0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c1003c0 con 0x7fd06c101d90 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd071f8c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c0fedb0 0x7fd06c101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd071f8c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c0fedb0 0x7fd06c101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59104/0 (socket says 172.21.15.2:59104) 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd071f8c700 1 -- 172.21.15.2:0/3150513992 learned_addr learned my addr 172.21.15.2:0/3150513992 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd07178b700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c101d90 0x7fd06c0ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd071f8c700 1 -- 172.21.15.2:0/3150513992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c101d90 msgr2=0x7fd06c0ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd071f8c700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c101d90 0x7fd06c0ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd071f8c700 1 -- 172.21.15.2:0/3150513992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c0095f0 con 0x7fd06c0fedb0 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd071f8c700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c0fedb0 0x7fd06c101650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd06800e8f0 tx=0x7fd06800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd062ffd700 1 -- 172.21.15.2:0/3150513992 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd068009790 con 0x7fd06c0fedb0 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd062ffd700 1 -- 172.21.15.2:0/3150513992 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd06800c810 con 0x7fd06c0fedb0 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c1006a0 con 0x7fd06c0fedb0 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd062ffd700 1 -- 172.21.15.2:0/3150513992 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd068003d20 con 0x7fd06c0fedb0 2024-03-16T10:47:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.580+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c19e5c0 con 0x7fd06c0fedb0 2024-03-16T10:47:43.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.581+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c190670 con 0x7fd06c0fedb0 2024-03-16T10:47:43.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.582+0000 7fd062ffd700 1 -- 172.21.15.2:0/3150513992 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fd0680041f0 con 0x7fd06c0fedb0 2024-03-16T10:47:43.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.582+0000 7fd062ffd700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd05806e0e0 0x7fd0580705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.582+0000 7fd062ffd700 1 -- 172.21.15.2:0/3150513992 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd068014070 con 0x7fd06c0fedb0 2024-03-16T10:47:43.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.582+0000 7fd07178b700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd05806e0e0 0x7fd0580705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.583+0000 7fd07178b700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd05806e0e0 0x7fd0580705a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00b800 tx=0x7fd05c009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:43.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.587+0000 7fd062ffd700 1 -- 172.21.15.2:0/3150513992 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd06805b020 con 0x7fd06c0fedb0 2024-03-16T10:47:43.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.843+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd06c02ccb0 con 0x7fd06c0fedb0 2024-03-16T10:47:43.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.843+0000 7fd062ffd700 1 -- 172.21.15.2:0/3150513992 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fd06805abb0 con 0x7fd06c0fedb0 2024-03-16T10:47:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:47:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:47:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:47:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:47:43.844 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:47:43.845 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:47:43.846 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:47:43.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd05806e0e0 msgr2=0x7fd0580705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd05806e0e0 0x7fd0580705a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00b800 tx=0x7fd05c009f70 comp rx=0 tx=0).stop 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c0fedb0 msgr2=0x7fd06c101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c0fedb0 0x7fd06c101650 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd06800e8f0 tx=0x7fd06800ec00 comp rx=0 tx=0).stop 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 shutdown_connections 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd05806e0e0 0x7fd0580705a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd06c0fedb0 0x7fd06c101650 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 --2- 172.21.15.2:0/3150513992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c101d90 0x7fd06c0ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 >> 172.21.15.2:0/3150513992 conn(0x7fd06c0fa710 msgr2=0x7fd06c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 shutdown_connections 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.845+0000 7fd0741f0700 1 -- 172.21.15.2:0/3150513992 wait complete. 2024-03-16T10:47:43.848 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.952+0000 7f1ce83c1700 1 -- 172.21.15.2:0/2791655121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce00ff9d0 msgr2=0x7f1ce00ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.952+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/2791655121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce00ff9d0 0x7f1ce00ffdf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009910 tx=0x7f1cdc009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.952+0000 7f1ce83c1700 1 -- 172.21.15.2:0/2791655121 shutdown_connections 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.952+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/2791655121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce0100bd0 0x7f1ce0101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.952+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/2791655121 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce00ff9d0 0x7f1ce00ffdf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 -- 172.21.15.2:0/2791655121 >> 172.21.15.2:0/2791655121 conn(0x7f1ce00faf70 msgr2=0x7f1ce00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 -- 172.21.15.2:0/2791655121 shutdown_connections 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 -- 172.21.15.2:0/2791655121 wait complete. 2024-03-16T10:47:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 Processor -- start 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 -- start start 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00ff9d0 0x7f1ce01964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce0100bd0 0x7f1ce0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0197030 con 0x7f1ce0100bd0 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce83c1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0197170 con 0x7f1ce00ff9d0 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce595c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce0100bd0 0x7f1ce0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce595c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce0100bd0 0x7f1ce0196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59132/0 (socket says 172.21.15.2:59132) 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce595c700 1 -- 172.21.15.2:0/1250146793 learned_addr learned my addr 172.21.15.2:0/1250146793 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.953+0000 7f1ce615d700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00ff9d0 0x7f1ce01964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1ce595c700 1 -- 172.21.15.2:0/1250146793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00ff9d0 msgr2=0x7f1ce01964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1ce595c700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00ff9d0 0x7f1ce01964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1ce595c700 1 -- 172.21.15.2:0/1250146793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0095f0 con 0x7f1ce0100bd0 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1ce595c700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce0100bd0 0x7f1ce0196a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000e920 tx=0x7f1cd000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:43.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1cd77fe700 1 -- 172.21.15.2:0/1250146793 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cd000c9d0 con 0x7f1ce0100bd0 2024-03-16T10:47:43.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1cd77fe700 1 -- 172.21.15.2:0/1250146793 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cd0010430 con 0x7f1ce0100bd0 2024-03-16T10:47:43.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1cd77fe700 1 -- 172.21.15.2:0/1250146793 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cd0018630 con 0x7f1ce0100bd0 2024-03-16T10:47:43.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce019bbd0 con 0x7f1ce0100bd0 2024-03-16T10:47:43.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.954+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce019c1a0 con 0x7f1ce0100bd0 2024-03-16T10:47:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.955+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce004ea80 con 0x7f1ce0100bd0 2024-03-16T10:47:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.956+0000 7f1cd77fe700 1 -- 172.21.15.2:0/1250146793 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7f1cd00105a0 con 0x7f1ce0100bd0 2024-03-16T10:47:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.956+0000 7f1cd77fe700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1ccc06e0e0 0x7f1ccc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.956+0000 7f1cd77fe700 1 -- 172.21.15.2:0/1250146793 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1cd0014070 con 0x7f1ce0100bd0 2024-03-16T10:47:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.956+0000 7f1ce615d700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1ccc06e0e0 0x7f1ccc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:43.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.956+0000 7f1ce615d700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1ccc06e0e0 0x7f1ccc0705a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00b800 tx=0x7f1cdc009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:43.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:43.961+0000 7f1cd77fe700 1 -- 172.21.15.2:0/1250146793 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1cd0053740 con 0x7f1ce0100bd0 2024-03-16T10:47:43.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:43 smithi026 ceph-mon[82600]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:43.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:43 smithi026 ceph-mon[82600]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:43.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:43 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3285894940' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:43.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:43 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3150513992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:47:44.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.185+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ce0068b90 con 0x7f1ccc06e0e0 2024-03-16T10:47:44.186 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:43 smithi002 ceph-mon[94239]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:44.186 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:43 smithi002 ceph-mon[94239]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:44.186 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:43 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3285894940' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:44.186 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:43 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3150513992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.186+0000 7f1cd77fe700 1 -- 172.21.15.2:0/1250146793 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f1ce0068b90 con 0x7f1ccc06e0e0 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:47:44.187 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:47:44.188 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "4/23 daemons upgraded", 2024-03-16T10:47:44.188 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading crash daemons", 2024-03-16T10:47:44.188 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:47:44.188 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:47:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1ccc06e0e0 msgr2=0x7f1ccc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1ccc06e0e0 0x7f1ccc0705a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00b800 tx=0x7f1cdc009f70 comp rx=0 tx=0).stop 2024-03-16T10:47:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce0100bd0 msgr2=0x7f1ce0196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce0100bd0 0x7f1ce0196a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000e920 tx=0x7f1cd000ec30 comp rx=0 tx=0).stop 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 shutdown_connections 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1ccc06e0e0 0x7f1ccc0705a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00ff9d0 0x7f1ce01964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 --2- 172.21.15.2:0/1250146793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ce0100bd0 0x7f1ce0196a10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.188+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 >> 172.21.15.2:0/1250146793 conn(0x7f1ce00faf70 msgr2=0x7f1ce0068480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.189+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 shutdown_connections 2024-03-16T10:47:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.189+0000 7f1ce83c1700 1 -- 172.21.15.2:0/1250146793 wait complete. 2024-03-16T10:47:44.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.294+0000 7fdd4c064700 1 -- 172.21.15.2:0/2012248548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 msgr2=0x7fdd44101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:44.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.294+0000 7fdd4c064700 1 --2- 172.21.15.2:0/2012248548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44101490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009910 tx=0x7fdd40009c20 comp rx=0 tx=0).stop 2024-03-16T10:47:44.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.294+0000 7fdd4c064700 1 -- 172.21.15.2:0/2012248548 shutdown_connections 2024-03-16T10:47:44.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.294+0000 7fdd4c064700 1 --2- 172.21.15.2:0/2012248548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd440feb70 0x7fdd440feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.294+0000 7fdd4c064700 1 --2- 172.21.15.2:0/2012248548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44101490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.294+0000 7fdd4c064700 1 -- 172.21.15.2:0/2012248548 >> 172.21.15.2:0/2012248548 conn(0x7fdd440fa730 msgr2=0x7fdd440fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:44.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.294+0000 7fdd4c064700 1 -- 172.21.15.2:0/2012248548 shutdown_connections 2024-03-16T10:47:44.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd4c064700 1 -- 172.21.15.2:0/2012248548 wait complete. 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd4c064700 1 Processor -- start 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd4c064700 1 -- start start 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd4c064700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd440feb70 0x7fdd441986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd4c064700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd4c064700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44199220 con 0x7fdd44101070 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd4c064700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44199360 con 0x7fdd440feb70 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd495ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:44.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd495ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59156/0 (socket says 172.21.15.2:59156) 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.295+0000 7fdd495ff700 1 -- 172.21.15.2:0/244701364 learned_addr learned my addr 172.21.15.2:0/244701364 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd49e00700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd440feb70 0x7fdd441986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd495ff700 1 -- 172.21.15.2:0/244701364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd440feb70 msgr2=0x7fdd441986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd495ff700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd440feb70 0x7fdd441986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd495ff700 1 -- 172.21.15.2:0/244701364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400095f0 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd49e00700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd440feb70 0x7fdd441986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd495ff700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44198c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400d8d0 tx=0x7fdd3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd3affd700 1 -- 172.21.15.2:0/244701364 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd34009940 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd3affd700 1 -- 172.21.15.2:0/244701364 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd34010460 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4419ddc0 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd3affd700 1 -- 172.21.15.2:0/244701364 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3400f5f0 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.296+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4419e360 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.297+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44192920 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.297+0000 7fdd3affd700 1 -- 172.21.15.2:0/244701364 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 37) v1 ==== 91941+0+0 (secure 0 0 0) 0x7fdd340105d0 con 0x7fdd44101070 2024-03-16T10:47:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.298+0000 7fdd3affd700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdd3006e0e0 0x7fdd300705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:47:44.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.298+0000 7fdd3affd700 1 -- 172.21.15.2:0/244701364 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdd3408dff0 con 0x7fdd44101070 2024-03-16T10:47:44.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.298+0000 7fdd49e00700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdd3006e0e0 0x7fdd300705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:47:44.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.298+0000 7fdd49e00700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdd3006e0e0 0x7fdd300705a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd400059d0 tx=0x7fdd40005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:47:44.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.303+0000 7fdd3affd700 1 -- 172.21.15.2:0/244701364 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdd3405ad70 con 0x7fdd44101070 2024-03-16T10:47:44.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.599+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdd4404ea80 con 0x7fdd44101070 2024-03-16T10:47:44.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.599+0000 7fdd3affd700 1 -- 172.21.15.2:0/244701364 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fdd3405a900 con 0x7fdd44101070 2024-03-16T10:47:44.600 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:47:44.600 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:47:44.600 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:47:44.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdd3006e0e0 msgr2=0x7fdd300705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:44.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdd3006e0e0 0x7fdd300705a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdd400059d0 tx=0x7fdd40005960 comp rx=0 tx=0).stop 2024-03-16T10:47:44.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 msgr2=0x7fdd44198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44198c00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400d8d0 tx=0x7fdd3400dc90 comp rx=0 tx=0).stop 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 shutdown_connections 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdd3006e0e0 0x7fdd300705a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd440feb70 0x7fdd441986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 --2- 172.21.15.2:0/244701364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd44101070 0x7fdd44198c00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 >> 172.21.15.2:0/244701364 conn(0x7fdd440fa730 msgr2=0x7fdd44108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.601+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 shutdown_connections 2024-03-16T10:47:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:47:44.602+0000 7fdd4c064700 1 -- 172.21.15.2:0/244701364 wait complete. 2024-03-16T10:47:44.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:44 smithi026 ceph-mon[82600]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:44.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:44 smithi026 ceph-mon[82600]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:44.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:44 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/244701364' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:47:44.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:44.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:44.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:47:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:44 smithi002 ceph-mon[94239]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:44 smithi002 ceph-mon[94239]: pgmap v45: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:44 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/244701364' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:47:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:47:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:45 smithi026 ceph-mon[82600]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:45 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:46.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:45 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:45 smithi002 ceph-mon[94239]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:47:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:45 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:46.260 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:45 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:47.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:46 smithi026 ceph-mon[82600]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:46 smithi002 ceph-mon[94239]: pgmap v46: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:48.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:48 smithi002 ceph-mon[94239]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:48.998 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:48 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:48.999 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:48 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:48 smithi026 ceph-mon[82600]: pgmap v47: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:48 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:48 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:50 smithi026 ceph-mon[82600]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:50.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:50 smithi002 ceph-mon[94239]: pgmap v48: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:47:50.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:47:52.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:51 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:52.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:51 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:52.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:51 smithi002 ceph-mon[94239]: mgrmap e38: smithi002.zevrtm(active, since 91s), standbys: smithi026.omjkpo 2024-03-16T10:47:52.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:51 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:52.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:51 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:52.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:51 smithi026 ceph-mon[82600]: mgrmap e38: smithi002.zevrtm(active, since 91s), standbys: smithi026.omjkpo 2024-03-16T10:47:53.003 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:52 smithi002 ceph-mon[94239]: pgmap v49: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:53.003 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:52 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:53.004 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:52 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:52 smithi026 ceph-mon[82600]: pgmap v49: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:52 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:52 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: pgmap v50: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:55.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:55.140 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:55.140 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.140 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi002"}]: dispatch 2024-03-16T10:47:55.140 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi002"}]': finished 2024-03-16T10:47:55.140 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi026"}]: dispatch 2024-03-16T10:47:55.140 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi026"}]': finished 2024-03-16T10:47:55.140 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T10:47:55.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: pgmap v50: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi002"}]: dispatch 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi002"}]': finished 2024-03-16T10:47:55.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi026"}]: dispatch 2024-03-16T10:47:55.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi026"}]': finished 2024-03-16T10:47:55.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T10:47:55.965 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:55 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all crash 2024-03-16T10:47:55.965 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:55 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T10:47:55.965 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:55 smithi002 ceph-mon[94239]: Upgrade: osd.0 is safe to restart 2024-03-16T10:47:56.145 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:55 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all crash 2024-03-16T10:47:56.145 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:55 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-16T10:47:56.145 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:55 smithi026 ceph-mon[82600]: Upgrade: osd.0 is safe to restart 2024-03-16T10:47:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:56 smithi026 ceph-mon[82600]: pgmap v51: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:57.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:56 smithi026 ceph-mon[82600]: Upgrade: Updating osd.0 2024-03-16T10:47:57.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:57 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:57.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:57 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T10:47:57.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:57 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:57.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:57 smithi026 ceph-mon[82600]: Deploying daemon osd.0 on smithi002 2024-03-16T10:47:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:56 smithi002 ceph-mon[94239]: pgmap v51: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:56 smithi002 ceph-mon[94239]: Upgrade: Updating osd.0 2024-03-16T10:47:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:56 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:47:57.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:56 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T10:47:57.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:56 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:47:57.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:56 smithi002 ceph-mon[94239]: Deploying daemon osd.0 on smithi002 2024-03-16T10:47:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:47:59 smithi002 ceph-mon[94239]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:59.299 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:47:59 smithi002 systemd[1]: Stopping Ceph osd.0 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:47:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:47:59 smithi026 ceph-mon[82600]: pgmap v52: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:47:59.798 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:47:59 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[48563]: 2024-03-16T10:47:59.407+0000 7f5595b37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:47:59.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:47:59 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[48563]: 2024-03-16T10:47:59.407+0000 7f5595b37700 -1 osd.0 51 *** Got signal Terminated *** 2024-03-16T10:47:59.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:47:59 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[48563]: 2024-03-16T10:47:59.407+0000 7f5595b37700 -1 osd.0 51 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:48:00.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:00 smithi002 ceph-mon[94239]: osd.0 marked itself down and dead 2024-03-16T10:48:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:00 smithi026 ceph-mon[82600]: osd.0 marked itself down and dead 2024-03-16T10:48:00.858 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:00 smithi002 podman[102433]: 2024-03-16 10:48:00.562600593 +0000 UTC m=+1.254762644 container died 5c3ea0b8d47f96b80153edb830104923bbce48751c43ed8ee23902d168293668 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-a752c2f) 2024-03-16T10:48:00.858 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:00 smithi002 podman[102433]: 2024-03-16 10:48:00.858403066 +0000 UTC m=+1.550565120 container cleanup 5c3ea0b8d47f96b80153edb830104923bbce48751c43ed8ee23902d168293668 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=reef-a752c2f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-16T10:48:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:01 smithi002 ceph-mon[94239]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:48:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:01 smithi002 ceph-mon[94239]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:48:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:01 smithi002 ceph-mon[94239]: osdmap e52: 6 total, 5 up, 6 in 2024-03-16T10:48:01.299 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:00 smithi002 bash[102433]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0 2024-03-16T10:48:01.299 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:01 smithi002 podman[102450]: 2024-03-16 10:48:01.093176984 +0000 UTC m=+0.534130971 container remove 5c3ea0b8d47f96b80153edb830104923bbce48751c43ed8ee23902d168293668 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-a752c2f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:48:01.383 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:01 smithi026 ceph-mon[82600]: pgmap v53: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-16T10:48:01.383 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:01 smithi026 ceph-mon[82600]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:48:01.383 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:01 smithi026 ceph-mon[82600]: osdmap e52: 6 total, 5 up, 6 in 2024-03-16T10:48:01.688 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:01 smithi002 podman[102544]: 2024-03-16 10:48:01.402521073 +0000 UTC m=+0.102532053 container create 6f28215c7960dd82c3b13e6b367a833cca6aa1f9c6b3d01a1ba8961d1dbbaea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-16T10:48:01.688 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:01 smithi002 podman[102544]: 2024-03-16 10:48:01.317566873 +0000 UTC m=+0.017577825 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:48:01.688 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:01 smithi002 podman[102544]: 2024-03-16 10:48:01.605382866 +0000 UTC m=+0.305393831 container init 6f28215c7960dd82c3b13e6b367a833cca6aa1f9c6b3d01a1ba8961d1dbbaea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-16T10:48:01.688 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:01 smithi002 podman[102544]: 2024-03-16 10:48:01.608664096 +0000 UTC m=+0.308675041 container start 6f28215c7960dd82c3b13e6b367a833cca6aa1f9c6b3d01a1ba8961d1dbbaea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:48:01.688 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:01 smithi002 podman[102544]: 2024-03-16 10:48:01.688396652 +0000 UTC m=+0.388407594 container attach 6f28215c7960dd82c3b13e6b367a833cca6aa1f9c6b3d01a1ba8961d1dbbaea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:48:02.012 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:01 smithi002 podman[102544]: 2024-03-16 10:48:01.908112619 +0000 UTC m=+0.608123556 container died 6f28215c7960dd82c3b13e6b367a833cca6aa1f9c6b3d01a1ba8961d1dbbaea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T10:48:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:02 smithi002 ceph-mon[94239]: osdmap e53: 6 total, 5 up, 6 in 2024-03-16T10:48:02.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:02 smithi026 ceph-mon[82600]: osdmap e53: 6 total, 5 up, 6 in 2024-03-16T10:48:02.684 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:02 smithi002 podman[102570]: 2024-03-16 10:48:02.467311571 +0000 UTC m=+0.553810366 container remove 6f28215c7960dd82c3b13e6b367a833cca6aa1f9c6b3d01a1ba8961d1dbbaea1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-16T10:48:02.684 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:02 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-16T10:48:02.684 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:02 smithi002 systemd[1]: Stopped Ceph osd.0 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:48:02.685 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:02 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.0.service: Consumed 51.835s CPU time. 2024-03-16T10:48:03.013 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:02 smithi002 systemd[1]: Starting Ceph osd.0 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:48:03.014 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:02 smithi002 podman[102677]: 2024-03-16 10:48:02.802589285 +0000 UTC m=+0.017885066 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:48:03.014 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:02 smithi002 podman[102677]: 2024-03-16 10:48:02.918527219 +0000 UTC m=+0.133823020 container create c0916b85a0f8f4b75dd197d2bc0ed184bd2ea7ab71890c3f767abc27fba45aa4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-16T10:48:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-mon[94239]: pgmap v56: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-mon[94239]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-16T10:48:03.299 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 podman[102677]: 2024-03-16 10:48:03.121476834 +0000 UTC m=+0.336772606 container init c0916b85a0f8f4b75dd197d2bc0ed184bd2ea7ab71890c3f767abc27fba45aa4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-16T10:48:03.299 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 podman[102677]: 2024-03-16 10:48:03.124750287 +0000 UTC m=+0.340046055 container start c0916b85a0f8f4b75dd197d2bc0ed184bd2ea7ab71890c3f767abc27fba45aa4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T10:48:03.299 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 podman[102677]: 2024-03-16 10:48:03.195814703 +0000 UTC m=+0.411110475 container attach c0916b85a0f8f4b75dd197d2bc0ed184bd2ea7ab71890c3f767abc27fba45aa4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-03-16T10:48:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:03 smithi026 ceph-mon[82600]: pgmap v56: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:03 smithi026 ceph-mon[82600]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate[102692]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate[102692]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 bash[102677]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 bash[102677]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 bash[102677]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 bash[102677]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 bash[102677]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 bash[102677]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 bash[102677]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate[102692]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate[102692]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate[102692]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-16T10:48:03.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate[102692]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T10:48:03.800 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate[102692]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-16T10:48:03.800 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:03 smithi002 podman[102677]: 2024-03-16 10:48:03.596950409 +0000 UTC m=+0.812246173 container died c0916b85a0f8f4b75dd197d2bc0ed184bd2ea7ab71890c3f767abc27fba45aa4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-16T10:48:04.484 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 podman[102816]: 2024-03-16 10:48:04.17499077 +0000 UTC m=+0.572628375 container remove c0916b85a0f8f4b75dd197d2bc0ed184bd2ea7ab71890c3f767abc27fba45aa4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:48:04.484 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 podman[102880]: 2024-03-16 10:48:04.384239735 +0000 UTC m=+0.114072856 container create 986db8fe457d758c30a063a1a747a0217bc515badb6e7f63e6f01dc5a8777055 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-16T10:48:04.484 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 podman[102880]: 2024-03-16 10:48:04.288537687 +0000 UTC m=+0.018370824 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:48:04.798 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 podman[102880]: 2024-03-16 10:48:04.569672233 +0000 UTC m=+0.299505362 container init 986db8fe457d758c30a063a1a747a0217bc515badb6e7f63e6f01dc5a8777055 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-16T10:48:04.798 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 podman[102880]: 2024-03-16 10:48:04.572715216 +0000 UTC m=+0.302548344 container start 986db8fe457d758c30a063a1a747a0217bc515badb6e7f63e6f01dc5a8777055 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-03-16T10:48:04.798 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 ceph-osd[102899]: -- 172.21.15.2:0/2646239468 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562af2131680 con 0x562af20fb000 2024-03-16T10:48:04.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 bash[102880]: 986db8fe457d758c30a063a1a747a0217bc515badb6e7f63e6f01dc5a8777055 2024-03-16T10:48:04.799 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:04 smithi002 systemd[1]: Started Ceph osd.0 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:48:05.298 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:05 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[102895]: 2024-03-16T10:48:05.128+0000 7f98ef9927c0 -1 Falling back to public interface 2024-03-16T10:48:05.383 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:05 smithi002 ceph-mon[94239]: pgmap v57: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:05.383 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:05.383 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:05.383 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:05.383 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:05.384 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:05 smithi026 ceph-mon[82600]: pgmap v57: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:05.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:05.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:05.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:05.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:06.341 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:06 smithi002 ceph-mon[94239]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:06.341 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:06 smithi002 ceph-mon[94239]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-03-16T10:48:06.341 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:06 smithi002 ceph-mon[94239]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-16T10:48:06.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:06 smithi026 ceph-mon[82600]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:06.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:06 smithi026 ceph-mon[82600]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-03-16T10:48:06.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:06 smithi026 ceph-mon[82600]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-16T10:48:09.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:08 smithi002 ceph-mon[94239]: pgmap v59: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:09.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:08 smithi026 ceph-mon[82600]: pgmap v59: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:09.798 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:09 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[102895]: 2024-03-16T10:48:09.639+0000 7f98ef9927c0 -1 osd.0 51 log_to_monitors true 2024-03-16T10:48:10.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:09 smithi026 ceph-mon[82600]: from='osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T10:48:10.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:09 smithi002 ceph-mon[94239]: from='osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T10:48:10.298 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 10:48:09 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[102895]: 2024-03-16T10:48:09.831+0000 7f98e0dea700 -1 osd.0 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:48:11.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:10 smithi026 ceph-mon[82600]: pgmap v60: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:11.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:10 smithi026 ceph-mon[82600]: from='osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T10:48:11.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:10 smithi026 ceph-mon[82600]: osdmap e54: 6 total, 5 up, 6 in 2024-03-16T10:48:11.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:10 smithi026 ceph-mon[82600]: from='osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:48:11.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:10 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:11.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:10 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:10 smithi002 ceph-mon[94239]: pgmap v60: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:10 smithi002 ceph-mon[94239]: from='osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T10:48:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:10 smithi002 ceph-mon[94239]: osdmap e54: 6 total, 5 up, 6 in 2024-03-16T10:48:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:10 smithi002 ceph-mon[94239]: from='osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:48:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:10 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:10 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:12.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:11 smithi002 ceph-mon[94239]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:48:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:11 smithi002 ceph-mon[94239]: osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518] boot 2024-03-16T10:48:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:11 smithi002 ceph-mon[94239]: osdmap e55: 6 total, 6 up, 6 in 2024-03-16T10:48:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:48:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:12.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:11 smithi002 ceph-mon[94239]: osdmap e56: 6 total, 6 up, 6 in 2024-03-16T10:48:12.094 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:11 smithi026 ceph-mon[82600]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:48:12.095 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:11 smithi026 ceph-mon[82600]: osd.0 [v2:172.21.15.2:6802/1147005518,v1:172.21.15.2:6803/1147005518] boot 2024-03-16T10:48:12.095 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:11 smithi026 ceph-mon[82600]: osdmap e55: 6 total, 6 up, 6 in 2024-03-16T10:48:12.095 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T10:48:12.095 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:12.095 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:12.095 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:11 smithi026 ceph-mon[82600]: osdmap e56: 6 total, 6 up, 6 in 2024-03-16T10:48:12.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:12 smithi026 ceph-mon[82600]: pgmap v63: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:13.044 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:12 smithi002 ceph-mon[94239]: pgmap v63: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.717+0000 7f2c97419700 1 -- 172.21.15.2:0/2566416698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 msgr2=0x7f2c900ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.717+0000 7f2c97419700 1 --2- 172.21.15.2:0/2566416698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c900ffe00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009910 tx=0x7f2c8c009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 -- 172.21.15.2:0/2566416698 shutdown_connections 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 --2- 172.21.15.2:0/2566416698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c90100be0 0x7f2c90101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 --2- 172.21.15.2:0/2566416698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c900ffe00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 -- 172.21.15.2:0/2566416698 >> 172.21.15.2:0/2566416698 conn(0x7f2c900faf80 msgr2=0x7f2c900fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 -- 172.21.15.2:0/2566416698 shutdown_connections 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 -- 172.21.15.2:0/2566416698 wait complete. 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 Processor -- start 2024-03-16T10:48:14.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 -- start start 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c901964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.718+0000 7f2c97419700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c90100be0 0x7f2c90196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c97419700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90196fc0 con 0x7f2c900ff9e0 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c97419700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90197100 con 0x7f2c90100be0 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c951b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c901964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c951b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c901964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36308/0 (socket says 172.21.15.2:36308) 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c951b5700 1 -- 172.21.15.2:0/3676894282 learned_addr learned my addr 172.21.15.2:0/3676894282 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c949b4700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c90100be0 0x7f2c90196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:14.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c951b5700 1 -- 172.21.15.2:0/3676894282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c90100be0 msgr2=0x7f2c90196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c951b5700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c90100be0 0x7f2c90196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c951b5700 1 -- 172.21.15.2:0/3676894282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0095f0 con 0x7f2c900ff9e0 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c951b5700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c901964f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0059d0 tx=0x7f2c8c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c867fc700 1 -- 172.21.15.2:0/3676894282 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c01d070 con 0x7f2c900ff9e0 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9019bb10 con 0x7f2c900ff9e0 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c867fc700 1 -- 172.21.15.2:0/3676894282 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c8c00f460 con 0x7f2c900ff9e0 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c867fc700 1 -- 172.21.15.2:0/3676894282 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c021640 con 0x7f2c900ff9e0 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.719+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9019c050 con 0x7f2c900ff9e0 2024-03-16T10:48:14.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.720+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9004ea80 con 0x7f2c900ff9e0 2024-03-16T10:48:14.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.721+0000 7f2c867fc700 1 -- 172.21.15.2:0/3676894282 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2c8c00f5d0 con 0x7f2c900ff9e0 2024-03-16T10:48:14.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.721+0000 7f2c867fc700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c7c06dfa0 0x7f2c7c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:14.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.721+0000 7f2c867fc700 1 -- 172.21.15.2:0/3676894282 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2c8c08f070 con 0x7f2c900ff9e0 2024-03-16T10:48:14.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.721+0000 7f2c949b4700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c7c06dfa0 0x7f2c7c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:14.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.722+0000 7f2c949b4700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c7c06dfa0 0x7f2c7c070460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000ac50 tx=0x7f2c8000a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:14.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.726+0000 7f2c867fc700 1 -- 172.21.15.2:0/3676894282 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c8c05c010 con 0x7f2c900ff9e0 2024-03-16T10:48:14.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.965+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c90069730 con 0x7f2c7c06dfa0 2024-03-16T10:48:14.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.966+0000 7f2c867fc700 1 -- 172.21.15.2:0/3676894282 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2c90069730 con 0x7f2c7c06dfa0 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c7c06dfa0 msgr2=0x7f2c7c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c7c06dfa0 0x7f2c7c070460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000ac50 tx=0x7f2c8000a380 comp rx=0 tx=0).stop 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 msgr2=0x7f2c901964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c901964f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0059d0 tx=0x7f2c8c00bd70 comp rx=0 tx=0).stop 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 shutdown_connections 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c7c06dfa0 0x7f2c7c070460 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c900ff9e0 0x7f2c901964f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 --2- 172.21.15.2:0/3676894282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c90100be0 0x7f2c90196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 >> 172.21.15.2:0/3676894282 conn(0x7f2c900faf80 msgr2=0x7f2c90069020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 shutdown_connections 2024-03-16T10:48:14.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:14.968+0000 7f2c97419700 1 -- 172.21.15.2:0/3676894282 wait complete. 2024-03-16T10:48:14.980 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:48:15.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.074+0000 7f4030395700 1 -- 172.21.15.2:0/4118769429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028103f50 msgr2=0x7f4028106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.074+0000 7f4030395700 1 --2- 172.21.15.2:0/4118769429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028103f50 0x7f4028106340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4018009910 tx=0x7f4018009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:15.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 -- 172.21.15.2:0/4118769429 shutdown_connections 2024-03-16T10:48:15.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 --2- 172.21.15.2:0/4118769429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028103f50 0x7f4028106340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 --2- 172.21.15.2:0/4118769429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4028101620 0x7f4028103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 -- 172.21.15.2:0/4118769429 >> 172.21.15.2:0/4118769429 conn(0x7f40280faf70 msgr2=0x7f40280fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 -- 172.21.15.2:0/4118769429 shutdown_connections 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 -- 172.21.15.2:0/4118769429 wait complete. 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 Processor -- start 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 -- start start 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028101620 0x7f4028196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4028103f50 0x7f40281969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028196ff0 con 0x7f4028101620 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.075+0000 7f4030395700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4028197130 con 0x7f4028103f50 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402e131700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028101620 0x7f4028196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402e131700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028101620 0x7f4028196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36318/0 (socket says 172.21.15.2:36318) 2024-03-16T10:48:15.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402e131700 1 -- 172.21.15.2:0/337929704 learned_addr learned my addr 172.21.15.2:0/337929704 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402d930700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4028103f50 0x7f40281969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402e131700 1 -- 172.21.15.2:0/337929704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4028103f50 msgr2=0x7f40281969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402e131700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4028103f50 0x7f40281969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402e131700 1 -- 172.21.15.2:0/337929704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40180095f0 con 0x7f4028101620 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f402e131700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028101620 0x7f4028196490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f402400d8d0 tx=0x7f402400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f401f7fe700 1 -- 172.21.15.2:0/337929704 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40240098a0 con 0x7f4028101620 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f401f7fe700 1 -- 172.21.15.2:0/337929704 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4024010460 con 0x7f4028101620 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402819bb90 con 0x7f4028101620 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f401f7fe700 1 -- 172.21.15.2:0/337929704 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f402400f5f0 con 0x7f4028101620 2024-03-16T10:48:15.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.076+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402819c0e0 con 0x7f4028101620 2024-03-16T10:48:15.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.077+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402804ea80 con 0x7f4028101620 2024-03-16T10:48:15.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.078+0000 7f401f7fe700 1 -- 172.21.15.2:0/337929704 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4024009a00 con 0x7f4028101620 2024-03-16T10:48:15.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.078+0000 7f401f7fe700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f401406dfa0 0x7f4014070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.078+0000 7f402d930700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f401406dfa0 0x7f4014070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.078+0000 7f401f7fe700 1 -- 172.21.15.2:0/337929704 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f402408dc00 con 0x7f4028101620 2024-03-16T10:48:15.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.079+0000 7f402d930700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f401406dfa0 0x7f4014070460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4018005090 tx=0x7f4018005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:15.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.083+0000 7f401f7fe700 1 -- 172.21.15.2:0/337929704 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4024093020 con 0x7f4028101620 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: pgmap v65: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:15.084 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:15.085 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:15.085 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:15.085 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: pgmap v65: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:15.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.309+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4028105e20 con 0x7f401406dfa0 2024-03-16T10:48:15.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.310+0000 7f401f7fe700 1 -- 172.21.15.2:0/337929704 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4028105e20 con 0x7f401406dfa0 2024-03-16T10:48:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.312+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f401406dfa0 msgr2=0x7f4014070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.312+0000 7f4030395700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f401406dfa0 0x7f4014070460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4018005090 tx=0x7f4018005960 comp rx=0 tx=0).stop 2024-03-16T10:48:15.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.312+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028101620 msgr2=0x7f4028196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.312+0000 7f4030395700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028101620 0x7f4028196490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f402400d8d0 tx=0x7f402400dbe0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.313+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 shutdown_connections 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.313+0000 7f4030395700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f401406dfa0 0x7f4014070460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.313+0000 7f4030395700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4028101620 0x7f4028196490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.313+0000 7f4030395700 1 --2- 172.21.15.2:0/337929704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4028103f50 0x7f40281969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.313+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 >> 172.21.15.2:0/337929704 conn(0x7f40280faf70 msgr2=0x7f40280fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.313+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 shutdown_connections 2024-03-16T10:48:15.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.313+0000 7f4030395700 1 -- 172.21.15.2:0/337929704 wait complete. 2024-03-16T10:48:15.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 -- 172.21.15.2:0/2598614737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 msgr2=0x7f88f40ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 --2- 172.21.15.2:0/2598614737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f40ffe30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009910 tx=0x7f88e4009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:15.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 -- 172.21.15.2:0/2598614737 shutdown_connections 2024-03-16T10:48:15.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 --2- 172.21.15.2:0/2598614737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88f4100c10 0x7f88f4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 --2- 172.21.15.2:0/2598614737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f40ffe30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 -- 172.21.15.2:0/2598614737 >> 172.21.15.2:0/2598614737 conn(0x7f88f40fafb0 msgr2=0x7f88f40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:15.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 -- 172.21.15.2:0/2598614737 shutdown_connections 2024-03-16T10:48:15.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.419+0000 7f88fa104700 1 -- 172.21.15.2:0/2598614737 wait complete. 2024-03-16T10:48:15.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88fa104700 1 Processor -- start 2024-03-16T10:48:15.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88fa104700 1 -- start start 2024-03-16T10:48:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88fa104700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f4103950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88fa104700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88f4100c10 0x7f88f4101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88fa104700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4102530 con 0x7f88f40ffa10 2024-03-16T10:48:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88fa104700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4102670 con 0x7f88f4100c10 2024-03-16T10:48:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88f37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f4103950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88f37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f4103950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36346/0 (socket says 172.21.15.2:36346) 2024-03-16T10:48:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.420+0000 7f88f37fe700 1 -- 172.21.15.2:0/71236387 learned_addr learned my addr 172.21.15.2:0/71236387 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f2ffd700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88f4100c10 0x7f88f4101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f37fe700 1 -- 172.21.15.2:0/71236387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88f4100c10 msgr2=0x7f88f4101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f37fe700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88f4100c10 0x7f88f4101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f37fe700 1 -- 172.21.15.2:0/71236387 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e40095f0 con 0x7f88f40ffa10 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f2ffd700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88f4100c10 0x7f88f4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f37fe700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f4103950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88e4005850 tx=0x7f88e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f0ff9700 1 -- 172.21.15.2:0/71236387 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88e401d070 con 0x7f88f40ffa10 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f0ff9700 1 -- 172.21.15.2:0/71236387 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88e400f460 con 0x7f88f40ffa10 2024-03-16T10:48:15.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88f0ff9700 1 -- 172.21.15.2:0/71236387 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88e4017610 con 0x7f88f40ffa10 2024-03-16T10:48:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f41028f0 con 0x7f88f40ffa10 2024-03-16T10:48:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.421+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f4102e10 con 0x7f88f40ffa10 2024-03-16T10:48:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.422+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f404ea80 con 0x7f88f40ffa10 2024-03-16T10:48:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.423+0000 7f88f0ff9700 1 -- 172.21.15.2:0/71236387 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f88e400fab0 con 0x7f88f40ffa10 2024-03-16T10:48:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.423+0000 7f88f0ff9700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f88dc06dfa0 0x7f88dc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.423+0000 7f88f0ff9700 1 -- 172.21.15.2:0/71236387 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f88e408f230 con 0x7f88f40ffa10 2024-03-16T10:48:15.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.423+0000 7f88f2ffd700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f88dc06dfa0 0x7f88dc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.424+0000 7f88f2ffd700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f88dc06dfa0 0x7f88dc070460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88e80099d0 tx=0x7f88e8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:15.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.428+0000 7f88f0ff9700 1 -- 172.21.15.2:0/71236387 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f88e405c1d0 con 0x7f88f40ffa10 2024-03-16T10:48:15.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.643+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f88f40697b0 con 0x7f88dc06dfa0 2024-03-16T10:48:15.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.654+0000 7f88f0ff9700 1 -- 172.21.15.2:0/71236387 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f88f40697b0 con 0x7f88dc06dfa0 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (10m) 5s ago 11m 24.8M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (12m) 5s ago 12m 7898k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (11m) 27s ago 11m 7922k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (37s) 5s ago 12m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (31s) 27s ago 11m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (10m) 5s ago 11m 86.1M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (6m) 5s ago 6m 18.5M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (6m) 5s ago 6m 15.5M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (6m) 27s ago 6m 153M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (6m) 27s ago 6m 16.7M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (3m) 5s ago 13m 502M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (2m) 27s ago 10m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:48:15.655 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (2m) 5s ago 13m 44.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (97s) 27s ago 10m 33.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (11m) 5s ago 11m 16.2M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (10m) 27s ago 10m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (11s) 5s ago 10m 11.5M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (9m) 5s ago 9m 199M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 a7bf6745e025 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (9m) 5s ago 9m 177M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (8m) 27s ago 8m 248M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (8m) 27s ago 8m 208M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (7m) 27s ago 7m 198M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:48:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (2m) 5s ago 11m 65.5M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f88dc06dfa0 msgr2=0x7f88dc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f88dc06dfa0 0x7f88dc070460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88e80099d0 tx=0x7f88e8008040 comp rx=0 tx=0).stop 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 msgr2=0x7f88f4103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f4103950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88e4005850 tx=0x7f88e400be20 comp rx=0 tx=0).stop 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 shutdown_connections 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f88dc06dfa0 0x7f88dc070460 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f40ffa10 0x7f88f4103950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 --2- 172.21.15.2:0/71236387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88f4100c10 0x7f88f4101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 >> 172.21.15.2:0/71236387 conn(0x7f88f40fafb0 msgr2=0x7f88f40690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 shutdown_connections 2024-03-16T10:48:15.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.656+0000 7f88fa104700 1 -- 172.21.15.2:0/71236387 wait complete. 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.760+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2293053658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 msgr2=0x7f6b70100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.760+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2293053658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70100f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009910 tx=0x7f6b60009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2293053658 shutdown_connections 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2293053658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b70101ce0 0x7f6b70102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2293053658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70100f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2293053658 >> 172.21.15.2:0/2293053658 conn(0x7f6b700fc0a0 msgr2=0x7f6b700fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2293053658 shutdown_connections 2024-03-16T10:48:15.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2293053658 wait complete. 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 Processor -- start 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.761+0000 7f6b74a5b700 1 -- start start 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b74a5b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b74a5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b70101ce0 0x7f6b700731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b74a5b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70075090 con 0x7f6b70100ae0 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b74a5b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700736e0 con 0x7f6b70101ce0 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36364/0 (socket says 172.21.15.2:36364) 2024-03-16T10:48:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6e59c700 1 -- 172.21.15.2:0/2187742430 learned_addr learned my addr 172.21.15.2:0/2187742430 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6dd9b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b70101ce0 0x7f6b700731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6e59c700 1 -- 172.21.15.2:0/2187742430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b70101ce0 msgr2=0x7f6b700731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6e59c700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b70101ce0 0x7f6b700731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6e59c700 1 -- 172.21.15.2:0/2187742430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600095f0 con 0x7f6b70100ae0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6dd9b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b70101ce0 0x7f6b700731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b6e59c700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70074b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b600059d0 tx=0x7f6b6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b5f7fe700 1 -- 172.21.15.2:0/2187742430 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b6001d070 con 0x7f6b70100ae0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b70073960 con 0x7f6b70100ae0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.762+0000 7f6b5f7fe700 1 -- 172.21.15.2:0/2187742430 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b6000f460 con 0x7f6b70100ae0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.763+0000 7f6b5f7fe700 1 -- 172.21.15.2:0/2187742430 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b60021620 con 0x7f6b70100ae0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.763+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b70073ed0 con 0x7f6b70100ae0 2024-03-16T10:48:15.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.763+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b70190660 con 0x7f6b70100ae0 2024-03-16T10:48:15.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.764+0000 7f6b5f7fe700 1 -- 172.21.15.2:0/2187742430 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f6b6000fa70 con 0x7f6b70100ae0 2024-03-16T10:48:15.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.764+0000 7f6b5f7fe700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b5806dfa0 0x7f6b58070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:15.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.764+0000 7f6b5f7fe700 1 -- 172.21.15.2:0/2187742430 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6b6008f0e0 con 0x7f6b70100ae0 2024-03-16T10:48:15.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.764+0000 7f6b6dd9b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b5806dfa0 0x7f6b58070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:15.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.765+0000 7f6b6dd9b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b5806dfa0 0x7f6b58070460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b64005fd0 tx=0x7f6b64005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:15.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:15.769+0000 7f6b5f7fe700 1 -- 172.21.15.2:0/2187742430 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b6005c080 con 0x7f6b70100ae0 2024-03-16T10:48:16.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:15 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:16.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:15 smithi002 ceph-mon[94239]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-16T10:48:16.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:15 smithi002 ceph-mon[94239]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.066+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b7002ccb0 con 0x7f6b70100ae0 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.066+0000 7f6b5f7fe700 1 -- 172.21.15.2:0/2187742430 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f6b6005bc10 con 0x7f6b70100ae0 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 5, 2024-03-16T10:48:16.067 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 9, 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 5 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:48:16.068 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b5806dfa0 msgr2=0x7f6b58070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b5806dfa0 0x7f6b58070460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6b64005fd0 tx=0x7f6b64005e30 comp rx=0 tx=0).stop 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 msgr2=0x7f6b70074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70074b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b600059d0 tx=0x7f6b6000be20 comp rx=0 tx=0).stop 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 shutdown_connections 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b5806dfa0 0x7f6b58070460 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b70100ae0 0x7f6b70074b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 --2- 172.21.15.2:0/2187742430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b70101ce0 0x7f6b700731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 >> 172.21.15.2:0/2187742430 conn(0x7f6b700fc0a0 msgr2=0x7f6b701052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 shutdown_connections 2024-03-16T10:48:16.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.068+0000 7f6b74a5b700 1 -- 172.21.15.2:0/2187742430 wait complete. 2024-03-16T10:48:16.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 -- 172.21.15.2:0/4158937309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 msgr2=0x7fee48103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/4158937309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee48103a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee30009910 tx=0x7fee30009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:16.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 -- 172.21.15.2:0/4158937309 shutdown_connections 2024-03-16T10:48:16.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/4158937309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee48103f50 0x7fee48106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/4158937309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee48103a10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 -- 172.21.15.2:0/4158937309 >> 172.21.15.2:0/4158937309 conn(0x7fee480faf70 msgr2=0x7fee480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:16.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 -- 172.21.15.2:0/4158937309 shutdown_connections 2024-03-16T10:48:16.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.173+0000 7fee4cbd3700 1 -- 172.21.15.2:0/4158937309 wait complete. 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4cbd3700 1 Processor -- start 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4cbd3700 1 -- start start 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4cbd3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee481965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4cbd3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee48103f50 0x7fee48196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4cbd3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48197100 con 0x7fee48101620 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4cbd3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48197240 con 0x7fee48103f50 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee481965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee481965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36372/0 (socket says 172.21.15.2:36372) 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4659c700 1 -- 172.21.15.2:0/348011292 learned_addr learned my addr 172.21.15.2:0/348011292 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:16.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4659c700 1 -- 172.21.15.2:0/348011292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee48103f50 msgr2=0x7fee48196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4659c700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee48103f50 0x7fee48196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4659c700 1 -- 172.21.15.2:0/348011292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee300095f0 con 0x7fee48101620 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.174+0000 7fee4659c700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee481965a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee30006010 tx=0x7fee3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.175+0000 7fee3f7fe700 1 -- 172.21.15.2:0/348011292 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee3001d070 con 0x7fee48101620 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.175+0000 7fee3f7fe700 1 -- 172.21.15.2:0/348011292 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee3000f460 con 0x7fee48101620 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.175+0000 7fee3f7fe700 1 -- 172.21.15.2:0/348011292 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee30021640 con 0x7fee48101620 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.175+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4819bc40 con 0x7fee48101620 2024-03-16T10:48:16.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.175+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4819c1b0 con 0x7fee48101620 2024-03-16T10:48:16.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.176+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4804ea80 con 0x7fee48101620 2024-03-16T10:48:16.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.176+0000 7fee3f7fe700 1 -- 172.21.15.2:0/348011292 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fee3000fab0 con 0x7fee48101620 2024-03-16T10:48:16.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.177+0000 7fee3f7fe700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fee3406dfa0 0x7fee34070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.177+0000 7fee3f7fe700 1 -- 172.21.15.2:0/348011292 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fee3008f0c0 con 0x7fee48101620 2024-03-16T10:48:16.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.177+0000 7fee45d9b700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fee3406dfa0 0x7fee34070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.177+0000 7fee45d9b700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fee3406dfa0 0x7fee34070460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee380097b0 tx=0x7fee38006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:16.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.181+0000 7fee3f7fe700 1 -- 172.21.15.2:0/348011292 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee3005c0e0 con 0x7fee48101620 2024-03-16T10:48:16.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:15 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:16.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:15 smithi026 ceph-mon[82600]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-16T10:48:16.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:15 smithi026 ceph-mon[82600]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:16.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.432+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fee4819c550 con 0x7fee48101620 2024-03-16T10:48:16.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.432+0000 7fee3f7fe700 1 -- 172.21.15.2:0/348011292 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fee3005bc70 con 0x7fee48101620 2024-03-16T10:48:16.433 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:48:16.433 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:48:16.434 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:48:16.435 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:16.436 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fee3406dfa0 msgr2=0x7fee34070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fee3406dfa0 0x7fee34070460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee380097b0 tx=0x7fee38006d20 comp rx=0 tx=0).stop 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 msgr2=0x7fee481965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee481965a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee30006010 tx=0x7fee3000be20 comp rx=0 tx=0).stop 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 shutdown_connections 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fee3406dfa0 0x7fee34070460 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee48101620 0x7fee481965a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 --2- 172.21.15.2:0/348011292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee48103f50 0x7fee48196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.434+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 >> 172.21.15.2:0/348011292 conn(0x7fee480faf70 msgr2=0x7fee480fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.435+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 shutdown_connections 2024-03-16T10:48:16.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.435+0000 7fee4cbd3700 1 -- 172.21.15.2:0/348011292 wait complete. 2024-03-16T10:48:16.438 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:48:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.540+0000 7fdef448c700 1 -- 172.21.15.2:0/3571466535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ff390 msgr2=0x7fdeec0ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.540+0000 7fdef448c700 1 --2- 172.21.15.2:0/3571466535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ff390 0x7fdeec0ff7b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009910 tx=0x7fdee8009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 -- 172.21.15.2:0/3571466535 shutdown_connections 2024-03-16T10:48:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 --2- 172.21.15.2:0/3571466535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeec0ffcf0 0x7fdeec100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 --2- 172.21.15.2:0/3571466535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ff390 0x7fdeec0ff7b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 -- 172.21.15.2:0/3571466535 >> 172.21.15.2:0/3571466535 conn(0x7fdeec0fafb0 msgr2=0x7fdeec0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 -- 172.21.15.2:0/3571466535 shutdown_connections 2024-03-16T10:48:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 -- 172.21.15.2:0/3571466535 wait complete. 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 Processor -- start 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 -- start start 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeec0ff390 0x7fdeec198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ffcf0 0x7fdeec198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec1992b0 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef448c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec1993f0 con 0x7fdeec0ff390 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.541+0000 7fdef1a27700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ffcf0 0x7fdeec198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef1a27700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ffcf0 0x7fdeec198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36386/0 (socket says 172.21.15.2:36386) 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef1a27700 1 -- 172.21.15.2:0/4005127071 learned_addr learned my addr 172.21.15.2:0/4005127071 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef2228700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeec0ff390 0x7fdeec198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef1a27700 1 -- 172.21.15.2:0/4005127071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeec0ff390 msgr2=0x7fdeec198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef1a27700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeec0ff390 0x7fdeec198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef1a27700 1 -- 172.21.15.2:0/4005127071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80095f0 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef1a27700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ffcf0 0x7fdeec198c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c8a0 tx=0x7fdedc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdee37fe700 1 -- 172.21.15.2:0/4005127071 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdedc0078e0 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdee37fe700 1 -- 172.21.15.2:0/4005127071 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdedc00f450 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec19de00 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdee37fe700 1 -- 172.21.15.2:0/4005127071 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdedc00e5e0 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.542+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec19e3a0 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.543+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeec192880 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.544+0000 7fdee37fe700 1 -- 172.21.15.2:0/4005127071 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdedc00f5c0 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.544+0000 7fdee37fe700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fded806dfb0 0x7fded8070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.544+0000 7fdee37fe700 1 -- 172.21.15.2:0/4005127071 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdedc08dc20 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.544+0000 7fdef2228700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fded806dfb0 0x7fded8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.544+0000 7fdef2228700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fded806dfb0 0x7fded8070470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005090 tx=0x7fdee801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:16.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.549+0000 7fdee37fe700 1 -- 172.21.15.2:0/4005127071 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdedc05ac40 con 0x7fdeec0ffcf0 2024-03-16T10:48:16.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.773+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdeec0611f0 con 0x7fded806dfb0 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.774+0000 7fdee37fe700 1 -- 172.21.15.2:0/4005127071 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdeec0611f0 con 0x7fded806dfb0 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:48:16.775 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:48:16.776 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "7/23 daemons upgraded", 2024-03-16T10:48:16.776 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T10:48:16.776 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:48:16.776 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:48:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fded806dfb0 msgr2=0x7fded8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fded806dfb0 0x7fded8070470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005090 tx=0x7fdee801a040 comp rx=0 tx=0).stop 2024-03-16T10:48:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ffcf0 msgr2=0x7fdeec198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ffcf0 0x7fdeec198c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c8a0 tx=0x7fdedc00cbb0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 shutdown_connections 2024-03-16T10:48:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fded806dfb0 0x7fded8070470 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeec0ff390 0x7fdeec198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 --2- 172.21.15.2:0/4005127071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdeec0ffcf0 0x7fdeec198c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 >> 172.21.15.2:0/4005127071 conn(0x7fdeec0fafb0 msgr2=0x7fdeec107d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 shutdown_connections 2024-03-16T10:48:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.776+0000 7fdef448c700 1 -- 172.21.15.2:0/4005127071 wait complete. 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 -- 172.21.15.2:0/332693532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 msgr2=0x7f3f240731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/332693532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f240731e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00b0a0 tx=0x7f3f1c00b3b0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 -- 172.21.15.2:0/332693532 shutdown_connections 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/332693532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f240737b0 0x7f3f24073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/332693532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f240731e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 -- 172.21.15.2:0/332693532 >> 172.21.15.2:0/332693532 conn(0x7f3f240fc040 msgr2=0x7f3f240fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 -- 172.21.15.2:0/332693532 shutdown_connections 2024-03-16T10:48:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.882+0000 7f3f2ba52700 1 -- 172.21.15.2:0/332693532 wait complete. 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f2ba52700 1 Processor -- start 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f2ba52700 1 -- start start 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f2ba52700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f240737b0 0x7f3f24198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f2ba52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f24198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f2ba52700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f241992e0 con 0x7f3f24074d80 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f2ba52700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24199420 con 0x7f3f240737b0 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f28fed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f24198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f28fed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f24198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36416/0 (socket says 172.21.15.2:36416) 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f28fed700 1 -- 172.21.15.2:0/531962124 learned_addr learned my addr 172.21.15.2:0/531962124 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:16.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f297ee700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f240737b0 0x7f3f24198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f28fed700 1 -- 172.21.15.2:0/531962124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f240737b0 msgr2=0x7f3f24198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f28fed700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f240737b0 0x7f3f24198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f28fed700 1 -- 172.21.15.2:0/531962124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c009d00 con 0x7f3f24074d80 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.883+0000 7f3f297ee700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f240737b0 0x7f3f24198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.884+0000 7f3f28fed700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f24198cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800b4b0 tx=0x7f3f1800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.884+0000 7f3f227fc700 1 -- 172.21.15.2:0/531962124 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f18011800 con 0x7f3f24074d80 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.884+0000 7f3f227fc700 1 -- 172.21.15.2:0/531962124 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f18011e20 con 0x7f3f24074d80 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.884+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2419de30 con 0x7f3f24074d80 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.884+0000 7f3f227fc700 1 -- 172.21.15.2:0/531962124 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f1800f2e0 con 0x7f3f24074d80 2024-03-16T10:48:16.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.884+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2419e3d0 con 0x7f3f24074d80 2024-03-16T10:48:16.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.885+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f24192890 con 0x7f3f24074d80 2024-03-16T10:48:16.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.885+0000 7f3f227fc700 1 -- 172.21.15.2:0/531962124 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3f18010380 con 0x7f3f24074d80 2024-03-16T10:48:16.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.886+0000 7f3f227fc700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f0c06dfa0 0x7f3f0c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:16.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.886+0000 7f3f297ee700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f0c06dfa0 0x7f3f0c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:16.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.886+0000 7f3f227fc700 1 -- 172.21.15.2:0/531962124 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3f1808d810 con 0x7f3f24074d80 2024-03-16T10:48:16.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.886+0000 7f3f297ee700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f0c06dfa0 0x7f3f0c070460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009480 tx=0x7f3f1c00b7d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:16.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:16.891+0000 7f3f227fc700 1 -- 172.21.15.2:0/531962124 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f1805a830 con 0x7f3f24074d80 2024-03-16T10:48:17.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:16 smithi002 ceph-mon[94239]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:16 smithi002 ceph-mon[94239]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:16 smithi002 ceph-mon[94239]: pgmap v66: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:16 smithi002 ceph-mon[94239]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded) 2024-03-16T10:48:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:16 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2187742430' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:16 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/348011292' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:48:17.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.181+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3f2404ea80 con 0x7f3f24074d80 2024-03-16T10:48:17.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.181+0000 7f3f227fc700 1 -- 172.21.15.2:0/531962124 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3f1805a3c0 con 0x7f3f24074d80 2024-03-16T10:48:17.182 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:48:17.182 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:48:17.182 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f0c06dfa0 msgr2=0x7f3f0c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f0c06dfa0 0x7f3f0c070460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009480 tx=0x7f3f1c00b7d0 comp rx=0 tx=0).stop 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 msgr2=0x7f3f24198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f24198cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800b4b0 tx=0x7f3f1800b870 comp rx=0 tx=0).stop 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 shutdown_connections 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f0c06dfa0 0x7f3f0c070460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f240737b0 0x7f3f24198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 --2- 172.21.15.2:0/531962124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f24074d80 0x7f3f24198cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 >> 172.21.15.2:0/531962124 conn(0x7f3f240fc040 msgr2=0x7f3f24106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 shutdown_connections 2024-03-16T10:48:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:17.183+0000 7f3f2ba52700 1 -- 172.21.15.2:0/531962124 wait complete. 2024-03-16T10:48:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:16 smithi026 ceph-mon[82600]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:16 smithi026 ceph-mon[82600]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:16 smithi026 ceph-mon[82600]: pgmap v66: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:16 smithi026 ceph-mon[82600]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded) 2024-03-16T10:48:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:16 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2187742430' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:16 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/348011292' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:48:18.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:17 smithi026 ceph-mon[82600]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:18.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:17 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/531962124' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:48:18.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:17 smithi002 ceph-mon[94239]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:18.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:17 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/531962124' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:48:19.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:18 smithi026 ceph-mon[82600]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:18 smithi002 ceph-mon[94239]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:20 smithi026 ceph-mon[82600]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:20 smithi002 ceph-mon[94239]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:22 smithi026 ceph-mon[82600]: pgmap v69: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:22 smithi002 ceph-mon[94239]: pgmap v69: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:25.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:24 smithi026 ceph-mon[82600]: pgmap v70: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:24 smithi002 ceph-mon[94239]: pgmap v70: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:26.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:25 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:26.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:25 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:27.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:26 smithi026 ceph-mon[82600]: pgmap v71: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:26 smithi002 ceph-mon[94239]: pgmap v71: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:28 smithi026 ceph-mon[82600]: pgmap v72: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:28 smithi002 ceph-mon[94239]: pgmap v72: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:29 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:30.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:29 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:30 smithi002 ceph-mon[94239]: pgmap v73: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:30 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:30 smithi002 ceph-mon[94239]: Upgrade: osd.1 is safe to restart 2024-03-16T10:48:31.133 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:30 smithi026 ceph-mon[82600]: pgmap v73: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:31.133 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:30 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-16T10:48:31.133 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:30 smithi026 ceph-mon[82600]: Upgrade: osd.1 is safe to restart 2024-03-16T10:48:32.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:32 smithi002 ceph-mon[94239]: Upgrade: Updating osd.1 2024-03-16T10:48:32.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:32.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:48:32.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:48:32.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:32 smithi002 ceph-mon[94239]: Deploying daemon osd.1 on smithi002 2024-03-16T10:48:32.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:32 smithi002 ceph-mon[94239]: pgmap v74: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:32.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:32 smithi026 ceph-mon[82600]: Upgrade: Updating osd.1 2024-03-16T10:48:32.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:32.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T10:48:32.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:48:32.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:32 smithi026 ceph-mon[82600]: Deploying daemon osd.1 on smithi002 2024-03-16T10:48:32.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:32 smithi026 ceph-mon[82600]: pgmap v74: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:34.709 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:34 smithi002 systemd[1]: Stopping Ceph osd.1 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:48:35.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:34 smithi002 ceph-mon[94239]: pgmap v75: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:34 smithi002 ceph-mon[94239]: osd.1 marked itself down and dead 2024-03-16T10:48:35.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:35.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:34 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[56344]: 2024-03-16T10:48:34.708+0000 7fe3354b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:48:35.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:34 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[56344]: 2024-03-16T10:48:34.708+0000 7fe3354b1700 -1 osd.1 56 *** Got signal Terminated *** 2024-03-16T10:48:35.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:34 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[56344]: 2024-03-16T10:48:34.708+0000 7fe3354b1700 -1 osd.1 56 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:48:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:34 smithi026 ceph-mon[82600]: pgmap v75: 65 pgs: 65 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:34 smithi026 ceph-mon[82600]: osd.1 marked itself down and dead 2024-03-16T10:48:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:35.676 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:35 smithi002 podman[107833]: 2024-03-16 10:48:35.382749026 +0000 UTC m=+0.784833531 container died a7bf6745e0258f3abec47e48c53e5a4cd920ac15eb13430d4c0765a7e9dbe61e (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=reef-a752c2f, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-16T10:48:36.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:35 smithi002 ceph-mon[94239]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:48:36.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:35 smithi002 ceph-mon[94239]: osdmap e57: 6 total, 5 up, 6 in 2024-03-16T10:48:36.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:35 smithi002 ceph-mon[94239]: osdmap e58: 6 total, 5 up, 6 in 2024-03-16T10:48:36.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:35 smithi002 podman[107833]: 2024-03-16 10:48:35.774949475 +0000 UTC m=+1.177033966 container cleanup a7bf6745e0258f3abec47e48c53e5a4cd920ac15eb13430d4c0765a7e9dbe61e (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=reef-a752c2f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:48:36.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:35 smithi002 bash[107833]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1 2024-03-16T10:48:36.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:35 smithi026 ceph-mon[82600]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:48:36.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:35 smithi026 ceph-mon[82600]: osdmap e57: 6 total, 5 up, 6 in 2024-03-16T10:48:36.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:35 smithi026 ceph-mon[82600]: osdmap e58: 6 total, 5 up, 6 in 2024-03-16T10:48:36.460 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:36 smithi002 podman[107852]: 2024-03-16 10:48:36.101291859 +0000 UTC m=+0.722341149 container remove a7bf6745e0258f3abec47e48c53e5a4cd920ac15eb13430d4c0765a7e9dbe61e (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, RELEASE=reef-a752c2f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:48:36.738 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:36 smithi002 podman[107941]: 2024-03-16 10:48:36.460460873 +0000 UTC m=+0.113950189 container create 2af2ef90559628796690ae6c11dda28758c4a7aeb0386bccdf930fb886cea39e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-03-16T10:48:36.738 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:36 smithi002 podman[107941]: 2024-03-16 10:48:36.364985081 +0000 UTC m=+0.018474402 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:48:36.738 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:36 smithi002 podman[107941]: 2024-03-16 10:48:36.671478468 +0000 UTC m=+0.324967779 container init 2af2ef90559628796690ae6c11dda28758c4a7aeb0386bccdf930fb886cea39e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-16T10:48:36.738 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:36 smithi002 podman[107941]: 2024-03-16 10:48:36.675533813 +0000 UTC m=+0.329023127 container start 2af2ef90559628796690ae6c11dda28758c4a7aeb0386bccdf930fb886cea39e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:48:36.738 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:36 smithi002 podman[107941]: 2024-03-16 10:48:36.738046575 +0000 UTC m=+0.391535894 container attach 2af2ef90559628796690ae6c11dda28758c4a7aeb0386bccdf930fb886cea39e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-16T10:48:36.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:36 smithi002 ceph-mon[94239]: pgmap v77: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:36.988 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:36 smithi002 ceph-mon[94239]: Health check failed: Reduced data availability: 12 pgs peering (PG_AVAILABILITY) 2024-03-16T10:48:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:36 smithi026 ceph-mon[82600]: pgmap v77: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:37.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:36 smithi026 ceph-mon[82600]: Health check failed: Reduced data availability: 12 pgs peering (PG_AVAILABILITY) 2024-03-16T10:48:37.298 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:36 smithi002 podman[107941]: 2024-03-16 10:48:36.988596193 +0000 UTC m=+0.642085521 container died 2af2ef90559628796690ae6c11dda28758c4a7aeb0386bccdf930fb886cea39e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-16T10:48:37.811 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:37 smithi002 podman[107973]: 2024-03-16 10:48:37.55812766 +0000 UTC m=+0.563440175 container remove 2af2ef90559628796690ae6c11dda28758c4a7aeb0386bccdf930fb886cea39e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:48:37.811 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:37 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-16T10:48:37.811 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:37 smithi002 systemd[1]: Stopped Ceph osd.1 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:48:37.812 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:37 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.1.service: Consumed 54.470s CPU time. 2024-03-16T10:48:37.812 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:37 smithi002 systemd[1]: Starting Ceph osd.1 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:48:38.178 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 podman[108083]: 2024-03-16 10:48:37.909735726 +0000 UTC m=+0.017900346 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:48:38.178 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 podman[108083]: 2024-03-16 10:48:38.03878958 +0000 UTC m=+0.146954182 container create 2e4fd6d7db6e788f5827afea98d9bf0e38ed308b87a2f442b368e628936f373c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-16T10:48:38.548 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 podman[108083]: 2024-03-16 10:48:38.278706106 +0000 UTC m=+0.386870718 container init 2e4fd6d7db6e788f5827afea98d9bf0e38ed308b87a2f442b368e628936f373c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:48:38.548 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 podman[108083]: 2024-03-16 10:48:38.283308686 +0000 UTC m=+0.391473290 container start 2e4fd6d7db6e788f5827afea98d9bf0e38ed308b87a2f442b368e628936f373c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-16T10:48:38.548 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 podman[108083]: 2024-03-16 10:48:38.369938188 +0000 UTC m=+0.478102810 container attach 2e4fd6d7db6e788f5827afea98d9bf0e38ed308b87a2f442b368e628936f373c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:48:39.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-mon[94239]: pgmap v79: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:39.048 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate[108098]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T10:48:39.048 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate[108098]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T10:48:39.048 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 bash[108083]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 bash[108083]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 bash[108083]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 bash[108083]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 bash[108083]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 bash[108083]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 bash[108083]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate[108098]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate[108098]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate[108098]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate[108098]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate[108098]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-16T10:48:39.049 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:38 smithi002 podman[108083]: 2024-03-16 10:48:38.755782819 +0000 UTC m=+0.863947422 container died 2e4fd6d7db6e788f5827afea98d9bf0e38ed308b87a2f442b368e628936f373c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:48:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:38 smithi026 ceph-mon[82600]: pgmap v79: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:39.798 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:39 smithi002 podman[108238]: 2024-03-16 10:48:39.507356783 +0000 UTC m=+0.746936179 container remove 2e4fd6d7db6e788f5827afea98d9bf0e38ed308b87a2f442b368e628936f373c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:48:39.799 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:39 smithi002 podman[108305]: 2024-03-16 10:48:39.724802735 +0000 UTC m=+0.115162929 container create dfff225aa9a34d4e170d9cd81230b2b00c41736935f0fe1fcef793864aa98c4d (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-16T10:48:39.799 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:39 smithi002 podman[108305]: 2024-03-16 10:48:39.628464435 +0000 UTC m=+0.018824635 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:48:40.299 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:39 smithi002 podman[108305]: 2024-03-16 10:48:39.969089892 +0000 UTC m=+0.359450086 container init dfff225aa9a34d4e170d9cd81230b2b00c41736935f0fe1fcef793864aa98c4d (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-16T10:48:40.299 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:39 smithi002 podman[108305]: 2024-03-16 10:48:39.972094308 +0000 UTC m=+0.362454501 container start dfff225aa9a34d4e170d9cd81230b2b00c41736935f0fe1fcef793864aa98c4d (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:48:40.299 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:39 smithi002 ceph-osd[108324]: -- 172.21.15.2:0/2936132388 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5ba407680 con 0x55e5ba3d1000 2024-03-16T10:48:40.299 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:40 smithi002 bash[108305]: dfff225aa9a34d4e170d9cd81230b2b00c41736935f0fe1fcef793864aa98c4d 2024-03-16T10:48:40.299 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:40 smithi002 systemd[1]: Started Ceph osd.1 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:48:40.798 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:40 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[108320]: 2024-03-16T10:48:40.509+0000 7f49686d27c0 -1 Falling back to public interface 2024-03-16T10:48:41.180 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:40 smithi002 ceph-mon[94239]: pgmap v80: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:41.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:41.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:41.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:41.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:40 smithi026 ceph-mon[82600]: pgmap v80: 65 pgs: 34 peering, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:41.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:42.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:41 smithi002 ceph-mon[94239]: Health check failed: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-03-16T10:48:42.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:41 smithi002 ceph-mon[94239]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 12 pgs peering) 2024-03-16T10:48:42.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:41 smithi026 ceph-mon[82600]: Health check failed: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-03-16T10:48:42.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:41 smithi026 ceph-mon[82600]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 12 pgs peering) 2024-03-16T10:48:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:42 smithi026 ceph-mon[82600]: pgmap v81: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:43.267 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:42 smithi002 ceph-mon[94239]: pgmap v81: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:45.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:44 smithi002 ceph-mon[94239]: pgmap v82: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:44 smithi026 ceph-mon[82600]: pgmap v82: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:45.798 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:45 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[108320]: 2024-03-16T10:48:45.333+0000 7f49686d27c0 -1 osd.1 56 log_to_monitors true 2024-03-16T10:48:46.653 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:46 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:46.653 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:46 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:46.653 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:46 smithi026 ceph-mon[82600]: from='osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:48:46.653 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:46 smithi026 ceph-mon[82600]: pgmap v83: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:46.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:46 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:46.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:46 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:46.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:46 smithi002 ceph-mon[94239]: from='osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T10:48:46.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:46 smithi002 ceph-mon[94239]: pgmap v83: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:46.799 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 10:48:46 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[108320]: 2024-03-16T10:48:46.327+0000 7f4959b2a700 -1 osd.1 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:48:47.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 -- 172.21.15.2:0/4199517599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0101560 msgr2=0x7f89f0103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 --2- 172.21.15.2:0/4199517599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0101560 0x7f89f0103950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009870 tx=0x7f89d8009b80 comp rx=0 tx=0).stop 2024-03-16T10:48:47.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 -- 172.21.15.2:0/4199517599 shutdown_connections 2024-03-16T10:48:47.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 --2- 172.21.15.2:0/4199517599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f0103e90 0x7f89f0106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 --2- 172.21.15.2:0/4199517599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0101560 0x7f89f0103950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 -- 172.21.15.2:0/4199517599 >> 172.21.15.2:0/4199517599 conn(0x7f89f00faf70 msgr2=0x7f89f00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:47.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 -- 172.21.15.2:0/4199517599 shutdown_connections 2024-03-16T10:48:47.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.301+0000 7f89f6401700 1 -- 172.21.15.2:0/4199517599 wait complete. 2024-03-16T10:48:47.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.302+0000 7f89f6401700 1 Processor -- start 2024-03-16T10:48:47.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.302+0000 7f89f6401700 1 -- start start 2024-03-16T10:48:47.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89f6401700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f0101560 0x7f89f0100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:47.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89f6401700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0103e90 0x7f89f00ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89f6401700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00ffa50 con 0x7f89f0103e90 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89f6401700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00ffb90 con 0x7f89f0101560 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89ef7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0103e90 0x7f89f00ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89ef7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0103e90 0x7f89f00ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43302/0 (socket says 172.21.15.2:43302) 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89ef7fe700 1 -- 172.21.15.2:0/1766469042 learned_addr learned my addr 172.21.15.2:0/1766469042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89effff700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f0101560 0x7f89f0100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89ef7fe700 1 -- 172.21.15.2:0/1766469042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f0101560 msgr2=0x7f89f0100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89ef7fe700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f0101560 0x7f89f0100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89ef7fe700 1 -- 172.21.15.2:0/1766469042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8009d00 con 0x7f89f0103e90 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.303+0000 7f89ef7fe700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0103e90 0x7f89f00ff510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e3c0 tx=0x7f89e800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.304+0000 7f89ed7fa700 1 -- 172.21.15.2:0/1766469042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e8018070 con 0x7f89f0103e90 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.304+0000 7f89ed7fa700 1 -- 172.21.15.2:0/1766469042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89e800f040 con 0x7f89f0103e90 2024-03-16T10:48:47.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.304+0000 7f89ed7fa700 1 -- 172.21.15.2:0/1766469042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e8004590 con 0x7f89f0103e90 2024-03-16T10:48:47.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.304+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d8009520 con 0x7f89f0103e90 2024-03-16T10:48:47.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.304+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f0100250 con 0x7f89f0103e90 2024-03-16T10:48:47.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.305+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f004ea80 con 0x7f89f0103e90 2024-03-16T10:48:47.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.305+0000 7f89ed7fa700 1 -- 172.21.15.2:0/1766469042 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f89e8003a00 con 0x7f89f0103e90 2024-03-16T10:48:47.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.306+0000 7f89ed7fa700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f89dc06dfa0 0x7f89dc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:47.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.306+0000 7f89ed7fa700 1 -- 172.21.15.2:0/1766469042 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f89e808db80 con 0x7f89f0103e90 2024-03-16T10:48:47.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.306+0000 7f89effff700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f89dc06dfa0 0x7f89dc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:47.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.306+0000 7f89effff700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f89dc06dfa0 0x7f89dc070460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009f50 tx=0x7f89d8019040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:47.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.311+0000 7f89ed7fa700 1 -- 172.21.15.2:0/1766469042 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f89e805aba0 con 0x7f89f0103e90 2024-03-16T10:48:47.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.551+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89f01047d0 con 0x7f89dc06dfa0 2024-03-16T10:48:47.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.552+0000 7f89ed7fa700 1 -- 172.21.15.2:0/1766469042 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f89f01047d0 con 0x7f89dc06dfa0 2024-03-16T10:48:47.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f89dc06dfa0 msgr2=0x7f89dc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f89dc06dfa0 0x7f89dc070460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009f50 tx=0x7f89d8019040 comp rx=0 tx=0).stop 2024-03-16T10:48:47.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0103e90 msgr2=0x7f89f00ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0103e90 0x7f89f00ff510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e3c0 tx=0x7f89e800e6d0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 shutdown_connections 2024-03-16T10:48:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f89dc06dfa0 0x7f89dc070460 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f0101560 0x7f89f0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 --2- 172.21.15.2:0/1766469042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f0103e90 0x7f89f00ff510 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 >> 172.21.15.2:0/1766469042 conn(0x7f89f00faf70 msgr2=0x7f89f00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 shutdown_connections 2024-03-16T10:48:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.554+0000 7f89f6401700 1 -- 172.21.15.2:0/1766469042 wait complete. 2024-03-16T10:48:47.566 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:48:47.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 -- 172.21.15.2:0/1763228140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78100ae0 msgr2=0x7fef78100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 --2- 172.21.15.2:0/1763228140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78100ae0 0x7fef78100f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef74009910 tx=0x7fef74009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:47.663 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:47 smithi002 ceph-mon[94239]: from='osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T10:48:47.663 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:47 smithi002 ceph-mon[94239]: osdmap e59: 6 total, 5 up, 6 in 2024-03-16T10:48:47.663 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:47 smithi002 ceph-mon[94239]: from='osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:48:47.663 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:47 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:47.663 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:47 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 -- 172.21.15.2:0/1763228140 shutdown_connections 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 --2- 172.21.15.2:0/1763228140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78101ce0 0x7fef78102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 --2- 172.21.15.2:0/1763228140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78100ae0 0x7fef78100f00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 -- 172.21.15.2:0/1763228140 >> 172.21.15.2:0/1763228140 conn(0x7fef780fc0a0 msgr2=0x7fef780fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 -- 172.21.15.2:0/1763228140 shutdown_connections 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 -- 172.21.15.2:0/1763228140 wait complete. 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 Processor -- start 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.662+0000 7fef7eede700 1 -- start start 2024-03-16T10:48:47.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef7eede700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78100ae0 0x7fef781964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef7eede700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78101ce0 0x7fef78196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef7eede700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78197030 con 0x7fef78101ce0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef7eede700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78197170 con 0x7fef78100ae0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78101ce0 0x7fef78196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78101ce0 0x7fef78196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43326/0 (socket says 172.21.15.2:43326) 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6ffff700 1 -- 172.21.15.2:0/3208274698 learned_addr learned my addr 172.21.15.2:0/3208274698 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef7cc7a700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78100ae0 0x7fef781964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6ffff700 1 -- 172.21.15.2:0/3208274698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78100ae0 msgr2=0x7fef781964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6ffff700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78100ae0 0x7fef781964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6ffff700 1 -- 172.21.15.2:0/3208274698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740095f0 con 0x7fef78101ce0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6ffff700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78101ce0 0x7fef78196a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d8d0 tx=0x7fef6800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6dffb700 1 -- 172.21.15.2:0/3208274698 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef680098a0 con 0x7fef78101ce0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef6dffb700 1 -- 172.21.15.2:0/3208274698 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef68010460 con 0x7fef78101ce0 2024-03-16T10:48:47.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.663+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7819bbd0 con 0x7fef78101ce0 2024-03-16T10:48:47.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.664+0000 7fef6dffb700 1 -- 172.21.15.2:0/3208274698 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6800f5f0 con 0x7fef78101ce0 2024-03-16T10:48:47.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.664+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7819c1a0 con 0x7fef78101ce0 2024-03-16T10:48:47.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.665+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78190660 con 0x7fef78101ce0 2024-03-16T10:48:47.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.665+0000 7fef6dffb700 1 -- 172.21.15.2:0/3208274698 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fef68009a00 con 0x7fef78101ce0 2024-03-16T10:48:47.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.665+0000 7fef6dffb700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fef6006dfa0 0x7fef60070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:47.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.666+0000 7fef6dffb700 1 -- 172.21.15.2:0/3208274698 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fef6808db30 con 0x7fef78101ce0 2024-03-16T10:48:47.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.666+0000 7fef7cc7a700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fef6006dfa0 0x7fef60070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:47.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.666+0000 7fef7cc7a700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fef6006dfa0 0x7fef60070460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef74000c00 tx=0x7fef74005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:47.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.670+0000 7fef6dffb700 1 -- 172.21.15.2:0/3208274698 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef6805ab50 con 0x7fef78101ce0 2024-03-16T10:48:47.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:47 smithi026 ceph-mon[82600]: from='osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T10:48:47.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:47 smithi026 ceph-mon[82600]: osdmap e59: 6 total, 5 up, 6 in 2024-03-16T10:48:47.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:47 smithi026 ceph-mon[82600]: from='osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:48:47.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:47 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:47.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:47 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:47.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.894+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef780611f0 con 0x7fef6006dfa0 2024-03-16T10:48:47.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.895+0000 7fef6dffb700 1 -- 172.21.15.2:0/3208274698 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fef780611f0 con 0x7fef6006dfa0 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fef6006dfa0 msgr2=0x7fef60070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fef6006dfa0 0x7fef60070460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef74000c00 tx=0x7fef74005960 comp rx=0 tx=0).stop 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78101ce0 msgr2=0x7fef78196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78101ce0 0x7fef78196a10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fef6800d8d0 tx=0x7fef6800dbe0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 shutdown_connections 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fef6006dfa0 0x7fef60070460 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef78100ae0 0x7fef781964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 --2- 172.21.15.2:0/3208274698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef78101ce0 0x7fef78196a10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 >> 172.21.15.2:0/3208274698 conn(0x7fef780fc0a0 msgr2=0x7fef781052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:47.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 shutdown_connections 2024-03-16T10:48:47.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:47.897+0000 7fef7eede700 1 -- 172.21.15.2:0/3208274698 wait complete. 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 -- 172.21.15.2:0/1172282394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540ff9e0 msgr2=0x7fcc540ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/1172282394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540ff9e0 0x7fcc540ffe00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400b0a0 tx=0x7fcc4400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 -- 172.21.15.2:0/1172282394 shutdown_connections 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/1172282394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54100be0 0x7fcc54101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/1172282394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540ff9e0 0x7fcc540ffe00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 -- 172.21.15.2:0/1172282394 >> 172.21.15.2:0/1172282394 conn(0x7fcc540faf80 msgr2=0x7fcc540fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 -- 172.21.15.2:0/1172282394 shutdown_connections 2024-03-16T10:48:48.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.004+0000 7fcc58ab7700 1 -- 172.21.15.2:0/1172282394 wait complete. 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc58ab7700 1 Processor -- start 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc58ab7700 1 -- start start 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc58ab7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc540ff9e0 0x7fcc54069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc58ab7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc54100be0 0x7fcc5419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc58ab7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5419b770 con 0x7fcc54100be0 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc58ab7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5419b8b0 con 0x7fcc540ff9e0 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc51d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc54100be0 0x7fcc5419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc51d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc54100be0 0x7fcc5419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43354/0 (socket says 172.21.15.2:43354) 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc51d9b700 1 -- 172.21.15.2:0/3244255612 learned_addr learned my addr 172.21.15.2:0/3244255612 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.005+0000 7fcc5259c700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc540ff9e0 0x7fcc54069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc51d9b700 1 -- 172.21.15.2:0/3244255612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc540ff9e0 msgr2=0x7fcc54069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc51d9b700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc540ff9e0 0x7fcc54069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc51d9b700 1 -- 172.21.15.2:0/3244255612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc44009d00 con 0x7fcc54100be0 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc5259c700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc540ff9e0 0x7fcc54069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc51d9b700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc54100be0 0x7fcc5419b230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c930 tx=0x7fcc3c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3244255612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c007ad0 con 0x7fcc54100be0 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3244255612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc3c007c30 con 0x7fcc54100be0 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5419bb40 con 0x7fcc54100be0 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3244255612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc3c018610 con 0x7fcc54100be0 2024-03-16T10:48:48.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.006+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5419c110 con 0x7fcc54100be0 2024-03-16T10:48:48.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.007+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5418f560 con 0x7fcc54100be0 2024-03-16T10:48:48.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.008+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3244255612 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fcc3c00f450 con 0x7fcc54100be0 2024-03-16T10:48:48.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.008+0000 7fcc4b7fe700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcc4006dfa0 0x7fcc40070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.008+0000 7fcc5259c700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcc4006dfa0 0x7fcc40070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.008+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3244255612 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcc3c08dfa0 con 0x7fcc54100be0 2024-03-16T10:48:48.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.009+0000 7fcc5259c700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcc4006dfa0 0x7fcc40070460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcc44000f80 tx=0x7fcc4400b7d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:48.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.013+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3244255612 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc3c05af40 con 0x7fcc54100be0 2024-03-16T10:48:48.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.231+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc540611f0 con 0x7fcc4006dfa0 2024-03-16T10:48:48.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.242+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3244255612 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcc540611f0 con 0x7fcc4006dfa0 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (10m) 2s ago 12m 24.8M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (12m) 2s ago 12m 8024k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (11m) 60s ago 11m 7922k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (69s) 2s ago 12m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (64s) 60s ago 11m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (10m) 2s ago 12m 86.4M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (6m) 2s ago 6m 18.7M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (6m) 2s ago 6m 15.7M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (6m) 60s ago 6m 153M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (6m) 60s ago 6m 16.7M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:48:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (4m) 2s ago 14m 502M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (2m) 60s ago 11m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (2m) 2s ago 14m 45.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (2m) 60s ago 11m 33.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (12m) 2s ago 12m 16.1M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (11m) 60s ago 11m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (43s) 2s ago 10m 81.1M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (8s) 2s ago 10m 15.0M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (9m) 2s ago 9m 178M 1217M 18.2.2-878-ga752c2f7 f692adb9a265 042aeb9681b1 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (9m) 60s ago 9m 248M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (8m) 60s ago 8m 208M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (8m) 60s ago 8m 198M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:48:48.244 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (3m) 2s ago 11m 68.1M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcc4006dfa0 msgr2=0x7fcc40070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcc4006dfa0 0x7fcc40070460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcc44000f80 tx=0x7fcc4400b7d0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc54100be0 msgr2=0x7fcc5419b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc54100be0 0x7fcc5419b230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c930 tx=0x7fcc3c00ccf0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 shutdown_connections 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcc4006dfa0 0x7fcc40070460 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc540ff9e0 0x7fcc54069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 --2- 172.21.15.2:0/3244255612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc54100be0 0x7fcc5419b230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 >> 172.21.15.2:0/3244255612 conn(0x7fcc540faf80 msgr2=0x7fcc54104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:48.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 shutdown_connections 2024-03-16T10:48:48.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.244+0000 7fcc58ab7700 1 -- 172.21.15.2:0/3244255612 wait complete. 2024-03-16T10:48:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 -- 172.21.15.2:0/2588189704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c0ff9d0 msgr2=0x7f2c9c0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/2588189704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c0ff9d0 0x7f2c9c0ffdf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009910 tx=0x7f2c98009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 -- 172.21.15.2:0/2588189704 shutdown_connections 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/2588189704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9c100bd0 0x7f2c9c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/2588189704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c0ff9d0 0x7f2c9c0ffdf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 -- 172.21.15.2:0/2588189704 >> 172.21.15.2:0/2588189704 conn(0x7f2c9c0faf70 msgr2=0x7f2c9c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 -- 172.21.15.2:0/2588189704 shutdown_connections 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 -- 172.21.15.2:0/2588189704 wait complete. 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 Processor -- start 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.366+0000 7f2ca41ec700 1 -- start start 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca41ec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9c0ff9d0 0x7f2c9c1038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca41ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c100bd0 0x7f2c9c101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca41ec700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c19a1d0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca41ec700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c19a310 con 0x7f2c9c0ff9d0 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1787700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c100bd0 0x7f2c9c101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1787700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c100bd0 0x7f2c9c101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43366/0 (socket says 172.21.15.2:43366) 2024-03-16T10:48:48.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1787700 1 -- 172.21.15.2:0/3991544705 learned_addr learned my addr 172.21.15.2:0/3991544705 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:48.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1f88700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9c0ff9d0 0x7f2c9c1038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1787700 1 -- 172.21.15.2:0/3991544705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9c0ff9d0 msgr2=0x7f2c9c1038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1787700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9c0ff9d0 0x7f2c9c1038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1787700 1 -- 172.21.15.2:0/3991544705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980095f0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca1787700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c100bd0 0x7f2c9c101fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d900 tx=0x7f2c8c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2c92ffd700 1 -- 172.21.15.2:0/3991544705 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c0041f0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2c92ffd700 1 -- 172.21.15.2:0/3991544705 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c8c004cf0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c1024f0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2c92ffd700 1 -- 172.21.15.2:0/3991544705 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8c00b750 con 0x7f2c9c100bd0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.367+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c102a10 con 0x7f2c9c100bd0 2024-03-16T10:48:48.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.369+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c04ea80 con 0x7f2c9c100bd0 2024-03-16T10:48:48.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.369+0000 7f2c92ffd700 1 -- 172.21.15.2:0/3991544705 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2c8c004750 con 0x7f2c9c100bd0 2024-03-16T10:48:48.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.370+0000 7f2c92ffd700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c8806dfb0 0x7f2c88070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.370+0000 7f2c92ffd700 1 -- 172.21.15.2:0/3991544705 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2c8c08dbc0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.370+0000 7f2ca1f88700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c8806dfb0 0x7f2c88070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.370+0000 7f2ca1f88700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c8806dfb0 0x7f2c88070470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c98005090 tx=0x7f2c98005180 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:48.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.374+0000 7f2c92ffd700 1 -- 172.21.15.2:0/3991544705 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c8c05abe0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:48 smithi002 ceph-mon[94239]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:48:48.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:48 smithi002 ceph-mon[94239]: osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700] boot 2024-03-16T10:48:48.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:48 smithi002 ceph-mon[94239]: osdmap e60: 6 total, 6 up, 6 in 2024-03-16T10:48:48.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:48 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:48:48.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:48 smithi002 ceph-mon[94239]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:48.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:48 smithi002 ceph-mon[94239]: pgmap v86: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:48.511 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:48 smithi002 ceph-mon[94239]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:48.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.674+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2c9c1a2dd0 con 0x7f2c9c100bd0 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.675+0000 7f2c92ffd700 1 -- 172.21.15.2:0/3991544705 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f2c8c016360 con 0x7f2c9c100bd0 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:48:48.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4, 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 8, 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:48:48.677 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:48:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.677+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c8806dfb0 msgr2=0x7f2c88070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.677+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c8806dfb0 0x7f2c88070470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c98005090 tx=0x7f2c98005180 comp rx=0 tx=0).stop 2024-03-16T10:48:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.677+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c100bd0 msgr2=0x7f2c9c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.677+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c100bd0 0x7f2c9c101fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d900 tx=0x7f2c8c00dc10 comp rx=0 tx=0).stop 2024-03-16T10:48:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.677+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 shutdown_connections 2024-03-16T10:48:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.678+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2c8806dfb0 0x7f2c88070470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.678+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9c0ff9d0 0x7f2c9c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.678+0000 7f2ca41ec700 1 --2- 172.21.15.2:0/3991544705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9c100bd0 0x7f2c9c101fb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.678+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 >> 172.21.15.2:0/3991544705 conn(0x7f2c9c0faf70 msgr2=0x7f2c9c068530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.678+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 shutdown_connections 2024-03-16T10:48:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.678+0000 7f2ca41ec700 1 -- 172.21.15.2:0/3991544705 wait complete. 2024-03-16T10:48:48.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:48 smithi026 ceph-mon[82600]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:48:48.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:48 smithi026 ceph-mon[82600]: osd.1 [v2:172.21.15.2:6810/648077700,v1:172.21.15.2:6811/648077700] boot 2024-03-16T10:48:48.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:48 smithi026 ceph-mon[82600]: osdmap e60: 6 total, 6 up, 6 in 2024-03-16T10:48:48.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:48 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T10:48:48.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:48 smithi026 ceph-mon[82600]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:48.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:48 smithi026 ceph-mon[82600]: pgmap v86: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:48.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:48 smithi026 ceph-mon[82600]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 -- 172.21.15.2:0/2447168429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ffd20 msgr2=0x7fc4841001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2447168429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ffd20 0x7fc4841001a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009910 tx=0x7fc46c009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 -- 172.21.15.2:0/2447168429 shutdown_connections 2024-03-16T10:48:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2447168429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ffd20 0x7fc4841001a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2447168429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ff3c0 0x7fc4840ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 -- 172.21.15.2:0/2447168429 >> 172.21.15.2:0/2447168429 conn(0x7fc4840faf80 msgr2=0x7fc4840fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 -- 172.21.15.2:0/2447168429 shutdown_connections 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.785+0000 7fc489d1e700 1 -- 172.21.15.2:0/2447168429 wait complete. 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc489d1e700 1 Processor -- start 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc489d1e700 1 -- start start 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc489d1e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ff3c0 0x7fc4841986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc489d1e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ffd20 0x7fc484198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc489d1e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484199190 con 0x7fc4840ff3c0 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc489d1e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841992d0 con 0x7fc4840ffd20 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc4837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ff3c0 0x7fc4841986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc482ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ffd20 0x7fc484198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc482ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ffd20 0x7fc484198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:49648/0 (socket says 172.21.15.2:49648) 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.786+0000 7fc4837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ff3c0 0x7fc4841986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43380/0 (socket says 172.21.15.2:43380) 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc482ffd700 1 -- 172.21.15.2:0/2561506540 learned_addr learned my addr 172.21.15.2:0/2561506540 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:48.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc4837fe700 1 -- 172.21.15.2:0/2561506540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ffd20 msgr2=0x7fc484198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc4837fe700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ffd20 0x7fc484198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc4837fe700 1 -- 172.21.15.2:0/2561506540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c0095f0 con 0x7fc4840ff3c0 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc482ffd700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ffd20 0x7fc484198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc4837fe700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ff3c0 0x7fc4841986c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c8f0 tx=0x7fc47400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc480ff9700 1 -- 172.21.15.2:0/2561506540 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc474004410 con 0x7fc4840ff3c0 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc480ff9700 1 -- 172.21.15.2:0/2561506540 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc474004570 con 0x7fc4840ff3c0 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48419dd40 con 0x7fc4840ff3c0 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc480ff9700 1 -- 172.21.15.2:0/2561506540 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4740038b0 con 0x7fc4840ff3c0 2024-03-16T10:48:48.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.787+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48419e280 con 0x7fc4840ff3c0 2024-03-16T10:48:48.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.789+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48404eaf0 con 0x7fc4840ff3c0 2024-03-16T10:48:48.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.789+0000 7fc480ff9700 1 -- 172.21.15.2:0/2561506540 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc474003b20 con 0x7fc4840ff3c0 2024-03-16T10:48:48.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.790+0000 7fc480ff9700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc47006dfa0 0x7fc470070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:48.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.790+0000 7fc480ff9700 1 -- 172.21.15.2:0/2561506540 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc47408d9b0 con 0x7fc4840ff3c0 2024-03-16T10:48:48.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.790+0000 7fc482ffd700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc47006dfa0 0x7fc470070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:48.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.791+0000 7fc482ffd700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc47006dfa0 0x7fc470070460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0051c0 tx=0x7fc46c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:48.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:48.795+0000 7fc480ff9700 1 -- 172.21.15.2:0/2561506540 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc47405a9f0 con 0x7fc4840ff3c0 2024-03-16T10:48:49.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.072+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc48402cd30 con 0x7fc4840ff3c0 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.072+0000 7fc480ff9700 1 -- 172.21.15.2:0/2561506540 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fc47401d770 con 0x7fc4840ff3c0 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:48:49.073 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:48:49.074 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:49.075 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:49.076 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:48:49.076 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:48:49.076 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:49.076 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:49.076 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc47006dfa0 msgr2=0x7fc470070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc47006dfa0 0x7fc470070460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0051c0 tx=0x7fc46c00b540 comp rx=0 tx=0).stop 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ff3c0 msgr2=0x7fc4841986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ff3c0 0x7fc4841986c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c8f0 tx=0x7fc47400cc00 comp rx=0 tx=0).stop 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 shutdown_connections 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc47006dfa0 0x7fc470070460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4840ff3c0 0x7fc4841986c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 --2- 172.21.15.2:0/2561506540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4840ffd20 0x7fc484198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 >> 172.21.15.2:0/2561506540 conn(0x7fc4840faf80 msgr2=0x7fc484107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.074+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 shutdown_connections 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.075+0000 7fc489d1e700 1 -- 172.21.15.2:0/2561506540 wait complete. 2024-03-16T10:48:49.077 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:48:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 -- 172.21.15.2:0/2378290320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 msgr2=0x7f94b4101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 --2- 172.21.15.2:0/2378290320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4101030 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009910 tx=0x7f94a4009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 -- 172.21.15.2:0/2378290320 shutdown_connections 2024-03-16T10:48:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 --2- 172.21.15.2:0/2378290320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4101030 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 --2- 172.21.15.2:0/2378290320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b40ff9d0 0x7f94b40ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 -- 172.21.15.2:0/2378290320 >> 172.21.15.2:0/2378290320 conn(0x7f94b40faf70 msgr2=0x7f94b40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:49.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 -- 172.21.15.2:0/2378290320 shutdown_connections 2024-03-16T10:48:49.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.183+0000 7f94bc651700 1 -- 172.21.15.2:0/2378290320 wait complete. 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94bc651700 1 Processor -- start 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94bc651700 1 -- start start 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94bc651700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b40ff9d0 0x7f94b4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94bc651700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94bc651700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4199270 con 0x7f94b4100bd0 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94bc651700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b41993b0 con 0x7f94b40ff9d0 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94b9bec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.184+0000 7f94b9bec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43384/0 (socket says 172.21.15.2:43384) 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94b9bec700 1 -- 172.21.15.2:0/4162798625 learned_addr learned my addr 172.21.15.2:0/4162798625 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94ba3ed700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b40ff9d0 0x7f94b4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:49.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94b9bec700 1 -- 172.21.15.2:0/4162798625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b40ff9d0 msgr2=0x7f94b4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94b9bec700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b40ff9d0 0x7f94b4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94b9bec700 1 -- 172.21.15.2:0/4162798625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a40095f0 con 0x7f94b4100bd0 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94b9bec700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4198c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94a4006010 tx=0x7f94a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94ab7fe700 1 -- 172.21.15.2:0/4162798625 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a401d070 con 0x7f94b4100bd0 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94ab7fe700 1 -- 172.21.15.2:0/4162798625 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94a4005dc0 con 0x7f94b4100bd0 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94ab7fe700 1 -- 172.21.15.2:0/4162798625 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a4021410 con 0x7f94b4100bd0 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b419ddb0 con 0x7f94b4100bd0 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.185+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b419e320 con 0x7f94b4100bd0 2024-03-16T10:48:49.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.186+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b404ea80 con 0x7f94b4100bd0 2024-03-16T10:48:49.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.187+0000 7f94ab7fe700 1 -- 172.21.15.2:0/4162798625 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f94a400f460 con 0x7f94b4100bd0 2024-03-16T10:48:49.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.188+0000 7f94ab7fe700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94a006dfb0 0x7f94a0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:49.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.188+0000 7f94ab7fe700 1 -- 172.21.15.2:0/4162798625 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f94a408eed0 con 0x7f94b4100bd0 2024-03-16T10:48:49.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.188+0000 7f94ba3ed700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94a006dfb0 0x7f94a0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:49.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.188+0000 7f94ba3ed700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94a006dfb0 0x7f94a0070470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94b4100a30 tx=0x7f94b000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:49.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.192+0000 7f94ab7fe700 1 -- 172.21.15.2:0/4162798625 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94a405bef0 con 0x7f94b4100bd0 2024-03-16T10:48:49.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.445+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94b4068c40 con 0x7f94a006dfb0 2024-03-16T10:48:49.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.446+0000 7f94ab7fe700 1 -- 172.21.15.2:0/4162798625 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f94b4068c40 con 0x7f94a006dfb0 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "8/23 daemons upgraded", 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:48:49.447 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94a006dfb0 msgr2=0x7f94a0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94a006dfb0 0x7f94a0070470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94b4100a30 tx=0x7f94b000b410 comp rx=0 tx=0).stop 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 msgr2=0x7f94b4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4198c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f94a4006010 tx=0x7f94a400bd70 comp rx=0 tx=0).stop 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 shutdown_connections 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94a006dfb0 0x7f94a0070470 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b40ff9d0 0x7f94b4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 --2- 172.21.15.2:0/4162798625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94b4100bd0 0x7f94b4198c50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 >> 172.21.15.2:0/4162798625 conn(0x7f94b40faf70 msgr2=0x7f94b4068530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:49.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 shutdown_connections 2024-03-16T10:48:49.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.448+0000 7f94bc651700 1 -- 172.21.15.2:0/4162798625 wait complete. 2024-03-16T10:48:49.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.557+0000 7faf9fe32700 1 -- 172.21.15.2:0/1677788652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98100b60 msgr2=0x7faf98100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.557+0000 7faf9fe32700 1 --2- 172.21.15.2:0/1677788652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98100b60 0x7faf98100fc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf88009910 tx=0x7faf88009c20 comp rx=0 tx=0).stop 2024-03-16T10:48:49.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 -- 172.21.15.2:0/1677788652 shutdown_connections 2024-03-16T10:48:49.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 --2- 172.21.15.2:0/1677788652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf98100b60 0x7faf98100fc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 --2- 172.21.15.2:0/1677788652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf980ff960 0x7faf980ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 -- 172.21.15.2:0/1677788652 >> 172.21.15.2:0/1677788652 conn(0x7faf980faf20 msgr2=0x7faf980fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:49.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 -- 172.21.15.2:0/1677788652 shutdown_connections 2024-03-16T10:48:49.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 -- 172.21.15.2:0/1677788652 wait complete. 2024-03-16T10:48:49.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 Processor -- start 2024-03-16T10:48:49.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 -- start start 2024-03-16T10:48:49.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf980ff960 0x7faf981038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:49.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf98100b60 0x7faf98101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98102480 con 0x7faf980ff960 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9fe32700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf981025c0 con 0x7faf98100b60 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.558+0000 7faf9dbce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf980ff960 0x7faf981038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9dbce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf980ff960 0x7faf981038a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43416/0 (socket says 172.21.15.2:43416) 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9dbce700 1 -- 172.21.15.2:0/984449112 learned_addr learned my addr 172.21.15.2:0/984449112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9d3cd700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf98100b60 0x7faf98101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9dbce700 1 -- 172.21.15.2:0/984449112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf98100b60 msgr2=0x7faf98101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9dbce700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf98100b60 0x7faf98101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9dbce700 1 -- 172.21.15.2:0/984449112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880095f0 con 0x7faf980ff960 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9dbce700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf980ff960 0x7faf981038a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e8f0 tx=0x7faf9400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf8effd700 1 -- 172.21.15.2:0/984449112 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf94009790 con 0x7faf980ff960 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf981028a0 con 0x7faf980ff960 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf8effd700 1 -- 172.21.15.2:0/984449112 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf9400c810 con 0x7faf980ff960 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf8effd700 1 -- 172.21.15.2:0/984449112 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf94003d20 con 0x7faf980ff960 2024-03-16T10:48:49.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.559+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9819e4b0 con 0x7faf980ff960 2024-03-16T10:48:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.560+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9804ea80 con 0x7faf980ff960 2024-03-16T10:48:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.561+0000 7faf8effd700 1 -- 172.21.15.2:0/984449112 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7faf940041f0 con 0x7faf980ff960 2024-03-16T10:48:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.561+0000 7faf8effd700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faf8406dfb0 0x7faf84070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:48:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.561+0000 7faf8effd700 1 -- 172.21.15.2:0/984449112 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6500+0+0 (secure 0 0 0) 0x7faf94014070 con 0x7faf980ff960 2024-03-16T10:48:49.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.561+0000 7faf9d3cd700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faf8406dfb0 0x7faf84070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:48:49.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.562+0000 7faf9d3cd700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faf8406dfb0 0x7faf84070470 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b800 tx=0x7faf88009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:48:49.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.566+0000 7faf8effd700 1 -- 172.21.15.2:0/984449112 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faf9405afc0 con 0x7faf980ff960 2024-03-16T10:48:49.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:49 smithi002 ceph-mon[94239]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:49.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:49 smithi002 ceph-mon[94239]: osdmap e61: 6 total, 6 up, 6 in 2024-03-16T10:48:49.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:49 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3991544705' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:49.726 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:49 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2561506540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:48:49.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:49 smithi026 ceph-mon[82600]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:49.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:49 smithi026 ceph-mon[82600]: osdmap e61: 6 total, 6 up, 6 in 2024-03-16T10:48:49.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:49 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3991544705' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:49.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:49 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2561506540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.868+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faf9819e640 con 0x7faf980ff960 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.868+0000 7faf8effd700 1 -- 172.21.15.2:0/984449112 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1093 (secure 0 0 0) 0x7faf9405ab50 con 0x7faf980ff960 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-16T10:48:49.869 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-16T10:48:49.870 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faf8406dfb0 msgr2=0x7faf84070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faf8406dfb0 0x7faf84070470 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b800 tx=0x7faf88009f70 comp rx=0 tx=0).stop 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf980ff960 msgr2=0x7faf981038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf980ff960 0x7faf981038a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e8f0 tx=0x7faf9400ec00 comp rx=0 tx=0).stop 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 shutdown_connections 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faf8406dfb0 0x7faf84070470 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf980ff960 0x7faf981038a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 --2- 172.21.15.2:0/984449112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf98100b60 0x7faf98101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 >> 172.21.15.2:0/984449112 conn(0x7faf980faf20 msgr2=0x7faf98068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:48:49.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 shutdown_connections 2024-03-16T10:48:49.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:48:49.870+0000 7faf9fe32700 1 -- 172.21.15.2:0/984449112 wait complete. 2024-03-16T10:48:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: pgmap v88: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/984449112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:51.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:51.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:48:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:48:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: pgmap v88: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/984449112' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:48:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:48:51.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:51 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:48:51.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:51 smithi026 ceph-mon[82600]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-16T10:48:52.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:51 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:48:52.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:51 smithi002 ceph-mon[94239]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-16T10:48:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:52 smithi026 ceph-mon[82600]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:53.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:52 smithi026 ceph-mon[82600]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded) 2024-03-16T10:48:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:52 smithi002 ceph-mon[94239]: pgmap v89: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:52 smithi002 ceph-mon[94239]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded) 2024-03-16T10:48:55.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:54 smithi026 ceph-mon[82600]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:54 smithi002 ceph-mon[94239]: pgmap v90: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:56 smithi026 ceph-mon[82600]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:56 smithi002 ceph-mon[94239]: pgmap v91: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:59.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:48:58 smithi026 ceph-mon[82600]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:48:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:48:58 smithi002 ceph-mon[94239]: pgmap v92: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:01.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:00 smithi002 ceph-mon[94239]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:01.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:00 smithi026 ceph-mon[82600]: pgmap v93: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:00 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:03.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:02 smithi026 ceph-mon[82600]: pgmap v94: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:02 smithi002 ceph-mon[94239]: pgmap v94: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:04 smithi026 ceph-mon[82600]: pgmap v95: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:04 smithi002 ceph-mon[94239]: pgmap v95: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:06.186 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:49:06.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:49:07.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:06 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:49:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:06 smithi026 ceph-mon[82600]: Upgrade: osd.2 is safe to restart 2024-03-16T10:49:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:06 smithi026 ceph-mon[82600]: pgmap v96: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:06 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-16T10:49:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:06 smithi002 ceph-mon[94239]: Upgrade: osd.2 is safe to restart 2024-03-16T10:49:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:06 smithi002 ceph-mon[94239]: pgmap v96: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:08.154 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:07 smithi002 ceph-mon[94239]: Upgrade: Updating osd.2 2024-03-16T10:49:08.155 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:07 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:08.155 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:07 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:49:08.155 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:07 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:49:08.155 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:07 smithi002 ceph-mon[94239]: Deploying daemon osd.2 on smithi002 2024-03-16T10:49:08.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:07 smithi026 ceph-mon[82600]: Upgrade: Updating osd.2 2024-03-16T10:49:08.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:07 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:08.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:07 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T10:49:08.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:07 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:49:08.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:07 smithi026 ceph-mon[82600]: Deploying daemon osd.2 on smithi002 2024-03-16T10:49:09.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:08 smithi026 ceph-mon[82600]: pgmap v97: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:08 smithi002 ceph-mon[94239]: pgmap v97: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:10.526 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:10 smithi002 systemd[1]: Stopping Ceph osd.2 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:49:10.798 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:10 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[63777]: 2024-03-16T10:49:10.524+0000 7f95eef50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:49:10.798 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:10 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[63777]: 2024-03-16T10:49:10.524+0000 7f95eef50700 -1 osd.2 61 *** Got signal Terminated *** 2024-03-16T10:49:10.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:10 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[63777]: 2024-03-16T10:49:10.524+0000 7f95eef50700 -1 osd.2 61 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:49:11.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:10 smithi026 ceph-mon[82600]: pgmap v98: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:11.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:10 smithi026 ceph-mon[82600]: osd.2 marked itself down and dead 2024-03-16T10:49:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:10 smithi002 ceph-mon[94239]: pgmap v98: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:10 smithi002 ceph-mon[94239]: osd.2 marked itself down and dead 2024-03-16T10:49:11.610 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:11 smithi002 podman[113273]: 2024-03-16 10:49:11.333106247 +0000 UTC m=+0.906878117 container died 042aeb9681b1635cbe58cf8c5815307519f88a95962ce1842f5be2627e29fbb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-a752c2f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-16T10:49:11.921 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:11 smithi002 podman[113273]: 2024-03-16 10:49:11.700083652 +0000 UTC m=+1.273855530 container cleanup 042aeb9681b1635cbe58cf8c5815307519f88a95962ce1842f5be2627e29fbb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, ceph=True, RELEASE=reef-a752c2f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:49:11.921 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:11 smithi002 bash[113273]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2 2024-03-16T10:49:12.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:11 smithi026 ceph-mon[82600]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:49:12.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:11 smithi026 ceph-mon[82600]: osdmap e62: 6 total, 5 up, 6 in 2024-03-16T10:49:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:11 smithi002 ceph-mon[94239]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:49:12.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:11 smithi002 ceph-mon[94239]: osdmap e62: 6 total, 5 up, 6 in 2024-03-16T10:49:12.299 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:12 smithi002 podman[113291]: 2024-03-16 10:49:12.01057383 +0000 UTC m=+0.674276729 container remove 042aeb9681b1635cbe58cf8c5815307519f88a95962ce1842f5be2627e29fbb5 (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=reef-a752c2f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-16T10:49:12.655 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:12 smithi002 podman[113383]: 2024-03-16 10:49:12.227903079 +0000 UTC m=+0.017778222 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:49:12.656 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:12 smithi002 podman[113383]: 2024-03-16 10:49:12.336238023 +0000 UTC m=+0.126113148 container create 89124e23ab54e6ba0a3c8d4ef29a2dca25e4b8afce058741d88c0bd213b6dc4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:49:12.656 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:12 smithi002 podman[113383]: 2024-03-16 10:49:12.55543874 +0000 UTC m=+0.345313864 container init 89124e23ab54e6ba0a3c8d4ef29a2dca25e4b8afce058741d88c0bd213b6dc4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:49:12.656 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:12 smithi002 podman[113383]: 2024-03-16 10:49:12.558456591 +0000 UTC m=+0.348331718 container start 89124e23ab54e6ba0a3c8d4ef29a2dca25e4b8afce058741d88c0bd213b6dc4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:49:12.656 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:12 smithi002 podman[113383]: 2024-03-16 10:49:12.655408714 +0000 UTC m=+0.445283837 container attach 89124e23ab54e6ba0a3c8d4ef29a2dca25e4b8afce058741d88c0bd213b6dc4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:49:12.922 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:12 smithi002 podman[113383]: 2024-03-16 10:49:12.859157997 +0000 UTC m=+0.649033128 container died 89124e23ab54e6ba0a3c8d4ef29a2dca25e4b8afce058741d88c0bd213b6dc4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-03-16T10:49:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:12 smithi026 ceph-mon[82600]: pgmap v100: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:12 smithi026 ceph-mon[82600]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-16T10:49:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:12 smithi026 ceph-mon[82600]: osdmap e63: 6 total, 5 up, 6 in 2024-03-16T10:49:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:12 smithi002 ceph-mon[94239]: pgmap v100: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:12 smithi002 ceph-mon[94239]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-16T10:49:13.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:12 smithi002 ceph-mon[94239]: osdmap e63: 6 total, 5 up, 6 in 2024-03-16T10:49:13.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:13 smithi002 podman[113410]: 2024-03-16 10:49:13.417446501 +0000 UTC m=+0.549521996 container remove 89124e23ab54e6ba0a3c8d4ef29a2dca25e4b8afce058741d88c0bd213b6dc4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-16T10:49:13.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:13 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-16T10:49:13.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:13 smithi002 systemd[1]: Stopped Ceph osd.2 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:49:13.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:13 smithi002 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.2.service: Consumed 45.653s CPU time. 2024-03-16T10:49:13.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:13 smithi002 systemd[1]: Starting Ceph osd.2 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:49:14.163 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:13 smithi002 podman[113515]: 2024-03-16 10:49:13.763827995 +0000 UTC m=+0.018704604 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:49:14.163 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:13 smithi002 podman[113515]: 2024-03-16 10:49:13.876887798 +0000 UTC m=+0.131764403 container create b6139617513631cd2333156a7fad040f4a30d842a98c93d5dba4c4e28b3e3950 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-16T10:49:14.163 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 podman[113515]: 2024-03-16 10:49:14.088174667 +0000 UTC m=+0.343051287 container init b6139617513631cd2333156a7fad040f4a30d842a98c93d5dba4c4e28b3e3950 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:49:14.163 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 podman[113515]: 2024-03-16 10:49:14.091767879 +0000 UTC m=+0.346644484 container start b6139617513631cd2333156a7fad040f4a30d842a98c93d5dba4c4e28b3e3950 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-16T10:49:14.163 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 podman[113515]: 2024-03-16 10:49:14.162672599 +0000 UTC m=+0.417549219 container attach b6139617513631cd2333156a7fad040f4a30d842a98c93d5dba4c4e28b3e3950 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-16T10:49:14.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate[113531]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T10:49:14.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate[113531]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T10:49:14.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 bash[113515]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T10:49:14.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 bash[113515]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T10:49:14.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 bash[113515]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T10:49:14.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 bash[113515]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T10:49:14.799 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 bash[113515]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 bash[113515]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 bash[113515]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate[113531]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate[113531]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate[113531]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate[113531]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate[113531]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-16T10:49:14.800 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:14 smithi002 podman[113515]: 2024-03-16 10:49:14.568100714 +0000 UTC m=+0.822977319 container died b6139617513631cd2333156a7fad040f4a30d842a98c93d5dba4c4e28b3e3950 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-16T10:49:15.150 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:14 smithi002 ceph-mon[94239]: pgmap v102: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:14 smithi026 ceph-mon[82600]: pgmap v102: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:15.466 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 podman[113655]: 2024-03-16 10:49:15.150130686 +0000 UTC m=+0.572487129 container remove b6139617513631cd2333156a7fad040f4a30d842a98c93d5dba4c4e28b3e3950 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-16T10:49:15.466 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 podman[113717]: 2024-03-16 10:49:15.270304922 +0000 UTC m=+0.018243384 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:49:15.466 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 podman[113717]: 2024-03-16 10:49:15.367862994 +0000 UTC m=+0.115801447 container create 803c7eeb943b58b37bed2d799a73a87a188b09b73217fa1a1d8f3d1f5f00e066 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:49:16.017 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 podman[113717]: 2024-03-16 10:49:15.561979782 +0000 UTC m=+0.309918237 container init 803c7eeb943b58b37bed2d799a73a87a188b09b73217fa1a1d8f3d1f5f00e066 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-16T10:49:16.017 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 podman[113717]: 2024-03-16 10:49:15.564909947 +0000 UTC m=+0.312848400 container start 803c7eeb943b58b37bed2d799a73a87a188b09b73217fa1a1d8f3d1f5f00e066 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-16T10:49:16.017 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 ceph-osd[113737]: -- 172.21.15.2:0/871454499 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561bb9eef680 con 0x561bb9eb8c00 2024-03-16T10:49:16.018 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 bash[113717]: 803c7eeb943b58b37bed2d799a73a87a188b09b73217fa1a1d8f3d1f5f00e066 2024-03-16T10:49:16.018 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:15 smithi002 systemd[1]: Started Ceph osd.2 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:49:16.298 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:16 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[113733]: 2024-03-16T10:49:16.108+0000 7f14ac3b67c0 -1 Falling back to public interface 2024-03-16T10:49:16.840 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:16.840 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:16.840 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:49:16.840 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:16 smithi002 ceph-mon[94239]: pgmap v103: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 20/267 objects degraded (7.491%) 2024-03-16T10:49:16.968 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:16.968 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:16.968 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:49:16.968 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:16 smithi026 ceph-mon[82600]: pgmap v103: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 20/267 objects degraded (7.491%) 2024-03-16T10:49:17.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:17 smithi026 ceph-mon[82600]: Health check failed: Degraded data redundancy: 20/267 objects degraded (7.491%), 9 pgs degraded (PG_DEGRADED) 2024-03-16T10:49:18.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:17 smithi002 ceph-mon[94239]: Health check failed: Degraded data redundancy: 20/267 objects degraded (7.491%), 9 pgs degraded (PG_DEGRADED) 2024-03-16T10:49:18.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:18 smithi026 ceph-mon[82600]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:18.997 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:18 smithi002 ceph-mon[94239]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:19.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:19 smithi002 ceph-mon[94239]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-16T10:49:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:19 smithi026 ceph-mon[82600]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-16T10:49:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 -- 172.21.15.2:0/1777375792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 msgr2=0x7f1790108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 --2- 172.21.15.2:0/1777375792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f1790108430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1780009910 tx=0x7f1780009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 -- 172.21.15.2:0/1777375792 shutdown_connections 2024-03-16T10:49:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 --2- 172.21.15.2:0/1777375792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010ae70 0x7f179010d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 --2- 172.21.15.2:0/1777375792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f1790108430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 -- 172.21.15.2:0/1777375792 >> 172.21.15.2:0/1777375792 conn(0x7f179006dd00 msgr2=0x7f1790070120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 -- 172.21.15.2:0/1777375792 shutdown_connections 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.986+0000 7f1795a1d700 1 -- 172.21.15.2:0/1777375792 wait complete. 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.987+0000 7f1795a1d700 1 Processor -- start 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.987+0000 7f1795a1d700 1 -- start start 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.987+0000 7f1795a1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f179006fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.987+0000 7f1795a1d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010ae70 0x7f179006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.987+0000 7f178effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f179006fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.987+0000 7f1795a1d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a2990 con 0x7f1790108010 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f178effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f179006fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46694/0 (socket says 172.21.15.2:46694) 2024-03-16T10:49:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f178effd700 1 -- 172.21.15.2:0/4185025996 learned_addr learned my addr 172.21.15.2:0/4185025996 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a2ad0 con 0x7f179010ae70 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f178e7fc700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010ae70 0x7f179006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f178effd700 1 -- 172.21.15.2:0/4185025996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010ae70 msgr2=0x7f179006e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f178effd700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010ae70 0x7f179006e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f178effd700 1 -- 172.21.15.2:0/4185025996 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800095f0 con 0x7f1790108010 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f178effd700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f179006fbb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1780006010 tx=0x7f178000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f1794a1b700 1 -- 172.21.15.2:0/4185025996 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178001d070 con 0x7f1790108010 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179006e7e0 con 0x7f1790108010 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.988+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179006ed20 con 0x7f1790108010 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.989+0000 7f1794a1b700 1 -- 172.21.15.2:0/4185025996 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f178000f460 con 0x7f1790108010 2024-03-16T10:49:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.989+0000 7f1794a1b700 1 -- 172.21.15.2:0/4185025996 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1780017610 con 0x7f1790108010 2024-03-16T10:49:19.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.989+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177c0052f0 con 0x7f1790108010 2024-03-16T10:49:19.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.990+0000 7f1794a1b700 1 -- 172.21.15.2:0/4185025996 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f1780017770 con 0x7f1790108010 2024-03-16T10:49:19.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.990+0000 7f1794a1b700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f177806dfa0 0x7f1778070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:19.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.991+0000 7f1794a1b700 1 -- 172.21.15.2:0/4185025996 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f178008ef00 con 0x7f1790108010 2024-03-16T10:49:19.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.991+0000 7f178e7fc700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f177806dfa0 0x7f1778070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:19.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.991+0000 7f178e7fc700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f177806dfa0 0x7f1778070460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1784005950 tx=0x7f178400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:19.996+0000 7f1794a1b700 1 -- 172.21.15.2:0/4185025996 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f178005bf50 con 0x7f1790108010 2024-03-16T10:49:20.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.253+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f177c000bc0 con 0x7f177806dfa0 2024-03-16T10:49:20.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.254+0000 7f1794a1b700 1 -- 172.21.15.2:0/4185025996 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f177c000bc0 con 0x7f177806dfa0 2024-03-16T10:49:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.257+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f177806dfa0 msgr2=0x7f1778070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.257+0000 7f1795a1d700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f177806dfa0 0x7f1778070460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1784005950 tx=0x7f178400b410 comp rx=0 tx=0).stop 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 msgr2=0x7f179006fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f179006fbb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1780006010 tx=0x7f178000bd70 comp rx=0 tx=0).stop 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 shutdown_connections 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f177806dfa0 0x7f1778070460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1790108010 0x7f179006fbb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 --2- 172.21.15.2:0/4185025996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010ae70 0x7f179006e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 >> 172.21.15.2:0/4185025996 conn(0x7f179006dd00 msgr2=0x7f1790195d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 shutdown_connections 2024-03-16T10:49:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.258+0000 7f1795a1d700 1 -- 172.21.15.2:0/4185025996 wait complete. 2024-03-16T10:49:20.270 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 -- 172.21.15.2:0/900360568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d40ff9d0 msgr2=0x7f94d40ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/900360568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d40ff9d0 0x7f94d40ffdf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009910 tx=0x7f94c8009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 -- 172.21.15.2:0/900360568 shutdown_connections 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/900360568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d4100bd0 0x7f94d4101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/900360568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d40ff9d0 0x7f94d40ffdf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 -- 172.21.15.2:0/900360568 >> 172.21.15.2:0/900360568 conn(0x7f94d40faf70 msgr2=0x7f94d40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 -- 172.21.15.2:0/900360568 shutdown_connections 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.376+0000 7f94d8d5a700 1 -- 172.21.15.2:0/900360568 wait complete. 2024-03-16T10:49:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d8d5a700 1 Processor -- start 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d8d5a700 1 -- start start 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d8d5a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d40ff9d0 0x7f94d41964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d8d5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4100bd0 0x7f94d41969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d8d5a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4197010 con 0x7f94d4100bd0 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d8d5a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d4197150 con 0x7f94d40ff9d0 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4100bd0 0x7f94d41969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4100bd0 0x7f94d41969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46718/0 (socket says 172.21.15.2:46718) 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d1d9b700 1 -- 172.21.15.2:0/1067017226 learned_addr learned my addr 172.21.15.2:0/1067017226 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d259c700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d40ff9d0 0x7f94d41964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d1d9b700 1 -- 172.21.15.2:0/1067017226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d40ff9d0 msgr2=0x7f94d41964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d1d9b700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d40ff9d0 0x7f94d41964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.377+0000 7f94d1d9b700 1 -- 172.21.15.2:0/1067017226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c80095f0 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.378+0000 7f94d1d9b700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4100bd0 0x7f94d41969f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c8a0 tx=0x7f94c400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.378+0000 7f94c37fe700 1 -- 172.21.15.2:0/1067017226 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c40078e0 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.378+0000 7f94c37fe700 1 -- 172.21.15.2:0/1067017226 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94c400f450 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.378+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d419bbb0 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.378+0000 7f94c37fe700 1 -- 172.21.15.2:0/1067017226 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c400e5e0 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.378+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d419c180 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.379+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4190650 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.379+0000 7f94c37fe700 1 -- 172.21.15.2:0/1067017226 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f94c400f5c0 con 0x7f94d4100bd0 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.380+0000 7f94c37fe700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94bc06dfa0 0x7f94bc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:20.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.380+0000 7f94c37fe700 1 -- 172.21.15.2:0/1067017226 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f94c408dc20 con 0x7f94d4100bd0 2024-03-16T10:49:20.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.380+0000 7f94d259c700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94bc06dfa0 0x7f94bc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:20.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.380+0000 7f94d259c700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94bc06dfa0 0x7f94bc070460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94c800b5c0 tx=0x7f94c8005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.384+0000 7f94c37fe700 1 -- 172.21.15.2:0/1067017226 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94c405ac40 con 0x7f94d4100bd0 2024-03-16T10:49:20.541 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:20 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[113733]: 2024-03-16T10:49:20.302+0000 7f14ac3b67c0 -1 osd.2 61 log_to_monitors true 2024-03-16T10:49:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.614+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d40611f0 con 0x7f94bc06dfa0 2024-03-16T10:49:20.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.616+0000 7f94c37fe700 1 -- 172.21.15.2:0/1067017226 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f94d40611f0 con 0x7f94bc06dfa0 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94bc06dfa0 msgr2=0x7f94bc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94bc06dfa0 0x7f94bc070460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94c800b5c0 tx=0x7f94c8005960 comp rx=0 tx=0).stop 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4100bd0 msgr2=0x7f94d41969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4100bd0 0x7f94d41969f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c8a0 tx=0x7f94c400cbb0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 shutdown_connections 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f94bc06dfa0 0x7f94bc070460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d40ff9d0 0x7f94d41964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 --2- 172.21.15.2:0/1067017226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94d4100bd0 0x7f94d41969f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 >> 172.21.15.2:0/1067017226 conn(0x7f94d40faf70 msgr2=0x7f94d4104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 shutdown_connections 2024-03-16T10:49:20.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.618+0000 7f94d8d5a700 1 -- 172.21.15.2:0/1067017226 wait complete. 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/1802360483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 msgr2=0x7f2e640feff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/1802360483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e640feff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009910 tx=0x7f2e58009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/1802360483 shutdown_connections 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/1802360483 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e640feff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/1802360483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e641020f0 0x7f2e64102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/1802360483 >> 172.21.15.2:0/1802360483 conn(0x7f2e640fa730 msgr2=0x7f2e640fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/1802360483 shutdown_connections 2024-03-16T10:49:20.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.723+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/1802360483 wait complete. 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e6bd0d700 1 Processor -- start 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e6bd0d700 1 -- start start 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e6bd0d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e64101750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e6bd0d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e641020f0 0x7f2e640ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e6bd0d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64101ce0 con 0x7f2e640feb70 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e6bd0d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64100380 con 0x7f2e641020f0 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e69aa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e64101750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e69aa9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e64101750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46740/0 (socket says 172.21.15.2:46740) 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e69aa9700 1 -- 172.21.15.2:0/337481707 learned_addr learned my addr 172.21.15.2:0/337481707 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e692a8700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e641020f0 0x7f2e640ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e69aa9700 1 -- 172.21.15.2:0/337481707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e641020f0 msgr2=0x7f2e640ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e69aa9700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e641020f0 0x7f2e640ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.724+0000 7f2e69aa9700 1 -- 172.21.15.2:0/337481707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580095f0 con 0x7f2e640feb70 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.725+0000 7f2e69aa9700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e64101750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400cfd0 tx=0x7f2e5400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.725+0000 7f2e62ffd700 1 -- 172.21.15.2:0/337481707 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e5400ad70 con 0x7f2e640feb70 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.725+0000 7f2e62ffd700 1 -- 172.21.15.2:0/337481707 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e54004020 con 0x7f2e640feb70 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.725+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e64100660 con 0x7f2e640feb70 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.725+0000 7f2e62ffd700 1 -- 172.21.15.2:0/337481707 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e54012650 con 0x7f2e640feb70 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.725+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e64100c30 con 0x7f2e640feb70 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.726+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e64190690 con 0x7f2e640feb70 2024-03-16T10:49:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.726+0000 7f2e62ffd700 1 -- 172.21.15.2:0/337481707 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2e54004190 con 0x7f2e640feb70 2024-03-16T10:49:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.727+0000 7f2e62ffd700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2e4c06dfa0 0x7f2e4c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.727+0000 7f2e62ffd700 1 -- 172.21.15.2:0/337481707 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2e5408dfd0 con 0x7f2e640feb70 2024-03-16T10:49:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.727+0000 7f2e692a8700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2e4c06dfa0 0x7f2e4c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.727+0000 7f2e692a8700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2e4c06dfa0 0x7f2e4c070460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e58006010 tx=0x7f2e5800b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:20.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:20.732+0000 7f2e62ffd700 1 -- 172.21.15.2:0/337481707 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e5405af70 con 0x7f2e640feb70 2024-03-16T10:49:21.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.005+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e640611f0 con 0x7f2e4c06dfa0 2024-03-16T10:49:21.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.026+0000 7f2e62ffd700 1 -- 172.21.15.2:0/337481707 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2e640611f0 con 0x7f2e4c06dfa0 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (11m) 35s ago 12m 24.8M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (13m) 35s ago 13m 8024k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (12m) 92s ago 12m 7922k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (102s) 35s ago 13m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (96s) 92s ago 12m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (11m) 35s ago 12m 86.4M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (7m) 35s ago 7m 18.7M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (7m) 35s ago 7m 15.7M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (7m) 92s ago 7m 153M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:49:21.027 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (7m) 92s ago 7m 16.7M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (4m) 35s ago 14m 502M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (3m) 92s ago 11m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (3m) 35s ago 14m 45.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (2m) 92s ago 11m 33.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (13m) 35s ago 13m 16.1M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (12m) 92s ago 12m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (76s) 35s ago 11m 81.1M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (41s) 35s ago 10m 15.0M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 starting - - - 1217M 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (9m) 92s ago 9m 248M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 eb4213762356 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (9m) 92s ago 9m 208M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (8m) 92s ago 8m 198M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:49:21.028 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (3m) 35s ago 12m 68.1M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:49:21.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2e4c06dfa0 msgr2=0x7f2e4c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2e4c06dfa0 0x7f2e4c070460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e58006010 tx=0x7f2e5800b580 comp rx=0 tx=0).stop 2024-03-16T10:49:21.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 msgr2=0x7f2e64101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e64101750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400cfd0 tx=0x7f2e5400a340 comp rx=0 tx=0).stop 2024-03-16T10:49:21.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 shutdown_connections 2024-03-16T10:49:21.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2e4c06dfa0 0x7f2e4c070460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e640feb70 0x7f2e64101750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 --2- 172.21.15.2:0/337481707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e641020f0 0x7f2e640ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 >> 172.21.15.2:0/337481707 conn(0x7f2e640fa730 msgr2=0x7f2e64105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:21.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.028+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 shutdown_connections 2024-03-16T10:49:21.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.029+0000 7f2e6bd0d700 1 -- 172.21.15.2:0/337481707 wait complete. 2024-03-16T10:49:21.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:20 smithi002 ceph-mon[94239]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:21.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:21.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:21.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:20 smithi002 ceph-mon[94239]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.180+0000 7ff9d4d37700 1 -- 172.21.15.2:0/2890098579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 msgr2=0x7ff9d00ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.180+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/2890098579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d00ffdf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009910 tx=0x7ff9c0009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.180+0000 7ff9d4d37700 1 -- 172.21.15.2:0/2890098579 shutdown_connections 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.180+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/2890098579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d0100bd0 0x7ff9d0101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.181+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/2890098579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d00ffdf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.181+0000 7ff9d4d37700 1 -- 172.21.15.2:0/2890098579 >> 172.21.15.2:0/2890098579 conn(0x7ff9d00faf70 msgr2=0x7ff9d00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.181+0000 7ff9d4d37700 1 -- 172.21.15.2:0/2890098579 shutdown_connections 2024-03-16T10:49:21.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.181+0000 7ff9d4d37700 1 -- 172.21.15.2:0/2890098579 wait complete. 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9d4d37700 1 Processor -- start 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9d4d37700 1 -- start start 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9d4d37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d0103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9d4d37700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d0100bd0 0x7ff9d0102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9ce59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d0103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9ce59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d0103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46762/0 (socket says 172.21.15.2:46762) 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9d4d37700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d0102870 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9d4d37700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01029b0 con 0x7ff9d0100bd0 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.182+0000 7ff9ce59c700 1 -- 172.21.15.2:0/1393811698 learned_addr learned my addr 172.21.15.2:0/1393811698 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.183+0000 7ff9cdd9b700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d0100bd0 0x7ff9d0102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.183+0000 7ff9ce59c700 1 -- 172.21.15.2:0/1393811698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d0100bd0 msgr2=0x7ff9d0102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.183+0000 7ff9ce59c700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d0100bd0 0x7ff9d0102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.183+0000 7ff9ce59c700 1 -- 172.21.15.2:0/1393811698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c00095f0 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.183+0000 7ff9ce59c700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d0103c90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000b800 tx=0x7ff9c0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:21.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.184+0000 7ff9bf7fe700 1 -- 172.21.15.2:0/1393811698 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c001d070 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.184+0000 7ff9bf7fe700 1 -- 172.21.15.2:0/1393811698 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9c00052a0 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.184+0000 7ff9bf7fe700 1 -- 172.21.15.2:0/1393811698 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c0022470 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.184+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d0102c30 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.185+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d01031a0 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.186+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d004ea80 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.187+0000 7ff9bf7fe700 1 -- 172.21.15.2:0/1393811698 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ff9c0003680 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.187+0000 7ff9bf7fe700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff9b806e000 0x7ff9b80704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:21.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.187+0000 7ff9bf7fe700 1 -- 172.21.15.2:0/1393811698 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff9c008e940 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.188+0000 7ff9cdd9b700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff9b806e000 0x7ff9b80704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:21.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.188+0000 7ff9cdd9b700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff9b806e000 0x7ff9b80704c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009c40 tx=0x7ff9c4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:21.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.195+0000 7ff9bf7fe700 1 -- 172.21.15.2:0/1393811698 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9c005ba00 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:20 smithi026 ceph-mon[82600]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:20 smithi026 ceph-mon[82600]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T10:49:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.610+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9d01031a0 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.611+0000 7ff9bf7fe700 1 -- 172.21.15.2:0/1393811698 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7ff9c005b590 con 0x7ff9d00ff9d0 2024-03-16T10:49:21.612 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:49:21.612 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:49:21.613 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:49:21.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:21.613 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:49:21.613 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:49:21.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:21.613 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:49:21.614 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 3, 2024-03-16T10:49:21.614 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:49:21.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:21.614 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:49:21.614 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:49:21.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:21.615 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:49:21.615 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 7, 2024-03-16T10:49:21.615 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:49:21.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:49:21.615 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:49:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff9b806e000 msgr2=0x7ff9b80704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff9b806e000 0x7ff9b80704c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009c40 tx=0x7ff9c4009260 comp rx=0 tx=0).stop 2024-03-16T10:49:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 msgr2=0x7ff9d0103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d0103c90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000b800 tx=0x7ff9c0005e30 comp rx=0 tx=0).stop 2024-03-16T10:49:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 shutdown_connections 2024-03-16T10:49:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff9b806e000 0x7ff9b80704c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9d00ff9d0 0x7ff9d0103c90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 --2- 172.21.15.2:0/1393811698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9d0100bd0 0x7ff9d0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 >> 172.21.15.2:0/1393811698 conn(0x7ff9d00faf70 msgr2=0x7ff9d0068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 shutdown_connections 2024-03-16T10:49:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.613+0000 7ff9d4d37700 1 -- 172.21.15.2:0/1393811698 wait complete. 2024-03-16T10:49:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 -- 172.21.15.2:0/875683632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 msgr2=0x7f3cf40ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/875683632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf40ff770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009910 tx=0x7f3ce4009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 -- 172.21.15.2:0/875683632 shutdown_connections 2024-03-16T10:49:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/875683632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf40ffcb0 0x7f3cf4100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/875683632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf40ff770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 -- 172.21.15.2:0/875683632 >> 172.21.15.2:0/875683632 conn(0x7f3cf40faf70 msgr2=0x7f3cf40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 -- 172.21.15.2:0/875683632 shutdown_connections 2024-03-16T10:49:21.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.727+0000 7f3cfa67b700 1 -- 172.21.15.2:0/875683632 wait complete. 2024-03-16T10:49:21.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.728+0000 7f3cfa67b700 1 Processor -- start 2024-03-16T10:49:21.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.728+0000 7f3cfa67b700 1 -- start start 2024-03-16T10:49:21.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.728+0000 7f3cfa67b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:21.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.728+0000 7f3cfa67b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf40ffcb0 0x7f3cf4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:21.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.728+0000 7f3cfa67b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4199300 con 0x7f3cf40ff350 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.728+0000 7f3cfa67b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf4199440 con 0x7f3cf40ffcb0 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf41987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46780/0 (socket says 172.21.15.2:46780) 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf3fff700 1 -- 172.21.15.2:0/2405719210 learned_addr learned my addr 172.21.15.2:0/2405719210 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf37fe700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf40ffcb0 0x7f3cf4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf3fff700 1 -- 172.21.15.2:0/2405719210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf40ffcb0 msgr2=0x7f3cf4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf3fff700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf40ffcb0 0x7f3cf4198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf3fff700 1 -- 172.21.15.2:0/2405719210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40095f0 con 0x7f3cf40ff350 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf3fff700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf41987a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4006010 tx=0x7f3ce400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf17fa700 1 -- 172.21.15.2:0/2405719210 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce401d070 con 0x7f3cf40ff350 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf419de40 con 0x7f3cf40ff350 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf17fa700 1 -- 172.21.15.2:0/2405719210 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ce400f460 con 0x7f3cf40ff350 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cf17fa700 1 -- 172.21.15.2:0/2405719210 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce4021620 con 0x7f3cf40ff350 2024-03-16T10:49:21.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.729+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf419e330 con 0x7f3cf40ff350 2024-03-16T10:49:21.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.731+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf41928c0 con 0x7f3cf40ff350 2024-03-16T10:49:21.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.731+0000 7f3cf17fa700 1 -- 172.21.15.2:0/2405719210 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3ce4003a60 con 0x7f3cf40ff350 2024-03-16T10:49:21.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.731+0000 7f3cf17fa700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3cdc06dfa0 0x7f3cdc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:21.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.731+0000 7f3cf17fa700 1 -- 172.21.15.2:0/2405719210 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3ce408f0b0 con 0x7f3cf40ff350 2024-03-16T10:49:21.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.731+0000 7f3cf37fe700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3cdc06dfa0 0x7f3cdc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:21.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.732+0000 7f3cf37fe700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3cdc06dfa0 0x7f3cdc070460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009a10 tx=0x7f3ce8009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:21.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.736+0000 7f3cf17fa700 1 -- 172.21.15.2:0/2405719210 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ce405c050 con 0x7f3cf40ff350 2024-03-16T10:49:21.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.998+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3cf404ea80 con 0x7f3cf40ff350 2024-03-16T10:49:21.999 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[113733]: 2024-03-16T10:49:21.630+0000 7f1499806700 -1 osd.2 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: osdmap e64: 6 total, 5 up, 6 in 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1393811698' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:22.000 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:21 smithi002 ceph-mon[94239]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:21.999+0000 7f3cf17fa700 1 -- 172.21.15.2:0/2405719210 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f3ce405bbe0 con 0x7f3cf40ff350 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:49:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:49:22.002 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:22.003 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:22.004 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:49:22.004 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:22.004 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:22.004 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:22.004 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3cdc06dfa0 msgr2=0x7f3cdc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3cdc06dfa0 0x7f3cdc070460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009a10 tx=0x7f3ce8009190 comp rx=0 tx=0).stop 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 msgr2=0x7f3cf41987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf41987a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4006010 tx=0x7f3ce400bd70 comp rx=0 tx=0).stop 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 shutdown_connections 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3cdc06dfa0 0x7f3cdc070460 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf40ff350 0x7f3cf41987a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 --2- 172.21.15.2:0/2405719210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cf40ffcb0 0x7f3cf4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 >> 172.21.15.2:0/2405719210 conn(0x7f3cf40faf70 msgr2=0x7f3cf4107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 shutdown_connections 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.002+0000 7f3cfa67b700 1 -- 172.21.15.2:0/2405719210 wait complete. 2024-03-16T10:49:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:49:22.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 -- 172.21.15.2:0/946211132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 msgr2=0x7f3540102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 --2- 172.21.15.2:0/946211132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540102140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3534009910 tx=0x7f3534009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:22.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 -- 172.21.15.2:0/946211132 shutdown_connections 2024-03-16T10:49:22.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 --2- 172.21.15.2:0/946211132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540102140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 --2- 172.21.15.2:0/946211132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3540100ae0 0x7f3540100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 -- 172.21.15.2:0/946211132 >> 172.21.15.2:0/946211132 conn(0x7f35400fc0a0 msgr2=0x7f35400fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:22.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 -- 172.21.15.2:0/946211132 shutdown_connections 2024-03-16T10:49:22.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.116+0000 7f354582d700 1 -- 172.21.15.2:0/946211132 wait complete. 2024-03-16T10:49:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.117+0000 7f354582d700 1 Processor -- start 2024-03-16T10:49:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.117+0000 7f354582d700 1 -- start start 2024-03-16T10:49:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.117+0000 7f354582d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3540100ae0 0x7f35401964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.117+0000 7f354582d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.117+0000 7f354582d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3540197020 con 0x7f3540101ce0 2024-03-16T10:49:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.117+0000 7f354582d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3540197160 con 0x7f3540100ae0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46810/0 (socket says 172.21.15.2:46810) 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353e7fc700 1 -- 172.21.15.2:0/3892900015 learned_addr learned my addr 172.21.15.2:0/3892900015 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353effd700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3540100ae0 0x7f35401964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353e7fc700 1 -- 172.21.15.2:0/3892900015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3540100ae0 msgr2=0x7f35401964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353e7fc700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3540100ae0 0x7f35401964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353e7fc700 1 -- 172.21.15.2:0/3892900015 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340095f0 con 0x7f3540101ce0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f353e7fc700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540196a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f353400bd70 tx=0x7f353400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f354482b700 1 -- 172.21.15.2:0/3892900015 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353401d070 con 0x7f3540101ce0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f354482b700 1 -- 172.21.15.2:0/3892900015 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f353400f460 con 0x7f3540101ce0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f354482b700 1 -- 172.21.15.2:0/3892900015 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3534021640 con 0x7f3540101ce0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.118+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354019bb60 con 0x7f3540101ce0 2024-03-16T10:49:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.119+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354019c030 con 0x7f3540101ce0 2024-03-16T10:49:22.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.120+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3540190660 con 0x7f3540101ce0 2024-03-16T10:49:22.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.121+0000 7f354482b700 1 -- 172.21.15.2:0/3892900015 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f353400fad0 con 0x7f3540101ce0 2024-03-16T10:49:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.121+0000 7f354482b700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f352806dfb0 0x7f3528070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.121+0000 7f354482b700 1 -- 172.21.15.2:0/3892900015 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f353408efe0 con 0x7f3540101ce0 2024-03-16T10:49:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.121+0000 7f353effd700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f352806dfb0 0x7f3528070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.122+0000 7f353effd700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f352806dfb0 0x7f3528070470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3540101b40 tx=0x7f353000a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:22.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.126+0000 7f354482b700 1 -- 172.21.15.2:0/3892900015 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f353405c000 con 0x7f3540101ce0 2024-03-16T10:49:22.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:22.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:22.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T10:49:22.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: osdmap e64: 6 total, 5 up, 6 in 2024-03-16T10:49:22.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-03-16T10:49:22.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:22.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:22.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:22.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1393811698' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:22.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:21 smithi026 ceph-mon[82600]: from='osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549]' entity='osd.2' 2024-03-16T10:49:22.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.353+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35400611f0 con 0x7f352806dfb0 2024-03-16T10:49:22.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.355+0000 7f354482b700 1 -- 172.21.15.2:0/3892900015 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f35400611f0 con 0x7f352806dfb0 2024-03-16T10:49:22.356 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:49:22.356 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:49:22.356 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:49:22.356 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:49:22.356 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:49:22.357 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:49:22.357 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:49:22.357 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:49:22.357 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:49:22.357 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "9/23 daemons upgraded", 2024-03-16T10:49:22.357 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T10:49:22.357 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:49:22.358 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:49:22.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f352806dfb0 msgr2=0x7f3528070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f352806dfb0 0x7f3528070470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3540101b40 tx=0x7f353000a380 comp rx=0 tx=0).stop 2024-03-16T10:49:22.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 msgr2=0x7f3540196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540196a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f353400bd70 tx=0x7f353400bda0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 shutdown_connections 2024-03-16T10:49:22.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f352806dfb0 0x7f3528070470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3540100ae0 0x7f35401964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 --2- 172.21.15.2:0/3892900015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3540101ce0 0x7f3540196a00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 >> 172.21.15.2:0/3892900015 conn(0x7f35400fc0a0 msgr2=0x7f3540105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:22.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 shutdown_connections 2024-03-16T10:49:22.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.357+0000 7f354582d700 1 -- 172.21.15.2:0/3892900015 wait complete. 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 -- 172.21.15.2:0/1413080554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 msgr2=0x7f4054100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 --2- 172.21.15.2:0/1413080554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054100ef0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4050009910 tx=0x7f4050009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 -- 172.21.15.2:0/1413080554 shutdown_connections 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 --2- 172.21.15.2:0/1413080554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4054101cd0 0x7f4054102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 --2- 172.21.15.2:0/1413080554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054100ef0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 -- 172.21.15.2:0/1413080554 >> 172.21.15.2:0/1413080554 conn(0x7f40540fc090 msgr2=0x7f40540fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 -- 172.21.15.2:0/1413080554 shutdown_connections 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.467+0000 7f405c79f700 1 -- 172.21.15.2:0/1413080554 wait complete. 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405c79f700 1 Processor -- start 2024-03-16T10:49:22.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405c79f700 1 -- start start 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405c79f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405c79f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4054101cd0 0x7f4054196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405c79f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054197190 con 0x7f4054100ad0 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405c79f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40541972d0 con 0x7f4054101cd0 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405a53b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405a53b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55948/0 (socket says 172.21.15.2:55948) 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405a53b700 1 -- 172.21.15.2:0/3729433549 learned_addr learned my addr 172.21.15.2:0/3729433549 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f4059d3a700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4054101cd0 0x7f4054196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405a53b700 1 -- 172.21.15.2:0/3729433549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4054101cd0 msgr2=0x7f4054196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405a53b700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4054101cd0 0x7f4054196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.468+0000 7f405a53b700 1 -- 172.21.15.2:0/3729433549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500095f0 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.469+0000 7f405a53b700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054196630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40500059d0 tx=0x7f405000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.469+0000 7f40477fe700 1 -- 172.21.15.2:0/3729433549 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405001d070 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.469+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405419bc80 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.469+0000 7f40477fe700 1 -- 172.21.15.2:0/3729433549 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4050005dc0 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.469+0000 7f40477fe700 1 -- 172.21.15.2:0/3729433549 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4050021410 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.469+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405419c170 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.470+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40541906d0 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.470+0000 7f40477fe700 1 -- 172.21.15.2:0/3729433549 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f405000f820 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.471+0000 7f40477fe700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f404006dfa0 0x7f4040070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.471+0000 7f40477fe700 1 -- 172.21.15.2:0/3729433549 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f405008efd0 con 0x7f4054100ad0 2024-03-16T10:49:22.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.471+0000 7f4059d3a700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f404006dfa0 0x7f4040070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:22.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.471+0000 7f4059d3a700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f404006dfa0 0x7f4040070460 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4048009a10 tx=0x7f4048009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:22.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.476+0000 7f40477fe700 1 -- 172.21.15.2:0/3729433549 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f405005bf70 con 0x7f4054100ad0 2024-03-16T10:49:22.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.782+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f405404ea80 con 0x7f4054100ad0 2024-03-16T10:49:22.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.782+0000 7f40477fe700 1 -- 172.21.15.2:0/3729433549 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1041 (secure 0 0 0) 0x7f405005bb00 con 0x7f4054100ad0 2024-03-16T10:49:22.786 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded 2024-03-16T10:49:22.786 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:49:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-03-16T10:49:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f404006dfa0 msgr2=0x7f4040070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f404006dfa0 0x7f4040070460 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4048009a10 tx=0x7f4048009190 comp rx=0 tx=0).stop 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 msgr2=0x7f4054196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054196630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40500059d0 tx=0x7f405000bd70 comp rx=0 tx=0).stop 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 shutdown_connections 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f404006dfa0 0x7f4040070460 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4054100ad0 0x7f4054196630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 --2- 172.21.15.2:0/3729433549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4054101cd0 0x7f4054196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 >> 172.21.15.2:0/3729433549 conn(0x7f40540fc090 msgr2=0x7f4054105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 shutdown_connections 2024-03-16T10:49:22.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:22.785+0000 7f405c79f700 1 -- 172.21.15.2:0/3729433549 wait complete. 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2405719210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549] boot 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: osdmap e65: 6 total, 6 up, 6 in 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:49:23.036 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:22 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3729433549' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:49:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2405719210' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: osd.2 [v2:172.21.15.2:6818/1082105549,v1:172.21.15.2:6819/1082105549] boot 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: osdmap e65: 6 total, 6 up, 6 in 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T10:49:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:22 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3729433549' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:49:24.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:23 smithi002 ceph-mon[94239]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:24.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:23 smithi002 ceph-mon[94239]: osdmap e66: 6 total, 6 up, 6 in 2024-03-16T10:49:24.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:23 smithi026 ceph-mon[82600]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:24.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:23 smithi026 ceph-mon[82600]: osdmap e66: 6 total, 6 up, 6 in 2024-03-16T10:49:25.230 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:24 smithi002 ceph-mon[94239]: pgmap v110: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:25.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:24 smithi026 ceph-mon[82600]: pgmap v110: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-03-16T10:49:26.120 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:25 smithi002 ceph-mon[94239]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded (PG_DEGRADED) 2024-03-16T10:49:26.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:25 smithi026 ceph-mon[82600]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded (PG_DEGRADED) 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: pgmap v111: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/267 objects degraded (7.865%) 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:27.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:27.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:27.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:27.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: pgmap v111: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21/267 objects degraded (7.865%) 2024-03-16T10:49:27.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:27.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:27.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:28.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:27 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:28.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:27 smithi026 ceph-mon[82600]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-16T10:49:28.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:27 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:28.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:27 smithi002 ceph-mon[94239]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-16T10:49:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:28 smithi026 ceph-mon[82600]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:28 smithi026 ceph-mon[82600]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/267 objects degraded (7.865%), 6 pgs degraded) 2024-03-16T10:49:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:28 smithi002 ceph-mon[94239]: pgmap v112: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:28 smithi002 ceph-mon[94239]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/267 objects degraded (7.865%), 6 pgs degraded) 2024-03-16T10:49:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:30 smithi026 ceph-mon[82600]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:30 smithi002 ceph-mon[94239]: pgmap v113: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:32 smithi026 ceph-mon[82600]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:32 smithi002 ceph-mon[94239]: pgmap v114: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:34 smithi026 ceph-mon[82600]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:34 smithi002 ceph-mon[94239]: pgmap v115: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:35.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:35.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:35.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:36 smithi002 ceph-mon[94239]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:36 smithi026 ceph-mon[82600]: pgmap v116: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:38 smithi002 ceph-mon[94239]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:39.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:38 smithi026 ceph-mon[82600]: pgmap v117: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:40 smithi002 ceph-mon[94239]: pgmap v118: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:41.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:40 smithi026 ceph-mon[82600]: pgmap v118: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:42.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:41 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:42.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:41 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:42 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:43.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:42 smithi026 ceph-mon[82600]: Upgrade: osd.3 is safe to restart 2024-03-16T10:49:43.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:42 smithi026 ceph-mon[82600]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:43.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:43.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:49:43.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:49:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:42 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-16T10:49:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:42 smithi002 ceph-mon[94239]: Upgrade: osd.3 is safe to restart 2024-03-16T10:49:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:42 smithi002 ceph-mon[94239]: pgmap v119: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T10:49:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:49:44.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:43 smithi026 ceph-mon[82600]: Upgrade: Updating osd.3 2024-03-16T10:49:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:43 smithi026 ceph-mon[82600]: Deploying daemon osd.3 on smithi026 2024-03-16T10:49:44.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:43 smithi002 ceph-mon[94239]: Upgrade: Updating osd.3 2024-03-16T10:49:44.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:43 smithi002 ceph-mon[94239]: Deploying daemon osd.3 on smithi026 2024-03-16T10:49:45.176 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:45 smithi026 ceph-mon[82600]: pgmap v120: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:45 smithi002 ceph-mon[94239]: pgmap v120: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:46.492 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:46 smithi026 systemd[1]: Stopping Ceph osd.3 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:49:46.981 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[43138]: 2024-03-16T10:49:46.490+0000 7f353617a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:49:46.981 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[43138]: 2024-03-16T10:49:46.490+0000 7f353617a700 -1 osd.3 66 *** Got signal Terminated *** 2024-03-16T10:49:46.981 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[43138]: 2024-03-16T10:49:46.490+0000 7f353617a700 -1 osd.3 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:49:47.270 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:47 smithi026 ceph-mon[82600]: pgmap v121: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:47.270 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:47 smithi026 ceph-mon[82600]: osd.3 marked itself down and dead 2024-03-16T10:49:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:47 smithi002 ceph-mon[94239]: pgmap v121: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:47.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:47 smithi002 ceph-mon[94239]: osd.3 marked itself down and dead 2024-03-16T10:49:47.731 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:47 smithi026 podman[91719]: 2024-03-16 10:49:47.493486532 +0000 UTC m=+1.107927198 container died eb4213762356510c6cd399fc5448ebb3e893c6465d1d03faa64320f220b15d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, RELEASE=reef-a752c2f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-16T10:49:48.018 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:47 smithi026 podman[91719]: 2024-03-16 10:49:47.849305802 +0000 UTC m=+1.463746520 container cleanup eb4213762356510c6cd399fc5448ebb3e893c6465d1d03faa64320f220b15d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=reef-a752c2f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:49:48.019 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:47 smithi026 bash[91719]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3 2024-03-16T10:49:48.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:48 smithi002 ceph-mon[94239]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:49:48.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:48 smithi002 ceph-mon[94239]: osdmap e67: 6 total, 5 up, 6 in 2024-03-16T10:49:48.468 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:48 smithi026 ceph-mon[82600]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:49:48.468 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:48 smithi026 ceph-mon[82600]: osdmap e67: 6 total, 5 up, 6 in 2024-03-16T10:49:48.468 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:48 smithi026 podman[91737]: 2024-03-16 10:49:48.125699599 +0000 UTC m=+0.626804844 container remove eb4213762356510c6cd399fc5448ebb3e893c6465d1d03faa64320f220b15d3d (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=reef-a752c2f, io.buildah.version=1.33.5) 2024-03-16T10:49:48.729 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:48 smithi026 podman[91826]: 2024-03-16 10:49:48.370973905 +0000 UTC m=+0.018213435 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:49:48.729 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:48 smithi026 podman[91826]: 2024-03-16 10:49:48.493317935 +0000 UTC m=+0.140557499 container create 40baf5cae77bad36e6ad5c0f85bf28972e3d53ec688c8003ebcc1dc6acf52ebd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-16T10:49:48.729 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:48 smithi026 podman[91826]: 2024-03-16 10:49:48.729255541 +0000 UTC m=+0.376495061 container init 40baf5cae77bad36e6ad5c0f85bf28972e3d53ec688c8003ebcc1dc6acf52ebd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:49:48.981 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:48 smithi026 podman[91826]: 2024-03-16 10:49:48.732473021 +0000 UTC m=+0.379712550 container start 40baf5cae77bad36e6ad5c0f85bf28972e3d53ec688c8003ebcc1dc6acf52ebd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:49:48.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:48 smithi026 podman[91826]: 2024-03-16 10:49:48.795773007 +0000 UTC m=+0.443012532 container attach 40baf5cae77bad36e6ad5c0f85bf28972e3d53ec688c8003ebcc1dc6acf52ebd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-03-16T10:49:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:49 smithi002 ceph-mon[94239]: pgmap v123: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:49 smithi002 ceph-mon[94239]: osdmap e68: 6 total, 5 up, 6 in 2024-03-16T10:49:49.374 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:49 smithi026 ceph-mon[82600]: pgmap v123: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:49.374 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:49 smithi026 ceph-mon[82600]: osdmap e68: 6 total, 5 up, 6 in 2024-03-16T10:49:49.374 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:49 smithi026 podman[91826]: 2024-03-16 10:49:49.026464905 +0000 UTC m=+0.673704427 container died 40baf5cae77bad36e6ad5c0f85bf28972e3d53ec688c8003ebcc1dc6acf52ebd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-16T10:49:49.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:49 smithi026 podman[91852]: 2024-03-16 10:49:49.632660107 +0000 UTC m=+0.601048172 container remove 40baf5cae77bad36e6ad5c0f85bf28972e3d53ec688c8003ebcc1dc6acf52ebd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-16T10:49:49.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:49 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-16T10:49:49.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:49 smithi026 systemd[1]: Stopped Ceph osd.3 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:49:49.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:49 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.3.service: Consumed 1min 2.832s CPU time. 2024-03-16T10:49:49.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:49 smithi026 systemd[1]: Starting Ceph osd.3 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:49:50.378 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 podman[91959]: 2024-03-16 10:49:50.075523971 +0000 UTC m=+0.111599981 container create 8072629a663f618ca7bc6980382b9d1fafc9785a652a8971831f9f06296c9eb8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-16T10:49:50.378 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 podman[91959]: 2024-03-16 10:49:49.982662579 +0000 UTC m=+0.018738589 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:49:50.378 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 podman[91959]: 2024-03-16 10:49:50.294868323 +0000 UTC m=+0.330944349 container init 8072629a663f618ca7bc6980382b9d1fafc9785a652a8971831f9f06296c9eb8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T10:49:50.378 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 podman[91959]: 2024-03-16 10:49:50.298012981 +0000 UTC m=+0.334088994 container start 8072629a663f618ca7bc6980382b9d1fafc9785a652a8971831f9f06296c9eb8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:49:50.378 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 podman[91959]: 2024-03-16 10:49:50.377870083 +0000 UTC m=+0.413946109 container attach 8072629a663f618ca7bc6980382b9d1fafc9785a652a8971831f9f06296c9eb8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:49:51.078 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-mon[82600]: pgmap v125: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:51.078 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:51.079 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:51.079 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate[91974]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T10:49:51.079 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate[91974]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T10:49:51.079 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 bash[91959]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T10:49:51.079 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 bash[91959]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-16T10:49:51.079 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 bash[91959]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T10:49:51.079 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 bash[91959]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T10:49:51.079 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 bash[91959]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 bash[91959]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 bash[91959]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate[91974]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate[91974]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate[91974]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate[91974]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate[91974]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-16T10:49:51.080 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:50 smithi026 podman[91959]: 2024-03-16 10:49:50.772867347 +0000 UTC m=+0.808943358 container died 8072629a663f618ca7bc6980382b9d1fafc9785a652a8971831f9f06296c9eb8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:49:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:50 smithi002 ceph-mon[94239]: pgmap v125: 65 pgs: 16 stale+active+clean, 49 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:49:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:49:51.616 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 podman[92100]: 2024-03-16 10:49:51.365247507 +0000 UTC m=+0.587175899 container remove 8072629a663f618ca7bc6980382b9d1fafc9785a652a8971831f9f06296c9eb8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:49:51.616 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 podman[92165]: 2024-03-16 10:49:51.48634871 +0000 UTC m=+0.019140105 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:49:51.616 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 podman[92165]: 2024-03-16 10:49:51.616073139 +0000 UTC m=+0.148864535 container create 7a6319cdca6d5a603694f841bebf83d8d1748cf80013fd8ca16c248446581a7a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:49:51.935 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:51 smithi026 ceph-mon[82600]: Health check failed: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded (PG_DEGRADED) 2024-03-16T10:49:51.935 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 podman[92165]: 2024-03-16 10:49:51.835239272 +0000 UTC m=+0.368030682 container init 7a6319cdca6d5a603694f841bebf83d8d1748cf80013fd8ca16c248446581a7a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T10:49:51.935 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 podman[92165]: 2024-03-16 10:49:51.838392411 +0000 UTC m=+0.371183811 container start 7a6319cdca6d5a603694f841bebf83d8d1748cf80013fd8ca16c248446581a7a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:49:51.935 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 ceph-osd[92184]: -- 172.21.15.26:0/2236088002 <== mon.1 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa2dc81680 con 0x55fa2dc4ac00 2024-03-16T10:49:51.936 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 bash[92165]: 7a6319cdca6d5a603694f841bebf83d8d1748cf80013fd8ca16c248446581a7a 2024-03-16T10:49:52.231 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:51 smithi026 systemd[1]: Started Ceph osd.3 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:49:52.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:51 smithi002 ceph-mon[94239]: Health check failed: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded (PG_DEGRADED) 2024-03-16T10:49:52.731 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:52 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T10:49:52.376+0000 7f39521127c0 -1 Falling back to public interface 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 -- 172.21.15.2:0/1373163037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784103f90 msgr2=0x7fb784106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 --2- 172.21.15.2:0/1373163037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784103f90 0x7fb784106380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb774009910 tx=0x7fb774009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 -- 172.21.15.2:0/1373163037 shutdown_connections 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 --2- 172.21.15.2:0/1373163037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784103f90 0x7fb784106380 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 --2- 172.21.15.2:0/1373163037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb784101660 0x7fb784103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 -- 172.21.15.2:0/1373163037 >> 172.21.15.2:0/1373163037 conn(0x7fb7840fafb0 msgr2=0x7fb7840fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 -- 172.21.15.2:0/1373163037 shutdown_connections 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.906+0000 7fb7890fe700 1 -- 172.21.15.2:0/1373163037 wait complete. 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.907+0000 7fb7890fe700 1 Processor -- start 2024-03-16T10:49:52.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.907+0000 7fb7890fe700 1 -- start start 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb7890fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784101660 0x7fb7841965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb7890fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb784103f90 0x7fb784196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb7890fe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784197120 con 0x7fb784101660 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb7890fe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784197260 con 0x7fb784103f90 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb782d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784101660 0x7fb7841965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb782d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784101660 0x7fb7841965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36562/0 (socket says 172.21.15.2:36562) 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb78259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb784103f90 0x7fb784196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb782d9d700 1 -- 172.21.15.2:0/950566709 learned_addr learned my addr 172.21.15.2:0/950566709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb78259c700 1 -- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784101660 msgr2=0x7fb7841965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb78259c700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784101660 0x7fb7841965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb78259c700 1 -- 172.21.15.2:0/950566709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740095f0 con 0x7fb784103f90 2024-03-16T10:49:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb78259c700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb784103f90 0x7fb784196b00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb774000c00 tx=0x7fb774005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:52.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.908+0000 7fb77bfff700 1 -- 172.21.15.2:0/950566709 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77401d070 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.909+0000 7fb77bfff700 1 -- 172.21.15.2:0/950566709 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7740052a0 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.909+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78419bc60 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.909+0000 7fb77bfff700 1 -- 172.21.15.2:0/950566709 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb774003ea0 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.909+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78419c150 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.909+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78404ea80 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.910+0000 7fb77bfff700 1 -- 172.21.15.2:0/950566709 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fb774003710 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.910+0000 7fb77bfff700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb77006dff0 0x7fb7700704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.911+0000 7fb77bfff700 1 -- 172.21.15.2:0/950566709 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb77408f170 con 0x7fb784103f90 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.911+0000 7fb782d9d700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb77006dff0 0x7fb7700704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.911+0000 7fb782d9d700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb77006dff0 0x7fb7700704b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009d20 tx=0x7fb76c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:52.916+0000 7fb77bfff700 1 -- 172.21.15.2:0/950566709 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb77405c070 con 0x7fb784103f90 2024-03-16T10:49:53.097 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:52 smithi026 ceph-mon[82600]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:53.097 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:52 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:53.097 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:52 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:53.097 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:52 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:49:53.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.140+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb784105f60 con 0x7fb77006dff0 2024-03-16T10:49:53.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.141+0000 7fb77bfff700 1 -- 172.21.15.2:0/950566709 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb784105f60 con 0x7fb77006dff0 2024-03-16T10:49:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb77006dff0 msgr2=0x7fb7700704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb77006dff0 0x7fb7700704b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009d20 tx=0x7fb76c009260 comp rx=0 tx=0).stop 2024-03-16T10:49:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb784103f90 msgr2=0x7fb784196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb784103f90 0x7fb784196b00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb774000c00 tx=0x7fb774005e30 comp rx=0 tx=0).stop 2024-03-16T10:49:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 shutdown_connections 2024-03-16T10:49:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb77006dff0 0x7fb7700704b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb784101660 0x7fb7841965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 --2- 172.21.15.2:0/950566709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb784103f90 0x7fb784196b00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 >> 172.21.15.2:0/950566709 conn(0x7fb7840fafb0 msgr2=0x7fb7840fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.143+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 shutdown_connections 2024-03-16T10:49:53.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.144+0000 7fb7890fe700 1 -- 172.21.15.2:0/950566709 wait complete. 2024-03-16T10:49:53.155 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 -- 172.21.15.2:0/738189365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff070100bd0 msgr2=0x7ff070101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 --2- 172.21.15.2:0/738189365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff070100bd0 0x7ff070101030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff064009910 tx=0x7ff064009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 -- 172.21.15.2:0/738189365 shutdown_connections 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 --2- 172.21.15.2:0/738189365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff070100bd0 0x7ff070101030 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 --2- 172.21.15.2:0/738189365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0700ff9d0 0x7ff0700ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 -- 172.21.15.2:0/738189365 >> 172.21.15.2:0/738189365 conn(0x7ff0700faf70 msgr2=0x7ff0700fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 -- 172.21.15.2:0/738189365 shutdown_connections 2024-03-16T10:49:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.250+0000 7ff075d65700 1 -- 172.21.15.2:0/738189365 wait complete. 2024-03-16T10:49:53.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:52 smithi002 ceph-mon[94239]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:53.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:52 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:53.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:52 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:53.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:52 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:49:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff075d65700 1 Processor -- start 2024-03-16T10:49:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff075d65700 1 -- start start 2024-03-16T10:49:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff075d65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0700ff9d0 0x7ff0701038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff075d65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff070100bd0 0x7ff070101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff075d65700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0701024f0 con 0x7ff0700ff9d0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff075d65700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff070102630 con 0x7ff070100bd0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff06f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0700ff9d0 0x7ff0701038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff06effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff070100bd0 0x7ff070101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff06effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff070100bd0 0x7ff070101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:50012/0 (socket says 172.21.15.2:50012) 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff06effd700 1 -- 172.21.15.2:0/2585023507 learned_addr learned my addr 172.21.15.2:0/2585023507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff06effd700 1 -- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0700ff9d0 msgr2=0x7ff0701038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff06effd700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0700ff9d0 0x7ff0701038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.251+0000 7ff06effd700 1 -- 172.21.15.2:0/2585023507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640095f0 con 0x7ff070100bd0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.252+0000 7ff06effd700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff070100bd0 0x7ff070101fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff0640092e0 tx=0x7ff06400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.252+0000 7ff06cff9700 1 -- 172.21.15.2:0/2585023507 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff06401d070 con 0x7ff070100bd0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.252+0000 7ff06cff9700 1 -- 172.21.15.2:0/2585023507 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff06400f460 con 0x7ff070100bd0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.252+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0701028b0 con 0x7ff070100bd0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.252+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff070102e20 con 0x7ff070100bd0 2024-03-16T10:49:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.252+0000 7ff06cff9700 1 -- 172.21.15.2:0/2585023507 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff064021620 con 0x7ff070100bd0 2024-03-16T10:49:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.253+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07004ea80 con 0x7ff070100bd0 2024-03-16T10:49:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.253+0000 7ff06cff9700 1 -- 172.21.15.2:0/2585023507 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ff06400f5d0 con 0x7ff070100bd0 2024-03-16T10:49:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.254+0000 7ff06cff9700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff05806dfa0 0x7ff058070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.254+0000 7ff06cff9700 1 -- 172.21.15.2:0/2585023507 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff06408ee20 con 0x7ff070100bd0 2024-03-16T10:49:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.254+0000 7ff06f7fe700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff05806dfa0 0x7ff058070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.254+0000 7ff06f7fe700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff05806dfa0 0x7ff058070460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff070100a30 tx=0x7ff06000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.259+0000 7ff06cff9700 1 -- 172.21.15.2:0/2585023507 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff06405bdc0 con 0x7ff070100bd0 2024-03-16T10:49:53.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.485+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff07019e5e0 con 0x7ff05806dfa0 2024-03-16T10:49:53.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.486+0000 7ff06cff9700 1 -- 172.21.15.2:0/2585023507 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff07019e5e0 con 0x7ff05806dfa0 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff05806dfa0 msgr2=0x7ff058070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff05806dfa0 0x7ff058070460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff070100a30 tx=0x7ff06000b410 comp rx=0 tx=0).stop 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff070100bd0 msgr2=0x7ff070101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff070100bd0 0x7ff070101fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff0640092e0 tx=0x7ff06400bd70 comp rx=0 tx=0).stop 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 shutdown_connections 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff05806dfa0 0x7ff058070460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0700ff9d0 0x7ff0701038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 --2- 172.21.15.2:0/2585023507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff070100bd0 0x7ff070101fb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 >> 172.21.15.2:0/2585023507 conn(0x7ff0700faf70 msgr2=0x7ff0700fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:53.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 shutdown_connections 2024-03-16T10:49:53.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.488+0000 7ff075d65700 1 -- 172.21.15.2:0/2585023507 wait complete. 2024-03-16T10:49:53.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/1325169964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac100ae0 msgr2=0x7fb3ac100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/1325169964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac100ae0 0x7fb3ac100f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009910 tx=0x7fb39c009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:53.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/1325169964 shutdown_connections 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/1325169964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac101ce0 0x7fb3ac102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/1325169964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac100ae0 0x7fb3ac100f00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/1325169964 >> 172.21.15.2:0/1325169964 conn(0x7fb3ac0fc0a0 msgr2=0x7fb3ac0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/1325169964 shutdown_connections 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.594+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/1325169964 wait complete. 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3b1dc2700 1 Processor -- start 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3b1dc2700 1 -- start start 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3b1dc2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac100ae0 0x7fb3ac1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3b1dc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac101ce0 0x7fb3ac196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3b1dc2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac197030 con 0x7fb3ac101ce0 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3b1dc2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac197170 con 0x7fb3ac100ae0 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3aaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac101ce0 0x7fb3ac196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3aaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac101ce0 0x7fb3ac196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36598/0 (socket says 172.21.15.2:36598) 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.595+0000 7fb3aaffd700 1 -- 172.21.15.2:0/3888991287 learned_addr learned my addr 172.21.15.2:0/3888991287 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:53.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3ab7fe700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac100ae0 0x7fb3ac1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3aaffd700 1 -- 172.21.15.2:0/3888991287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac100ae0 msgr2=0x7fb3ac1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3aaffd700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac100ae0 0x7fb3ac1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3aaffd700 1 -- 172.21.15.2:0/3888991287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0095f0 con 0x7fb3ac101ce0 2024-03-16T10:49:53.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3aaffd700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac101ce0 0x7fb3ac196a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e920 tx=0x7fb3a000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3a8ff9700 1 -- 172.21.15.2:0/3888991287 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a0009790 con 0x7fb3ac101ce0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3a8ff9700 1 -- 172.21.15.2:0/3888991287 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3a000cba0 con 0x7fb3ac101ce0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac19bbd0 con 0x7fb3ac101ce0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3a8ff9700 1 -- 172.21.15.2:0/3888991287 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a0018650 con 0x7fb3ac101ce0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.596+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac19c1a0 con 0x7fb3ac101ce0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.597+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac190660 con 0x7fb3ac101ce0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.598+0000 7fb3a8ff9700 1 -- 172.21.15.2:0/3888991287 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fb3a000cd10 con 0x7fb3ac101ce0 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.598+0000 7fb3a8ff9700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb39406dfb0 0x7fb394070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.598+0000 7fb3a8ff9700 1 -- 172.21.15.2:0/3888991287 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb3a0014070 con 0x7fb3ac101ce0 2024-03-16T10:49:53.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.598+0000 7fb3ab7fe700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb39406dfb0 0x7fb394070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.599+0000 7fb3ab7fe700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb39406dfb0 0x7fb394070470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009fd0 tx=0x7fb39c005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.603+0000 7fb3a8ff9700 1 -- 172.21.15.2:0/3888991287 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb3a0053740 con 0x7fb3ac101ce0 2024-03-16T10:49:53.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.817+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3ac0611f0 con 0x7fb39406dfb0 2024-03-16T10:49:53.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.828+0000 7fb3a8ff9700 1 -- 172.21.15.2:0/3888991287 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb3ac0611f0 con 0x7fb39406dfb0 2024-03-16T10:49:53.828 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (12m) 32s ago 13m 24.8M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (13m) 32s ago 13m 8238k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (12m) 2m ago 12m 7922k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (2m) 32s ago 13m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (2m) 2m ago 12m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (11m) 32s ago 13m 86.7M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (7m) 32s ago 7m 19.0M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (7m) 32s ago 7m 16.0M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (7m) 2m ago 7m 153M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (7m) 2m ago 7m 16.7M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (5m) 32s ago 15m 504M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (4m) 2m ago 12m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:49:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (3m) 32s ago 15m 46.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (3m) 2m ago 12m 33.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (13m) 32s ago 13m 16.4M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (12m) 2m ago 12m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (109s) 32s ago 11m 81.8M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (74s) 32s ago 11m 85.2M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (38s) 32s ago 10m 13.5M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 starting - - - 2241M 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (9m) 2m ago 9m 208M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 1f2c398304b3 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (9m) 2m ago 9m 198M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:49:53.830 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (4m) 32s ago 13m 74.3M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb39406dfb0 msgr2=0x7fb394070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb39406dfb0 0x7fb394070470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009fd0 tx=0x7fb39c005fb0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac101ce0 msgr2=0x7fb3ac196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac101ce0 0x7fb3ac196a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000e920 tx=0x7fb3a000ece0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 shutdown_connections 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb39406dfb0 0x7fb394070470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ac100ae0 0x7fb3ac1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 --2- 172.21.15.2:0/3888991287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3ac101ce0 0x7fb3ac196a10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 >> 172.21.15.2:0/3888991287 conn(0x7fb3ac0fc0a0 msgr2=0x7fb3ac1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:53.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 shutdown_connections 2024-03-16T10:49:53.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.830+0000 7fb3b1dc2700 1 -- 172.21.15.2:0/3888991287 wait complete. 2024-03-16T10:49:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 -- 172.21.15.2:0/3804434841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4103f30 msgr2=0x7fdfe4106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 --2- 172.21.15.2:0/3804434841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4103f30 0x7fdfe4106320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009910 tx=0x7fdfd4009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 -- 172.21.15.2:0/3804434841 shutdown_connections 2024-03-16T10:49:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 --2- 172.21.15.2:0/3804434841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4103f30 0x7fdfe4106320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 --2- 172.21.15.2:0/3804434841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe4101600 0x7fdfe41039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 -- 172.21.15.2:0/3804434841 >> 172.21.15.2:0/3804434841 conn(0x7fdfe40faf70 msgr2=0x7fdfe40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 -- 172.21.15.2:0/3804434841 shutdown_connections 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 -- 172.21.15.2:0/3804434841 wait complete. 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 Processor -- start 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 -- start start 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.934+0000 7fdfec372700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4101600 0x7fdfe41964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfec372700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe4103f30 0x7fdfe41969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfec372700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe4197010 con 0x7fdfe4101600 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfec372700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe4197150 con 0x7fdfe4103f30 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfea10e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4101600 0x7fdfe41964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfea10e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4101600 0x7fdfe41964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36616/0 (socket says 172.21.15.2:36616) 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfea10e700 1 -- 172.21.15.2:0/532064391 learned_addr learned my addr 172.21.15.2:0/532064391 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfe990d700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe4103f30 0x7fdfe41969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfea10e700 1 -- 172.21.15.2:0/532064391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe4103f30 msgr2=0x7fdfe41969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfea10e700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe4103f30 0x7fdfe41969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfea10e700 1 -- 172.21.15.2:0/532064391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40095f0 con 0x7fdfe4101600 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfe990d700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe4103f30 0x7fdfe41969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfea10e700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4101600 0x7fdfe41964b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000e920 tx=0x7fdfe000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfdb7fe700 1 -- 172.21.15.2:0/532064391 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe0009790 con 0x7fdfe4101600 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfdb7fe700 1 -- 172.21.15.2:0/532064391 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfe000cba0 con 0x7fdfe4101600 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.935+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe419bbb0 con 0x7fdfe4101600 2024-03-16T10:49:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.936+0000 7fdfdb7fe700 1 -- 172.21.15.2:0/532064391 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe0018650 con 0x7fdfe4101600 2024-03-16T10:49:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.936+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe419c100 con 0x7fdfe4101600 2024-03-16T10:49:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.937+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe404ea80 con 0x7fdfe4101600 2024-03-16T10:49:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.938+0000 7fdfdb7fe700 1 -- 172.21.15.2:0/532064391 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fdfe000cd10 con 0x7fdfe4101600 2024-03-16T10:49:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.938+0000 7fdfdb7fe700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdfd006dfb0 0x7fdfd0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.938+0000 7fdfdb7fe700 1 -- 172.21.15.2:0/532064391 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdfe0014070 con 0x7fdfe4101600 2024-03-16T10:49:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.938+0000 7fdfe990d700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdfd006dfb0 0x7fdfd0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.939+0000 7fdfe990d700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdfd006dfb0 0x7fdfd0070470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b800 tx=0x7fdfd4009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:53.945+0000 7fdfdb7fe700 1 -- 172.21.15.2:0/532064391 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdfe0053740 con 0x7fdfe4101600 2024-03-16T10:49:54.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.240+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdfe419ca10 con 0x7fdfe4101600 2024-03-16T10:49:54.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.240+0000 7fdfdb7fe700 1 -- 172.21.15.2:0/532064391 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7fdfe0055e60 con 0x7fdfe4101600 2024-03-16T10:49:54.241 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:49:54.241 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:49:54.241 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2, 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 3 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:49:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 6, 2024-03-16T10:49:54.243 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 7 2024-03-16T10:49:54.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:49:54.243 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:49:54.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.242+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdfd006dfb0 msgr2=0x7fdfd0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.242+0000 7fdfec372700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdfd006dfb0 0x7fdfd0070470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b800 tx=0x7fdfd4009f70 comp rx=0 tx=0).stop 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4101600 msgr2=0x7fdfe41964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4101600 0x7fdfe41964b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000e920 tx=0x7fdfe000ece0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 shutdown_connections 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fdfd006dfb0 0x7fdfd0070470 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfe4101600 0x7fdfe41964b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 --2- 172.21.15.2:0/532064391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfe4103f30 0x7fdfe41969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 >> 172.21.15.2:0/532064391 conn(0x7fdfe40faf70 msgr2=0x7fdfe40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 shutdown_connections 2024-03-16T10:49:54.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.243+0000 7fdfec372700 1 -- 172.21.15.2:0/532064391 wait complete. 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 -- 172.21.15.2:0/1794345730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214103d90 msgr2=0x7ff214106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 --2- 172.21.15.2:0/1794345730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214103d90 0x7ff214106180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009910 tx=0x7ff1fc009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 -- 172.21.15.2:0/1794345730 shutdown_connections 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 --2- 172.21.15.2:0/1794345730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214103d90 0x7ff214106180 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 --2- 172.21.15.2:0/1794345730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff214101460 0x7ff214103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 -- 172.21.15.2:0/1794345730 >> 172.21.15.2:0/1794345730 conn(0x7ff2140fadf0 msgr2=0x7ff2140fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 -- 172.21.15.2:0/1794345730 shutdown_connections 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 -- 172.21.15.2:0/1794345730 wait complete. 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 Processor -- start 2024-03-16T10:49:54.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.348+0000 7ff213fff700 1 -- start start 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff213fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214101460 0x7ff214196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff213fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff214103d90 0x7ff214196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff213fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214196f70 con 0x7ff214101460 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff213fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2141970b0 con 0x7ff214103d90 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff212ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214101460 0x7ff214196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff212ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214101460 0x7ff214196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36630/0 (socket says 172.21.15.2:36630) 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff212ffd700 1 -- 172.21.15.2:0/1911000342 learned_addr learned my addr 172.21.15.2:0/1911000342 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff2127fc700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff214103d90 0x7ff214196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff212ffd700 1 -- 172.21.15.2:0/1911000342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff214103d90 msgr2=0x7ff214196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff212ffd700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff214103d90 0x7ff214196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff212ffd700 1 -- 172.21.15.2:0/1911000342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc0095f0 con 0x7ff214101460 2024-03-16T10:49:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff212ffd700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214101460 0x7ff214196410 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff20400ba50 tx=0x7ff20400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff20bfff700 1 -- 172.21.15.2:0/1911000342 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20400c900 con 0x7ff214101460 2024-03-16T10:49:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21419bb10 con 0x7ff214101460 2024-03-16T10:49:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff20bfff700 1 -- 172.21.15.2:0/1911000342 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff204014440 con 0x7ff214101460 2024-03-16T10:49:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.349+0000 7ff20bfff700 1 -- 172.21.15.2:0/1911000342 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2040075a0 con 0x7ff214101460 2024-03-16T10:49:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.350+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21419c060 con 0x7ff214101460 2024-03-16T10:49:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.350+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21404f2d0 con 0x7ff214101460 2024-03-16T10:49:54.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.351+0000 7ff20bfff700 1 -- 172.21.15.2:0/1911000342 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ff2040145b0 con 0x7ff214101460 2024-03-16T10:49:54.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.351+0000 7ff20bfff700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff20006dfa0 0x7ff200070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:54.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.351+0000 7ff20bfff700 1 -- 172.21.15.2:0/1911000342 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff204006e40 con 0x7ff214101460 2024-03-16T10:49:54.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.351+0000 7ff2127fc700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff20006dfa0 0x7ff200070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:54.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.352+0000 7ff2127fc700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff20006dfa0 0x7ff200070460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009fd0 tx=0x7ff1fc005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:54.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.356+0000 7ff20bfff700 1 -- 172.21.15.2:0/1911000342 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff20405a480 con 0x7ff214101460 2024-03-16T10:49:54.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.608+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff21419c310 con 0x7ff214101460 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.609+0000 7ff20bfff700 1 -- 172.21.15.2:0/1911000342 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7ff20405a010 con 0x7ff214101460 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:49:54.610 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:49:54.611 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:54.613 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:49:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:49:54.613 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:54.613 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:54.613 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff20006dfa0 msgr2=0x7ff200070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff20006dfa0 0x7ff200070460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009fd0 tx=0x7ff1fc005960 comp rx=0 tx=0).stop 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214101460 msgr2=0x7ff214196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214101460 0x7ff214196410 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff20400ba50 tx=0x7ff20400bd60 comp rx=0 tx=0).stop 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 shutdown_connections 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff20006dfa0 0x7ff200070460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff214101460 0x7ff214196410 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 --2- 172.21.15.2:0/1911000342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff214103d90 0x7ff214196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 >> 172.21.15.2:0/1911000342 conn(0x7ff2140fadf0 msgr2=0x7ff2140fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 shutdown_connections 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.611+0000 7ff213fff700 1 -- 172.21.15.2:0/1911000342 wait complete. 2024-03-16T10:49:54.614 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:49:54.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.720+0000 7fce42139700 1 -- 172.21.15.2:0/3272967277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 msgr2=0x7fce3c0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.720+0000 7fce42139700 1 --2- 172.21.15.2:0/3272967277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c0ffe00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009910 tx=0x7fce2c009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.720+0000 7fce42139700 1 -- 172.21.15.2:0/3272967277 shutdown_connections 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.720+0000 7fce42139700 1 --2- 172.21.15.2:0/3272967277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3c100be0 0x7fce3c101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.720+0000 7fce42139700 1 --2- 172.21.15.2:0/3272967277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c0ffe00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.720+0000 7fce42139700 1 -- 172.21.15.2:0/3272967277 >> 172.21.15.2:0/3272967277 conn(0x7fce3c0faf80 msgr2=0x7fce3c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.720+0000 7fce42139700 1 -- 172.21.15.2:0/3272967277 shutdown_connections 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce42139700 1 -- 172.21.15.2:0/3272967277 wait complete. 2024-03-16T10:49:54.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce42139700 1 Processor -- start 2024-03-16T10:49:54.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce42139700 1 -- start start 2024-03-16T10:49:54.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce42139700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:54.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce42139700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3c100be0 0x7fce3c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:54.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce42139700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c102500 con 0x7fce3c0ff9e0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce42139700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c102640 con 0x7fce3c100be0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.721+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36642/0 (socket says 172.21.15.2:36642) 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3b7fe700 1 -- 172.21.15.2:0/3301338930 learned_addr learned my addr 172.21.15.2:0/3301338930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3affd700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3c100be0 0x7fce3c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3affd700 1 -- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 msgr2=0x7fce3c103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3affd700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3affd700 1 -- 172.21.15.2:0/3301338930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0095f0 con 0x7fce3c100be0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3b7fe700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce3affd700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3c100be0 0x7fce3c101fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e970 tx=0x7fce3000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce38ff9700 1 -- 172.21.15.2:0/3301338930 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3000c9d0 con 0x7fce3c100be0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c102920 con 0x7fce3c100be0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce38ff9700 1 -- 172.21.15.2:0/3301338930 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce30010430 con 0x7fce3c100be0 2024-03-16T10:49:54.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce38ff9700 1 -- 172.21.15.2:0/3301338930 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce30018630 con 0x7fce3c100be0 2024-03-16T10:49:54.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.722+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c19e550 con 0x7fce3c100be0 2024-03-16T10:49:54.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.723+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c04ea80 con 0x7fce3c100be0 2024-03-16T10:49:54.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.724+0000 7fce38ff9700 1 -- 172.21.15.2:0/3301338930 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fce300105a0 con 0x7fce3c100be0 2024-03-16T10:49:54.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.724+0000 7fce38ff9700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fce2406dff0 0x7fce240704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:54.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.724+0000 7fce38ff9700 1 -- 172.21.15.2:0/3301338930 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fce30014070 con 0x7fce3c100be0 2024-03-16T10:49:54.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.724+0000 7fce3b7fe700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fce2406dff0 0x7fce240704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:54.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.725+0000 7fce3b7fe700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fce2406dff0 0x7fce240704b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00b800 tx=0x7fce2c009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:54.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.729+0000 7fce38ff9700 1 -- 172.21.15.2:0/3301338930 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce30053740 con 0x7fce3c100be0 2024-03-16T10:49:54.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.956+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce3c069750 con 0x7fce2406dff0 2024-03-16T10:49:54.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.958+0000 7fce38ff9700 1 -- 172.21.15.2:0/3301338930 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fce3c069750 con 0x7fce2406dff0 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "9/23 daemons upgraded", 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:49:54.959 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fce2406dff0 msgr2=0x7fce240704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fce2406dff0 0x7fce240704b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00b800 tx=0x7fce2c009f70 comp rx=0 tx=0).stop 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3c100be0 msgr2=0x7fce3c101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3c100be0 0x7fce3c101fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fce3000e970 tx=0x7fce3000ec80 comp rx=0 tx=0).stop 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 shutdown_connections 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fce2406dff0 0x7fce240704b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce3c0ff9e0 0x7fce3c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 --2- 172.21.15.2:0/3301338930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3c100be0 0x7fce3c101fc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:54.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 >> 172.21.15.2:0/3301338930 conn(0x7fce3c0faf80 msgr2=0x7fce3c069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:54.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 shutdown_connections 2024-03-16T10:49:54.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:54.960+0000 7fce42139700 1 -- 172.21.15.2:0/3301338930 wait complete. 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 -- 172.21.15.2:0/98247955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 msgr2=0x7f7064100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 --2- 172.21.15.2:0/98247955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064100130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7060009910 tx=0x7f7060009c20 comp rx=0 tx=0).stop 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 -- 172.21.15.2:0/98247955 shutdown_connections 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 --2- 172.21.15.2:0/98247955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064100130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 --2- 172.21.15.2:0/98247955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70640ff350 0x7f70640ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 -- 172.21.15.2:0/98247955 >> 172.21.15.2:0/98247955 conn(0x7f70640faf70 msgr2=0x7f70640fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 -- 172.21.15.2:0/98247955 shutdown_connections 2024-03-16T10:49:55.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 -- 172.21.15.2:0/98247955 wait complete. 2024-03-16T10:49:55.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.067+0000 7f706c00d700 1 Processor -- start 2024-03-16T10:49:55.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f706c00d700 1 -- start start 2024-03-16T10:49:55.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f706c00d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70640ff350 0x7f7064198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f706c00d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f706c00d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641991d0 con 0x7f70640ffcb0 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f706c00d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064199310 con 0x7f70640ff350 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f70695a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f70695a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36662/0 (socket says 172.21.15.2:36662) 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f70695a8700 1 -- 172.21.15.2:0/2012563285 learned_addr learned my addr 172.21.15.2:0/2012563285 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f7069da9700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70640ff350 0x7f7064198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f70695a8700 1 -- 172.21.15.2:0/2012563285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70640ff350 msgr2=0x7f7064198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f70695a8700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70640ff350 0x7f7064198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f70695a8700 1 -- 172.21.15.2:0/2012563285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600095f0 con 0x7f70640ffcb0 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f70695a8700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064198bb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f706000b800 tx=0x7f7060005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f705affd700 1 -- 172.21.15.2:0/2012563285 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f706001d070 con 0x7f70640ffcb0 2024-03-16T10:49:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706419dd10 con 0x7f70640ffcb0 2024-03-16T10:49:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f705affd700 1 -- 172.21.15.2:0/2012563285 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70600052a0 con 0x7f70640ffcb0 2024-03-16T10:49:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.068+0000 7f705affd700 1 -- 172.21.15.2:0/2012563285 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7060017400 con 0x7f70640ffcb0 2024-03-16T10:49:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.069+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706419e280 con 0x7f70640ffcb0 2024-03-16T10:49:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.069+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7064192840 con 0x7f70640ffcb0 2024-03-16T10:49:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.070+0000 7f705affd700 1 -- 172.21.15.2:0/2012563285 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f7060003780 con 0x7f70640ffcb0 2024-03-16T10:49:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.070+0000 7f705affd700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f705006dfa0 0x7f7050070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:49:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.070+0000 7f705affd700 1 -- 172.21.15.2:0/2012563285 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f706008f1a0 con 0x7f70640ffcb0 2024-03-16T10:49:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.070+0000 7f7069da9700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f705006dfa0 0x7f7050070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:49:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.071+0000 7f7069da9700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f705006dfa0 0x7f7050070460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7054009c60 tx=0x7f7054009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:49:55.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.075+0000 7f705affd700 1 -- 172.21.15.2:0/2012563285 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f706005c140 con 0x7f70640ffcb0 2024-03-16T10:49:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:54 smithi002 ceph-mon[94239]: from='client.44211 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:54 smithi002 ceph-mon[94239]: from='client.44215 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:54 smithi002 ceph-mon[94239]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:55.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:54 smithi002 ceph-mon[94239]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:55.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:54 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/532064391' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:55.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:54 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1911000342' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:49:55.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.385+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f706404ea80 con 0x7f70640ffcb0 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.386+0000 7f705affd700 1 -- 172.21.15.2:0/2012563285 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1461 (secure 0 0 0) 0x7f706005bcd0 con 0x7f70640ffcb0 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stdout: osd.3 (root=default,host=smithi026) is down 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-03-16T10:49:55.387 INFO:teuthology.orchestra.run.smithi002.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-03-16T10:49:55.388 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-16T10:49:55.389 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-03-16T10:49:55.389 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-16T10:49:55.389 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-16T10:49:55.389 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-16T10:49:55.389 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-16T10:49:55.389 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-16T10:49:55.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f705006dfa0 msgr2=0x7f7050070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f705006dfa0 0x7f7050070460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7054009c60 tx=0x7f7054009260 comp rx=0 tx=0).stop 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 msgr2=0x7f7064198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064198bb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f706000b800 tx=0x7f7060005f00 comp rx=0 tx=0).stop 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 shutdown_connections 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f705006dfa0 0x7f7050070460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70640ff350 0x7f7064198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 --2- 172.21.15.2:0/2012563285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70640ffcb0 0x7f7064198bb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 >> 172.21.15.2:0/2012563285 conn(0x7f70640faf70 msgr2=0x7f7064107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 shutdown_connections 2024-03-16T10:49:55.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:49:55.389+0000 7f706c00d700 1 -- 172.21.15.2:0/2012563285 wait complete. 2024-03-16T10:49:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:54 smithi026 ceph-mon[82600]: from='client.44211 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:54 smithi026 ceph-mon[82600]: from='client.44215 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:55.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:54 smithi026 ceph-mon[82600]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:55.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:54 smithi026 ceph-mon[82600]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:55.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:54 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/532064391' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:49:55.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:54 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1911000342' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:49:56.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:55 smithi026 ceph-mon[82600]: from='client.44229 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:56.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:55 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:56.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:55 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:56.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:55 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2012563285' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:49:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:55 smithi002 ceph-mon[94239]: from='client.44229 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:49:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:55 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:55 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:55 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2012563285' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:49:57.278 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:56 smithi026 ceph-mon[82600]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:57.278 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:56 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:57.278 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:56 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:56 smithi002 ceph-mon[94239]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:56 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:56 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:49:57.608 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:57 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T10:49:57.276+0000 7f39521127c0 -1 osd.3 66 log_to_monitors true 2024-03-16T10:49:58.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:57 smithi026 ceph-mon[82600]: from='osd.3 [v2:172.21.15.26:6800/3946506757,v1:172.21.15.26:6801/3946506757]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:49:58.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:57 smithi026 ceph-mon[82600]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:49:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:57 smithi002 ceph-mon[94239]: from='osd.3 [v2:172.21.15.26:6800/3946506757,v1:172.21.15.26:6801/3946506757]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:49:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:57 smithi002 ceph-mon[94239]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T10:49:58.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 10:49:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T10:49:58.700+0000 7f393f562700 -1 osd.3 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:49:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:58 smithi002 ceph-mon[94239]: pgmap v129: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:58 smithi002 ceph-mon[94239]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T10:49:59.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:58 smithi002 ceph-mon[94239]: from='osd.3 [v2:172.21.15.26:6800/3946506757,v1:172.21.15.26:6801/3946506757]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:49:59.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:58 smithi002 ceph-mon[94239]: osdmap e69: 6 total, 5 up, 6 in 2024-03-16T10:49:59.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:58 smithi002 ceph-mon[94239]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:49:59.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:58 smithi002 ceph-mon[94239]: from='osd.3 ' entity='osd.3' 2024-03-16T10:49:59.395 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:58 smithi026 ceph-mon[82600]: pgmap v129: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:49:59.395 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:58 smithi026 ceph-mon[82600]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T10:49:59.395 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:58 smithi026 ceph-mon[82600]: from='osd.3 [v2:172.21.15.26:6800/3946506757,v1:172.21.15.26:6801/3946506757]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:49:59.395 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:58 smithi026 ceph-mon[82600]: osdmap e69: 6 total, 5 up, 6 in 2024-03-16T10:49:59.395 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:58 smithi026 ceph-mon[82600]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:49:59.395 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:58 smithi026 ceph-mon[82600]: from='osd.3 ' entity='osd.3' 2024-03-16T10:50:00.264 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:59 smithi002 ceph-mon[94239]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:50:00.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:59 smithi002 ceph-mon[94239]: osd.3 [v2:172.21.15.26:6800/3946506757,v1:172.21.15.26:6801/3946506757] boot 2024-03-16T10:50:00.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:59 smithi002 ceph-mon[94239]: osdmap e70: 6 total, 6 up, 6 in 2024-03-16T10:50:00.265 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:49:59 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:50:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:59 smithi026 ceph-mon[82600]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:50:00.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:59 smithi026 ceph-mon[82600]: osd.3 [v2:172.21.15.26:6800/3946506757,v1:172.21.15.26:6801/3946506757] boot 2024-03-16T10:50:00.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:59 smithi026 ceph-mon[82600]: osdmap e70: 6 total, 6 up, 6 in 2024-03-16T10:50:00.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:49:59 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T10:50:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pgmap v132: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:50:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-03-16T10:50:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:50:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:50:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: [WRN] PG_DEGRADED: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-03-16T10:50:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-16T10:50:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.c is active+undersized+degraded, acting [2,0] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.e is active+undersized+degraded, acting [2,0] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-16T10:50:01.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:01.301 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:00 smithi002 ceph-mon[94239]: osdmap e71: 6 total, 6 up, 6 in 2024-03-16T10:50:01.384 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pgmap v132: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-03-16T10:50:01.384 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-03-16T10:50:01.384 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:50:01.384 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:50:01.384 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: [WRN] PG_DEGRADED: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-03-16T10:50:01.384 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-16T10:50:01.384 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.c is active+undersized+degraded, acting [2,0] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.e is active+undersized+degraded, acting [2,0] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:00 smithi026 ceph-mon[82600]: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-16T10:50:01.385 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:01.386 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:01 smithi026 ceph-mon[82600]: osdmap e71: 6 total, 6 up, 6 in 2024-03-16T10:50:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:02 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:02.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:02 smithi002 ceph-mon[94239]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-16T10:50:02.401 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:02 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:02.401 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:02 smithi026 ceph-mon[82600]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-16T10:50:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:03 smithi002 ceph-mon[94239]: pgmap v134: 65 pgs: 4 peering, 9 active+undersized+degraded, 7 active+undersized, 45 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-03-16T10:50:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:03 smithi002 ceph-mon[94239]: Health check update: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:03 smithi026 ceph-mon[82600]: pgmap v134: 65 pgs: 4 peering, 9 active+undersized+degraded, 7 active+undersized, 45 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29/267 objects degraded (10.861%) 2024-03-16T10:50:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:03 smithi026 ceph-mon[82600]: Health check update: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:04.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:04 smithi026 ceph-mon[82600]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded) 2024-03-16T10:50:04.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:04 smithi002 ceph-mon[94239]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 29/267 objects degraded (10.861%), 9 pgs degraded) 2024-03-16T10:50:05.449 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:05 smithi002 ceph-mon[94239]: pgmap v135: 65 pgs: 4 peering, 61 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:05.449 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:05.449 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:05 smithi026 ceph-mon[82600]: pgmap v135: 65 pgs: 4 peering, 61 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:07.470 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:07 smithi026 ceph-mon[82600]: pgmap v136: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:07 smithi002 ceph-mon[94239]: pgmap v136: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:09.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:09 smithi026 ceph-mon[82600]: pgmap v137: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:09 smithi002 ceph-mon[94239]: pgmap v137: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:11 smithi026 ceph-mon[82600]: pgmap v138: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:11.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:11 smithi002 ceph-mon[94239]: pgmap v138: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:13 smithi026 ceph-mon[82600]: pgmap v139: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:13 smithi002 ceph-mon[94239]: pgmap v139: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:15.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:15 smithi026 ceph-mon[82600]: pgmap v140: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:15.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:15 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:15.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:15 smithi002 ceph-mon[94239]: pgmap v140: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:15.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:15 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:16.148 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:16.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:17.435 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:17.435 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: Upgrade: osd.4 is safe to restart 2024-03-16T10:50:17.435 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: pgmap v141: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:17.435 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: Upgrade: Updating osd.4 2024-03-16T10:50:17.436 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:17.436 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:50:17.436 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:17.436 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:17 smithi026 ceph-mon[82600]: Deploying daemon osd.4 on smithi026 2024-03-16T10:50:17.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-16T10:50:17.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: Upgrade: osd.4 is safe to restart 2024-03-16T10:50:17.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: pgmap v141: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:17.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: Upgrade: Updating osd.4 2024-03-16T10:50:17.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:17.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T10:50:17.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:17.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:17 smithi002 ceph-mon[94239]: Deploying daemon osd.4 on smithi026 2024-03-16T10:50:18.425 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:18 smithi026 ceph-mon[82600]: pgmap v142: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:18.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:18 smithi002 ceph-mon[94239]: pgmap v142: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:19.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:20.212 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:20.354 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:20 smithi026 systemd[1]: Stopping Ceph osd.4 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:50:20.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:20 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[49782]: 2024-03-16T10:50:20.352+0000 7f7982afa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:50:20.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:20 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[49782]: 2024-03-16T10:50:20.352+0000 7f7982afa700 -1 osd.4 71 *** Got signal Terminated *** 2024-03-16T10:50:20.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:20 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[49782]: 2024-03-16T10:50:20.352+0000 7f7982afa700 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:50:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:20 smithi026 ceph-mon[82600]: pgmap v143: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:20 smithi026 ceph-mon[82600]: osd.4 marked itself down and dead 2024-03-16T10:50:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:20 smithi002 ceph-mon[94239]: pgmap v143: 65 pgs: 65 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:20 smithi002 ceph-mon[94239]: osd.4 marked itself down and dead 2024-03-16T10:50:21.621 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:21 smithi026 podman[96615]: 2024-03-16 10:50:21.355992035 +0000 UTC m=+1.113569059 container died 1f2c398304b320492024fe707293f2127511a9df0a757a7c0b449f990253613c (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-a752c2f, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T10:50:21.887 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:21 smithi026 ceph-mon[82600]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:50:21.887 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:21 smithi026 ceph-mon[82600]: osdmap e72: 6 total, 5 up, 6 in 2024-03-16T10:50:21.888 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:21 smithi026 podman[96615]: 2024-03-16 10:50:21.71957689 +0000 UTC m=+1.477153924 container cleanup 1f2c398304b320492024fe707293f2127511a9df0a757a7c0b449f990253613c (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=reef-a752c2f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T10:50:21.888 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:21 smithi026 bash[96615]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4 2024-03-16T10:50:22.231 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:21 smithi026 podman[96632]: 2024-03-16 10:50:21.8874697 +0000 UTC m=+0.528494087 container remove 1f2c398304b320492024fe707293f2127511a9df0a757a7c0b449f990253613c (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-a752c2f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T10:50:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:21 smithi002 ceph-mon[94239]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:50:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:21 smithi002 ceph-mon[94239]: osdmap e72: 6 total, 5 up, 6 in 2024-03-16T10:50:22.541 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:22 smithi026 podman[96721]: 2024-03-16 10:50:22.190507344 +0000 UTC m=+0.017814272 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:50:22.541 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:22 smithi026 podman[96721]: 2024-03-16 10:50:22.313686051 +0000 UTC m=+0.140992967 container create 151f6b09303e886dfc7da70c87db6ed8495a9e3c1c0e6a9e8c04def3da53ec2e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True) 2024-03-16T10:50:22.541 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:22 smithi026 podman[96721]: 2024-03-16 10:50:22.541223125 +0000 UTC m=+0.368530044 container init 151f6b09303e886dfc7da70c87db6ed8495a9e3c1c0e6a9e8c04def3da53ec2e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-16T10:50:22.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:22 smithi026 podman[96721]: 2024-03-16 10:50:22.544630635 +0000 UTC m=+0.371937559 container start 151f6b09303e886dfc7da70c87db6ed8495a9e3c1c0e6a9e8c04def3da53ec2e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T10:50:22.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:22 smithi026 podman[96721]: 2024-03-16 10:50:22.624474133 +0000 UTC m=+0.451781049 container attach 151f6b09303e886dfc7da70c87db6ed8495a9e3c1c0e6a9e8c04def3da53ec2e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-16T10:50:23.144 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:22 smithi026 podman[96721]: 2024-03-16 10:50:22.852328943 +0000 UTC m=+0.679635865 container died 151f6b09303e886dfc7da70c87db6ed8495a9e3c1c0e6a9e8c04def3da53ec2e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-16T10:50:23.144 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:22 smithi026 ceph-mon[82600]: pgmap v145: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:23.144 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:22 smithi026 ceph-mon[82600]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-16T10:50:23.144 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:22 smithi026 ceph-mon[82600]: osdmap e73: 6 total, 5 up, 6 in 2024-03-16T10:50:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:22 smithi002 ceph-mon[94239]: pgmap v145: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:23.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:22 smithi002 ceph-mon[94239]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-16T10:50:23.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:22 smithi002 ceph-mon[94239]: osdmap e73: 6 total, 5 up, 6 in 2024-03-16T10:50:23.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:23 smithi026 podman[96746]: 2024-03-16 10:50:23.42801298 +0000 UTC m=+0.566679443 container remove 151f6b09303e886dfc7da70c87db6ed8495a9e3c1c0e6a9e8c04def3da53ec2e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:50:23.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:23 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-16T10:50:23.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:23 smithi026 systemd[1]: Stopped Ceph osd.4 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:50:23.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:23 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.4.service: Consumed 51.208s CPU time. 2024-03-16T10:50:23.732 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:23 smithi026 systemd[1]: Starting Ceph osd.4 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:50:24.174 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:23 smithi026 ceph-mon[82600]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:24.174 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:23 smithi026 podman[96844]: 2024-03-16 10:50:23.772133116 +0000 UTC m=+0.017882682 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:50:24.174 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:23 smithi026 podman[96844]: 2024-03-16 10:50:23.870646231 +0000 UTC m=+0.116395777 container create 31828a85e84bd481653b0c0590080b864873ecf25c6d3f807b104b248fe21866 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T10:50:24.174 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 podman[96844]: 2024-03-16 10:50:24.098397039 +0000 UTC m=+0.344146593 container init 31828a85e84bd481653b0c0590080b864873ecf25c6d3f807b104b248fe21866 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-16T10:50:24.174 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 podman[96844]: 2024-03-16 10:50:24.101533579 +0000 UTC m=+0.347283127 container start 31828a85e84bd481653b0c0590080b864873ecf25c6d3f807b104b248fe21866 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T10:50:24.174 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 podman[96844]: 2024-03-16 10:50:24.17343286 +0000 UTC m=+0.419182419 container attach 31828a85e84bd481653b0c0590080b864873ecf25c6d3f807b104b248fe21866 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-16T10:50:24.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:23 smithi002 ceph-mon[94239]: Health check failed: Degraded data redundancy: 22/267 objects degraded (8.240%), 5 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate[96859]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate[96859]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 bash[96844]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 bash[96844]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 bash[96844]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 bash[96844]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 bash[96844]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 bash[96844]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 bash[96844]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate[96859]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-16T10:50:24.852 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate[96859]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-16T10:50:24.853 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate[96859]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-16T10:50:24.853 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate[96859]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T10:50:24.853 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate[96859]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-16T10:50:24.853 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:24 smithi026 podman[96844]: 2024-03-16 10:50:24.57763712 +0000 UTC m=+0.823386667 container died 31828a85e84bd481653b0c0590080b864873ecf25c6d3f807b104b248fe21866 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:50:25.160 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:24 smithi026 ceph-mon[82600]: pgmap v147: 65 pgs: 5 active+undersized+degraded, 8 peering, 4 stale+active+clean, 6 active+undersized, 42 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-03-16T10:50:25.266 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:24 smithi002 ceph-mon[94239]: pgmap v147: 65 pgs: 5 active+undersized+degraded, 8 peering, 4 stale+active+clean, 6 active+undersized, 42 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/267 objects degraded (8.240%) 2024-03-16T10:50:25.481 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 podman[96984]: 2024-03-16 10:50:25.160732972 +0000 UTC m=+0.574909891 container remove 31828a85e84bd481653b0c0590080b864873ecf25c6d3f807b104b248fe21866 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-activate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:50:25.482 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 podman[97047]: 2024-03-16 10:50:25.279124288 +0000 UTC m=+0.017799071 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:50:25.482 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 podman[97047]: 2024-03-16 10:50:25.395010197 +0000 UTC m=+0.133685004 container create ae5f63e37d7725bd75601956314b75ae0f8bdcb4f044466a0800396375e6b13e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 -- 172.21.15.2:0/2408842672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920103ec0 msgr2=0x7fc9201062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 --2- 172.21.15.2:0/2408842672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920103ec0 0x7fc9201062b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc910009910 tx=0x7fc910009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 -- 172.21.15.2:0/2408842672 shutdown_connections 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 --2- 172.21.15.2:0/2408842672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920103ec0 0x7fc9201062b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 --2- 172.21.15.2:0/2408842672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc920101590 0x7fc920103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 -- 172.21.15.2:0/2408842672 >> 172.21.15.2:0/2408842672 conn(0x7fc9200fafc0 msgr2=0x7fc9200fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 -- 172.21.15.2:0/2408842672 shutdown_connections 2024-03-16T10:50:25.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 -- 172.21.15.2:0/2408842672 wait complete. 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.506+0000 7fc92737a700 1 Processor -- start 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc92737a700 1 -- start start 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc92737a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920101590 0x7fc920100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc92737a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc920103ec0 0x7fc9200ff540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc92737a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200ffa80 con 0x7fc920101590 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc92737a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200ffbc0 con 0x7fc920103ec0 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc925116700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920101590 0x7fc920100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc925116700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920101590 0x7fc920100ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35284/0 (socket says 172.21.15.2:35284) 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc925116700 1 -- 172.21.15.2:0/3731578240 learned_addr learned my addr 172.21.15.2:0/3731578240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc924915700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc920103ec0 0x7fc9200ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc925116700 1 -- 172.21.15.2:0/3731578240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc920103ec0 msgr2=0x7fc9200ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:25.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc925116700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc920103ec0 0x7fc9200ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc925116700 1 -- 172.21.15.2:0/3731578240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100095f0 con 0x7fc920101590 2024-03-16T10:50:25.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc925116700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920101590 0x7fc920100ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00e920 tx=0x7fc91c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:25.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.507+0000 7fc9167fc700 1 -- 172.21.15.2:0/3731578240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc91c00c9d0 con 0x7fc920101590 2024-03-16T10:50:25.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.508+0000 7fc9167fc700 1 -- 172.21.15.2:0/3731578240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc91c010430 con 0x7fc920101590 2024-03-16T10:50:25.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.508+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9200ffea0 con 0x7fc920101590 2024-03-16T10:50:25.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.508+0000 7fc9167fc700 1 -- 172.21.15.2:0/3731578240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc91c018630 con 0x7fc920101590 2024-03-16T10:50:25.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.508+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92019e500 con 0x7fc920101590 2024-03-16T10:50:25.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.509+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92004ea80 con 0x7fc920101590 2024-03-16T10:50:25.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.509+0000 7fc9167fc700 1 -- 172.21.15.2:0/3731578240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc91c0105a0 con 0x7fc920101590 2024-03-16T10:50:25.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.509+0000 7fc9167fc700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc90c06dfa0 0x7fc90c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:25.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.509+0000 7fc9167fc700 1 -- 172.21.15.2:0/3731578240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc91c014070 con 0x7fc920101590 2024-03-16T10:50:25.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.510+0000 7fc924915700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc90c06dfa0 0x7fc90c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:25.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.510+0000 7fc924915700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc90c06dfa0 0x7fc90c070460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b6f0 tx=0x7fc910005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.514+0000 7fc9167fc700 1 -- 172.21.15.2:0/3731578240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc91c053740 con 0x7fc920101590 2024-03-16T10:50:25.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.737+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc920105da0 con 0x7fc90c06dfa0 2024-03-16T10:50:25.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.738+0000 7fc9167fc700 1 -- 172.21.15.2:0/3731578240 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc920105da0 con 0x7fc90c06dfa0 2024-03-16T10:50:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc90c06dfa0 msgr2=0x7fc90c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc90c06dfa0 0x7fc90c070460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b6f0 tx=0x7fc910005fb0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920101590 msgr2=0x7fc920100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:25.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920101590 0x7fc920100ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc91c00e920 tx=0x7fc91c00ec30 comp rx=0 tx=0).stop 2024-03-16T10:50:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 shutdown_connections 2024-03-16T10:50:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc90c06dfa0 0x7fc90c070460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc920101590 0x7fc920100ea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 --2- 172.21.15.2:0/3731578240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc920103ec0 0x7fc9200ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 >> 172.21.15.2:0/3731578240 conn(0x7fc9200fafc0 msgr2=0x7fc9200fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 shutdown_connections 2024-03-16T10:50:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.740+0000 7fc92737a700 1 -- 172.21.15.2:0/3731578240 wait complete. 2024-03-16T10:50:25.752 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:50:25.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 -- 172.21.15.2:0/2798954363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58101cd0 msgr2=0x7fab58102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:25.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 --2- 172.21.15.2:0/2798954363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58101cd0 0x7fab58102130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab44009910 tx=0x7fab44009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:25.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 -- 172.21.15.2:0/2798954363 shutdown_connections 2024-03-16T10:50:25.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 --2- 172.21.15.2:0/2798954363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58101cd0 0x7fab58102130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 --2- 172.21.15.2:0/2798954363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58100ad0 0x7fab58100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 -- 172.21.15.2:0/2798954363 >> 172.21.15.2:0/2798954363 conn(0x7fab580fc090 msgr2=0x7fab580fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:25.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 -- 172.21.15.2:0/2798954363 shutdown_connections 2024-03-16T10:50:25.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.854+0000 7fab5eb21700 1 -- 172.21.15.2:0/2798954363 wait complete. 2024-03-16T10:50:25.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5eb21700 1 Processor -- start 2024-03-16T10:50:25.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5eb21700 1 -- start start 2024-03-16T10:50:25.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5eb21700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58100ad0 0x7fab58196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:25.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5eb21700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58101cd0 0x7fab58196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:25.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5eb21700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab58197080 con 0x7fab58100ad0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5eb21700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab581971c0 con 0x7fab58101cd0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5c8bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58100ad0 0x7fab58196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5c8bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58100ad0 0x7fab58196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35310/0 (socket says 172.21.15.2:35310) 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab5c8bd700 1 -- 172.21.15.2:0/4144061816 learned_addr learned my addr 172.21.15.2:0/4144061816 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.855+0000 7fab57fff700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58101cd0 0x7fab58196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab5c8bd700 1 -- 172.21.15.2:0/4144061816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58101cd0 msgr2=0x7fab58196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab5c8bd700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58101cd0 0x7fab58196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab5c8bd700 1 -- 172.21.15.2:0/4144061816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440095f0 con 0x7fab58100ad0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab5c8bd700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58100ad0 0x7fab58196520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00d900 tx=0x7fab4c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab55ffb700 1 -- 172.21.15.2:0/4144061816 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab4c00b920 con 0x7fab58100ad0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab55ffb700 1 -- 172.21.15.2:0/4144061816 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab4c010460 con 0x7fab58100ad0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab55ffb700 1 -- 172.21.15.2:0/4144061816 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab4c00f590 con 0x7fab58100ad0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.856+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5819bc20 con 0x7fab58100ad0 2024-03-16T10:50:25.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.857+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5819c170 con 0x7fab58100ad0 2024-03-16T10:50:25.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.857+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab58190650 con 0x7fab58100ad0 2024-03-16T10:50:25.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.858+0000 7fab55ffb700 1 -- 172.21.15.2:0/4144061816 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fab4c0105d0 con 0x7fab58100ad0 2024-03-16T10:50:25.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.858+0000 7fab55ffb700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fab4806dfa0 0x7fab48070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:25.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.859+0000 7fab55ffb700 1 -- 172.21.15.2:0/4144061816 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fab4c08da00 con 0x7fab58100ad0 2024-03-16T10:50:25.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.859+0000 7fab57fff700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fab4806dfa0 0x7fab48070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:25.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.859+0000 7fab57fff700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fab4806dfa0 0x7fab48070460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab44009fd0 tx=0x7fab44005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:25.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:25.864+0000 7fab55ffb700 1 -- 172.21.15.2:0/4144061816 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab4c05a9a0 con 0x7fab58100ad0 2024-03-16T10:50:25.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 podman[97047]: 2024-03-16 10:50:25.622578976 +0000 UTC m=+0.361253752 container init ae5f63e37d7725bd75601956314b75ae0f8bdcb4f044466a0800396375e6b13e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-16T10:50:25.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 podman[97047]: 2024-03-16 10:50:25.625571797 +0000 UTC m=+0.364246562 container start ae5f63e37d7725bd75601956314b75ae0f8bdcb4f044466a0800396375e6b13e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-16T10:50:25.982 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 ceph-osd[97066]: -- 172.21.15.26:0/3471205136 <== mon.1 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c27103680 con 0x560c270ccc00 2024-03-16T10:50:25.982 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 bash[97047]: ae5f63e37d7725bd75601956314b75ae0f8bdcb4f044466a0800396375e6b13e 2024-03-16T10:50:25.982 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:25 smithi026 systemd[1]: Started Ceph osd.4 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:50:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.087+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab580611f0 con 0x7fab4806dfa0 2024-03-16T10:50:26.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.088+0000 7fab55ffb700 1 -- 172.21.15.2:0/4144061816 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fab580611f0 con 0x7fab4806dfa0 2024-03-16T10:50:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fab4806dfa0 msgr2=0x7fab48070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fab4806dfa0 0x7fab48070460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab44009fd0 tx=0x7fab44005960 comp rx=0 tx=0).stop 2024-03-16T10:50:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58100ad0 msgr2=0x7fab58196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58100ad0 0x7fab58196520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00d900 tx=0x7fab4c00dc10 comp rx=0 tx=0).stop 2024-03-16T10:50:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 shutdown_connections 2024-03-16T10:50:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fab4806dfa0 0x7fab48070460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58100ad0 0x7fab58196520 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 --2- 172.21.15.2:0/4144061816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab58101cd0 0x7fab58196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 >> 172.21.15.2:0/4144061816 conn(0x7fab580fc090 msgr2=0x7fab58105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 shutdown_connections 2024-03-16T10:50:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.090+0000 7fab5eb21700 1 -- 172.21.15.2:0/4144061816 wait complete. 2024-03-16T10:50:26.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.197+0000 7f86992f2700 1 -- 172.21.15.2:0/1487257074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86940fedb0 msgr2=0x7f86940ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.197+0000 7f86992f2700 1 --2- 172.21.15.2:0/1487257074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86940fedb0 0x7f86940ff1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8684009910 tx=0x7f8684009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:26.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 -- 172.21.15.2:0/1487257074 shutdown_connections 2024-03-16T10:50:26.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 --2- 172.21.15.2:0/1487257074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694101d90 0x7f86941021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 --2- 172.21.15.2:0/1487257074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86940fedb0 0x7f86940ff1d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 -- 172.21.15.2:0/1487257074 >> 172.21.15.2:0/1487257074 conn(0x7f86940fa710 msgr2=0x7f86940fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 -- 172.21.15.2:0/1487257074 shutdown_connections 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 -- 172.21.15.2:0/1487257074 wait complete. 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 Processor -- start 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.198+0000 7f86992f2700 1 -- start start 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86992f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86940fedb0 0x7f8694196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86992f2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8694101d90 0x7f8694196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86992f2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941970a0 con 0x7f8694101d90 2024-03-16T10:50:26.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86992f2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941971e0 con 0x7f86940fedb0 2024-03-16T10:50:26.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86927fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8694101d90 0x7f8694196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86927fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8694101d90 0x7f8694196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35330/0 (socket says 172.21.15.2:35330) 2024-03-16T10:50:26.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86927fc700 1 -- 172.21.15.2:0/2834241215 learned_addr learned my addr 172.21.15.2:0/2834241215 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:26.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f8692ffd700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86940fedb0 0x7f8694196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86927fc700 1 -- 172.21.15.2:0/2834241215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86940fedb0 msgr2=0x7f8694196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86927fc700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86940fedb0 0x7f8694196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86927fc700 1 -- 172.21.15.2:0/2834241215 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840095f0 con 0x7f8694101d90 2024-03-16T10:50:26.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f8692ffd700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86940fedb0 0x7f8694196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:50:26.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86927fc700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8694101d90 0x7f8694196a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f868800e920 tx=0x7f868800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:26.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f867bfff700 1 -- 172.21.15.2:0/2834241215 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f868800c9d0 con 0x7f8694101d90 2024-03-16T10:50:26.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f867bfff700 1 -- 172.21.15.2:0/2834241215 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8688010430 con 0x7f8694101d90 2024-03-16T10:50:26.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869419bc40 con 0x7f8694101d90 2024-03-16T10:50:26.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.199+0000 7f867bfff700 1 -- 172.21.15.2:0/2834241215 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8688018630 con 0x7f8694101d90 2024-03-16T10:50:26.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.200+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869419c210 con 0x7f8694101d90 2024-03-16T10:50:26.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.201+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8694190670 con 0x7f8694101d90 2024-03-16T10:50:26.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.201+0000 7f867bfff700 1 -- 172.21.15.2:0/2834241215 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f868800cb30 con 0x7f8694101d90 2024-03-16T10:50:26.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.201+0000 7f867bfff700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f867c06dfb0 0x7f867c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.201+0000 7f867bfff700 1 -- 172.21.15.2:0/2834241215 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8688014070 con 0x7f8694101d90 2024-03-16T10:50:26.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.201+0000 7f8692ffd700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f867c06dfb0 0x7f867c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.202+0000 7f8692ffd700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f867c06dfb0 0x7f867c070470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f868400b800 tx=0x7f8684009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:26.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.206+0000 7f867bfff700 1 -- 172.21.15.2:0/2834241215 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8688053740 con 0x7f8694101d90 2024-03-16T10:50:26.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.427+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f86940611f0 con 0x7f867c06dfb0 2024-03-16T10:50:26.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.438+0000 7f867bfff700 1 -- 172.21.15.2:0/2834241215 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f86940611f0 con 0x7f867c06dfb0 2024-03-16T10:50:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:50:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (12m) 65s ago 13m 24.8M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:50:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (14m) 65s ago 14m 8238k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:50:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (13m) 31s ago 13m 8094k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:50:26.439 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (2m) 65s ago 14m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (2m) 31s ago 13m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (12m) 65s ago 13m 86.7M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (8m) 65s ago 8m 19.0M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (8m) 65s ago 8m 16.0M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (8m) 31s ago 8m 153M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (8m) 31s ago 8m 17.5M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (5m) 65s ago 15m 504M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (4m) 31s ago 13m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (4m) 65s ago 15m 46.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (3m) 31s ago 12m 38.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (14m) 65s ago 14m 16.4M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (13m) 31s ago 13m 15.7M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (2m) 65s ago 12m 81.8M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:50:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (106s) 65s ago 11m 85.2M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:50:26.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (71s) 65s ago 11m 13.5M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:50:26.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (34s) 31s ago 10m 12.7M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:50:26.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 starting - - - 2241M 2024-03-16T10:50:26.441 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (9m) 31s ago 9m 199M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:50:26.441 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (4m) 65s ago 13m 74.3M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f867c06dfb0 msgr2=0x7f867c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f867c06dfb0 0x7f867c070470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f868400b800 tx=0x7f8684009f70 comp rx=0 tx=0).stop 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8694101d90 msgr2=0x7f8694196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8694101d90 0x7f8694196a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f868800e920 tx=0x7f868800ec30 comp rx=0 tx=0).stop 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 shutdown_connections 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f867c06dfb0 0x7f867c070470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86940fedb0 0x7f8694196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 --2- 172.21.15.2:0/2834241215 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8694101d90 0x7f8694196a80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 >> 172.21.15.2:0/2834241215 conn(0x7f86940fa710 msgr2=0x7f8694105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:26.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 shutdown_connections 2024-03-16T10:50:26.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.441+0000 7f86992f2700 1 -- 172.21.15.2:0/2834241215 wait complete. 2024-03-16T10:50:26.481 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:26 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T10:50:26.163+0000 7fca823ae7c0 -1 Falling back to public interface 2024-03-16T10:50:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 -- 172.21.15.2:0/2355698160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 msgr2=0x7f2a44102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/2355698160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a44102140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009910 tx=0x7f2a38009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 -- 172.21.15.2:0/2355698160 shutdown_connections 2024-03-16T10:50:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/2355698160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a44102140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/2355698160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a44100ae0 0x7f2a44100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 -- 172.21.15.2:0/2355698160 >> 172.21.15.2:0/2355698160 conn(0x7f2a440fc0a0 msgr2=0x7f2a440fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:26.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 -- 172.21.15.2:0/2355698160 shutdown_connections 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.546+0000 7f2a4ae88700 1 -- 172.21.15.2:0/2355698160 wait complete. 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a4ae88700 1 Processor -- start 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a4ae88700 1 -- start start 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a4ae88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a44100ae0 0x7f2a44074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a4ae88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a440731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a4ae88700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440736e0 con 0x7f2a44101ce0 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a4ae88700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44073820 con 0x7f2a44100ae0 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a43fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a440731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a43fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a440731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35350/0 (socket says 172.21.15.2:35350) 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a43fff700 1 -- 172.21.15.2:0/3489647439 learned_addr learned my addr 172.21.15.2:0/3489647439 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a48c24700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a44100ae0 0x7f2a44074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a43fff700 1 -- 172.21.15.2:0/3489647439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a44100ae0 msgr2=0x7f2a44074ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a43fff700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a44100ae0 0x7f2a44074ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a43fff700 1 -- 172.21.15.2:0/3489647439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380095f0 con 0x7f2a44101ce0 2024-03-16T10:50:26.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a48c24700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a44100ae0 0x7f2a44074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:50:26.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.547+0000 7f2a43fff700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a440731a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006010 tx=0x7f2a3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:26.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.548+0000 7f2a41ffb700 1 -- 172.21.15.2:0/3489647439 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3801d070 con 0x7f2a44101ce0 2024-03-16T10:50:26.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.548+0000 7f2a41ffb700 1 -- 172.21.15.2:0/3489647439 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a3800f460 con 0x7f2a44101ce0 2024-03-16T10:50:26.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.548+0000 7f2a41ffb700 1 -- 172.21.15.2:0/3489647439 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a38021640 con 0x7f2a44101ce0 2024-03-16T10:50:26.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.548+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a44073aa0 con 0x7f2a44101ce0 2024-03-16T10:50:26.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.548+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a44073fc0 con 0x7f2a44101ce0 2024-03-16T10:50:26.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.549+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a44190680 con 0x7f2a44101ce0 2024-03-16T10:50:26.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.550+0000 7f2a41ffb700 1 -- 172.21.15.2:0/3489647439 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2a3800fab0 con 0x7f2a44101ce0 2024-03-16T10:50:26.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.550+0000 7f2a41ffb700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2a2c06dfa0 0x7f2a2c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.550+0000 7f2a41ffb700 1 -- 172.21.15.2:0/3489647439 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2a3808f0b0 con 0x7f2a44101ce0 2024-03-16T10:50:26.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.550+0000 7f2a48c24700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2a2c06dfa0 0x7f2a2c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.550+0000 7f2a48c24700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2a2c06dfa0 0x7f2a2c070460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005950 tx=0x7f2a340058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:26.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.555+0000 7f2a41ffb700 1 -- 172.21.15.2:0/3489647439 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a3805c050 con 0x7f2a44101ce0 2024-03-16T10:50:26.846 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:26 smithi026 ceph-mon[82600]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:26.847 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:26.847 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:26.847 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:26 smithi026 ceph-mon[82600]: pgmap v148: 65 pgs: 11 active+undersized+degraded, 8 peering, 1 stale+active+clean, 13 active+undersized, 32 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-03-16T10:50:26.847 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:50:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.849+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2a4404ea80 con 0x7f2a44101ce0 2024-03-16T10:50:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.850+0000 7f2a41ffb700 1 -- 172.21.15.2:0/3489647439 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f2a3805bbe0 con 0x7f2a44101ce0 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 1, 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:50:26.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:26.852 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:50:26.852 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 5, 2024-03-16T10:50:26.852 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 8 2024-03-16T10:50:26.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:50:26.852 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2a2c06dfa0 msgr2=0x7f2a2c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2a2c06dfa0 0x7f2a2c070460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005950 tx=0x7f2a340058e0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 msgr2=0x7f2a440731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a440731a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006010 tx=0x7f2a3800be20 comp rx=0 tx=0).stop 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 shutdown_connections 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2a2c06dfa0 0x7f2a2c070460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a44100ae0 0x7f2a44074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 --2- 172.21.15.2:0/3489647439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a44101ce0 0x7f2a440731a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 >> 172.21.15.2:0/3489647439 conn(0x7f2a440fc0a0 msgr2=0x7f2a441052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 shutdown_connections 2024-03-16T10:50:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.852+0000 7f2a4ae88700 1 -- 172.21.15.2:0/3489647439 wait complete. 2024-03-16T10:50:26.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.957+0000 7f8e635f2700 1 -- 172.21.15.2:0/2585213738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 msgr2=0x7f8e5c101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.957+0000 7f8e635f2700 1 --2- 172.21.15.2:0/2585213738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c101790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009910 tx=0x7f8e58009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:26.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.957+0000 7f8e635f2700 1 -- 172.21.15.2:0/2585213738 shutdown_connections 2024-03-16T10:50:26.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.957+0000 7f8e635f2700 1 --2- 172.21.15.2:0/2585213738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5c101cd0 0x7f8e5c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.957+0000 7f8e635f2700 1 --2- 172.21.15.2:0/2585213738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c101790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.957+0000 7f8e635f2700 1 -- 172.21.15.2:0/2585213738 >> 172.21.15.2:0/2585213738 conn(0x7f8e5c0faf80 msgr2=0x7f8e5c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:26.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 -- 172.21.15.2:0/2585213738 shutdown_connections 2024-03-16T10:50:26.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 -- 172.21.15.2:0/2585213738 wait complete. 2024-03-16T10:50:26.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 Processor -- start 2024-03-16T10:50:26.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 -- start start 2024-03-16T10:50:26.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5c101cd0 0x7f8e5c19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19d660 con 0x7f8e5c0ff360 2024-03-16T10:50:26.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e635f2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19d7a0 con 0x7f8e5c101cd0 2024-03-16T10:50:26.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.958+0000 7f8e6138e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e6138e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35368/0 (socket says 172.21.15.2:35368) 2024-03-16T10:50:26.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e6138e700 1 -- 172.21.15.2:0/237160332 learned_addr learned my addr 172.21.15.2:0/237160332 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:26.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e60b8d700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5c101cd0 0x7f8e5c19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e6138e700 1 -- 172.21.15.2:0/237160332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5c101cd0 msgr2=0x7f8e5c19d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:26.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e6138e700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5c101cd0 0x7f8e5c19d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:26.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e6138e700 1 -- 172.21.15.2:0/237160332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580095f0 con 0x7f8e5c0ff360 2024-03-16T10:50:26.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e6138e700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c19cb00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800bd70 tx=0x7f8e5800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:26.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e527fc700 1 -- 172.21.15.2:0/237160332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5801d070 con 0x7f8e5c0ff360 2024-03-16T10:50:26.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e527fc700 1 -- 172.21.15.2:0/237160332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e58005dc0 con 0x7f8e5c0ff360 2024-03-16T10:50:26.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c1a2150 con 0x7f8e5c0ff360 2024-03-16T10:50:26.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e527fc700 1 -- 172.21.15.2:0/237160332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e58021410 con 0x7f8e5c0ff360 2024-03-16T10:50:26.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.959+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c1a2640 con 0x7f8e5c0ff360 2024-03-16T10:50:26.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.960+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c04ea80 con 0x7f8e5c0ff360 2024-03-16T10:50:26.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.961+0000 7f8e527fc700 1 -- 172.21.15.2:0/237160332 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f8e5800f820 con 0x7f8e5c0ff360 2024-03-16T10:50:26.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.961+0000 7f8e527fc700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8e4806dfb0 0x7f8e48070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:26.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.961+0000 7f8e527fc700 1 -- 172.21.15.2:0/237160332 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8e5808efc0 con 0x7f8e5c0ff360 2024-03-16T10:50:26.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.961+0000 7f8e60b8d700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8e4806dfb0 0x7f8e48070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:26.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.962+0000 7f8e60b8d700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8e4806dfb0 0x7f8e48070470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c005fd0 tx=0x7f8e4c005f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:26.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:26.966+0000 7f8e527fc700 1 -- 172.21.15.2:0/237160332 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e5805bf60 con 0x7f8e5c0ff360 2024-03-16T10:50:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:26 smithi002 ceph-mon[94239]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:26 smithi002 ceph-mon[94239]: pgmap v148: 65 pgs: 11 active+undersized+degraded, 8 peering, 1 stale+active+clean, 13 active+undersized, 32 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-03-16T10:50:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:50:27.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.215+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e5c1a28b0 con 0x7f8e5c0ff360 2024-03-16T10:50:27.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.215+0000 7f8e527fc700 1 -- 172.21.15.2:0/237160332 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f8e5805baf0 con 0x7f8e5c0ff360 2024-03-16T10:50:27.217 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:50:27.217 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:50:27.217 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:50:27.217 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:50:27.217 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:27.217 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:50:27.218 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:50:27.218 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:50:27.218 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:50:27.218 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:50:27.218 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:50:27.219 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:50:27.219 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:50:27.219 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:50:27.219 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:50:27.219 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:50:27.219 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:50:27.220 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:50:27.220 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:50:27.220 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:50:27.220 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:50:27.220 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:50:27.220 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:50:27.221 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:50:27.221 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:50:27.221 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:50:27.221 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:50:27.221 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:50:27.221 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:50:27.222 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:50:27.222 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:50:27.222 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:50:27.222 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:50:27.222 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:50:27.222 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:27.223 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:27.223 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:27.223 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:50:27.223 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:27.223 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:27.223 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:27.224 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:27.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8e4806dfb0 msgr2=0x7f8e48070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8e4806dfb0 0x7f8e48070470 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c005fd0 tx=0x7f8e4c005f60 comp rx=0 tx=0).stop 2024-03-16T10:50:27.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 msgr2=0x7f8e5c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c19cb00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800bd70 tx=0x7f8e5800be50 comp rx=0 tx=0).stop 2024-03-16T10:50:27.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 shutdown_connections 2024-03-16T10:50:27.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8e4806dfb0 0x7f8e48070470 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e5c0ff360 0x7f8e5c19cb00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 --2- 172.21.15.2:0/237160332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5c101cd0 0x7f8e5c19d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 >> 172.21.15.2:0/237160332 conn(0x7f8e5c0faf80 msgr2=0x7f8e5c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:27.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 shutdown_connections 2024-03-16T10:50:27.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.217+0000 7f8e635f2700 1 -- 172.21.15.2:0/237160332 wait complete. 2024-03-16T10:50:27.227 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:50:27.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 -- 172.21.15.2:0/136744352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 msgr2=0x7f99f40ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 --2- 172.21.15.2:0/136744352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f40ffe00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009910 tx=0x7f99e4009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 -- 172.21.15.2:0/136744352 shutdown_connections 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 --2- 172.21.15.2:0/136744352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f4100be0 0x7f99f4101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 --2- 172.21.15.2:0/136744352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f40ffe00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 -- 172.21.15.2:0/136744352 >> 172.21.15.2:0/136744352 conn(0x7f99f40faf80 msgr2=0x7f99f40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 -- 172.21.15.2:0/136744352 shutdown_connections 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.325+0000 7f99fa24a700 1 -- 172.21.15.2:0/136744352 wait complete. 2024-03-16T10:50:27.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99fa24a700 1 Processor -- start 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99fa24a700 1 -- start start 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99fa24a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f4103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99fa24a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f4100be0 0x7f99f4101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99fa24a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f419a140 con 0x7f99f40ff9e0 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99fa24a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f419a280 con 0x7f99f4100be0 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99f37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f4103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99f37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f4103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35388/0 (socket says 172.21.15.2:35388) 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99f37fe700 1 -- 172.21.15.2:0/705323502 learned_addr learned my addr 172.21.15.2:0/705323502 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99f2ffd700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f4100be0 0x7f99f4101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99f37fe700 1 -- 172.21.15.2:0/705323502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f4100be0 msgr2=0x7f99f4101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99f37fe700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f4100be0 0x7f99f4101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.326+0000 7f99f37fe700 1 -- 172.21.15.2:0/705323502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e40095f0 con 0x7f99f40ff9e0 2024-03-16T10:50:27.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.327+0000 7f99f37fe700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f4103920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f99e40059d0 tx=0x7f99e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:27.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.327+0000 7f99f0ff9700 1 -- 172.21.15.2:0/705323502 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99e401d070 con 0x7f99f40ff9e0 2024-03-16T10:50:27.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.327+0000 7f99f0ff9700 1 -- 172.21.15.2:0/705323502 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99e400f460 con 0x7f99f40ff9e0 2024-03-16T10:50:27.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.327+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f4102500 con 0x7f99f40ff9e0 2024-03-16T10:50:27.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.327+0000 7f99f0ff9700 1 -- 172.21.15.2:0/705323502 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99e4021640 con 0x7f99f40ff9e0 2024-03-16T10:50:27.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.327+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f41029c0 con 0x7f99f40ff9e0 2024-03-16T10:50:27.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.328+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f404ea80 con 0x7f99f40ff9e0 2024-03-16T10:50:27.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.329+0000 7f99f0ff9700 1 -- 172.21.15.2:0/705323502 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f99e400f5d0 con 0x7f99f40ff9e0 2024-03-16T10:50:27.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.329+0000 7f99f0ff9700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f99dc06dfa0 0x7f99dc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:27.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.329+0000 7f99f0ff9700 1 -- 172.21.15.2:0/705323502 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f99e408eeb0 con 0x7f99f40ff9e0 2024-03-16T10:50:27.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.329+0000 7f99f2ffd700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f99dc06dfa0 0x7f99dc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:27.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.329+0000 7f99f2ffd700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f99dc06dfa0 0x7f99dc070460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007c60 tx=0x7f99e80073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:27.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.334+0000 7f99f0ff9700 1 -- 172.21.15.2:0/705323502 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99e405bed0 con 0x7f99f40ff9e0 2024-03-16T10:50:27.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.556+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99f4069770 con 0x7f99dc06dfa0 2024-03-16T10:50:27.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.557+0000 7f99f0ff9700 1 -- 172.21.15.2:0/705323502 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f99f4069770 con 0x7f99dc06dfa0 2024-03-16T10:50:27.558 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:50:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:50:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:50:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:50:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:50:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:50:27.559 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:50:27.559 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:50:27.559 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:50:27.559 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "10/23 daemons upgraded", 2024-03-16T10:50:27.559 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T10:50:27.559 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:50:27.559 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:50:27.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f99dc06dfa0 msgr2=0x7f99dc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f99dc06dfa0 0x7f99dc070460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99e8007c60 tx=0x7f99e80073d0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 msgr2=0x7f99f4103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f4103920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f99e40059d0 tx=0x7f99e400bd70 comp rx=0 tx=0).stop 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 shutdown_connections 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f99dc06dfa0 0x7f99dc070460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99f40ff9e0 0x7f99f4103920 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 --2- 172.21.15.2:0/705323502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99f4100be0 0x7f99f4101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 >> 172.21.15.2:0/705323502 conn(0x7f99f40faf80 msgr2=0x7f99f4069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 shutdown_connections 2024-03-16T10:50:27.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.560+0000 7f99fa24a700 1 -- 172.21.15.2:0/705323502 wait complete. 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 -- 172.21.15.2:0/3945405007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f4100bd0 msgr2=0x7ff8f4101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/3945405007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f4100bd0 0x7ff8f4101030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009910 tx=0x7ff8f0009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 -- 172.21.15.2:0/3945405007 shutdown_connections 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/3945405007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f4100bd0 0x7ff8f4101030 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/3945405007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f40ff9d0 0x7ff8f40ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 -- 172.21.15.2:0/3945405007 >> 172.21.15.2:0/3945405007 conn(0x7ff8f40faf70 msgr2=0x7ff8f40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 -- 172.21.15.2:0/3945405007 shutdown_connections 2024-03-16T10:50:27.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.666+0000 7ff8fbdba700 1 -- 172.21.15.2:0/3945405007 wait complete. 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8fbdba700 1 Processor -- start 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8fbdba700 1 -- start start 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8fbdba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f40ff9d0 0x7ff8f41953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8fbdba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f4100bd0 0x7ff8f4195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8fbdba700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4195f40 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8fbdba700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4196080 con 0x7ff8f4100bd0 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8f9b56700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f40ff9d0 0x7ff8f41953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8f9b56700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f40ff9d0 0x7ff8f41953e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35412/0 (socket says 172.21.15.2:35412) 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8f9b56700 1 -- 172.21.15.2:0/890018531 learned_addr learned my addr 172.21.15.2:0/890018531 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8f9355700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f4100bd0 0x7ff8f4195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:27.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8f9b56700 1 -- 172.21.15.2:0/890018531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f4100bd0 msgr2=0x7ff8f4195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8f9b56700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f4100bd0 0x7ff8f4195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.667+0000 7ff8f9b56700 1 -- 172.21.15.2:0/890018531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00095f0 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.668+0000 7ff8f9b56700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f40ff9d0 0x7ff8f41953e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80098f0 tx=0x7ff8e8009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.668+0000 7ff8e6ffd700 1 -- 172.21.15.2:0/890018531 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e8012070 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.668+0000 7ff8e6ffd700 1 -- 172.21.15.2:0/890018531 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8e80044e0 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.668+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f4068d70 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.668+0000 7ff8e6ffd700 1 -- 172.21.15.2:0/890018531 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8e8007630 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.668+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f4069340 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.669+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f404ea80 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.669+0000 7ff8e6ffd700 1 -- 172.21.15.2:0/890018531 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ff8e8016030 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.670+0000 7ff8e6ffd700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff8e006dfa0 0x7ff8e0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:27.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.670+0000 7ff8e6ffd700 1 -- 172.21.15.2:0/890018531 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff8e808f0f0 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.670+0000 7ff8f9355700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff8e006dfa0 0x7ff8e0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:27.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.670+0000 7ff8f9355700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff8e006dfa0 0x7ff8e0070460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00051c0 tx=0x7ff8f0005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:27.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.674+0000 7ff8e6ffd700 1 -- 172.21.15.2:0/890018531 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8e805c110 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.968+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff8f418ef30 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.969 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:27 smithi002 ceph-mon[94239]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:27.969 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:27 smithi002 ceph-mon[94239]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:27.970 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:27 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3489647439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:27.970 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:27 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/237160332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:50:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.968+0000 7ff8e6ffd700 1 -- 172.21.15.2:0/890018531 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1258 (secure 0 0 0) 0x7ff8e805bca0 con 0x7ff8f40ff9d0 2024-03-16T10:50:27.970 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 1 pg peering; Degraded data redundancy: 40/267 objects degraded (14.981%), 11 pgs degraded 2024-03-16T10:50:27.971 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:50:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:50:27.971 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-16T10:50:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: osd.4 (root=default,host=smithi026) is down 2024-03-16T10:50:27.971 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 1 pg peering 2024-03-16T10:50:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: pg 3.1a is stuck peering for 3m, current state peering, last acting [1,2] 2024-03-16T10:50:27.972 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 40/267 objects degraded (14.981%), 11 pgs degraded 2024-03-16T10:50:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-03-16T10:50:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-03-16T10:50:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-03-16T10:50:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-03-16T10:50:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-03-16T10:50:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-03-16T10:50:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-03-16T10:50:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-03-16T10:50:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-03-16T10:50:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-03-16T10:50:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.970+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff8e006dfa0 msgr2=0x7ff8e0070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff8e006dfa0 0x7ff8e0070460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00051c0 tx=0x7ff8f0005a90 comp rx=0 tx=0).stop 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f40ff9d0 msgr2=0x7ff8f41953e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f40ff9d0 0x7ff8f41953e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80098f0 tx=0x7ff8e8009cb0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 shutdown_connections 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff8e006dfa0 0x7ff8e0070460 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8f40ff9d0 0x7ff8f41953e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 --2- 172.21.15.2:0/890018531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f4100bd0 0x7ff8f4195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:27.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 >> 172.21.15.2:0/890018531 conn(0x7ff8f40faf70 msgr2=0x7ff8f4104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:27.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 shutdown_connections 2024-03-16T10:50:27.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:27.971+0000 7ff8fbdba700 1 -- 172.21.15.2:0/890018531 wait complete. 2024-03-16T10:50:28.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:27 smithi026 ceph-mon[82600]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:28.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:27 smithi026 ceph-mon[82600]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:28.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:27 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3489647439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:28.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:27 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/237160332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:50:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:28 smithi002 ceph-mon[94239]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:29.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:28 smithi002 ceph-mon[94239]: pgmap v149: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-03-16T10:50:29.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:28 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/890018531' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:50:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:28 smithi026 ceph-mon[82600]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:50:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:28 smithi026 ceph-mon[82600]: pgmap v149: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-03-16T10:50:29.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:28 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/890018531' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:50:30.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:29 smithi002 ceph-mon[94239]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:50:30.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:29 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:30.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:29 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:29 smithi026 ceph-mon[82600]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-16T10:50:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:29 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:29 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:31.044 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:30 smithi026 ceph-mon[82600]: pgmap v150: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-03-16T10:50:31.044 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:30 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:31.045 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:30 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:31.045 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:30 smithi026 ceph-mon[82600]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:30 smithi002 ceph-mon[94239]: pgmap v150: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-03-16T10:50:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:30 smithi002 ceph-mon[94239]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:31.310 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:31 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T10:50:31.043+0000 7fca823ae7c0 -1 osd.4 71 log_to_monitors true 2024-03-16T10:50:32.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:31 smithi002 ceph-mon[94239]: from='osd.4 [v2:172.21.15.26:6808/2367920652,v1:172.21.15.26:6809/2367920652]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:50:32.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:31 smithi002 ceph-mon[94239]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:50:32.115 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:31 smithi026 ceph-mon[82600]: from='osd.4 [v2:172.21.15.26:6808/2367920652,v1:172.21.15.26:6809/2367920652]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:50:32.115 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:31 smithi026 ceph-mon[82600]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T10:50:32.115 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 10:50:31 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T10:50:31.812+0000 7fca73806700 -1 osd.4 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:50:33.103 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:32 smithi026 ceph-mon[82600]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-03-16T10:50:33.103 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:32 smithi026 ceph-mon[82600]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T10:50:33.104 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:32 smithi026 ceph-mon[82600]: osdmap e74: 6 total, 5 up, 6 in 2024-03-16T10:50:33.104 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:32 smithi026 ceph-mon[82600]: from='osd.4 [v2:172.21.15.26:6808/2367920652,v1:172.21.15.26:6809/2367920652]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:50:33.104 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:32 smithi026 ceph-mon[82600]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:50:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:32 smithi002 ceph-mon[94239]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-03-16T10:50:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:32 smithi002 ceph-mon[94239]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T10:50:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:32 smithi002 ceph-mon[94239]: osdmap e74: 6 total, 5 up, 6 in 2024-03-16T10:50:33.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:32 smithi002 ceph-mon[94239]: from='osd.4 [v2:172.21.15.26:6808/2367920652,v1:172.21.15.26:6809/2367920652]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:50:33.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:32 smithi002 ceph-mon[94239]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:50:34.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:33 smithi026 ceph-mon[82600]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:50:34.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:33 smithi026 ceph-mon[82600]: osd.4 [v2:172.21.15.26:6808/2367920652,v1:172.21.15.26:6809/2367920652] boot 2024-03-16T10:50:34.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:33 smithi026 ceph-mon[82600]: osdmap e75: 6 total, 6 up, 6 in 2024-03-16T10:50:34.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:33 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:50:34.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:33 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:34.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:33 smithi026 ceph-mon[82600]: osdmap e76: 6 total, 6 up, 6 in 2024-03-16T10:50:34.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:33 smithi002 ceph-mon[94239]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:50:34.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:33 smithi002 ceph-mon[94239]: osd.4 [v2:172.21.15.26:6808/2367920652,v1:172.21.15.26:6809/2367920652] boot 2024-03-16T10:50:34.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:33 smithi002 ceph-mon[94239]: osdmap e75: 6 total, 6 up, 6 in 2024-03-16T10:50:34.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:33 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T10:50:34.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:33 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:34.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:33 smithi002 ceph-mon[94239]: osdmap e76: 6 total, 6 up, 6 in 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: pgmap v154: 65 pgs: 10 peering, 10 active+undersized+degraded, 14 active+undersized, 31 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25/267 objects degraded (9.363%) 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:35.200 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:35.201 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:35.201 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:35.201 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:34 smithi002 ceph-mon[94239]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: pgmap v154: 65 pgs: 10 peering, 10 active+undersized+degraded, 14 active+undersized, 31 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 25/267 objects degraded (9.363%) 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:35.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:34 smithi026 ceph-mon[82600]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-16T10:50:36.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:36.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:36.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:35 smithi026 ceph-mon[82600]: Health check update: Degraded data redundancy: 25/267 objects degraded (9.363%), 10 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:36.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:36.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:36.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:35 smithi002 ceph-mon[94239]: Health check update: Degraded data redundancy: 25/267 objects degraded (9.363%), 10 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:37.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:36 smithi026 ceph-mon[82600]: pgmap v156: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12/267 objects degraded (4.494%) 2024-03-16T10:50:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:36 smithi002 ceph-mon[94239]: pgmap v156: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12/267 objects degraded (4.494%) 2024-03-16T10:50:38.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:37 smithi026 ceph-mon[82600]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/267 objects degraded (4.494%), 6 pgs degraded) 2024-03-16T10:50:38.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:37 smithi002 ceph-mon[94239]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/267 objects degraded (4.494%), 6 pgs degraded) 2024-03-16T10:50:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:38 smithi026 ceph-mon[82600]: pgmap v157: 65 pgs: 10 peering, 55 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:38 smithi002 ceph-mon[94239]: pgmap v157: 65 pgs: 10 peering, 55 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:41.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:40 smithi026 ceph-mon[82600]: pgmap v158: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:40 smithi002 ceph-mon[94239]: pgmap v158: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:43.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:42 smithi026 ceph-mon[82600]: pgmap v159: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:42 smithi002 ceph-mon[94239]: pgmap v159: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:44 smithi026 ceph-mon[82600]: pgmap v160: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:44 smithi002 ceph-mon[94239]: pgmap v160: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:47.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:46 smithi026 ceph-mon[82600]: pgmap v161: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:46 smithi002 ceph-mon[94239]: pgmap v161: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:48 smithi026 ceph-mon[82600]: pgmap v162: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:48 smithi002 ceph-mon[94239]: pgmap v162: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:50.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:49 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:50.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:49 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:50.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:49 smithi026 ceph-mon[82600]: Upgrade: osd.5 is safe to restart 2024-03-16T10:50:50.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:49 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:50.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:49 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:49 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:50.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:49 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-16T10:50:50.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:49 smithi002 ceph-mon[94239]: Upgrade: osd.5 is safe to restart 2024-03-16T10:50:50.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:49 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:50.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:49 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:50:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:50 smithi026 ceph-mon[82600]: pgmap v163: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:50:51.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:50 smithi002 ceph-mon[94239]: pgmap v163: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:50:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T10:50:51.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:50:52.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:51 smithi026 ceph-mon[82600]: Upgrade: Updating osd.5 2024-03-16T10:50:52.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:51 smithi026 ceph-mon[82600]: Deploying daemon osd.5 on smithi026 2024-03-16T10:50:52.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:51 smithi002 ceph-mon[94239]: Upgrade: Updating osd.5 2024-03-16T10:50:52.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:51 smithi002 ceph-mon[94239]: Deploying daemon osd.5 on smithi026 2024-03-16T10:50:53.110 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:52 smithi026 ceph-mon[82600]: pgmap v164: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:52 smithi002 ceph-mon[94239]: pgmap v164: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:54.418 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:54 smithi026 systemd[1]: Stopping Ceph osd.5 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:50:54.731 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:54 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[56653]: 2024-03-16T10:50:54.516+0000 7fe99e587700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T10:50:54.732 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:54 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[56653]: 2024-03-16T10:50:54.516+0000 7fe99e587700 -1 osd.5 76 *** Got signal Terminated *** 2024-03-16T10:50:54.732 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:54 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[56653]: 2024-03-16T10:50:54.516+0000 7fe99e587700 -1 osd.5 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T10:50:55.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:54 smithi026 ceph-mon[82600]: pgmap v165: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:55.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:54 smithi026 ceph-mon[82600]: osd.5 marked itself down and dead 2024-03-16T10:50:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:54 smithi002 ceph-mon[94239]: pgmap v165: 65 pgs: 65 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:54 smithi002 ceph-mon[94239]: osd.5 marked itself down and dead 2024-03-16T10:50:55.659 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:55 smithi026 podman[101784]: 2024-03-16 10:50:55.35550487 +0000 UTC m=+0.968343389 container died 50b8b62eb70151422034e08ab1b1500d346e860441c3bf6868e86e121f529f0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=reef-a752c2f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T10:50:55.920 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:55 smithi026 podman[101784]: 2024-03-16 10:50:55.780447246 +0000 UTC m=+1.393285762 container cleanup 50b8b62eb70151422034e08ab1b1500d346e860441c3bf6868e86e121f529f0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=reef-a752c2f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-16T10:50:55.920 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:55 smithi026 bash[101784]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5 2024-03-16T10:50:56.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:55 smithi026 podman[101803]: 2024-03-16 10:50:55.976275238 +0000 UTC m=+0.613459559 container remove 50b8b62eb70151422034e08ab1b1500d346e860441c3bf6868e86e121f529f0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d9aa4dc425b2cbb995ed46bd336bb1bfa20e568f5dfd1f8fbc9944eacb1ef704, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=reef-a752c2f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True) 2024-03-16T10:50:56.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:55 smithi026 ceph-mon[82600]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:50:56.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:55 smithi026 ceph-mon[82600]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-16T10:50:56.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:55 smithi026 ceph-mon[82600]: osdmap e77: 6 total, 5 up, 6 in 2024-03-16T10:50:56.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:55 smithi026 ceph-mon[82600]: osdmap e78: 6 total, 5 up, 6 in 2024-03-16T10:50:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:55 smithi002 ceph-mon[94239]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-16T10:50:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:55 smithi002 ceph-mon[94239]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-16T10:50:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:55 smithi002 ceph-mon[94239]: osdmap e77: 6 total, 5 up, 6 in 2024-03-16T10:50:56.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:55 smithi002 ceph-mon[94239]: osdmap e78: 6 total, 5 up, 6 in 2024-03-16T10:50:56.729 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:56 smithi026 podman[101892]: 2024-03-16 10:50:56.302983191 +0000 UTC m=+0.018539561 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:50:56.730 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:56 smithi026 podman[101892]: 2024-03-16 10:50:56.410665214 +0000 UTC m=+0.126221572 container create 6eb82a83f302b5cd5172260b3c7a5190db5e70bed966a659e83640f3f971c8d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-16T10:50:56.730 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:56 smithi026 podman[101892]: 2024-03-16 10:50:56.638234287 +0000 UTC m=+0.353790643 container init 6eb82a83f302b5cd5172260b3c7a5190db5e70bed966a659e83640f3f971c8d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-16T10:50:56.730 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:56 smithi026 podman[101892]: 2024-03-16 10:50:56.641238215 +0000 UTC m=+0.356794574 container start 6eb82a83f302b5cd5172260b3c7a5190db5e70bed966a659e83640f3f971c8d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T10:50:56.730 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:56 smithi026 podman[101892]: 2024-03-16 10:50:56.729813212 +0000 UTC m=+0.445369577 container attach 6eb82a83f302b5cd5172260b3c7a5190db5e70bed966a659e83640f3f971c8d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T10:50:56.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:56 smithi026 ceph-mon[82600]: pgmap v167: 65 pgs: 13 stale+active+clean, 52 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:56.982 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:56 smithi026 podman[101892]: 2024-03-16 10:50:56.941002912 +0000 UTC m=+0.656559261 container died 6eb82a83f302b5cd5172260b3c7a5190db5e70bed966a659e83640f3f971c8d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:50:57.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:56 smithi002 ceph-mon[94239]: pgmap v167: 65 pgs: 13 stale+active+clean, 52 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-03-16T10:50:57.493 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:57 smithi026 podman[101919]: 2024-03-16 10:50:57.483430325 +0000 UTC m=+0.533155381 container remove 6eb82a83f302b5cd5172260b3c7a5190db5e70bed966a659e83640f3f971c8d7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T10:50:57.910 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:57 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-16T10:50:57.910 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:57 smithi026 systemd[1]: Stopped Ceph osd.5 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:50:57.910 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:57 smithi026 systemd[1]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.5.service: Consumed 53.090s CPU time. 2024-03-16T10:50:57.910 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:57 smithi026 systemd[1]: Starting Ceph osd.5 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 -- 172.21.15.2:0/1809229287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8100b00 msgr2=0x7faca8100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 --2- 172.21.15.2:0/1809229287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8100b00 0x7faca8100f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fac98009910 tx=0x7fac98009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 -- 172.21.15.2:0/1809229287 shutdown_connections 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 --2- 172.21.15.2:0/1809229287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca8101d00 0x7faca8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 --2- 172.21.15.2:0/1809229287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8100b00 0x7faca8100f20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 -- 172.21.15.2:0/1809229287 >> 172.21.15.2:0/1809229287 conn(0x7faca80fc0c0 msgr2=0x7faca80fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 -- 172.21.15.2:0/1809229287 shutdown_connections 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 -- 172.21.15.2:0/1809229287 wait complete. 2024-03-16T10:50:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.091+0000 7facad03a700 1 Processor -- start 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7facad03a700 1 -- start start 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7facad03a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca8100b00 0x7faca8196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7facad03a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8101d00 0x7faca8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7facad03a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8197070 con 0x7faca8101d00 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7facad03a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca81971b0 con 0x7faca8100b00 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7faca659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8101d00 0x7faca8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7faca659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8101d00 0x7faca8196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36198/0 (socket says 172.21.15.2:36198) 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7faca6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca8100b00 0x7faca8196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7faca659c700 1 -- 172.21.15.2:0/2231424459 learned_addr learned my addr 172.21.15.2:0/2231424459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7faca659c700 1 -- 172.21.15.2:0/2231424459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca8100b00 msgr2=0x7faca8196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7faca659c700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca8100b00 0x7faca8196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.092+0000 7faca659c700 1 -- 172.21.15.2:0/2231424459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980095f0 con 0x7faca8101d00 2024-03-16T10:50:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.093+0000 7faca659c700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8101d00 0x7faca8196a50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00d8d0 tx=0x7fac9c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.093+0000 7fac8ffff700 1 -- 172.21.15.2:0/2231424459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c0098a0 con 0x7faca8101d00 2024-03-16T10:50:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.093+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca819bbc0 con 0x7faca8101d00 2024-03-16T10:50:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.093+0000 7fac8ffff700 1 -- 172.21.15.2:0/2231424459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac9c010460 con 0x7faca8101d00 2024-03-16T10:50:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.093+0000 7fac8ffff700 1 -- 172.21.15.2:0/2231424459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac9c00f5f0 con 0x7faca8101d00 2024-03-16T10:50:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.093+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca819c190 con 0x7faca8101d00 2024-03-16T10:50:58.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.094+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8190680 con 0x7faca8101d00 2024-03-16T10:50:58.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.095+0000 7fac8ffff700 1 -- 172.21.15.2:0/2231424459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fac9c009a00 con 0x7faca8101d00 2024-03-16T10:50:58.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.095+0000 7fac8ffff700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fac9006dfa0 0x7fac90070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.095+0000 7fac8ffff700 1 -- 172.21.15.2:0/2231424459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fac9c08dc00 con 0x7faca8101d00 2024-03-16T10:50:58.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.095+0000 7faca6d9d700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fac9006dfa0 0x7fac90070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.096+0000 7faca6d9d700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fac9006dfa0 0x7fac90070460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac98005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:58.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.100+0000 7fac8ffff700 1 -- 172.21.15.2:0/2231424459 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac9c093020 con 0x7faca8101d00 2024-03-16T10:50:58.220 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:57 smithi026 ceph-mon[82600]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-16T10:50:58.220 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:57 smithi026 ceph-mon[82600]: Health check failed: Degraded data redundancy: 14/267 objects degraded (5.243%), 5 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:58.220 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:57 smithi026 podman[102021]: 2024-03-16 10:50:57.909338438 +0000 UTC m=+0.104405559 container create 2093e6716b8fdac99d2f55065e1ebef7070d61215ad8f6e53c5550cd67b6da1c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:50:58.220 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:57 smithi026 podman[102021]: 2024-03-16 10:50:57.823178729 +0000 UTC m=+0.018245808 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:50:58.220 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 podman[102021]: 2024-03-16 10:50:58.146091093 +0000 UTC m=+0.341158188 container init 2093e6716b8fdac99d2f55065e1ebef7070d61215ad8f6e53c5550cd67b6da1c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True) 2024-03-16T10:50:58.221 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 podman[102021]: 2024-03-16 10:50:58.149427092 +0000 UTC m=+0.344494174 container start 2093e6716b8fdac99d2f55065e1ebef7070d61215ad8f6e53c5550cd67b6da1c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:50:58.221 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 podman[102021]: 2024-03-16 10:50:58.220361118 +0000 UTC m=+0.415428196 container attach 2093e6716b8fdac99d2f55065e1ebef7070d61215ad8f6e53c5550cd67b6da1c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T10:50:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:57 smithi002 ceph-mon[94239]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-16T10:50:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:57 smithi002 ceph-mon[94239]: Health check failed: Degraded data redundancy: 14/267 objects degraded (5.243%), 5 pgs degraded (PG_DEGRADED) 2024-03-16T10:50:58.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.330+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faca80611f0 con 0x7fac9006dfa0 2024-03-16T10:50:58.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.331+0000 7fac8ffff700 1 -- 172.21.15.2:0/2231424459 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7faca80611f0 con 0x7fac9006dfa0 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fac9006dfa0 msgr2=0x7fac90070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fac9006dfa0 0x7fac90070460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac98000c00 tx=0x7fac98005960 comp rx=0 tx=0).stop 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8101d00 msgr2=0x7faca8196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8101d00 0x7faca8196a50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00d8d0 tx=0x7fac9c00dbe0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 shutdown_connections 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fac9006dfa0 0x7fac90070460 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca8100b00 0x7faca8196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 --2- 172.21.15.2:0/2231424459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faca8101d00 0x7faca8196a50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 >> 172.21.15.2:0/2231424459 conn(0x7faca80fc0c0 msgr2=0x7faca8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 shutdown_connections 2024-03-16T10:50:58.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.333+0000 7facad03a700 1 -- 172.21.15.2:0/2231424459 wait complete. 2024-03-16T10:50:58.345 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:50:58.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 -- 172.21.15.2:0/3300370586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9430100bd0 msgr2=0x7f9430101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3300370586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9430100bd0 0x7f9430101030 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9418009910 tx=0x7f9418009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:58.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 -- 172.21.15.2:0/3300370586 shutdown_connections 2024-03-16T10:50:58.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3300370586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9430100bd0 0x7f9430101030 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3300370586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94300ff9d0 0x7f94300ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 -- 172.21.15.2:0/3300370586 >> 172.21.15.2:0/3300370586 conn(0x7f94300faf70 msgr2=0x7f94300fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:58.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 -- 172.21.15.2:0/3300370586 shutdown_connections 2024-03-16T10:50:58.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.439+0000 7f9435fe9700 1 -- 172.21.15.2:0/3300370586 wait complete. 2024-03-16T10:50:58.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f9435fe9700 1 Processor -- start 2024-03-16T10:50:58.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f9435fe9700 1 -- start start 2024-03-16T10:50:58.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f9435fe9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94300ff9d0 0x7f94301953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f9435fe9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9430100bd0 0x7f9430195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f9435fe9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9430195f40 con 0x7f94300ff9d0 2024-03-16T10:50:58.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f9435fe9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9430196080 con 0x7f9430100bd0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94300ff9d0 0x7f94301953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94300ff9d0 0x7f94301953e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36212/0 (socket says 172.21.15.2:36212) 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942f7fe700 1 -- 172.21.15.2:0/3668553539 learned_addr learned my addr 172.21.15.2:0/3668553539 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942effd700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9430100bd0 0x7f9430195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942f7fe700 1 -- 172.21.15.2:0/3668553539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9430100bd0 msgr2=0x7f9430195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942f7fe700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9430100bd0 0x7f9430195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942f7fe700 1 -- 172.21.15.2:0/3668553539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180095f0 con 0x7f94300ff9d0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.440+0000 7f942effd700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9430100bd0 0x7f9430195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.441+0000 7f942f7fe700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94300ff9d0 0x7f94301953e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f942000efc0 tx=0x7f942000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.441+0000 7f942cff9700 1 -- 172.21.15.2:0/3668553539 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9420018070 con 0x7f94300ff9d0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.441+0000 7f942cff9700 1 -- 172.21.15.2:0/3668553539 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f942000f040 con 0x7f94300ff9d0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.441+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943019aa30 con 0x7f94300ff9d0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.441+0000 7f942cff9700 1 -- 172.21.15.2:0/3668553539 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94200044b0 con 0x7f94300ff9d0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.441+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9430068d10 con 0x7f94300ff9d0 2024-03-16T10:50:58.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.442+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943004ea80 con 0x7f94300ff9d0 2024-03-16T10:50:58.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.442+0000 7f942cff9700 1 -- 172.21.15.2:0/3668553539 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f94200046e0 con 0x7f94300ff9d0 2024-03-16T10:50:58.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.443+0000 7f942cff9700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f941c0701b0 0x7f941c072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.443+0000 7f942cff9700 1 -- 172.21.15.2:0/3668553539 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f942008de70 con 0x7f94300ff9d0 2024-03-16T10:50:58.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.443+0000 7f942effd700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f941c0701b0 0x7f941c072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.443+0000 7f942effd700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f941c0701b0 0x7f941c072670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f941800b800 tx=0x7f9418009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:58.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.448+0000 7f942cff9700 1 -- 172.21.15.2:0/3668553539 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f942005ae90 con 0x7f94300ff9d0 2024-03-16T10:50:58.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.672+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9430068f80 con 0x7f941c0701b0 2024-03-16T10:50:58.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.673+0000 7f942cff9700 1 -- 172.21.15.2:0/3668553539 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9430068f80 con 0x7f941c0701b0 2024-03-16T10:50:58.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.676+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f941c0701b0 msgr2=0x7f941c072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f941c0701b0 0x7f941c072670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f941800b800 tx=0x7f9418009f70 comp rx=0 tx=0).stop 2024-03-16T10:50:58.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94300ff9d0 msgr2=0x7f94301953e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94300ff9d0 0x7f94301953e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f942000efc0 tx=0x7f942000c3a0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 shutdown_connections 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f941c0701b0 0x7f941c072670 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94300ff9d0 0x7f94301953e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 --2- 172.21.15.2:0/3668553539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9430100bd0 0x7f9430195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 >> 172.21.15.2:0/3668553539 conn(0x7f94300faf70 msgr2=0x7f9430104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 shutdown_connections 2024-03-16T10:50:58.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.677+0000 7f9435fe9700 1 -- 172.21.15.2:0/3668553539 wait complete. 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.781+0000 7f54cb675700 1 -- 172.21.15.2:0/3575023205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4101d10 msgr2=0x7f54c4102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.781+0000 7f54cb675700 1 --2- 172.21.15.2:0/3575023205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4101d10 0x7f54c4102170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009910 tx=0x7f54b4009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 -- 172.21.15.2:0/3575023205 shutdown_connections 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 --2- 172.21.15.2:0/3575023205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4101d10 0x7f54c4102170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 --2- 172.21.15.2:0/3575023205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54c4100b10 0x7f54c4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 -- 172.21.15.2:0/3575023205 >> 172.21.15.2:0/3575023205 conn(0x7f54c40fc0d0 msgr2=0x7f54c40fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 -- 172.21.15.2:0/3575023205 shutdown_connections 2024-03-16T10:50:58.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 -- 172.21.15.2:0/3575023205 wait complete. 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 Processor -- start 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 -- start start 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4100b10 0x7f54c4074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54c4101d10 0x7f54c40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40736e0 con 0x7f54c4100b10 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.782+0000 7f54cb675700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c4073820 con 0x7f54c4101d10 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c9411700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4100b10 0x7f54c4074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c9411700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4100b10 0x7f54c4074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36230/0 (socket says 172.21.15.2:36230) 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c9411700 1 -- 172.21.15.2:0/863136735 learned_addr learned my addr 172.21.15.2:0/863136735 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c8c10700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54c4101d10 0x7f54c40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c9411700 1 -- 172.21.15.2:0/863136735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54c4101d10 msgr2=0x7f54c40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c9411700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54c4101d10 0x7f54c40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c9411700 1 -- 172.21.15.2:0/863136735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40095f0 con 0x7f54c4100b10 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54c9411700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4100b10 0x7f54c4074ab0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54c000d900 tx=0x7f54c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54ba7fc700 1 -- 172.21.15.2:0/863136735 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54c000b920 con 0x7f54c4100b10 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54ba7fc700 1 -- 172.21.15.2:0/863136735 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54c0010460 con 0x7f54c4100b10 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c4073b00 con 0x7f54c4100b10 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54ba7fc700 1 -- 172.21.15.2:0/863136735 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54c000f590 con 0x7f54c4100b10 2024-03-16T10:50:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.783+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c419e530 con 0x7f54c4100b10 2024-03-16T10:50:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.784+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c4190690 con 0x7f54c4100b10 2024-03-16T10:50:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.785+0000 7f54ba7fc700 1 -- 172.21.15.2:0/863136735 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f54c00105d0 con 0x7f54c4100b10 2024-03-16T10:50:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.785+0000 7f54ba7fc700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54b006dfa0 0x7f54b0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.785+0000 7f54c8c10700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54b006dfa0 0x7f54b0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.785+0000 7f54ba7fc700 1 -- 172.21.15.2:0/863136735 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f54c008dd30 con 0x7f54c4100b10 2024-03-16T10:50:58.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.786+0000 7f54c8c10700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54b006dfa0 0x7f54b0070460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009fd0 tx=0x7f54b4005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:58.790+0000 7f54ba7fc700 1 -- 172.21.15.2:0/863136735 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54c005acd0 con 0x7f54c4100b10 2024-03-16T10:50:58.933 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate[102035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T10:50:58.933 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate[102035]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T10:50:58.933 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 bash[102021]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 bash[102021]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 bash[102021]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 bash[102021]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 bash[102021]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 bash[102021]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 bash[102021]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate[102035]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate[102035]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate[102035]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate[102035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate[102035]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-16T10:50:58.934 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:58 smithi026 podman[102021]: 2024-03-16 10:50:58.624885814 +0000 UTC m=+0.819952886 container died 2093e6716b8fdac99d2f55065e1ebef7070d61215ad8f6e53c5550cd67b6da1c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-16T10:50:59.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.006+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f54c402d060 con 0x7f54b006dfa0 2024-03-16T10:50:59.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.017+0000 7f54ba7fc700 1 -- 172.21.15.2:0/863136735 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f54c402d060 con 0x7f54b006dfa0 2024-03-16T10:50:59.019 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:50:59.019 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (13m) 97s ago 14m 24.8M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:50:59.019 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (14m) 97s ago 14m 8238k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:50:59.019 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (13m) 30s ago 13m 8334k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (3m) 97s ago 14m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (3m) 30s ago 13m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (13m) 97s ago 14m 86.7M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (8m) 97s ago 8m 19.0M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (9m) 97s ago 9m 16.0M - 18.2.2-878-ga752c2f7 f692adb9a265 de18d4498362 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (9m) 30s ago 9m 152M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (8m) 30s ago 8m 17.7M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (6m) 97s ago 16m 504M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (5m) 30s ago 13m 451M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (4m) 97s ago 16m 46.9M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (4m) 30s ago 13m 37.0M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (14m) 97s ago 14m 16.4M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (13m) 30s ago 13m 15.8M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (2m) 97s ago 12m 81.8M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:50:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (2m) 97s ago 12m 85.2M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:50:59.021 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (103s) 97s ago 11m 13.5M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:50:59.021 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (67s) 30s ago 11m 97.7M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:50:59.021 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (33s) 30s ago 10m 11.7M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:50:59.021 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (10m) 30s ago 10m 200M 2241M 18.2.2-878-ga752c2f7 f692adb9a265 50b8b62eb701 2024-03-16T10:50:59.021 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (5m) 97s ago 14m 74.3M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.019+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54b006dfa0 msgr2=0x7f54b0070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.019+0000 7f54cb675700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54b006dfa0 0x7f54b0070460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009fd0 tx=0x7f54b4005960 comp rx=0 tx=0).stop 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.019+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4100b10 msgr2=0x7f54c4074ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.019+0000 7f54cb675700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4100b10 0x7f54c4074ab0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f54c000d900 tx=0x7f54c000dc10 comp rx=0 tx=0).stop 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.020+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 shutdown_connections 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.020+0000 7f54cb675700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54b006dfa0 0x7f54b0070460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.020+0000 7f54cb675700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54c4100b10 0x7f54c4074ab0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.020+0000 7f54cb675700 1 --2- 172.21.15.2:0/863136735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54c4101d10 0x7f54c40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.020+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 >> 172.21.15.2:0/863136735 conn(0x7f54c40fc0d0 msgr2=0x7f54c4105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.020+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 shutdown_connections 2024-03-16T10:50:59.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.020+0000 7f54cb675700 1 -- 172.21.15.2:0/863136735 wait complete. 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 -- 172.21.15.2:0/1236906410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 msgr2=0x7f5dac101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 --2- 172.21.15.2:0/1236906410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009910 tx=0x7f5d94009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 -- 172.21.15.2:0/1236906410 shutdown_connections 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 --2- 172.21.15.2:0/1236906410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 --2- 172.21.15.2:0/1236906410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dac0ff9d0 0x7f5dac0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 -- 172.21.15.2:0/1236906410 >> 172.21.15.2:0/1236906410 conn(0x7f5dac0faf70 msgr2=0x7f5dac0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 -- 172.21.15.2:0/1236906410 shutdown_connections 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 -- 172.21.15.2:0/1236906410 wait complete. 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 Processor -- start 2024-03-16T10:50:59.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.124+0000 7f5db2832700 1 -- start start 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5db2832700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dac0ff9d0 0x7f5dac103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5db2832700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5db2832700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac1024f0 con 0x7f5dac100bd0 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5db2832700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac102630 con 0x7f5dac0ff9d0 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36256/0 (socket says 172.21.15.2:36256) 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dab7fe700 1 -- 172.21.15.2:0/2451338258 learned_addr learned my addr 172.21.15.2:0/2451338258 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dabfff700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dac0ff9d0 0x7f5dac103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dab7fe700 1 -- 172.21.15.2:0/2451338258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dac0ff9d0 msgr2=0x7f5dac103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dab7fe700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dac0ff9d0 0x7f5dac103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dab7fe700 1 -- 172.21.15.2:0/2451338258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940095f0 con 0x7f5dac100bd0 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5dab7fe700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5d940092e0 tx=0x7f5d9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5da97fa700 1 -- 172.21.15.2:0/2451338258 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d9401d070 con 0x7f5dac100bd0 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5da97fa700 1 -- 172.21.15.2:0/2451338258 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d9400f460 con 0x7f5dac100bd0 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dac1028b0 con 0x7f5dac100bd0 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5da97fa700 1 -- 172.21.15.2:0/2451338258 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d94021640 con 0x7f5dac100bd0 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.125+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac102e20 con 0x7f5dac100bd0 2024-03-16T10:50:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.126+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dac04ea80 con 0x7f5dac100bd0 2024-03-16T10:50:59.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.127+0000 7f5da97fa700 1 -- 172.21.15.2:0/2451338258 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f5d9400fad0 con 0x7f5dac100bd0 2024-03-16T10:50:59.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.127+0000 7f5da97fa700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d9806dfa0 0x7f5d98070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.127+0000 7f5da97fa700 1 -- 172.21.15.2:0/2451338258 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5d9408ec70 con 0x7f5dac100bd0 2024-03-16T10:50:59.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.127+0000 7f5dabfff700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d9806dfa0 0x7f5d98070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.128+0000 7f5dabfff700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d9806dfa0 0x7f5d98070460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c005950 tx=0x7f5d9c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:59.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.132+0000 7f5da97fa700 1 -- 172.21.15.2:0/2451338258 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d9405bd40 con 0x7f5dac100bd0 2024-03-16T10:50:59.216 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:58 smithi026 ceph-mon[82600]: pgmap v169: 65 pgs: 5 active+undersized+degraded, 5 peering, 5 stale+active+clean, 5 active+undersized, 45 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14/267 objects degraded (5.243%) 2024-03-16T10:50:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:58 smithi002 ceph-mon[94239]: pgmap v169: 65 pgs: 5 active+undersized+degraded, 5 peering, 5 stale+active+clean, 5 active+undersized, 45 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 14/267 objects degraded (5.243%) 2024-03-16T10:50:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.428+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5dac102e20 con 0x7f5dac100bd0 2024-03-16T10:50:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.429+0000 7f5da97fa700 1 -- 172.21.15.2:0/2451338258 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+728 (secure 0 0 0) 0x7f5d9405b8d0 con 0x7f5dac100bd0 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 5 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:50:59.431 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 4, 2024-03-16T10:50:59.432 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 9 2024-03-16T10:50:59.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:50:59.432 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:50:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d9806dfa0 msgr2=0x7f5d98070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d9806dfa0 0x7f5d98070460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c005950 tx=0x7f5d9c0058e0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 msgr2=0x7f5dac101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5d940092e0 tx=0x7f5d9400be20 comp rx=0 tx=0).stop 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 shutdown_connections 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d9806dfa0 0x7f5d98070460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dac0ff9d0 0x7f5dac103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 --2- 172.21.15.2:0/2451338258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5dac100bd0 0x7f5dac101fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 >> 172.21.15.2:0/2451338258 conn(0x7f5dac0faf70 msgr2=0x7f5dac069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 shutdown_connections 2024-03-16T10:50:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.431+0000 7f5db2832700 1 -- 172.21.15.2:0/2451338258 wait complete. 2024-03-16T10:50:59.481 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 podman[102172]: 2024-03-16 10:50:59.215994822 +0000 UTC m=+0.583159199 container remove 2093e6716b8fdac99d2f55065e1ebef7070d61215ad8f6e53c5550cd67b6da1c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-activate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T10:50:59.481 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 podman[102240]: 2024-03-16 10:50:59.334797334 +0000 UTC m=+0.018478848 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T10:50:59.482 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 podman[102240]: 2024-03-16 10:50:59.433355504 +0000 UTC m=+0.117037017 container create cf40ac20a9779aed7177f0cc5e3a26750e1dacc971da9236d55c5e8fb6aff802 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T10:50:59.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 -- 172.21.15.2:0/1949202905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8100ae0 msgr2=0x7f0ea8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 --2- 172.21.15.2:0/1949202905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8100ae0 0x7f0ea8100f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009910 tx=0x7f0e98009c20 comp rx=0 tx=0).stop 2024-03-16T10:50:59.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 -- 172.21.15.2:0/1949202905 shutdown_connections 2024-03-16T10:50:59.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 --2- 172.21.15.2:0/1949202905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8101ce0 0x7f0ea8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 --2- 172.21.15.2:0/1949202905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8100ae0 0x7f0ea8100f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 -- 172.21.15.2:0/1949202905 >> 172.21.15.2:0/1949202905 conn(0x7f0ea80fc0a0 msgr2=0x7f0ea80fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:59.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 -- 172.21.15.2:0/1949202905 shutdown_connections 2024-03-16T10:50:59.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.536+0000 7f0ead831700 1 -- 172.21.15.2:0/1949202905 wait complete. 2024-03-16T10:50:59.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ead831700 1 Processor -- start 2024-03-16T10:50:59.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ead831700 1 -- start start 2024-03-16T10:50:59.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ead831700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8100ae0 0x7f0ea8074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ead831700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8101ce0 0x7f0ea80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ead831700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80750b0 con 0x7f0ea8101ce0 2024-03-16T10:50:59.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ead831700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80736e0 con 0x7f0ea8100ae0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ea67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8101ce0 0x7f0ea80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ea67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8101ce0 0x7f0ea80731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36276/0 (socket says 172.21.15.2:36276) 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ea67fc700 1 -- 172.21.15.2:0/1598503516 learned_addr learned my addr 172.21.15.2:0/1598503516 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ea6ffd700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8100ae0 0x7f0ea8074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ea67fc700 1 -- 172.21.15.2:0/1598503516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8100ae0 msgr2=0x7f0ea8074a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ea67fc700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8100ae0 0x7f0ea8074a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.537+0000 7f0ea67fc700 1 -- 172.21.15.2:0/1598503516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980095f0 con 0x7f0ea8101ce0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.538+0000 7f0ea67fc700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8101ce0 0x7f0ea80731a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000d8d0 tx=0x7f0e9000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.538+0000 7f0eac82f700 1 -- 172.21.15.2:0/1598503516 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e900098a0 con 0x7f0ea8101ce0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.538+0000 7f0eac82f700 1 -- 172.21.15.2:0/1598503516 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e90010460 con 0x7f0ea8101ce0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.538+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea80739c0 con 0x7f0ea8101ce0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.538+0000 7f0eac82f700 1 -- 172.21.15.2:0/1598503516 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9000f5f0 con 0x7f0ea8101ce0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.538+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea8073f90 con 0x7f0ea8101ce0 2024-03-16T10:50:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.539+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea8190660 con 0x7f0ea8101ce0 2024-03-16T10:50:59.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.539+0000 7f0eac82f700 1 -- 172.21.15.2:0/1598503516 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0e90009a00 con 0x7f0ea8101ce0 2024-03-16T10:50:59.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.540+0000 7f0eac82f700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0e9406dfb0 0x7f0e94070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.540+0000 7f0eac82f700 1 -- 172.21.15.2:0/1598503516 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0e9008dcb0 con 0x7f0ea8101ce0 2024-03-16T10:50:59.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.540+0000 7f0ea6ffd700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0e9406dfb0 0x7f0e94070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.540+0000 7f0ea6ffd700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0e9406dfb0 0x7f0e94070470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005090 tx=0x7f0e9800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:59.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.544+0000 7f0eac82f700 1 -- 172.21.15.2:0/1598503516 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e9005abb0 con 0x7f0ea8101ce0 2024-03-16T10:50:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.802+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0ea804ea80 con 0x7f0ea8101ce0 2024-03-16T10:50:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.802+0000 7f0eac82f700 1 -- 172.21.15.2:0/1598503516 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f0e9005a740 con 0x7f0ea8101ce0 2024-03-16T10:50:59.821 INFO:teuthology.orchestra.run.smithi002.stdout:e17 2024-03-16T10:50:59.821 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:50:59.822 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:50:59.823 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:14492} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.2:6828/2260596709,v1:172.21.15.2:6829/2260596709] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:14504} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/1847480131,v1:172.21.15.2:6827/1847480131] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:59.824 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0e9406dfb0 msgr2=0x7f0e94070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0e9406dfb0 0x7f0e94070470 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005090 tx=0x7f0e9800b540 comp rx=0 tx=0).stop 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8101ce0 msgr2=0x7f0ea80731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8101ce0 0x7f0ea80731a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000d8d0 tx=0x7f0e9000dbe0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 shutdown_connections 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0e9406dfb0 0x7f0e94070470 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8100ae0 0x7f0ea8074a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 --2- 172.21.15.2:0/1598503516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ea8101ce0 0x7f0ea80731a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 >> 172.21.15.2:0/1598503516 conn(0x7f0ea80fc0a0 msgr2=0x7f0ea81052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 shutdown_connections 2024-03-16T10:50:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.805+0000 7f0ead831700 1 -- 172.21.15.2:0/1598503516 wait complete. 2024-03-16T10:50:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:50:59.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 -- 172.21.15.2:0/1631721772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c074d60 msgr2=0x7f273c0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 --2- 172.21.15.2:0/1631721772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c074d60 0x7f273c0731c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f273400b0a0 tx=0x7f273400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 -- 172.21.15.2:0/1631721772 shutdown_connections 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 --2- 172.21.15.2:0/1631721772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c073790 0x7f273c073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 --2- 172.21.15.2:0/1631721772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c074d60 0x7f273c0731c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 -- 172.21.15.2:0/1631721772 >> 172.21.15.2:0/1631721772 conn(0x7f273c0fc0a0 msgr2=0x7f273c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 -- 172.21.15.2:0/1631721772 shutdown_connections 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 -- 172.21.15.2:0/1631721772 wait complete. 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.915+0000 7f2741532700 1 Processor -- start 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f2741532700 1 -- start start 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f2741532700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c073790 0x7f273c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f2741532700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c074d60 0x7f273c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f2741532700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c199260 con 0x7f273c073790 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f2741532700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c1993a0 con 0x7f273c074d60 2024-03-16T10:50:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c073790 0x7f273c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c073790 0x7f273c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36294/0 (socket says 172.21.15.2:36294) 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273affd700 1 -- 172.21.15.2:0/1789476323 learned_addr learned my addr 172.21.15.2:0/1789476323 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273a7fc700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c074d60 0x7f273c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273affd700 1 -- 172.21.15.2:0/1789476323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c074d60 msgr2=0x7f273c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273affd700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c074d60 0x7f273c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273affd700 1 -- 172.21.15.2:0/1789476323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2734009d00 con 0x7f273c073790 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f273affd700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c073790 0x7f273c198700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2734007950 tx=0x7f2734003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.916+0000 7f2733fff700 1 -- 172.21.15.2:0/1789476323 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2734021070 con 0x7f273c073790 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.917+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c19dda0 con 0x7f273c073790 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.917+0000 7f2733fff700 1 -- 172.21.15.2:0/1789476323 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f273400ce60 con 0x7f273c073790 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.917+0000 7f2733fff700 1 -- 172.21.15.2:0/1789476323 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273400e040 con 0x7f273c073790 2024-03-16T10:50:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.917+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c19e230 con 0x7f273c073790 2024-03-16T10:50:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.918+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c192870 con 0x7f273c073790 2024-03-16T10:50:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.918+0000 7f2733fff700 1 -- 172.21.15.2:0/1789476323 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2734004250 con 0x7f273c073790 2024-03-16T10:50:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.918+0000 7f2733fff700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f272806dfb0 0x7f2728070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:50:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.918+0000 7f2733fff700 1 -- 172.21.15.2:0/1789476323 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f273408ee60 con 0x7f273c073790 2024-03-16T10:50:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.918+0000 7f273a7fc700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f272806dfb0 0x7f2728070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:50:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.919+0000 7f273a7fc700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f272806dfb0 0x7f2728070470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2724007c60 tx=0x7f27240073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:50:59.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:50:59.923+0000 7f2733fff700 1 -- 172.21.15.2:0/1789476323 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f273405be00 con 0x7f273c073790 2024-03-16T10:50:59.974 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 podman[102240]: 2024-03-16 10:50:59.636127866 +0000 UTC m=+0.319809393 container init cf40ac20a9779aed7177f0cc5e3a26750e1dacc971da9236d55c5e8fb6aff802 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T10:50:59.974 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 podman[102240]: 2024-03-16 10:50:59.638927785 +0000 UTC m=+0.322609298 container start cf40ac20a9779aed7177f0cc5e3a26750e1dacc971da9236d55c5e8fb6aff802 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-16T10:50:59.974 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-osd[102259]: -- 172.21.15.26:0/3858988142 <== mon.1 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd982ed680 con 0x55fd982b6c00 2024-03-16T10:50:59.975 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 bash[102240]: cf40ac20a9779aed7177f0cc5e3a26750e1dacc971da9236d55c5e8fb6aff802 2024-03-16T10:50:59.975 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:50:59 smithi026 systemd[1]: Started Ceph osd.5 for b3aed5e6-e380-11ee-95c9-87774f69a715. 2024-03-16T10:51:00.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.147+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f273c0611f0 con 0x7f272806dfb0 2024-03-16T10:51:00.149 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:00.149 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:00.149 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:00.150 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2451338258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:00.150 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:00.150 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:00.150 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:00.150 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:50:59 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1598503516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:51:00.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.149+0000 7f2733fff700 1 -- 172.21.15.2:0/1789476323 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f273c0611f0 con 0x7f272806dfb0 2024-03-16T10:51:00.150 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "11/23 daemons upgraded", 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading osd daemons", 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:51:00.151 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f272806dfb0 msgr2=0x7f2728070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f272806dfb0 0x7f2728070470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2724007c60 tx=0x7f27240073d0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c073790 msgr2=0x7f273c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c073790 0x7f273c198700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2734007950 tx=0x7f2734003c30 comp rx=0 tx=0).stop 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 shutdown_connections 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f272806dfb0 0x7f2728070470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f273c073790 0x7f273c198700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 --2- 172.21.15.2:0/1789476323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273c074d60 0x7f273c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 >> 172.21.15.2:0/1789476323 conn(0x7f273c0fc0a0 msgr2=0x7f273c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 shutdown_connections 2024-03-16T10:51:00.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.152+0000 7f2741532700 1 -- 172.21.15.2:0/1789476323 wait complete. 2024-03-16T10:51:00.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:51:00 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T10:51:00.189+0000 7f3eb084e7c0 -1 Falling back to public interface 2024-03-16T10:51:00.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:00.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:00.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:00.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2451338258' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:00.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:00.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:00.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:00.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:50:59 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1598503516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:51:00.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 -- 172.21.15.2:0/1845500499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 msgr2=0x7fed380ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 --2- 172.21.15.2:0/1845500499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed380ff210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fed34009910 tx=0x7fed34009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 -- 172.21.15.2:0/1845500499 shutdown_connections 2024-03-16T10:51:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 --2- 172.21.15.2:0/1845500499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed38101e70 0x7fed381022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 --2- 172.21.15.2:0/1845500499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed380ff210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 -- 172.21.15.2:0/1845500499 >> 172.21.15.2:0/1845500499 conn(0x7fed380fa730 msgr2=0x7fed380fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 -- 172.21.15.2:0/1845500499 shutdown_connections 2024-03-16T10:51:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 -- 172.21.15.2:0/1845500499 wait complete. 2024-03-16T10:51:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 Processor -- start 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.257+0000 7fed3fecf700 1 -- start start 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3fecf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed38196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3fecf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed38101e70 0x7fed38196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3fecf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381970e0 con 0x7fed380fedf0 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3fecf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38197220 con 0x7fed38101e70 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3dc6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed38196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3dc6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed38196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36310/0 (socket says 172.21.15.2:36310) 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3dc6b700 1 -- 172.21.15.2:0/3440132183 learned_addr learned my addr 172.21.15.2:0/3440132183 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3d46a700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed38101e70 0x7fed38196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3dc6b700 1 -- 172.21.15.2:0/3440132183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed38101e70 msgr2=0x7fed38196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3dc6b700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed38101e70 0x7fed38196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3dc6b700 1 -- 172.21.15.2:0/3440132183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed340095f0 con 0x7fed380fedf0 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3dc6b700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed38196580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed340092e0 tx=0x7fed3400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:00.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed2effd700 1 -- 172.21.15.2:0/3440132183 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed3401d070 con 0x7fed380fedf0 2024-03-16T10:51:00.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed2effd700 1 -- 172.21.15.2:0/3440132183 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed34005dc0 con 0x7fed380fedf0 2024-03-16T10:51:00.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3819bbd0 con 0x7fed380fedf0 2024-03-16T10:51:00.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.258+0000 7fed2effd700 1 -- 172.21.15.2:0/3440132183 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed34017400 con 0x7fed380fedf0 2024-03-16T10:51:00.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.259+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3819c0c0 con 0x7fed380fedf0 2024-03-16T10:51:00.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.259+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed38190690 con 0x7fed380fedf0 2024-03-16T10:51:00.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.260+0000 7fed2effd700 1 -- 172.21.15.2:0/3440132183 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fed3400f6f0 con 0x7fed380fedf0 2024-03-16T10:51:00.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.260+0000 7fed2effd700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fed2406dfa0 0x7fed24070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:00.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.260+0000 7fed2effd700 1 -- 172.21.15.2:0/3440132183 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fed3408f0f0 con 0x7fed380fedf0 2024-03-16T10:51:00.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.260+0000 7fed3d46a700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fed2406dfa0 0x7fed24070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:00.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.261+0000 7fed3d46a700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fed2406dfa0 0x7fed24070460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed28007c60 tx=0x7fed280073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:00.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.265+0000 7fed2effd700 1 -- 172.21.15.2:0/3440132183 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fed3405c090 con 0x7fed380fedf0 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.555+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fed3804ea80 con 0x7fed380fedf0 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.555+0000 7fed2effd700 1 -- 172.21.15.2:0/3440132183 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1313 (secure 0 0 0) 0x7fed3405bc20 con 0x7fed380fedf0 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Reduced data availability: 3 pgs peering; Degraded data redundancy: 14/267 objects degraded (5.243%), 5 pgs degraded 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout: osd.5 (root=default,host=smithi026) is down 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 3 pgs peering 2024-03-16T10:51:00.557 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.3 is stuck peering for 4m, current state peering, last acting [2,1] 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.17 is stuck peering for 4m, current state peering, last acting [2,1] 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout: pg 3.5 is stuck peering for 3m, current state peering, last acting [2,3] 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 14/267 objects degraded (5.243%), 5 pgs degraded 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-03-16T10:51:00.558 INFO:teuthology.orchestra.run.smithi002.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fed2406dfa0 msgr2=0x7fed24070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fed2406dfa0 0x7fed24070460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fed28007c60 tx=0x7fed280073d0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 msgr2=0x7fed38196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed38196580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed340092e0 tx=0x7fed3400bd70 comp rx=0 tx=0).stop 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 shutdown_connections 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fed2406dfa0 0x7fed24070460 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed380fedf0 0x7fed38196580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 --2- 172.21.15.2:0/3440132183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed38101e70 0x7fed38196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 >> 172.21.15.2:0/3440132183 conn(0x7fed380fa730 msgr2=0x7fed381054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 shutdown_connections 2024-03-16T10:51:00.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:00.557+0000 7fed3fecf700 1 -- 172.21.15.2:0/3440132183 wait complete. 2024-03-16T10:51:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:00 smithi026 ceph-mon[82600]: pgmap v170: 65 pgs: 11 active+undersized+degraded, 5 peering, 2 stale+active+clean, 9 active+undersized, 38 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-03-16T10:51:01.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:00 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3440132183' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:51:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:00 smithi002 ceph-mon[94239]: pgmap v170: 65 pgs: 11 active+undersized+degraded, 5 peering, 2 stale+active+clean, 9 active+undersized, 38 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-03-16T10:51:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:00 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3440132183' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:51:02.129 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:01 smithi026 ceph-mon[82600]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:01 smithi002 ceph-mon[94239]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:03.055 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:02 smithi026 ceph-mon[82600]: pgmap v171: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-03-16T10:51:03.055 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:02 smithi026 ceph-mon[82600]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-16T10:51:03.055 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:03.056 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:02 smithi002 ceph-mon[94239]: pgmap v171: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-03-16T10:51:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:02 smithi002 ceph-mon[94239]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-16T10:51:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:03.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:04.663 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T10:51:04.407+0000 7f3eb084e7c0 -1 osd.5 76 log_to_monitors true 2024-03-16T10:51:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-mon[82600]: pgmap v172: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-03-16T10:51:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-mon[82600]: from='osd.5 [v2:172.21.15.26:6816/2348241985,v1:172.21.15.26:6817/2348241985]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:51:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-mon[82600]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:51:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:05.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:04 smithi002 ceph-mon[94239]: pgmap v172: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-03-16T10:51:05.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:05.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:05.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:04 smithi002 ceph-mon[94239]: from='osd.5 [v2:172.21.15.26:6816/2348241985,v1:172.21.15.26:6817/2348241985]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:51:05.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:04 smithi002 ceph-mon[94239]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T10:51:05.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:05.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:05.931 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 10:51:05 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T10:51:05.713+0000 7f3e9dc9e700 -1 osd.5 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:51:06.216 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:05 smithi026 ceph-mon[82600]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T10:51:06.217 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:05 smithi026 ceph-mon[82600]: from='osd.5 [v2:172.21.15.26:6816/2348241985,v1:172.21.15.26:6817/2348241985]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:51:06.217 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:05 smithi026 ceph-mon[82600]: osdmap e79: 6 total, 5 up, 6 in 2024-03-16T10:51:06.217 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:05 smithi026 ceph-mon[82600]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:51:06.217 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:05 smithi026 ceph-mon[82600]: Health check update: Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded (PG_DEGRADED) 2024-03-16T10:51:06.217 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:05 smithi026 ceph-mon[82600]: from='osd.5 ' entity='osd.5' 2024-03-16T10:51:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:05 smithi002 ceph-mon[94239]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T10:51:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:05 smithi002 ceph-mon[94239]: from='osd.5 [v2:172.21.15.26:6816/2348241985,v1:172.21.15.26:6817/2348241985]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:51:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:05 smithi002 ceph-mon[94239]: osdmap e79: 6 total, 5 up, 6 in 2024-03-16T10:51:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:05 smithi002 ceph-mon[94239]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-03-16T10:51:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:05 smithi002 ceph-mon[94239]: Health check update: Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded (PG_DEGRADED) 2024-03-16T10:51:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:05 smithi002 ceph-mon[94239]: from='osd.5 ' entity='osd.5' 2024-03-16T10:51:07.228 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:06 smithi026 ceph-mon[82600]: pgmap v174: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-03-16T10:51:07.228 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:06 smithi026 ceph-mon[82600]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:51:07.228 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:06 smithi026 ceph-mon[82600]: osd.5 [v2:172.21.15.26:6816/2348241985,v1:172.21.15.26:6817/2348241985] boot 2024-03-16T10:51:07.228 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:06 smithi026 ceph-mon[82600]: osdmap e80: 6 total, 6 up, 6 in 2024-03-16T10:51:07.228 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:51:07.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:06 smithi002 ceph-mon[94239]: pgmap v174: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-03-16T10:51:07.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:06 smithi002 ceph-mon[94239]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-16T10:51:07.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:06 smithi002 ceph-mon[94239]: osd.5 [v2:172.21.15.26:6816/2348241985,v1:172.21.15.26:6817/2348241985] boot 2024-03-16T10:51:07.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:06 smithi002 ceph-mon[94239]: osdmap e80: 6 total, 6 up, 6 in 2024-03-16T10:51:07.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T10:51:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:51:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:08.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: osdmap e81: 6 total, 6 up, 6 in 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all osd 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-16T10:51:08.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-16T10:51:08.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-16T10:51:08.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-16T10:51:08.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-16T10:51:08.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-16T10:51:08.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: Upgrade: Setting require_osd_release to 19 squid 2024-03-16T10:51:08.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-16T10:51:08.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:08 smithi026 ceph-mon[82600]: pgmap v177: 65 pgs: 3 peering, 11 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 37/267 objects degraded (13.858%) 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: osdmap e81: 6 total, 6 up, 6 in 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:09.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all osd 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-16T10:51:09.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-16T10:51:09.051 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-16T10:51:09.051 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-16T10:51:09.051 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: Upgrade: Setting require_osd_release to 19 squid 2024-03-16T10:51:09.051 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-16T10:51:09.051 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:08 smithi002 ceph-mon[94239]: pgmap v177: 65 pgs: 3 peering, 11 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 37/267 objects degraded (13.858%) 2024-03-16T10:51:09.714 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:09 smithi002 ceph-mon[94239]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-16T10:51:09.714 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-16T10:51:09.714 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:09 smithi002 ceph-mon[94239]: osdmap e82: 6 total, 6 up, 6 in 2024-03-16T10:51:09.714 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi002.qvbseu"]}]: dispatch 2024-03-16T10:51:09.714 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:09 smithi002 ceph-mon[94239]: Upgrade: It appears safe to stop mds.cephfs.smithi002.qvbseu 2024-03-16T10:51:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:09 smithi026 ceph-mon[82600]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-16T10:51:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-16T10:51:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:09 smithi026 ceph-mon[82600]: osdmap e82: 6 total, 6 up, 6 in 2024-03-16T10:51:09.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi002.qvbseu"]}]: dispatch 2024-03-16T10:51:09.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:09 smithi026 ceph-mon[82600]: Upgrade: It appears safe to stop mds.cephfs.smithi002.qvbseu 2024-03-16T10:51:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:10 smithi026 ceph-mon[82600]: pgmap v179: 65 pgs: 3 peering, 6 active+undersized+degraded, 12 active+undersized, 44 active+clean; 259 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 17/267 objects degraded (6.367%) 2024-03-16T10:51:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:10 smithi026 ceph-mon[82600]: Health check update: Degraded data redundancy: 37/267 objects degraded (13.858%), 11 pgs degraded (PG_DEGRADED) 2024-03-16T10:51:11.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:10 smithi002 ceph-mon[94239]: pgmap v179: 65 pgs: 3 peering, 6 active+undersized+degraded, 12 active+undersized, 44 active+clean; 259 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 17/267 objects degraded (6.367%) 2024-03-16T10:51:11.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:10 smithi002 ceph-mon[94239]: Health check update: Degraded data redundancy: 37/267 objects degraded (13.858%), 11 pgs degraded (PG_DEGRADED) 2024-03-16T10:51:11.801 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:11 smithi002 ceph-mon[94239]: Upgrade: Updating mds.cephfs.smithi002.qvbseu 2024-03-16T10:51:11.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:11.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:51:11.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:11.802 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:11 smithi002 ceph-mon[94239]: Deploying daemon mds.cephfs.smithi002.qvbseu on smithi002 2024-03-16T10:51:11.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:11 smithi026 ceph-mon[82600]: Upgrade: Updating mds.cephfs.smithi002.qvbseu 2024-03-16T10:51:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.qvbseu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:51:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:11.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:11 smithi026 ceph-mon[82600]: Deploying daemon mds.cephfs.smithi002.qvbseu on smithi002 2024-03-16T10:51:12.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:12 smithi026 ceph-mon[82600]: pgmap v180: 65 pgs: 3 peering, 62 active+clean; 259 MiB data, 990 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:13.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:12 smithi002 ceph-mon[94239]: pgmap v180: 65 pgs: 3 peering, 62 active+clean; 259 MiB data, 990 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:13.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:13 smithi026 ceph-mon[82600]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/267 objects degraded (6.367%), 6 pgs degraded) 2024-03-16T10:51:14.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:13 smithi002 ceph-mon[94239]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/267 objects degraded (6.367%), 6 pgs degraded) 2024-03-16T10:51:14.873 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:14 smithi002 ceph-mon[94239]: osdmap e83: 6 total, 6 up, 6 in 2024-03-16T10:51:14.873 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:14 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 2 up:standby 2024-03-16T10:51:14.873 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:14 smithi002 ceph-mon[94239]: pgmap v182: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:14.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:14 smithi026 ceph-mon[82600]: osdmap e83: 6 total, 6 up, 6 in 2024-03-16T10:51:14.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:14 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 2 up:standby 2024-03-16T10:51:14.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:14 smithi026 ceph-mon[82600]: pgmap v182: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:17.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:16 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:boot 2024-03-16T10:51:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:16 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:51:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:51:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:16 smithi002 ceph-mon[94239]: pgmap v183: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:17.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:17.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:16 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:boot 2024-03-16T10:51:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:16 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:51:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:51:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:16 smithi026 ceph-mon[82600]: pgmap v183: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:17.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:19.187 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:18 smithi002 ceph-mon[94239]: pgmap v184: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:18 smithi026 ceph-mon[82600]: pgmap v184: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:20 smithi026 ceph-mon[82600]: pgmap v185: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:21.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:20 smithi002 ceph-mon[94239]: pgmap v185: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:22.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:21 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:22.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:21 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:21 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:21 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:22 smithi002 ceph-mon[94239]: pgmap v186: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:23.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:22 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:22 smithi026 ceph-mon[82600]: pgmap v186: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:23.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:22 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:25.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:24 smithi026 ceph-mon[82600]: pgmap v187: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:25.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:24 smithi002 ceph-mon[94239]: pgmap v187: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: pgmap v188: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi002.ijcmqs"]}]: dispatch 2024-03-16T10:51:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:26 smithi026 ceph-mon[82600]: Upgrade: It appears safe to stop mds.cephfs.smithi002.ijcmqs 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: pgmap v188: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi002.ijcmqs"]}]: dispatch 2024-03-16T10:51:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:26 smithi002 ceph-mon[94239]: Upgrade: It appears safe to stop mds.cephfs.smithi002.ijcmqs 2024-03-16T10:51:28.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:28 smithi026 ceph-mon[82600]: Upgrade: Updating mds.cephfs.smithi002.ijcmqs 2024-03-16T10:51:28.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:28.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:51:28.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:28.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:28 smithi026 ceph-mon[82600]: Deploying daemon mds.cephfs.smithi002.ijcmqs on smithi002 2024-03-16T10:51:28.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:28 smithi026 ceph-mon[82600]: pgmap v189: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:28.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:28 smithi002 ceph-mon[94239]: Upgrade: Updating mds.cephfs.smithi002.ijcmqs 2024-03-16T10:51:28.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:28.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi002.ijcmqs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:51:28.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:28.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:28 smithi002 ceph-mon[94239]: Deploying daemon mds.cephfs.smithi002.ijcmqs on smithi002 2024-03-16T10:51:28.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:28 smithi002 ceph-mon[94239]: pgmap v189: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 -- 172.21.15.2:0/739881471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 msgr2=0x7f3a6c0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 --2- 172.21.15.2:0/739881471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c0ffe00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009910 tx=0x7f3a5c009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 -- 172.21.15.2:0/739881471 shutdown_connections 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 --2- 172.21.15.2:0/739881471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c100be0 0x7f3a6c101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 --2- 172.21.15.2:0/739881471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c0ffe00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 -- 172.21.15.2:0/739881471 >> 172.21.15.2:0/739881471 conn(0x7f3a6c0faf80 msgr2=0x7f3a6c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 -- 172.21.15.2:0/739881471 shutdown_connections 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.689+0000 7f3a723fc700 1 -- 172.21.15.2:0/739881471 wait complete. 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.690+0000 7f3a723fc700 1 Processor -- start 2024-03-16T10:51:30.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.690+0000 7f3a723fc700 1 -- start start 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a723fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c1953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a723fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c100be0 0x7f3a6c195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a723fc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c195f30 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a723fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c196070 con 0x7f3a6c100be0 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c1953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c1953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42094/0 (socket says 172.21.15.2:42094) 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6bfff700 1 -- 172.21.15.2:0/3656339212 learned_addr learned my addr 172.21.15.2:0/3656339212 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6b7fe700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c100be0 0x7f3a6c195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6bfff700 1 -- 172.21.15.2:0/3656339212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c100be0 msgr2=0x7f3a6c195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6bfff700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c100be0 0x7f3a6c195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:30.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6bfff700 1 -- 172.21.15.2:0/3656339212 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0095f0 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.691+0000 7f3a6bfff700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c1953d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0059d0 tx=0x7f3a5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.692+0000 7f3a697fa700 1 -- 172.21.15.2:0/3656339212 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5c01d070 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.692+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c068ff0 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.692+0000 7f3a697fa700 1 -- 172.21.15.2:0/3656339212 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a5c005dc0 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.692+0000 7f3a697fa700 1 -- 172.21.15.2:0/3656339212 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a5c021410 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.692+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c069560 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.693+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c04ea80 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.693+0000 7f3a697fa700 1 -- 172.21.15.2:0/3656339212 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3a5c00f460 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.694+0000 7f3a697fa700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a5406dfa0 0x7f3a54070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:30.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.694+0000 7f3a697fa700 1 -- 172.21.15.2:0/3656339212 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3a5c08eef0 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.694+0000 7f3a6b7fe700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a5406dfa0 0x7f3a54070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.694+0000 7f3a6b7fe700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a5406dfa0 0x7f3a54070460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005950 tx=0x7f3a6000d620 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:30.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.699+0000 7f3a697fa700 1 -- 172.21.15.2:0/3656339212 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a5c05be90 con 0x7f3a6c0ff9e0 2024-03-16T10:51:30.910 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:30 smithi002 ceph-mon[94239]: osdmap e84: 6 total, 6 up, 6 in 2024-03-16T10:51:30.910 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:30 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 2 up:standby 2024-03-16T10:51:30.910 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:30 smithi002 ceph-mon[94239]: pgmap v191: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:30.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.927+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a6c069780 con 0x7f3a5406dfa0 2024-03-16T10:51:30.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.929+0000 7f3a697fa700 1 -- 172.21.15.2:0/3656339212 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3a6c069780 con 0x7f3a5406dfa0 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a5406dfa0 msgr2=0x7f3a54070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a5406dfa0 0x7f3a54070460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3a60005950 tx=0x7f3a6000d620 comp rx=0 tx=0).stop 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 msgr2=0x7f3a6c1953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c1953d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0059d0 tx=0x7f3a5c00bd70 comp rx=0 tx=0).stop 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 shutdown_connections 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a5406dfa0 0x7f3a54070460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a6c0ff9e0 0x7f3a6c1953d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 --2- 172.21.15.2:0/3656339212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6c100be0 0x7f3a6c195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 >> 172.21.15.2:0/3656339212 conn(0x7f3a6c0faf80 msgr2=0x7f3a6c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 shutdown_connections 2024-03-16T10:51:30.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:30.931+0000 7f3a723fc700 1 -- 172.21.15.2:0/3656339212 wait complete. 2024-03-16T10:51:30.942 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:51:30.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:30 smithi026 ceph-mon[82600]: osdmap e84: 6 total, 6 up, 6 in 2024-03-16T10:51:30.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:30 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 2 up:standby 2024-03-16T10:51:30.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:30 smithi026 ceph-mon[82600]: pgmap v191: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:31.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 -- 172.21.15.2:0/3341453116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 msgr2=0x7f4184101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 --2- 172.21.15.2:0/3341453116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101040 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4174009910 tx=0x7f4174009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 -- 172.21.15.2:0/3341453116 shutdown_connections 2024-03-16T10:51:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 --2- 172.21.15.2:0/3341453116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101040 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 --2- 172.21.15.2:0/3341453116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41840ff9e0 0x7f41840ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 -- 172.21.15.2:0/3341453116 >> 172.21.15.2:0/3341453116 conn(0x7f41840faf80 msgr2=0x7f41840fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 -- 172.21.15.2:0/3341453116 shutdown_connections 2024-03-16T10:51:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 -- 172.21.15.2:0/3341453116 wait complete. 2024-03-16T10:51:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 Processor -- start 2024-03-16T10:51:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.040+0000 7f418883f700 1 -- start start 2024-03-16T10:51:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f418883f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41840ff9e0 0x7f41841038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f418883f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f418883f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4184102500 con 0x7f4184100be0 2024-03-16T10:51:31.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f418883f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4184102640 con 0x7f41840ff9e0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f4181d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f4181d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42120/0 (socket says 172.21.15.2:42120) 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f4181d9b700 1 -- 172.21.15.2:0/2154182065 learned_addr learned my addr 172.21.15.2:0/2154182065 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f418259c700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41840ff9e0 0x7f41841038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f4181d9b700 1 -- 172.21.15.2:0/2154182065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41840ff9e0 msgr2=0x7f41841038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f4181d9b700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41840ff9e0 0x7f41841038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f4181d9b700 1 -- 172.21.15.2:0/2154182065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41740095f0 con 0x7f4184100be0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f4181d9b700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101fc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41740092e0 tx=0x7f417400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.041+0000 7f417b7fe700 1 -- 172.21.15.2:0/2154182065 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f417401d070 con 0x7f4184100be0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.042+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41841028c0 con 0x7f4184100be0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.042+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4184102e30 con 0x7f4184100be0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.042+0000 7f417b7fe700 1 -- 172.21.15.2:0/2154182065 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f417400f460 con 0x7f4184100be0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.042+0000 7f417b7fe700 1 -- 172.21.15.2:0/2154182065 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4174021640 con 0x7f4184100be0 2024-03-16T10:51:31.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.043+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41640052f0 con 0x7f4184100be0 2024-03-16T10:51:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.043+0000 7f417b7fe700 1 -- 172.21.15.2:0/2154182065 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4174003a60 con 0x7f4184100be0 2024-03-16T10:51:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.044+0000 7f417b7fe700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f417006dfa0 0x7f4170070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.044+0000 7f418259c700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f417006dfa0 0x7f4170070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.044+0000 7f417b7fe700 1 -- 172.21.15.2:0/2154182065 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f417408f150 con 0x7f4184100be0 2024-03-16T10:51:31.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.044+0000 7f418259c700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f417006dfa0 0x7f4170070460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f416c005950 tx=0x7f416c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:31.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.048+0000 7f417b7fe700 1 -- 172.21.15.2:0/2154182065 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f417405c180 con 0x7f4184100be0 2024-03-16T10:51:31.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.291+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4164000bc0 con 0x7f417006dfa0 2024-03-16T10:51:31.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.292+0000 7f417b7fe700 1 -- 172.21.15.2:0/2154182065 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4164000bc0 con 0x7f417006dfa0 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f417006dfa0 msgr2=0x7f4170070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f417006dfa0 0x7f4170070460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f416c005950 tx=0x7f416c0058e0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 msgr2=0x7f4184101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101fc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41740092e0 tx=0x7f417400be20 comp rx=0 tx=0).stop 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 shutdown_connections 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f417006dfa0 0x7f4170070460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41840ff9e0 0x7f41841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 --2- 172.21.15.2:0/2154182065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4184100be0 0x7f4184101fc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 >> 172.21.15.2:0/2154182065 conn(0x7f41840faf80 msgr2=0x7f4184069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 shutdown_connections 2024-03-16T10:51:31.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.294+0000 7f418883f700 1 -- 172.21.15.2:0/2154182065 wait complete. 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 -- 172.21.15.2:0/1315988427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c100ae0 msgr2=0x7ff77c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 --2- 172.21.15.2:0/1315988427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c100ae0 0x7ff77c100f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff778009910 tx=0x7ff778009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 -- 172.21.15.2:0/1315988427 shutdown_connections 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 --2- 172.21.15.2:0/1315988427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c101ce0 0x7ff77c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 --2- 172.21.15.2:0/1315988427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c100ae0 0x7ff77c100f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 -- 172.21.15.2:0/1315988427 >> 172.21.15.2:0/1315988427 conn(0x7ff77c0fc0a0 msgr2=0x7ff77c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 -- 172.21.15.2:0/1315988427 shutdown_connections 2024-03-16T10:51:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 -- 172.21.15.2:0/1315988427 wait complete. 2024-03-16T10:51:31.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 Processor -- start 2024-03-16T10:51:31.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.400+0000 7ff78410c700 1 -- start start 2024-03-16T10:51:31.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff78410c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c100ae0 0x7ff77c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff78410c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c101ce0 0x7ff77c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff78410c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c197030 con 0x7ff77c101ce0 2024-03-16T10:51:31.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff78410c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c197170 con 0x7ff77c100ae0 2024-03-16T10:51:31.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff7816a7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c101ce0 0x7ff77c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff7816a7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c101ce0 0x7ff77c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42146/0 (socket says 172.21.15.2:42146) 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff7816a7700 1 -- 172.21.15.2:0/2706763123 learned_addr learned my addr 172.21.15.2:0/2706763123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff781ea8700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c100ae0 0x7ff77c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff7816a7700 1 -- 172.21.15.2:0/2706763123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c100ae0 msgr2=0x7ff77c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff7816a7700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c100ae0 0x7ff77c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff7816a7700 1 -- 172.21.15.2:0/2706763123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7780095f0 con 0x7ff77c101ce0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff7816a7700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c101ce0 0x7ff77c196a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00e920 tx=0x7ff76c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff772ffd700 1 -- 172.21.15.2:0/2706763123 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff76c00c9d0 con 0x7ff77c101ce0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff772ffd700 1 -- 172.21.15.2:0/2706763123 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff76c010430 con 0x7ff77c101ce0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c19bbd0 con 0x7ff77c101ce0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff772ffd700 1 -- 172.21.15.2:0/2706763123 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff76c018630 con 0x7ff77c101ce0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.401+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c19c1a0 con 0x7ff77c101ce0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.402+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c190660 con 0x7ff77c101ce0 2024-03-16T10:51:31.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.403+0000 7ff772ffd700 1 -- 172.21.15.2:0/2706763123 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ff76c0105a0 con 0x7ff77c101ce0 2024-03-16T10:51:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.403+0000 7ff772ffd700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff76806dff0 0x7ff7680704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.403+0000 7ff772ffd700 1 -- 172.21.15.2:0/2706763123 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff76c014070 con 0x7ff77c101ce0 2024-03-16T10:51:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.403+0000 7ff781ea8700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff76806dff0 0x7ff7680704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.404+0000 7ff781ea8700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff76806dff0 0x7ff7680704b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff77800b800 tx=0x7ff778009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:31.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.408+0000 7ff772ffd700 1 -- 172.21.15.2:0/2706763123 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff76c053740 con 0x7ff77c101ce0 2024-03-16T10:51:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.662+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff77c0611f0 con 0x7ff76806dff0 2024-03-16T10:51:31.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.673+0000 7ff772ffd700 1 -- 172.21.15.2:0/2706763123 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff77c0611f0 con 0x7ff76806dff0 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (13m) 10s ago 15m 24.9M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (15m) 10s ago 15m 8614k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (14m) 28s ago 14m 8556k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (3m) 10s ago 15m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (3m) 28s ago 14m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (13m) 10s ago 14m 85.3M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (9m) 10s ago 9m 19.8M - 18.2.2-878-ga752c2f7 f692adb9a265 7cc0f0137650 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (16s) 10s ago 9m 15.5M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a68e60da4931 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (9m) 28s ago 9m 152M - 18.2.2-878-ga752c2f7 f692adb9a265 f84d0cc49ac0 2024-03-16T10:51:31.674 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (9m) 28s ago 9m 18.0M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (6m) 10s ago 16m 507M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (5m) 28s ago 14m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (5m) 10s ago 16m 49.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (4m) 28s ago 14m 38.2M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (15m) 10s ago 15m 16.5M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (14m) 28s ago 14m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (3m) 10s ago 13m 86.2M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (2m) 10s ago 12m 90.0M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (2m) 10s ago 12m 81.8M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (100s) 28s ago 11m 98.3M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (66s) 28s ago 11m 83.4M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (32s) 28s ago 11m 13.6M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b cf40ac20a977 2024-03-16T10:51:31.675 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (5m) 10s ago 14m 77.1M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:51:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff76806dff0 msgr2=0x7ff7680704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff76806dff0 0x7ff7680704b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff77800b800 tx=0x7ff778009f70 comp rx=0 tx=0).stop 2024-03-16T10:51:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c101ce0 msgr2=0x7ff77c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c101ce0 0x7ff77c196a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00e920 tx=0x7ff76c00ec30 comp rx=0 tx=0).stop 2024-03-16T10:51:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 shutdown_connections 2024-03-16T10:51:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff76806dff0 0x7ff7680704b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77c100ae0 0x7ff77c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 --2- 172.21.15.2:0/2706763123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff77c101ce0 0x7ff77c196a10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 >> 172.21.15.2:0/2706763123 conn(0x7ff77c0fc0a0 msgr2=0x7ff77c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 shutdown_connections 2024-03-16T10:51:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.675+0000 7ff78410c700 1 -- 172.21.15.2:0/2706763123 wait complete. 2024-03-16T10:51:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 -- 172.21.15.2:0/741192774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc100b00 msgr2=0x7f38cc100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 --2- 172.21.15.2:0/741192774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc100b00 0x7f38cc100f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009910 tx=0x7f38bc009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 -- 172.21.15.2:0/741192774 shutdown_connections 2024-03-16T10:51:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 --2- 172.21.15.2:0/741192774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc101d00 0x7f38cc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 --2- 172.21.15.2:0/741192774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc100b00 0x7f38cc100f20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 -- 172.21.15.2:0/741192774 >> 172.21.15.2:0/741192774 conn(0x7f38cc0fc0a0 msgr2=0x7f38cc0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:31.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 -- 172.21.15.2:0/741192774 shutdown_connections 2024-03-16T10:51:31.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.785+0000 7f38d1b62700 1 -- 172.21.15.2:0/741192774 wait complete. 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38d1b62700 1 Processor -- start 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38d1b62700 1 -- start start 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38d1b62700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc100b00 0x7f38cc1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38d1b62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc101d00 0x7f38cc196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38d1b62700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc197020 con 0x7f38cc101d00 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38d1b62700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc197160 con 0x7f38cc100b00 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38caffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc101d00 0x7f38cc196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.786+0000 7f38caffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc101d00 0x7f38cc196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42158/0 (socket says 172.21.15.2:42158) 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38caffd700 1 -- 172.21.15.2:0/2686173974 learned_addr learned my addr 172.21.15.2:0/2686173974 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38cb7fe700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc100b00 0x7f38cc1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38caffd700 1 -- 172.21.15.2:0/2686173974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc100b00 msgr2=0x7f38cc1964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38caffd700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc100b00 0x7f38cc1964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:31.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38caffd700 1 -- 172.21.15.2:0/2686173974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0095f0 con 0x7f38cc101d00 2024-03-16T10:51:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38caffd700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc101d00 0x7f38cc196a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38c000d8d0 tx=0x7f38c000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38c8ff9700 1 -- 172.21.15.2:0/2686173974 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c00098a0 con 0x7f38cc101d00 2024-03-16T10:51:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc19bbc0 con 0x7f38cc101d00 2024-03-16T10:51:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38c8ff9700 1 -- 172.21.15.2:0/2686173974 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38c0010460 con 0x7f38cc101d00 2024-03-16T10:51:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc19c190 con 0x7f38cc101d00 2024-03-16T10:51:31.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.787+0000 7f38c8ff9700 1 -- 172.21.15.2:0/2686173974 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38c000f5f0 con 0x7f38cc101d00 2024-03-16T10:51:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.789+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc190680 con 0x7f38cc101d00 2024-03-16T10:51:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.789+0000 7f38c8ff9700 1 -- 172.21.15.2:0/2686173974 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f38c0009a00 con 0x7f38cc101d00 2024-03-16T10:51:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.790+0000 7f38c8ff9700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f38b406dff0 0x7f38b40704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.790+0000 7f38c8ff9700 1 -- 172.21.15.2:0/2686173974 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f38c008dbb0 con 0x7f38cc101d00 2024-03-16T10:51:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.790+0000 7f38cb7fe700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f38b406dff0 0x7f38b40704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:31.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.791+0000 7f38cb7fe700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f38b406dff0 0x7f38b40704b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f38bc000c00 tx=0x7f38bc005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:31.795+0000 7f38c8ff9700 1 -- 172.21.15.2:0/2686173974 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38c005ab30 con 0x7f38cc101d00 2024-03-16T10:51:31.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:31 smithi026 ceph-mon[82600]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:31.987 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:31 smithi002 ceph-mon[94239]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:32.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.103+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38cc04eab0 con 0x7f38cc101d00 2024-03-16T10:51:32.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.104+0000 7f38c8ff9700 1 -- 172.21.15.2:0/2686173974 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f38c005a6c0 con 0x7f38cc101d00 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:51:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2, 2024-03-16T10:51:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 1 2024-03-16T10:51:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:51:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:51:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 2, 2024-03-16T10:51:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 11 2024-03-16T10:51:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:51:32.106 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:51:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f38b406dff0 msgr2=0x7f38b40704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f38b406dff0 0x7f38b40704b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f38bc000c00 tx=0x7f38bc005960 comp rx=0 tx=0).stop 2024-03-16T10:51:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc101d00 msgr2=0x7f38cc196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc101d00 0x7f38cc196a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38c000d8d0 tx=0x7f38c000dbe0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 shutdown_connections 2024-03-16T10:51:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f38b406dff0 0x7f38b40704b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38cc100b00 0x7f38cc1964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 --2- 172.21.15.2:0/2686173974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38cc101d00 0x7f38cc196a00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 >> 172.21.15.2:0/2686173974 conn(0x7f38cc0fc0a0 msgr2=0x7f38cc1052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 shutdown_connections 2024-03-16T10:51:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.106+0000 7f38d1b62700 1 -- 172.21.15.2:0/2686173974 wait complete. 2024-03-16T10:51:32.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 -- 172.21.15.2:0/153703465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628101cd0 msgr2=0x7f4628102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 --2- 172.21.15.2:0/153703465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628101cd0 0x7f4628102130 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f461c009910 tx=0x7f461c009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:32.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 -- 172.21.15.2:0/153703465 shutdown_connections 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 --2- 172.21.15.2:0/153703465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628101cd0 0x7f4628102130 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 --2- 172.21.15.2:0/153703465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4628100ad0 0x7f4628100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 -- 172.21.15.2:0/153703465 >> 172.21.15.2:0/153703465 conn(0x7f46280fc090 msgr2=0x7f46280fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 -- 172.21.15.2:0/153703465 shutdown_connections 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.215+0000 7f462e872700 1 -- 172.21.15.2:0/153703465 wait complete. 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f462e872700 1 Processor -- start 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f462e872700 1 -- start start 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f462e872700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628100ad0 0x7f4628196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f462e872700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4628101cd0 0x7f4628196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f462e872700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628197080 con 0x7f4628100ad0 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f462e872700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46281971c0 con 0x7f4628101cd0 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f4627fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628100ad0 0x7f4628196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f4627fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628100ad0 0x7f4628196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42176/0 (socket says 172.21.15.2:42176) 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f4627fff700 1 -- 172.21.15.2:0/1321440109 learned_addr learned my addr 172.21.15.2:0/1321440109 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f46277fe700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4628101cd0 0x7f4628196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f4627fff700 1 -- 172.21.15.2:0/1321440109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4628101cd0 msgr2=0x7f4628196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f4627fff700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4628101cd0 0x7f4628196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.216+0000 7f4627fff700 1 -- 172.21.15.2:0/1321440109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0095f0 con 0x7f4628100ad0 2024-03-16T10:51:32.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.217+0000 7f4627fff700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628100ad0 0x7f4628196520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f461800d900 tx=0x7f461800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:32.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.217+0000 7f46257fa700 1 -- 172.21.15.2:0/1321440109 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461800b920 con 0x7f4628100ad0 2024-03-16T10:51:32.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.217+0000 7f46257fa700 1 -- 172.21.15.2:0/1321440109 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4618010460 con 0x7f4628100ad0 2024-03-16T10:51:32.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.217+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462819bc20 con 0x7f4628100ad0 2024-03-16T10:51:32.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.217+0000 7f46257fa700 1 -- 172.21.15.2:0/1321440109 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461800f590 con 0x7f4628100ad0 2024-03-16T10:51:32.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.217+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462819c170 con 0x7f4628100ad0 2024-03-16T10:51:32.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.218+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4628190650 con 0x7f4628100ad0 2024-03-16T10:51:32.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.219+0000 7f46257fa700 1 -- 172.21.15.2:0/1321440109 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f46180105d0 con 0x7f4628100ad0 2024-03-16T10:51:32.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.219+0000 7f46257fa700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f461006dfa0 0x7f4610070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.219+0000 7f46277fe700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f461006dfa0 0x7f4610070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.219+0000 7f46257fa700 1 -- 172.21.15.2:0/1321440109 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f461808dd30 con 0x7f4628100ad0 2024-03-16T10:51:32.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.220+0000 7f46277fe700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f461006dfa0 0x7f4610070460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f461c009fd0 tx=0x7f461c005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:32.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.224+0000 7f46257fa700 1 -- 172.21.15.2:0/1321440109 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f461805acd0 con 0x7f4628100ad0 2024-03-16T10:51:32.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.494+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46280619c0 con 0x7f4628100ad0 2024-03-16T10:51:32.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.495+0000 7f46257fa700 1 -- 172.21.15.2:0/1321440109 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1698 (secure 0 0 0) 0x7f461805a860 con 0x7f4628100ad0 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:e20 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 16 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:42:09.214374+0000 2024-03-16T10:51:32.496 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 47 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24255} 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:51:32.497 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 0 members: 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{0:24255} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.26:6824/2074466615,v1:172.21.15.26:6825/2074466615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:24267} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:51:32.498 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:51:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f461006dfa0 msgr2=0x7f4610070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f461006dfa0 0x7f4610070460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f461c009fd0 tx=0x7f461c005960 comp rx=0 tx=0).stop 2024-03-16T10:51:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628100ad0 msgr2=0x7f4628196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628100ad0 0x7f4628196520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f461800d900 tx=0x7f461800dc10 comp rx=0 tx=0).stop 2024-03-16T10:51:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 shutdown_connections 2024-03-16T10:51:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f461006dfa0 0x7f4610070460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4628100ad0 0x7f4628196520 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 --2- 172.21.15.2:0/1321440109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4628101cd0 0x7f4628196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 >> 172.21.15.2:0/1321440109 conn(0x7f46280fc090 msgr2=0x7f4628105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 shutdown_connections 2024-03-16T10:51:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.497+0000 7f462e872700 1 -- 172.21.15.2:0/1321440109 wait complete. 2024-03-16T10:51:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 20 2024-03-16T10:51:32.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.607+0000 7fa8422bb700 1 -- 172.21.15.2:0/2491151165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 msgr2=0x7fa83c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.607+0000 7fa8422bb700 1 --2- 172.21.15.2:0/2491151165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c102160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa830009910 tx=0x7fa830009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:32.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.607+0000 7fa8422bb700 1 -- 172.21.15.2:0/2491151165 shutdown_connections 2024-03-16T10:51:32.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.607+0000 7fa8422bb700 1 --2- 172.21.15.2:0/2491151165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c102160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.607+0000 7fa8422bb700 1 --2- 172.21.15.2:0/2491151165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa83c100b00 0x7fa83c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.607+0000 7fa8422bb700 1 -- 172.21.15.2:0/2491151165 >> 172.21.15.2:0/2491151165 conn(0x7fa83c0fc0a0 msgr2=0x7fa83c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:32.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.608+0000 7fa8422bb700 1 -- 172.21.15.2:0/2491151165 shutdown_connections 2024-03-16T10:51:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.608+0000 7fa8422bb700 1 -- 172.21.15.2:0/2491151165 wait complete. 2024-03-16T10:51:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.608+0000 7fa8422bb700 1 Processor -- start 2024-03-16T10:51:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.608+0000 7fa8422bb700 1 -- start start 2024-03-16T10:51:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.608+0000 7fa8422bb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa83c100b00 0x7fa83c074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa8422bb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa8422bb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0736e0 con 0x7fa83c101d00 2024-03-16T10:51:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa8422bb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c073820 con 0x7fa83c100b00 2024-03-16T10:51:32.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48436/0 (socket says 172.21.15.2:48436) 2024-03-16T10:51:32.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83b7fe700 1 -- 172.21.15.2:0/333175081 learned_addr learned my addr 172.21.15.2:0/333175081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:32.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83bfff700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa83c100b00 0x7fa83c074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83b7fe700 1 -- 172.21.15.2:0/333175081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa83c100b00 msgr2=0x7fa83c074ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83b7fe700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa83c100b00 0x7fa83c074ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83b7fe700 1 -- 172.21.15.2:0/333175081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300095f0 con 0x7fa83c101d00 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa83b7fe700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c0731a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa830006010 tx=0x7fa83000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa8397fa700 1 -- 172.21.15.2:0/333175081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83001d070 con 0x7fa83c101d00 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa8397fa700 1 -- 172.21.15.2:0/333175081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa83000f460 con 0x7fa83c101d00 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c073aa0 con 0x7fa83c101d00 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.609+0000 7fa8397fa700 1 -- 172.21.15.2:0/333175081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa830021620 con 0x7fa83c101d00 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.610+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c074010 con 0x7fa83c101d00 2024-03-16T10:51:32.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.610+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c190680 con 0x7fa83c101d00 2024-03-16T10:51:32.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.611+0000 7fa8397fa700 1 -- 172.21.15.2:0/333175081 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa83000f5d0 con 0x7fa83c101d00 2024-03-16T10:51:32.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.611+0000 7fa8397fa700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa82406dfa0 0x7fa824070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.612+0000 7fa8397fa700 1 -- 172.21.15.2:0/333175081 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa83008edd0 con 0x7fa83c101d00 2024-03-16T10:51:32.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.612+0000 7fa83bfff700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa82406dfa0 0x7fa824070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.612+0000 7fa83bfff700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa82406dfa0 0x7fa824070460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa83c101b60 tx=0x7fa82c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:32.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.616+0000 7fa8397fa700 1 -- 172.21.15.2:0/333175081 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa83005bd70 con 0x7fa83c101d00 2024-03-16T10:51:32.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.847+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa83c074010 con 0x7fa82406dfa0 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.848+0000 7fa8397fa700 1 -- 172.21.15.2:0/333175081 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa83c074010 con 0x7fa82406dfa0 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "osd", 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:51:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:51:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:51:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "13/23 daemons upgraded", 2024-03-16T10:51:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading mds daemons", 2024-03-16T10:51:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:51:32.850 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:51:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa82406dfa0 msgr2=0x7fa824070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa82406dfa0 0x7fa824070460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa83c101b60 tx=0x7fa82c00b410 comp rx=0 tx=0).stop 2024-03-16T10:51:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 msgr2=0x7fa83c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c0731a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa830006010 tx=0x7fa83000bd70 comp rx=0 tx=0).stop 2024-03-16T10:51:32.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 shutdown_connections 2024-03-16T10:51:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa82406dfa0 0x7fa824070460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa83c100b00 0x7fa83c074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 --2- 172.21.15.2:0/333175081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa83c101d00 0x7fa83c0731a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 >> 172.21.15.2:0/333175081 conn(0x7fa83c0fc0a0 msgr2=0x7fa83c1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 shutdown_connections 2024-03-16T10:51:32.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.850+0000 7fa8422bb700 1 -- 172.21.15.2:0/333175081 wait complete. 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: pgmap v192: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2686173974' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:32.966 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:32 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1321440109' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.964+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2022184901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 msgr2=0x7f54b0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.964+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2022184901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101030 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009910 tx=0x7f54a0009c20 comp rx=0 tx=0).stop 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.964+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2022184901 shutdown_connections 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.964+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2022184901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101030 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.964+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2022184901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b00ff9d0 0x7f54b00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.964+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2022184901 >> 172.21.15.2:0/2022184901 conn(0x7f54b00faf70 msgr2=0x7f54b00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.965+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2022184901 shutdown_connections 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.965+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2022184901 wait complete. 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.965+0000 7f54b7e8d700 1 Processor -- start 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.965+0000 7f54b7e8d700 1 -- start start 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b7e8d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b00ff9d0 0x7f54b01038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b7e8d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b7e8d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01024f0 con 0x7f54b0100bd0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b7e8d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0102630 con 0x7f54b00ff9d0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5428700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5c29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b00ff9d0 0x7f54b01038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5428700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48454/0 (socket says 172.21.15.2:48454) 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5428700 1 -- 172.21.15.2:0/2843175014 learned_addr learned my addr 172.21.15.2:0/2843175014 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5428700 1 -- 172.21.15.2:0/2843175014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b00ff9d0 msgr2=0x7f54b01038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5428700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b00ff9d0 0x7f54b01038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5428700 1 -- 172.21.15.2:0/2843175014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a00095f0 con 0x7f54b0100bd0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5c29700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b00ff9d0 0x7f54b01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.966+0000 7f54b5428700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101fb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f54a000b800 tx=0x7f54a0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.967+0000 7f54a6ffd700 1 -- 172.21.15.2:0/2843175014 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a001d070 con 0x7f54b0100bd0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.967+0000 7f54a6ffd700 1 -- 172.21.15.2:0/2843175014 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54a00052a0 con 0x7f54b0100bd0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.967+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b01028b0 con 0x7f54b0100bd0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.967+0000 7f54a6ffd700 1 -- 172.21.15.2:0/2843175014 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a0017400 con 0x7f54b0100bd0 2024-03-16T10:51:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.967+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b0102e20 con 0x7f54b0100bd0 2024-03-16T10:51:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.968+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b004ea80 con 0x7f54b0100bd0 2024-03-16T10:51:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.968+0000 7f54a6ffd700 1 -- 172.21.15.2:0/2843175014 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f54a000f820 con 0x7f54b0100bd0 2024-03-16T10:51:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.969+0000 7f54a6ffd700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f549c06dfa0 0x7f549c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:51:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.969+0000 7f54a6ffd700 1 -- 172.21.15.2:0/2843175014 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f54a008eed0 con 0x7f54b0100bd0 2024-03-16T10:51:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.969+0000 7f54b5c29700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f549c06dfa0 0x7f549c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:51:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.969+0000 7f54b5c29700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f549c06dfa0 0x7f549c070460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54b0100a30 tx=0x7f54ac00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:51:32.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:32.975+0000 7f54a6ffd700 1 -- 172.21.15.2:0/2843175014 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54a005bfa0 con 0x7f54b0100bd0 2024-03-16T10:51:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: pgmap v192: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2686173974' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:32.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:32.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:32 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1321440109' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:51:33.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.270+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f54b0102e20 con 0x7f54b0100bd0 2024-03-16T10:51:33.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.271+0000 7f54a6ffd700 1 -- 172.21.15.2:0/2843175014 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f54a005bb30 con 0x7f54b0100bd0 2024-03-16T10:51:33.272 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:51:33.272 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:51:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f549c06dfa0 msgr2=0x7f549c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f549c06dfa0 0x7f549c070460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54b0100a30 tx=0x7f54ac00b040 comp rx=0 tx=0).stop 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 msgr2=0x7f54b0101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101fb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f54a000b800 tx=0x7f54a0005f00 comp rx=0 tx=0).stop 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 shutdown_connections 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f549c06dfa0 0x7f549c070460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b00ff9d0 0x7f54b01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 --2- 172.21.15.2:0/2843175014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54b0100bd0 0x7f54b0101fb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:51:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 >> 172.21.15.2:0/2843175014 conn(0x7f54b00faf70 msgr2=0x7f54b0068d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:51:33.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 shutdown_connections 2024-03-16T10:51:33.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:51:33.273+0000 7f54b7e8d700 1 -- 172.21.15.2:0/2843175014 wait complete. 2024-03-16T10:51:33.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:33 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.2:6828/1125443118,v1:172.21.15.2:6829/1125443118] up:boot 2024-03-16T10:51:33.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:33 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:51:33.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:33 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:51:33.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:33 smithi026 ceph-mon[82600]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:33.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:33 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2843175014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:51:34.029 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:33 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.2:6828/1125443118,v1:172.21.15.2:6829/1125443118] up:boot 2024-03-16T10:51:34.029 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:33 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.dzzagf=up:active} 3 up:standby 2024-03-16T10:51:34.030 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:33 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:51:34.030 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:33 smithi002 ceph-mon[94239]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:51:34.030 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:33 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2843175014' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:51:34.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:34 smithi026 ceph-mon[82600]: pgmap v193: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:35.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:34 smithi002 ceph-mon[94239]: pgmap v193: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:35.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:36.009 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:36.943 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:36 smithi002 ceph-mon[94239]: pgmap v194: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:36.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:36 smithi026 ceph-mon[82600]: pgmap v194: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:38.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:38 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:38.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:38 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:38.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:38 smithi026 ceph-mon[82600]: pgmap v195: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:39.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:38 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:39.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:38 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:39.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:38 smithi002 ceph-mon[94239]: pgmap v195: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:39.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:39 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:39.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:39 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:40.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:39 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:40.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:39 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:40.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:40 smithi026 ceph-mon[82600]: pgmap v196: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:41.040 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:40 smithi002 ceph-mon[94239]: pgmap v196: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:43.184 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: pgmap v197: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.185 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:42 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi026.dzzagf"]}]: dispatch 2024-03-16T10:51:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: pgmap v197: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:51:43.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:42 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi026.dzzagf"]}]: dispatch 2024-03-16T10:51:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:43 smithi026 ceph-mon[82600]: Upgrade: It appears safe to stop mds.cephfs.smithi026.dzzagf 2024-03-16T10:51:44.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:43 smithi002 ceph-mon[94239]: Upgrade: It appears safe to stop mds.cephfs.smithi026.dzzagf 2024-03-16T10:51:45.177 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:44 smithi026 ceph-mon[82600]: pgmap v198: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:45.178 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:45.178 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:51:45.178 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:44 smithi002 ceph-mon[94239]: pgmap v198: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.dzzagf", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:51:45.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:51:46.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:45 smithi002 ceph-mon[94239]: Upgrade: Updating mds.cephfs.smithi026.dzzagf 2024-03-16T10:51:46.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:45 smithi002 ceph-mon[94239]: Deploying daemon mds.cephfs.smithi026.dzzagf on smithi026 2024-03-16T10:51:46.125 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:45 smithi026 ceph-mon[82600]: Upgrade: Updating mds.cephfs.smithi026.dzzagf 2024-03-16T10:51:46.125 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:45 smithi026 ceph-mon[82600]: Deploying daemon mds.cephfs.smithi026.dzzagf on smithi026 2024-03-16T10:51:47.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[94235]: 2024-03-16T10:51:46.662+0000 7fa1164d9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:51:47.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: pgmap v199: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:47.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:51:47.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:51:47.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: osdmap e85: 6 total, 6 up, 6 in 2024-03-16T10:51:47.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: Standby daemon mds.cephfs.smithi026.ojzcsw assigned to filesystem cephfs as rank 0 2024-03-16T10:51:47.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:51:47.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-16T10:51:47.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:46 smithi002 ceph-mon[94239]: fsmap cephfs:1/1 {0=cephfs.smithi026.ojzcsw=up:replay} 2 up:standby 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: pgmap v199: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: osdmap e85: 6 total, 6 up, 6 in 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: Standby daemon mds.cephfs.smithi026.ojzcsw assigned to filesystem cephfs as rank 0 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-16T10:51:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:46 smithi026 ceph-mon[82600]: fsmap cephfs:1/1 {0=cephfs.smithi026.ojzcsw=up:replay} 2 up:standby 2024-03-16T10:51:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:48 smithi026 ceph-mon[82600]: pgmap v201: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-16T10:51:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:48 smithi002 ceph-mon[94239]: pgmap v201: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-16T10:51:51.164 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:50 smithi002 ceph-mon[94239]: pgmap v202: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-03-16T10:51:51.164 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:51.164 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:50 smithi026 ceph-mon[82600]: pgmap v202: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-03-16T10:51:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:51:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:52 smithi026 ceph-mon[82600]: pgmap v203: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-03-16T10:51:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:52 smithi002 ceph-mon[94239]: pgmap v203: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-03-16T10:51:55.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:54 smithi026 ceph-mon[82600]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-16T10:51:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:54 smithi002 ceph-mon[94239]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-16T10:51:56.979 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:56 smithi026 ceph-mon[82600]: pgmap v205: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-03-16T10:51:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:56 smithi002 ceph-mon[94239]: pgmap v205: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-03-16T10:51:58.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:57 smithi026 ceph-mon[82600]: reconnect by client.24305 192.168.0.1:0/3141458606 after 0.000999985 2024-03-16T10:51:58.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:57 smithi026 ceph-mon[82600]: reconnect by client.14544 192.168.0.1:0/676582552 after 0.000999985 2024-03-16T10:51:58.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:57 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:reconnect 2024-03-16T10:51:58.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:57 smithi026 ceph-mon[82600]: fsmap cephfs:1/1 {0=cephfs.smithi026.ojzcsw=up:reconnect} 2 up:standby 2024-03-16T10:51:58.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:57 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:58.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:57 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:58.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:57 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:57 smithi002 ceph-mon[94239]: reconnect by client.24305 192.168.0.1:0/3141458606 after 0.000999985 2024-03-16T10:51:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:57 smithi002 ceph-mon[94239]: reconnect by client.14544 192.168.0.1:0/676582552 after 0.000999985 2024-03-16T10:51:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:57 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:reconnect 2024-03-16T10:51:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:57 smithi002 ceph-mon[94239]: fsmap cephfs:1/1 {0=cephfs.smithi026.ojzcsw=up:reconnect} 2 up:standby 2024-03-16T10:51:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:57 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:57 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:51:58.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:57 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:51:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:58 smithi026 ceph-mon[82600]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-03-16T10:51:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:58 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:rejoin 2024-03-16T10:51:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:58 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.26:6824/2953126152,v1:172.21.15.26:6825/2953126152] up:boot 2024-03-16T10:51:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:58 smithi026 ceph-mon[82600]: fsmap cephfs:1/1 {0=cephfs.smithi026.ojzcsw=up:rejoin} 3 up:standby 2024-03-16T10:51:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:58 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:51:58.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:58 smithi026 ceph-mon[82600]: daemon mds.cephfs.smithi026.ojzcsw is now active in filesystem cephfs as rank 0 2024-03-16T10:51:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:58 smithi002 ceph-mon[94239]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-03-16T10:51:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:58 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:rejoin 2024-03-16T10:51:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:58 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.26:6824/2953126152,v1:172.21.15.26:6825/2953126152] up:boot 2024-03-16T10:51:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:58 smithi002 ceph-mon[94239]: fsmap cephfs:1/1 {0=cephfs.smithi026.ojzcsw=up:rejoin} 3 up:standby 2024-03-16T10:51:59.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:58 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:51:59.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:58 smithi002 ceph-mon[94239]: daemon mds.cephfs.smithi026.ojzcsw is now active in filesystem cephfs as rank 0 2024-03-16T10:52:00.104 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:59 smithi026 ceph-mon[82600]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:52:00.104 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:59 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:active 2024-03-16T10:52:00.104 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:51:59 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi026.ojzcsw=up:active} 3 up:standby 2024-03-16T10:52:00.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:59 smithi002 ceph-mon[94239]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:52:00.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:59 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] up:active 2024-03-16T10:52:00.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:51:59 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi026.ojzcsw=up:active} 3 up:standby 2024-03-16T10:52:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:00 smithi026 ceph-mon[82600]: pgmap v207: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T10:52:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:00 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:00 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:01.270 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:00 smithi002 ceph-mon[94239]: pgmap v207: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T10:52:01.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:01.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:00 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:03.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:02 smithi026 ceph-mon[82600]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T10:52:03.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:03.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:02 smithi002 ceph-mon[94239]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T10:52:03.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:03.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 -- 172.21.15.2:0/2601144760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 msgr2=0x7f2b3c104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 --2- 172.21.15.2:0/2601144760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c104150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009910 tx=0x7f2b24009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 -- 172.21.15.2:0/2601144760 shutdown_connections 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 --2- 172.21.15.2:0/2601144760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c104150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 --2- 172.21.15.2:0/2601144760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c0ff350 0x7f2b3c101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 -- 172.21.15.2:0/2601144760 >> 172.21.15.2:0/2601144760 conn(0x7f2b3c0faf70 msgr2=0x7f2b3c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 -- 172.21.15.2:0/2601144760 shutdown_connections 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.393+0000 7f2b412ce700 1 -- 172.21.15.2:0/2601144760 wait complete. 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.394+0000 7f2b412ce700 1 Processor -- start 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.394+0000 7f2b412ce700 1 -- start start 2024-03-16T10:52:03.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.394+0000 7f2b412ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c0ff350 0x7f2b3c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.394+0000 7f2b412ce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.394+0000 7f2b412ce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c19d610 con 0x7f2b3c101cc0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.394+0000 7f2b412ce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c19d750 con 0x7f2b3c0ff350 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.394+0000 7f2b3a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c19cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54692/0 (socket says 172.21.15.2:54692) 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3a7fc700 1 -- 172.21.15.2:0/309374735 learned_addr learned my addr 172.21.15.2:0/309374735 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3affd700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c0ff350 0x7f2b3c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3a7fc700 1 -- 172.21.15.2:0/309374735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c0ff350 msgr2=0x7f2b3c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3a7fc700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c0ff350 0x7f2b3c19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3a7fc700 1 -- 172.21.15.2:0/309374735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b240095f0 con 0x7f2b3c101cc0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3affd700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c0ff350 0x7f2b3c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b3a7fc700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c19cff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b24006010 tx=0x7f2b2400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b33fff700 1 -- 172.21.15.2:0/309374735 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b2401d070 con 0x7f2b3c101cc0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c1a2150 con 0x7f2b3c101cc0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b33fff700 1 -- 172.21.15.2:0/309374735 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b2400f460 con 0x7f2b3c101cc0 2024-03-16T10:52:03.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b33fff700 1 -- 172.21.15.2:0/309374735 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b24021640 con 0x7f2b3c101cc0 2024-03-16T10:52:03.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.395+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c1a26c0 con 0x7f2b3c101cc0 2024-03-16T10:52:03.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.397+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c04ea80 con 0x7f2b3c101cc0 2024-03-16T10:52:03.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.397+0000 7f2b33fff700 1 -- 172.21.15.2:0/309374735 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2b2400fab0 con 0x7f2b3c101cc0 2024-03-16T10:52:03.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.398+0000 7f2b33fff700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b2806dfa0 0x7f2b28070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:03.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.398+0000 7f2b3affd700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b2806dfa0 0x7f2b28070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:03.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.398+0000 7f2b33fff700 1 -- 172.21.15.2:0/309374735 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2b2408f190 con 0x7f2b3c101cc0 2024-03-16T10:52:03.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.398+0000 7f2b3affd700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b2806dfa0 0x7f2b28070460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009dd0 tx=0x7f2b2c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:03.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.403+0000 7f2b33fff700 1 -- 172.21.15.2:0/309374735 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b2405c040 con 0x7f2b3c101cc0 2024-03-16T10:52:03.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.633+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b3c102700 con 0x7f2b2806dfa0 2024-03-16T10:52:03.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.634+0000 7f2b33fff700 1 -- 172.21.15.2:0/309374735 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2b3c102700 con 0x7f2b2806dfa0 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b2806dfa0 msgr2=0x7f2b28070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b2806dfa0 0x7f2b28070460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009dd0 tx=0x7f2b2c009040 comp rx=0 tx=0).stop 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 msgr2=0x7f2b3c19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c19cff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b24006010 tx=0x7f2b2400be20 comp rx=0 tx=0).stop 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 shutdown_connections 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b2806dfa0 0x7f2b28070460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c0ff350 0x7f2b3c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 --2- 172.21.15.2:0/309374735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2b3c101cc0 0x7f2b3c19cff0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 >> 172.21.15.2:0/309374735 conn(0x7f2b3c0faf70 msgr2=0x7f2b3c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:03.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 shutdown_connections 2024-03-16T10:52:03.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.636+0000 7f2b412ce700 1 -- 172.21.15.2:0/309374735 wait complete. 2024-03-16T10:52:03.647 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.749+0000 7faaf74ef700 1 -- 172.21.15.2:0/1686871921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0101ce0 msgr2=0x7faaf0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.749+0000 7faaf74ef700 1 --2- 172.21.15.2:0/1686871921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0101ce0 0x7faaf0102140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faae0009910 tx=0x7faae0009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 -- 172.21.15.2:0/1686871921 shutdown_connections 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 --2- 172.21.15.2:0/1686871921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0101ce0 0x7faaf0102140 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 --2- 172.21.15.2:0/1686871921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf0100ae0 0x7faaf0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 -- 172.21.15.2:0/1686871921 >> 172.21.15.2:0/1686871921 conn(0x7faaf00fc0a0 msgr2=0x7faaf00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 -- 172.21.15.2:0/1686871921 shutdown_connections 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 -- 172.21.15.2:0/1686871921 wait complete. 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 Processor -- start 2024-03-16T10:52:03.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 -- start start 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0100ae0 0x7faaf0074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf0101ce0 0x7faaf00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00736e0 con 0x7faaf0100ae0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.750+0000 7faaf74ef700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0073820 con 0x7faaf0101ce0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf528b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0100ae0 0x7faaf0074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf528b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0100ae0 0x7faaf0074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54718/0 (socket says 172.21.15.2:54718) 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf528b700 1 -- 172.21.15.2:0/2222387458 learned_addr learned my addr 172.21.15.2:0/2222387458 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf4a8a700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf0101ce0 0x7faaf00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf528b700 1 -- 172.21.15.2:0/2222387458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf0101ce0 msgr2=0x7faaf00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf528b700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf0101ce0 0x7faaf00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf528b700 1 -- 172.21.15.2:0/2222387458 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae00095f0 con 0x7faaf0100ae0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf528b700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0100ae0 0x7faaf0074b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faaec00d900 tx=0x7faaec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faae67fc700 1 -- 172.21.15.2:0/2222387458 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaec0041f0 con 0x7faaf0100ae0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faae67fc700 1 -- 172.21.15.2:0/2222387458 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faaec004350 con 0x7faaf0100ae0 2024-03-16T10:52:03.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf0073b00 con 0x7faaf0100ae0 2024-03-16T10:52:03.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faae67fc700 1 -- 172.21.15.2:0/2222387458 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaec003cf0 con 0x7faaf0100ae0 2024-03-16T10:52:03.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.751+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf019e520 con 0x7faaf0100ae0 2024-03-16T10:52:03.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.752+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf0190660 con 0x7faaf0100ae0 2024-03-16T10:52:03.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.753+0000 7faae67fc700 1 -- 172.21.15.2:0/2222387458 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7faaec009730 con 0x7faaf0100ae0 2024-03-16T10:52:03.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.753+0000 7faae67fc700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faadc06dfa0 0x7faadc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:03.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.753+0000 7faae67fc700 1 -- 172.21.15.2:0/2222387458 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faaec08dbc0 con 0x7faaf0100ae0 2024-03-16T10:52:03.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.753+0000 7faaf4a8a700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faadc06dfa0 0x7faadc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:03.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.754+0000 7faaf4a8a700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faadc06dfa0 0x7faadc070460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faae0009fd0 tx=0x7faae0005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.758+0000 7faae67fc700 1 -- 172.21.15.2:0/2222387458 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faaec05ab90 con 0x7faaf0100ae0 2024-03-16T10:52:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.986+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faaf00611f0 con 0x7faadc06dfa0 2024-03-16T10:52:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.988+0000 7faae67fc700 1 -- 172.21.15.2:0/2222387458 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faaf00611f0 con 0x7faadc06dfa0 2024-03-16T10:52:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faadc06dfa0 msgr2=0x7faadc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faadc06dfa0 0x7faadc070460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faae0009fd0 tx=0x7faae0005960 comp rx=0 tx=0).stop 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0100ae0 msgr2=0x7faaf0074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0100ae0 0x7faaf0074b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faaec00d900 tx=0x7faaec00dc10 comp rx=0 tx=0).stop 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 shutdown_connections 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7faadc06dfa0 0x7faadc070460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaf0100ae0 0x7faaf0074b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 --2- 172.21.15.2:0/2222387458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaf0101ce0 0x7faaf00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 >> 172.21.15.2:0/2222387458 conn(0x7faaf00fc0a0 msgr2=0x7faaf0105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 shutdown_connections 2024-03-16T10:52:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:03.990+0000 7faaf74ef700 1 -- 172.21.15.2:0/2222387458 wait complete. 2024-03-16T10:52:04.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 -- 172.21.15.2:0/3709167066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9940fedb0 msgr2=0x7fc9940ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 --2- 172.21.15.2:0/3709167066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9940fedb0 0x7fc9940ff1d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc990009910 tx=0x7fc990009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:04.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 -- 172.21.15.2:0/3709167066 shutdown_connections 2024-03-16T10:52:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 --2- 172.21.15.2:0/3709167066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc994101d90 0x7fc9941021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 --2- 172.21.15.2:0/3709167066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9940fedb0 0x7fc9940ff1d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 -- 172.21.15.2:0/3709167066 >> 172.21.15.2:0/3709167066 conn(0x7fc9940fa730 msgr2=0x7fc9940fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 -- 172.21.15.2:0/3709167066 shutdown_connections 2024-03-16T10:52:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.095+0000 7fc99c250700 1 -- 172.21.15.2:0/3709167066 wait complete. 2024-03-16T10:52:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc99c250700 1 Processor -- start 2024-03-16T10:52:04.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc99c250700 1 -- start start 2024-03-16T10:52:04.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc99c250700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9940fedb0 0x7fc9941016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc99c250700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994101d90 0x7fc9940ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc99c250700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994100280 con 0x7fc994101d90 2024-03-16T10:52:04.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc99c250700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941003c0 con 0x7fc9940fedb0 2024-03-16T10:52:04.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc9997eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994101d90 0x7fc9940ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc9997eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994101d90 0x7fc9940ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54728/0 (socket says 172.21.15.2:54728) 2024-03-16T10:52:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc9997eb700 1 -- 172.21.15.2:0/1387395226 learned_addr learned my addr 172.21.15.2:0/1387395226 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.096+0000 7fc999fec700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9940fedb0 0x7fc9941016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc9997eb700 1 -- 172.21.15.2:0/1387395226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9940fedb0 msgr2=0x7fc9941016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc9997eb700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9940fedb0 0x7fc9941016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc9997eb700 1 -- 172.21.15.2:0/1387395226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900095f0 con 0x7fc994101d90 2024-03-16T10:52:04.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc9997eb700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994101d90 0x7fc9940ffd40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc98400d8d0 tx=0x7fc98400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:04.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc98affd700 1 -- 172.21.15.2:0/1387395226 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9840098a0 con 0x7fc994101d90 2024-03-16T10:52:04.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc98affd700 1 -- 172.21.15.2:0/1387395226 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc984010460 con 0x7fc994101d90 2024-03-16T10:52:04.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc98affd700 1 -- 172.21.15.2:0/1387395226 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98400f5f0 con 0x7fc994101d90 2024-03-16T10:52:04.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9941006a0 con 0x7fc994101d90 2024-03-16T10:52:04.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.097+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99419e530 con 0x7fc994101d90 2024-03-16T10:52:04.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.098+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994190670 con 0x7fc994101d90 2024-03-16T10:52:04.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.098+0000 7fc98affd700 1 -- 172.21.15.2:0/1387395226 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc984009a00 con 0x7fc994101d90 2024-03-16T10:52:04.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.099+0000 7fc98affd700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc98006dfa0 0x7fc980070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.099+0000 7fc98affd700 1 -- 172.21.15.2:0/1387395226 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc98408dc60 con 0x7fc994101d90 2024-03-16T10:52:04.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.099+0000 7fc999fec700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc98006dfa0 0x7fc980070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.099+0000 7fc999fec700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc98006dfa0 0x7fc980070460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc990000c00 tx=0x7fc990005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:04.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.104+0000 7fc98affd700 1 -- 172.21.15.2:0/1387395226 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc98405abb0 con 0x7fc994101d90 2024-03-16T10:52:04.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.317+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9940611f0 con 0x7fc98006dfa0 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.328+0000 7fc98affd700 1 -- 172.21.15.2:0/1387395226 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc9940611f0 con 0x7fc98006dfa0 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (14m) 26s ago 15m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (15m) 26s ago 15m 8632k - 18.2.2-878-ga752c2f7 f692adb9a265 cadf3f013e03 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (14m) 3s ago 14m 8892k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (4m) 26s ago 15m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (4m) 3s ago 14m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (14m) 26s ago 15m 85.3M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:52:04.329 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (32s) 26s ago 10m 11.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a6b65504c2dc 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (48s) 26s ago 10m 15.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a68e60da4931 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (7s) 3s ago 10m 11.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 745c119b7ff5 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (9m) 3s ago 9m 181M - 18.2.2-878-ga752c2f7 f692adb9a265 1dadc2309b82 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (7m) 26s ago 17m 508M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (6m) 3s ago 14m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (5m) 26s ago 17m 50.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (5m) 3s ago 14m 41.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (15m) 26s ago 15m 17.0M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (14m) 3s ago 14m 16.3M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (4m) 26s ago 13m 86.3M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (3m) 26s ago 13m 88.7M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (2m) 26s ago 12m 81.9M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (2m) 3s ago 12m 109M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (99s) 3s ago 12m 86.6M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:52:04.330 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (64s) 3s ago 11m 90.7M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b cf40ac20a977 2024-03-16T10:52:04.331 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (6m) 26s ago 15m 77.1M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:52:04.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc98006dfa0 msgr2=0x7fc980070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc98006dfa0 0x7fc980070460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc990000c00 tx=0x7fc990005960 comp rx=0 tx=0).stop 2024-03-16T10:52:04.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994101d90 msgr2=0x7fc9940ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994101d90 0x7fc9940ffd40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc98400d8d0 tx=0x7fc98400dbe0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 shutdown_connections 2024-03-16T10:52:04.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc98006dfa0 0x7fc980070460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9940fedb0 0x7fc9941016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 --2- 172.21.15.2:0/1387395226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994101d90 0x7fc9940ffd40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.330+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 >> 172.21.15.2:0/1387395226 conn(0x7fc9940fa730 msgr2=0x7fc994105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:04.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.331+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 shutdown_connections 2024-03-16T10:52:04.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.331+0000 7fc99c250700 1 -- 172.21.15.2:0/1387395226 wait complete. 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 -- 172.21.15.2:0/2881533184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 msgr2=0x7f17f40ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 --2- 172.21.15.2:0/2881533184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f40ffe00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009910 tx=0x7f17e8009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 -- 172.21.15.2:0/2881533184 shutdown_connections 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 --2- 172.21.15.2:0/2881533184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17f4100be0 0x7f17f4101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 --2- 172.21.15.2:0/2881533184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f40ffe00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 -- 172.21.15.2:0/2881533184 >> 172.21.15.2:0/2881533184 conn(0x7f17f40faf80 msgr2=0x7f17f40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 -- 172.21.15.2:0/2881533184 shutdown_connections 2024-03-16T10:52:04.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.435+0000 7f17fa727700 1 -- 172.21.15.2:0/2881533184 wait complete. 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17fa727700 1 Processor -- start 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17fa727700 1 -- start start 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17fa727700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17fa727700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17f4100be0 0x7f17f4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17fa727700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4197030 con 0x7f17f40ff9e0 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17fa727700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4197170 con 0x7f17f4100be0 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f41964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54746/0 (socket says 172.21.15.2:54746) 2024-03-16T10:52:04.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17f3fff700 1 -- 172.21.15.2:0/1289223389 learned_addr learned my addr 172.21.15.2:0/1289223389 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.436+0000 7f17f37fe700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17f4100be0 0x7f17f4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17f3fff700 1 -- 172.21.15.2:0/1289223389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17f4100be0 msgr2=0x7f17f4196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17f3fff700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17f4100be0 0x7f17f4196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17f3fff700 1 -- 172.21.15.2:0/1289223389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e80095f0 con 0x7f17f40ff9e0 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17f3fff700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f41964d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17e80059d0 tx=0x7f17e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17f17fa700 1 -- 172.21.15.2:0/1289223389 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e801d070 con 0x7f17f40ff9e0 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17f17fa700 1 -- 172.21.15.2:0/1289223389 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17e800f460 con 0x7f17f40ff9e0 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f419bb70 con 0x7f17f40ff9e0 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17f17fa700 1 -- 172.21.15.2:0/1289223389 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e8021640 con 0x7f17f40ff9e0 2024-03-16T10:52:04.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.437+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f419c060 con 0x7f17f40ff9e0 2024-03-16T10:52:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.438+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f4190660 con 0x7f17f40ff9e0 2024-03-16T10:52:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.439+0000 7f17f17fa700 1 -- 172.21.15.2:0/1289223389 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f17e800f5d0 con 0x7f17f40ff9e0 2024-03-16T10:52:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.439+0000 7f17f17fa700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f17dc06dfa0 0x7f17dc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.439+0000 7f17f17fa700 1 -- 172.21.15.2:0/1289223389 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f17e808f000 con 0x7f17f40ff9e0 2024-03-16T10:52:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.439+0000 7f17f37fe700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f17dc06dfa0 0x7f17dc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.439+0000 7f17f37fe700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f17dc06dfa0 0x7f17dc070460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f17e4007c60 tx=0x7f17e40073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:04.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.444+0000 7f17f17fa700 1 -- 172.21.15.2:0/1289223389 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17e805bf50 con 0x7f17f40ff9e0 2024-03-16T10:52:04.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.737+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f17f404ea80 con 0x7f17f40ff9e0 2024-03-16T10:52:04.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.737+0000 7f17f17fa700 1 -- 172.21.15.2:0/1289223389 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f17e805bae0 con 0x7f17f40ff9e0 2024-03-16T10:52:04.738 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:52:04.738 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:52:04.738 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:52:04.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:04.738 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 1, 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 3 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 18.2.2-878-ga752c2f7 (a752c2f7898f53056d3000f2350a5865d1d67887) reef (stable)": 1, 2024-03-16T10:52:04.739 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 13 2024-03-16T10:52:04.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:52:04.740 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:52:04.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f17dc06dfa0 msgr2=0x7f17dc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f17dc06dfa0 0x7f17dc070460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f17e4007c60 tx=0x7f17e40073d0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 msgr2=0x7f17f41964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f41964d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17e80059d0 tx=0x7f17e800bd70 comp rx=0 tx=0).stop 2024-03-16T10:52:04.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 shutdown_connections 2024-03-16T10:52:04.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f17dc06dfa0 0x7f17dc070460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f40ff9e0 0x7f17f41964d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 --2- 172.21.15.2:0/1289223389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17f4100be0 0x7f17f4196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.740+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 >> 172.21.15.2:0/1289223389 conn(0x7f17f40faf80 msgr2=0x7f17f4104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:04.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.741+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 shutdown_connections 2024-03-16T10:52:04.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.741+0000 7f17fa727700 1 -- 172.21.15.2:0/1289223389 wait complete. 2024-03-16T10:52:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.857+0000 7f3630cce700 1 -- 172.21.15.2:0/3477675134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c100bd0 msgr2=0x7f362c101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.857+0000 7f3630cce700 1 --2- 172.21.15.2:0/3477675134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c100bd0 0x7f362c101030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3620009910 tx=0x7f3620009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 -- 172.21.15.2:0/3477675134 shutdown_connections 2024-03-16T10:52:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 --2- 172.21.15.2:0/3477675134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c100bd0 0x7f362c101030 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 --2- 172.21.15.2:0/3477675134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362c0ff9d0 0x7f362c0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 -- 172.21.15.2:0/3477675134 >> 172.21.15.2:0/3477675134 conn(0x7f362c0faf70 msgr2=0x7f362c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 -- 172.21.15.2:0/3477675134 shutdown_connections 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 -- 172.21.15.2:0/3477675134 wait complete. 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 Processor -- start 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 -- start start 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.858+0000 7f3630cce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c0ff9d0 0x7f362c196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f3630cce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362c100bd0 0x7f362c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f3630cce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1970e0 con 0x7f362c0ff9d0 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f3630cce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c197220 con 0x7f362c100bd0 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f362a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c0ff9d0 0x7f362c196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f362a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c0ff9d0 0x7f362c196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54764/0 (socket says 172.21.15.2:54764) 2024-03-16T10:52:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f362a59c700 1 -- 172.21.15.2:0/3874203973 learned_addr learned my addr 172.21.15.2:0/3874203973 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f3629d9b700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362c100bd0 0x7f362c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f362a59c700 1 -- 172.21.15.2:0/3874203973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362c100bd0 msgr2=0x7f362c196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f362a59c700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362c100bd0 0x7f362c196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f362a59c700 1 -- 172.21.15.2:0/3874203973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200095f0 con 0x7f362c0ff9d0 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f362a59c700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c0ff9d0 0x7f362c196580 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f361c00e8f0 tx=0x7f361c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f361b7fe700 1 -- 172.21.15.2:0/3874203973 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c009790 con 0x7f362c0ff9d0 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c19bc30 con 0x7f362c0ff9d0 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f361b7fe700 1 -- 172.21.15.2:0/3874203973 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f361c00c810 con 0x7f362c0ff9d0 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f361b7fe700 1 -- 172.21.15.2:0/3874203973 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c003d20 con 0x7f362c0ff9d0 2024-03-16T10:52:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.859+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c19c180 con 0x7f362c0ff9d0 2024-03-16T10:52:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.860+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c04ea80 con 0x7f362c0ff9d0 2024-03-16T10:52:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.861+0000 7f361b7fe700 1 -- 172.21.15.2:0/3874203973 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f361c0041f0 con 0x7f362c0ff9d0 2024-03-16T10:52:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.861+0000 7f361b7fe700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f361406dfb0 0x7f3614070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.861+0000 7f361b7fe700 1 -- 172.21.15.2:0/3874203973 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f361c014070 con 0x7f362c0ff9d0 2024-03-16T10:52:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.861+0000 7f3629d9b700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f361406dfb0 0x7f3614070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.862+0000 7f3629d9b700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f361406dfb0 0x7f3614070470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f362000b800 tx=0x7f3620009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:04.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:04.866+0000 7f361b7fe700 1 -- 172.21.15.2:0/3874203973 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f361c05afc0 con 0x7f362c0ff9d0 2024-03-16T10:52:05.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:04 smithi002 ceph-mon[94239]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:05.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:04 smithi002 ceph-mon[94239]: pgmap v209: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-03-16T10:52:05.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:04 smithi002 ceph-mon[94239]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:05.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:04 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1289223389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:05.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:05.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.119+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f362c19c400 con 0x7f362c0ff9d0 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.119+0000 7f361b7fe700 1 -- 172.21.15.2:0/3874203973 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1885 (secure 0 0 0) 0x7f361c05ab50 con 0x7f362c0ff9d0 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:e26 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 26 2024-03-16T10:52:05.120 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:51:58.832187+0000 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 85 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:52:05.121 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24267} 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 24267 members: 24267 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{0:24267} state up:active seq 151 join_fscid=1 addr [v2:172.21.15.26:6826/2911421579,v1:172.21.15.26:6827/2911421579] compat {c=[1],r=[1],i=[7ff]}] 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:05.122 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:52:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:05.123 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{-1:34408} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:52:05.123 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6828/1125443118,v1:172.21.15.2:6829/1125443118] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:52:05.123 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{-1:44337} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6824/2953126152,v1:172.21.15.26:6825/2953126152] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.121+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f361406dfb0 msgr2=0x7f3614070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.121+0000 7f3630cce700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f361406dfb0 0x7f3614070470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f362000b800 tx=0x7f3620009f70 comp rx=0 tx=0).stop 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.121+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c0ff9d0 msgr2=0x7f362c196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.121+0000 7f3630cce700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c0ff9d0 0x7f362c196580 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f361c00e8f0 tx=0x7f361c00ec00 comp rx=0 tx=0).stop 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.122+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 shutdown_connections 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.122+0000 7f3630cce700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f361406dfb0 0x7f3614070470 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.122+0000 7f3630cce700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f362c0ff9d0 0x7f362c196580 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.122+0000 7f3630cce700 1 --2- 172.21.15.2:0/3874203973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f362c100bd0 0x7f362c196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.122+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 >> 172.21.15.2:0/3874203973 conn(0x7f362c0faf70 msgr2=0x7f362c0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.122+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 shutdown_connections 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.122+0000 7f3630cce700 1 -- 172.21.15.2:0/3874203973 wait complete. 2024-03-16T10:52:05.124 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-03-16T10:52:05.222 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:04 smithi026 ceph-mon[82600]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:05.222 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:04 smithi026 ceph-mon[82600]: pgmap v209: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-03-16T10:52:05.222 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:04 smithi026 ceph-mon[82600]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:05.222 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:04 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1289223389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:05.222 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:05.222 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 -- 172.21.15.2:0/3404999768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c100370 msgr2=0x7f7d7c1007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 --2- 172.21.15.2:0/3404999768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c100370 0x7f7d7c1007d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009910 tx=0x7f7d78009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 -- 172.21.15.2:0/3404999768 shutdown_connections 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 --2- 172.21.15.2:0/3404999768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c100370 0x7f7d7c1007d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 --2- 172.21.15.2:0/3404999768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d7c0ff170 0x7f7d7c0ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 -- 172.21.15.2:0/3404999768 >> 172.21.15.2:0/3404999768 conn(0x7f7d7c0fa730 msgr2=0x7f7d7c0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 -- 172.21.15.2:0/3404999768 shutdown_connections 2024-03-16T10:52:05.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.229+0000 7f7d83900700 1 -- 172.21.15.2:0/3404999768 wait complete. 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d83900700 1 Processor -- start 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d83900700 1 -- start start 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d83900700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c0ff170 0x7f7d7c106020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d83900700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d7c100370 0x7f7d7c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d83900700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c19b830 con 0x7f7d7c0ff170 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d83900700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c19b970 con 0x7f7d7c100370 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d8169c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c0ff170 0x7f7d7c106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d8169c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c0ff170 0x7f7d7c106020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54776/0 (socket says 172.21.15.2:54776) 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d8169c700 1 -- 172.21.15.2:0/1468810604 learned_addr learned my addr 172.21.15.2:0/1468810604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d80e9b700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d7c100370 0x7f7d7c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d80e9b700 1 -- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c0ff170 msgr2=0x7f7d7c106020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d80e9b700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c0ff170 0x7f7d7c106020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.230+0000 7f7d80e9b700 1 -- 172.21.15.2:0/1468810604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780095f0 con 0x7f7d7c100370 2024-03-16T10:52:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.231+0000 7f7d80e9b700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d7c100370 0x7f7d7c19b2f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d78006010 tx=0x7f7d78005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.231+0000 7f7d727fc700 1 -- 172.21.15.2:0/1468810604 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d7801d070 con 0x7f7d7c100370 2024-03-16T10:52:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.231+0000 7f7d727fc700 1 -- 172.21.15.2:0/1468810604 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d780052a0 con 0x7f7d7c100370 2024-03-16T10:52:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.231+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c19bb70 con 0x7f7d7c100370 2024-03-16T10:52:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.231+0000 7f7d727fc700 1 -- 172.21.15.2:0/1468810604 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d78017400 con 0x7f7d7c100370 2024-03-16T10:52:05.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.231+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c19c0e0 con 0x7f7d7c100370 2024-03-16T10:52:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.232+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c18f570 con 0x7f7d7c100370 2024-03-16T10:52:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.233+0000 7f7d727fc700 1 -- 172.21.15.2:0/1468810604 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f7d78003780 con 0x7f7d7c100370 2024-03-16T10:52:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.233+0000 7f7d727fc700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7d6806dfa0 0x7f7d68070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.233+0000 7f7d727fc700 1 -- 172.21.15.2:0/1468810604 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7d7808f050 con 0x7f7d7c100370 2024-03-16T10:52:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.233+0000 7f7d8169c700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7d6806dfa0 0x7f7d68070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:05.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.233+0000 7f7d8169c700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7d6806dfa0 0x7f7d68070460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005950 tx=0x7f7d6c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:05.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.238+0000 7f7d727fc700 1 -- 172.21.15.2:0/1468810604 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d7805bfa0 con 0x7f7d7c100370 2024-03-16T10:52:05.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.465+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d7c0611f0 con 0x7f7d6806dfa0 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.466+0000 7f7d727fc700 1 -- 172.21.15.2:0/1468810604 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7d7c0611f0 con 0x7f7d6806dfa0 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "osd", 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "15/23 daemons upgraded", 2024-03-16T10:52:05.467 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading mds daemons", 2024-03-16T10:52:05.468 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:52:05.468 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:52:05.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7d6806dfa0 msgr2=0x7f7d68070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7d6806dfa0 0x7f7d68070460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c005950 tx=0x7f7d6c009260 comp rx=0 tx=0).stop 2024-03-16T10:52:05.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d7c100370 msgr2=0x7f7d7c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d7c100370 0x7f7d7c19b2f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7d78006010 tx=0x7f7d78005f00 comp rx=0 tx=0).stop 2024-03-16T10:52:05.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 shutdown_connections 2024-03-16T10:52:05.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7d6806dfa0 0x7f7d68070460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7c0ff170 0x7f7d7c106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 --2- 172.21.15.2:0/1468810604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d7c100370 0x7f7d7c19b2f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 >> 172.21.15.2:0/1468810604 conn(0x7f7d7c0fa730 msgr2=0x7f7d7c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:05.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 shutdown_connections 2024-03-16T10:52:05.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.468+0000 7f7d83900700 1 -- 172.21.15.2:0/1468810604 wait complete. 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 -- 172.21.15.2:0/374130505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380103ea0 msgr2=0x7f4380106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 --2- 172.21.15.2:0/374130505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380103ea0 0x7f4380106290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4370009910 tx=0x7f4370009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 -- 172.21.15.2:0/374130505 shutdown_connections 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 --2- 172.21.15.2:0/374130505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380103ea0 0x7f4380106290 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 --2- 172.21.15.2:0/374130505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4380101570 0x7f4380103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 -- 172.21.15.2:0/374130505 >> 172.21.15.2:0/374130505 conn(0x7f43800faf80 msgr2=0x7f43800fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 -- 172.21.15.2:0/374130505 shutdown_connections 2024-03-16T10:52:05.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.577+0000 7f4385b32700 1 -- 172.21.15.2:0/374130505 wait complete. 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f4385b32700 1 Processor -- start 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f4385b32700 1 -- start start 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f4385b32700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101570 0x7f43801964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f4385b32700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4380103ea0 0x7f4380196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f4385b32700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4380196fb0 con 0x7f4380101570 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f4385b32700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43801970f0 con 0x7f4380103ea0 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101570 0x7f43801964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f437f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101570 0x7f43801964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54790/0 (socket says 172.21.15.2:54790) 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f437f7fe700 1 -- 172.21.15.2:0/991830244 learned_addr learned my addr 172.21.15.2:0/991830244 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:05.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.578+0000 7f437effd700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4380103ea0 0x7f4380196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.579+0000 7f437f7fe700 1 -- 172.21.15.2:0/991830244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4380103ea0 msgr2=0x7f4380196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.579+0000 7f437f7fe700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4380103ea0 0x7f4380196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.579+0000 7f437f7fe700 1 -- 172.21.15.2:0/991830244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700095f0 con 0x7f4380101570 2024-03-16T10:52:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.579+0000 7f437f7fe700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101570 0x7f43801964e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f437400c930 tx=0x7f437400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.579+0000 7f437cff9700 1 -- 172.21.15.2:0/991830244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4374007ad0 con 0x7f4380101570 2024-03-16T10:52:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.579+0000 7f437cff9700 1 -- 172.21.15.2:0/991830244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f437400ce80 con 0x7f4380101570 2024-03-16T10:52:05.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.579+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438019bb60 con 0x7f4380101570 2024-03-16T10:52:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.580+0000 7f437cff9700 1 -- 172.21.15.2:0/991830244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4374018630 con 0x7f4380101570 2024-03-16T10:52:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.580+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438019c080 con 0x7f4380101570 2024-03-16T10:52:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.581+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438004ea80 con 0x7f4380101570 2024-03-16T10:52:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.581+0000 7f437cff9700 1 -- 172.21.15.2:0/991830244 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4374007c30 con 0x7f4380101570 2024-03-16T10:52:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.581+0000 7f437cff9700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f436806dfa0 0x7f4368070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.581+0000 7f437cff9700 1 -- 172.21.15.2:0/991830244 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f437408dcc0 con 0x7f4380101570 2024-03-16T10:52:05.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.581+0000 7f437effd700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f436806dfa0 0x7f4368070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:05.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.582+0000 7f437effd700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f436806dfa0 0x7f4368070460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4370009fd0 tx=0x7f4370005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:05.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.590+0000 7f437cff9700 1 -- 172.21.15.2:0/991830244 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f437405ac90 con 0x7f4380101570 2024-03-16T10:52:05.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.892+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f438019ca00 con 0x7f4380101570 2024-03-16T10:52:05.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.892+0000 7f437cff9700 1 -- 172.21.15.2:0/991830244 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f437405a820 con 0x7f4380101570 2024-03-16T10:52:05.893 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:52:05.893 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:52:05.893 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:52:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f436806dfa0 msgr2=0x7f4368070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f436806dfa0 0x7f4368070460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4370009fd0 tx=0x7f4370005960 comp rx=0 tx=0).stop 2024-03-16T10:52:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101570 msgr2=0x7f43801964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101570 0x7f43801964e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f437400c930 tx=0x7f437400cc40 comp rx=0 tx=0).stop 2024-03-16T10:52:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 shutdown_connections 2024-03-16T10:52:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f436806dfa0 0x7f4368070460 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4380101570 0x7f43801964e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 --2- 172.21.15.2:0/991830244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4380103ea0 0x7f4380196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:05.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 >> 172.21.15.2:0/991830244 conn(0x7f43800faf80 msgr2=0x7f43800fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:05.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 shutdown_connections 2024-03-16T10:52:05.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:05.894+0000 7f4385b32700 1 -- 172.21.15.2:0/991830244 wait complete. 2024-03-16T10:52:06.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3874203973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi026.ojzcsw"]}]: dispatch 2024-03-16T10:52:06.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:05 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/991830244' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:52:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3874203973' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:52:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:06.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi026.ojzcsw"]}]: dispatch 2024-03-16T10:52:06.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:05 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/991830244' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:52:07.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:06 smithi026 ceph-mon[82600]: from='client.44361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:07.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:06 smithi026 ceph-mon[82600]: Upgrade: It appears safe to stop mds.cephfs.smithi026.ojzcsw 2024-03-16T10:52:07.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:06 smithi026 ceph-mon[82600]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 6.3 MiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-03-16T10:52:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:06 smithi002 ceph-mon[94239]: from='client.44361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:06 smithi002 ceph-mon[94239]: Upgrade: It appears safe to stop mds.cephfs.smithi026.ojzcsw 2024-03-16T10:52:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:06 smithi002 ceph-mon[94239]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 6.3 MiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-03-16T10:52:08.620 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:08 smithi026 ceph-mon[82600]: Upgrade: Updating mds.cephfs.smithi026.ojzcsw 2024-03-16T10:52:08.621 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:08.621 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:52:08.621 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:08.621 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:08 smithi026 ceph-mon[82600]: Deploying daemon mds.cephfs.smithi026.ojzcsw on smithi026 2024-03-16T10:52:08.621 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:08 smithi026 ceph-mon[82600]: pgmap v211: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-03-16T10:52:08.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:08 smithi002 ceph-mon[94239]: Upgrade: Updating mds.cephfs.smithi026.ojzcsw 2024-03-16T10:52:08.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:08.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi026.ojzcsw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T10:52:08.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:08.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:08 smithi002 ceph-mon[94239]: Deploying daemon mds.cephfs.smithi026.ojzcsw on smithi026 2024-03-16T10:52:08.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:08 smithi002 ceph-mon[94239]: pgmap v211: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 4.6 KiB/s wr, 6 op/s 2024-03-16T10:52:09.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[94235]: 2024-03-16T10:52:09.467+0000 7fa1164d9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:52:09.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-mon[94239]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:52:09.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-mon[94239]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:52:09.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-mon[94239]: osdmap e86: 6 total, 6 up, 6 in 2024-03-16T10:52:09.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-mon[94239]: Standby daemon mds.cephfs.smithi002.qvbseu assigned to filesystem cephfs as rank 0 2024-03-16T10:52:09.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-mon[94239]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:52:09.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-mon[94239]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-16T10:52:09.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:09 smithi002 ceph-mon[94239]: fsmap cephfs:1/1 {0=cephfs.smithi002.qvbseu=up:replay} 2 up:standby 2024-03-16T10:52:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:09 smithi026 ceph-mon[82600]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T10:52:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:09 smithi026 ceph-mon[82600]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:52:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:09 smithi026 ceph-mon[82600]: osdmap e86: 6 total, 6 up, 6 in 2024-03-16T10:52:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:09 smithi026 ceph-mon[82600]: Standby daemon mds.cephfs.smithi002.qvbseu assigned to filesystem cephfs as rank 0 2024-03-16T10:52:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:09 smithi026 ceph-mon[82600]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T10:52:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:09 smithi026 ceph-mon[82600]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-16T10:52:09.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:09 smithi026 ceph-mon[82600]: fsmap cephfs:1/1 {0=cephfs.smithi002.qvbseu=up:replay} 2 up:standby 2024-03-16T10:52:10.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:10 smithi002 ceph-mon[94239]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 5.3 KiB/s wr, 6 op/s 2024-03-16T10:52:10.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:10 smithi026 ceph-mon[82600]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 5.3 KiB/s wr, 6 op/s 2024-03-16T10:52:11.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:11 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:52:11.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:11 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:52:12.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:12 smithi026 ceph-mon[82600]: pgmap v214: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-03-16T10:52:13.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:12 smithi002 ceph-mon[94239]: pgmap v214: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 5.2 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-03-16T10:52:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:14 smithi026 ceph-mon[82600]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-16T10:52:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:14 smithi002 ceph-mon[94239]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-16T10:52:17.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:16 smithi026 ceph-mon[82600]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-16T10:52:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:16 smithi002 ceph-mon[94239]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-16T10:52:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:18 smithi026 ceph-mon[82600]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-03-16T10:52:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:18 smithi002 ceph-mon[94239]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-03-16T10:52:20.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: reconnect by client.24305 192.168.0.1:0/3141458606 after 0.000999978 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: reconnect by client.14544 192.168.0.1:0/676582552 after 0.000999978 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:reconnect 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: fsmap cephfs:1/1 {0=cephfs.smithi002.qvbseu=up:reconnect} 2 up:standby 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:20.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: reconnect by client.24305 192.168.0.1:0/3141458606 after 0.000999978 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: reconnect by client.14544 192.168.0.1:0/676582552 after 0.000999978 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:reconnect 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: fsmap cephfs:1/1 {0=cephfs.smithi002.qvbseu=up:reconnect} 2 up:standby 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:21.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:22.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:21 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:rejoin 2024-03-16T10:52:22.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:21 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.26:6826/1298103624,v1:172.21.15.26:6827/1298103624] up:boot 2024-03-16T10:52:22.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:21 smithi026 ceph-mon[82600]: fsmap cephfs:1/1 {0=cephfs.smithi002.qvbseu=up:rejoin} 3 up:standby 2024-03-16T10:52:22.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:21 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:52:22.202 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:21 smithi026 ceph-mon[82600]: daemon mds.cephfs.smithi002.qvbseu is now active in filesystem cephfs as rank 0 2024-03-16T10:52:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:21 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:rejoin 2024-03-16T10:52:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:21 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.26:6826/1298103624,v1:172.21.15.26:6827/1298103624] up:boot 2024-03-16T10:52:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:21 smithi002 ceph-mon[94239]: fsmap cephfs:1/1 {0=cephfs.smithi002.qvbseu=up:rejoin} 3 up:standby 2024-03-16T10:52:22.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:21 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:52:22.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:21 smithi002 ceph-mon[94239]: daemon mds.cephfs.smithi002.qvbseu is now active in filesystem cephfs as rank 0 2024-03-16T10:52:23.164 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:22 smithi026 ceph-mon[82600]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-03-16T10:52:23.164 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:22 smithi026 ceph-mon[82600]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:52:23.164 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:22 smithi026 ceph-mon[82600]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:active 2024-03-16T10:52:23.164 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:22 smithi026 ceph-mon[82600]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 3 up:standby 2024-03-16T10:52:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:22 smithi002 ceph-mon[94239]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-03-16T10:52:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:22 smithi002 ceph-mon[94239]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T10:52:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:22 smithi002 ceph-mon[94239]: mds.? [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] up:active 2024-03-16T10:52:23.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:22 smithi002 ceph-mon[94239]: fsmap cephfs:1 {0=cephfs.smithi002.qvbseu=up:active} 3 up:standby 2024-03-16T10:52:24.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:24 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:24.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:24 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:24.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:24 smithi026 ceph-mon[82600]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T10:52:25.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:24 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:25.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:24 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:25.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:24 smithi002 ceph-mon[94239]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T10:52:26.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:25 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:26.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:25 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:26.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:25 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:26.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:25 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:27.226 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:26 smithi026 ceph-mon[82600]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-16T10:52:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:26 smithi002 ceph-mon[94239]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-16T10:52:29.056 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T10:52:29.056 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.056 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.ijcmqs"}]': finished 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:52:29.057 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.qvbseu"}]': finished 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.dzzagf"}]': finished 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.ojzcsw"}]': finished 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.058 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:28 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.ijcmqs"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.ijcmqs"}]': finished 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.qvbseu"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi002.qvbseu"}]': finished 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.dzzagf"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.dzzagf"}]': finished 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.ojzcsw"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi026.ojzcsw"}]': finished 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:29.300 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:28 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:29 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all mds 2024-03-16T10:52:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:29 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all rgw 2024-03-16T10:52:30.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:29 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all rbd-mirror 2024-03-16T10:52:30.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:29 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all mds 2024-03-16T10:52:30.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:29 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all rgw 2024-03-16T10:52:30.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:29 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all rbd-mirror 2024-03-16T10:52:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:30 smithi026 ceph-mon[82600]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-16T10:52:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:30 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:31.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:30 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:52:31.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:30 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:31.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:30 smithi002 ceph-mon[94239]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-16T10:52:31.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:31.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi002", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:52:31.254 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:30 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:32.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:31 smithi026 ceph-mon[82600]: Upgrade: Updating ceph-exporter.smithi002 (1/2) 2024-03-16T10:52:32.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:31 smithi026 ceph-mon[82600]: Deploying daemon ceph-exporter.smithi002 on smithi002 2024-03-16T10:52:32.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:31 smithi002 ceph-mon[94239]: Upgrade: Updating ceph-exporter.smithi002 (1/2) 2024-03-16T10:52:32.273 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:31 smithi002 ceph-mon[94239]: Deploying daemon ceph-exporter.smithi002 on smithi002 2024-03-16T10:52:33.120 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:32 smithi002 ceph-mon[94239]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-16T10:52:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:32 smithi026 ceph-mon[82600]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-16T10:52:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:34 smithi026 ceph-mon[82600]: pgmap v225: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-16T10:52:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:34 smithi002 ceph-mon[94239]: pgmap v225: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-16T10:52:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 -- 172.21.15.2:0/276358007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518100b00 msgr2=0x7fe518100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/276358007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518100b00 0x7fe518100f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe508009910 tx=0x7fe508009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 -- 172.21.15.2:0/276358007 shutdown_connections 2024-03-16T10:52:36.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/276358007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe518101d00 0x7fe518102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/276358007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518100b00 0x7fe518100f20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 -- 172.21.15.2:0/276358007 >> 172.21.15.2:0/276358007 conn(0x7fe5180fc0a0 msgr2=0x7fe5180fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 -- 172.21.15.2:0/276358007 shutdown_connections 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.008+0000 7fe51dbcc700 1 -- 172.21.15.2:0/276358007 wait complete. 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe51dbcc700 1 Processor -- start 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe51dbcc700 1 -- start start 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe51dbcc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe518100b00 0x7fe5181964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe51dbcc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518101d00 0x7fe518196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe51dbcc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe518197020 con 0x7fe518101d00 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe51dbcc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe518197160 con 0x7fe518100b00 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe516ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518101d00 0x7fe518196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe5177fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe518100b00 0x7fe5181964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe516ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518101d00 0x7fe518196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55808/0 (socket says 172.21.15.2:55808) 2024-03-16T10:52:36.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.009+0000 7fe516ffd700 1 -- 172.21.15.2:0/734305708 learned_addr learned my addr 172.21.15.2:0/734305708 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe516ffd700 1 -- 172.21.15.2:0/734305708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe518100b00 msgr2=0x7fe5181964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe516ffd700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe518100b00 0x7fe5181964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe516ffd700 1 -- 172.21.15.2:0/734305708 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080095f0 con 0x7fe518101d00 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe5177fe700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe518100b00 0x7fe5181964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe516ffd700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518101d00 0x7fe518196a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00e920 tx=0x7fe50c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe514ff9700 1 -- 172.21.15.2:0/734305708 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe50c00c9d0 con 0x7fe518101d00 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51819bbc0 con 0x7fe518101d00 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe514ff9700 1 -- 172.21.15.2:0/734305708 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe50c010430 con 0x7fe518101d00 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe514ff9700 1 -- 172.21.15.2:0/734305708 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe50c018630 con 0x7fe518101d00 2024-03-16T10:52:36.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.010+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51819c190 con 0x7fe518101d00 2024-03-16T10:52:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.011+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe518190680 con 0x7fe518101d00 2024-03-16T10:52:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.012+0000 7fe514ff9700 1 -- 172.21.15.2:0/734305708 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fe50c0105a0 con 0x7fe518101d00 2024-03-16T10:52:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.012+0000 7fe514ff9700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe50006dfb0 0x7fe500070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.013+0000 7fe514ff9700 1 -- 172.21.15.2:0/734305708 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe50c014070 con 0x7fe518101d00 2024-03-16T10:52:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.013+0000 7fe5177fe700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe50006dfb0 0x7fe500070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.013+0000 7fe5177fe700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe50006dfb0 0x7fe500070470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b6f0 tx=0x7fe508005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:36.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.017+0000 7fe514ff9700 1 -- 172.21.15.2:0/734305708 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe50c053740 con 0x7fe518101d00 2024-03-16T10:52:36.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.244+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5180611f0 con 0x7fe50006dfb0 2024-03-16T10:52:36.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.245+0000 7fe514ff9700 1 -- 172.21.15.2:0/734305708 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fe5180611f0 con 0x7fe50006dfb0 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe50006dfb0 msgr2=0x7fe500070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe50006dfb0 0x7fe500070470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b6f0 tx=0x7fe508005fb0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518101d00 msgr2=0x7fe518196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518101d00 0x7fe518196a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00e920 tx=0x7fe50c00ec30 comp rx=0 tx=0).stop 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 shutdown_connections 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe50006dfb0 0x7fe500070470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe518100b00 0x7fe5181964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 --2- 172.21.15.2:0/734305708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe518101d00 0x7fe518196a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 >> 172.21.15.2:0/734305708 conn(0x7fe5180fc0a0 msgr2=0x7fe5181052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 shutdown_connections 2024-03-16T10:52:36.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.247+0000 7fe51dbcc700 1 -- 172.21.15.2:0/734305708 wait complete. 2024-03-16T10:52:36.258 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:52:36.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 -- 172.21.15.2:0/3603369193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ff3b0 msgr2=0x7f4a900ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 --2- 172.21.15.2:0/3603369193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ff3b0 0x7f4a900ff7d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009910 tx=0x7f4a8c009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:36.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 -- 172.21.15.2:0/3603369193 shutdown_connections 2024-03-16T10:52:36.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 --2- 172.21.15.2:0/3603369193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a900ffd10 0x7f4a90100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 --2- 172.21.15.2:0/3603369193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ff3b0 0x7f4a900ff7d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 -- 172.21.15.2:0/3603369193 >> 172.21.15.2:0/3603369193 conn(0x7f4a900faf70 msgr2=0x7f4a900fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:36.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 -- 172.21.15.2:0/3603369193 shutdown_connections 2024-03-16T10:52:36.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.375+0000 7f4a97e19700 1 -- 172.21.15.2:0/3603369193 wait complete. 2024-03-16T10:52:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a97e19700 1 Processor -- start 2024-03-16T10:52:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a97e19700 1 -- start start 2024-03-16T10:52:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a97e19700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a900ff3b0 0x7f4a901986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a97e19700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ffd10 0x7f4a90198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a97e19700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90199250 con 0x7f4a900ffd10 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a97e19700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a90199390 con 0x7f4a900ff3b0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a953b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ffd10 0x7f4a90198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a953b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ffd10 0x7f4a90198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55814/0 (socket says 172.21.15.2:55814) 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a953b4700 1 -- 172.21.15.2:0/2777429574 learned_addr learned my addr 172.21.15.2:0/2777429574 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.376+0000 7f4a95bb5700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a900ff3b0 0x7f4a901986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a95bb5700 1 -- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ffd10 msgr2=0x7f4a90198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a95bb5700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ffd10 0x7f4a90198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a95bb5700 1 -- 172.21.15.2:0/2777429574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0095f0 con 0x7f4a900ff3b0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a95bb5700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a900ff3b0 0x7f4a901986f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c006010 tx=0x7f4a8c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a86ffd700 1 -- 172.21.15.2:0/2777429574 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a8c01d070 con 0x7f4a900ff3b0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a86ffd700 1 -- 172.21.15.2:0/2777429574 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a8c0052a0 con 0x7f4a900ff3b0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a86ffd700 1 -- 172.21.15.2:0/2777429574 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a8c017400 con 0x7f4a900ff3b0 2024-03-16T10:52:36.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9019dd40 con 0x7f4a900ff3b0 2024-03-16T10:52:36.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.377+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9019e210 con 0x7f4a900ff3b0 2024-03-16T10:52:36.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.379+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a90192850 con 0x7f4a900ff3b0 2024-03-16T10:52:36.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.379+0000 7f4a86ffd700 1 -- 172.21.15.2:0/2777429574 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4a8c003780 con 0x7f4a900ff3b0 2024-03-16T10:52:36.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.379+0000 7f4a86ffd700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4a7c06dfa0 0x7f4a7c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.379+0000 7f4a86ffd700 1 -- 172.21.15.2:0/2777429574 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4a8c08f0a0 con 0x7f4a900ff3b0 2024-03-16T10:52:36.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.379+0000 7f4a953b4700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4a7c06dfa0 0x7f4a7c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.380+0000 7f4a953b4700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4a7c06dfa0 0x7f4a7c070460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009d20 tx=0x7f4a80009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:36.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.384+0000 7f4a86ffd700 1 -- 172.21.15.2:0/2777429574 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a8c05bfa0 con 0x7f4a900ff3b0 2024-03-16T10:52:36.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.610+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a900611f0 con 0x7f4a7c06dfa0 2024-03-16T10:52:36.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.611+0000 7f4a86ffd700 1 -- 172.21.15.2:0/2777429574 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f4a900611f0 con 0x7f4a7c06dfa0 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4a7c06dfa0 msgr2=0x7f4a7c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4a7c06dfa0 0x7f4a7c070460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009d20 tx=0x7f4a80009260 comp rx=0 tx=0).stop 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a900ff3b0 msgr2=0x7f4a901986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a900ff3b0 0x7f4a901986f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c006010 tx=0x7f4a8c005f00 comp rx=0 tx=0).stop 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 shutdown_connections 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4a7c06dfa0 0x7f4a7c070460 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a900ff3b0 0x7f4a901986f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 --2- 172.21.15.2:0/2777429574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a900ffd10 0x7f4a90198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 >> 172.21.15.2:0/2777429574 conn(0x7f4a900faf70 msgr2=0x7f4a90107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 shutdown_connections 2024-03-16T10:52:36.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.614+0000 7f4a97e19700 1 -- 172.21.15.2:0/2777429574 wait complete. 2024-03-16T10:52:36.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 -- 172.21.15.2:0/551534271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 msgr2=0x7f22e81022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 --2- 172.21.15.2:0/551534271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e81022d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009910 tx=0x7f22dc009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:36.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 -- 172.21.15.2:0/551534271 shutdown_connections 2024-03-16T10:52:36.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 --2- 172.21.15.2:0/551534271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e81022d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 --2- 172.21.15.2:0/551534271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22e80fedf0 0x7f22e80ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 -- 172.21.15.2:0/551534271 >> 172.21.15.2:0/551534271 conn(0x7f22e80fa730 msgr2=0x7f22e80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:36.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 -- 172.21.15.2:0/551534271 shutdown_connections 2024-03-16T10:52:36.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.720+0000 7f22efe50700 1 -- 172.21.15.2:0/551534271 wait complete. 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22efe50700 1 Processor -- start 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22efe50700 1 -- start start 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22efe50700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22e80fedf0 0x7f22e8196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22efe50700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22efe50700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e81970c0 con 0x7f22e8101e70 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22efe50700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8197200 con 0x7f22e80fedf0 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22ed3eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22edbec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22e80fedf0 0x7f22e8196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22ed3eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e8196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55830/0 (socket says 172.21.15.2:55830) 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22edbec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22e80fedf0 0x7f22e8196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:34548/0 (socket says 172.21.15.2:34548) 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.721+0000 7f22ed3eb700 1 -- 172.21.15.2:0/2114207421 learned_addr learned my addr 172.21.15.2:0/2114207421 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22ed3eb700 1 -- 172.21.15.2:0/2114207421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22e80fedf0 msgr2=0x7f22e8196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22ed3eb700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22e80fedf0 0x7f22e8196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22ed3eb700 1 -- 172.21.15.2:0/2114207421 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0095f0 con 0x7f22e8101e70 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22ed3eb700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e8196aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005b40 tx=0x7f22dc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22daffd700 1 -- 172.21.15.2:0/2114207421 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22dc01d070 con 0x7f22e8101e70 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22daffd700 1 -- 172.21.15.2:0/2114207421 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22dc005dc0 con 0x7f22e8101e70 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22daffd700 1 -- 172.21.15.2:0/2114207421 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22dc017400 con 0x7f22e8101e70 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e819bc00 con 0x7f22e8101e70 2024-03-16T10:52:36.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.722+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e819c120 con 0x7f22e8101e70 2024-03-16T10:52:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.724+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e8190690 con 0x7f22e8101e70 2024-03-16T10:52:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.724+0000 7f22daffd700 1 -- 172.21.15.2:0/2114207421 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f22dc00f6d0 con 0x7f22e8101e70 2024-03-16T10:52:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.725+0000 7f22daffd700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f22d406dfb0 0x7f22d4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.725+0000 7f22daffd700 1 -- 172.21.15.2:0/2114207421 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f22dc08f230 con 0x7f22e8101e70 2024-03-16T10:52:36.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.725+0000 7f22edbec700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f22d406dfb0 0x7f22d4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:36.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.725+0000 7f22edbec700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f22d406dfb0 0x7f22d4070470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f22e8068d50 tx=0x7f22e400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:36.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.730+0000 7f22daffd700 1 -- 172.21.15.2:0/2114207421 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f22dc05c130 con 0x7f22e8101e70 2024-03-16T10:52:36.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.973+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f22e80611f0 con 0x7f22d406dfb0 2024-03-16T10:52:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.985+0000 7f22daffd700 1 -- 172.21.15.2:0/2114207421 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f22e80611f0 con 0x7f22d406dfb0 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (14m) 59s ago 16m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 starting - - - - 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (15m) 13s ago 15m 8954k - 18.2.2-878-ga752c2f7 f692adb9a265 1dc71f078208 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (4m) 59s ago 16m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (4m) 13s ago 15m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (14m) 59s ago 15m 85.3M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (65s) 59s ago 10m 11.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a6b65504c2dc 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (81s) 59s ago 10m 15.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a68e60da4931 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (39s) 13s ago 10m 12.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 745c119b7ff5 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (16s) 13s ago 10m 13.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 1cb95b8df6a0 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (7m) 59s ago 17m 508M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:52:36.986 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (6m) 13s ago 15m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (6m) 59s ago 17m 50.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (5m) 13s ago 15m 41.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (16m) 59s ago 16m 17.0M - 1.5.0 0da6a335fe13 dd2603bfde8f 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (15m) 13s ago 15m 15.9M - 1.5.0 0da6a335fe13 272141885db3 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (4m) 59s ago 14m 86.3M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (3m) 59s ago 13m 88.7M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (3m) 59s ago 13m 81.9M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (2m) 13s ago 13m 111M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (2m) 13s ago 12m 87.9M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (97s) 13s ago 12m 92.4M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b cf40ac20a977 2024-03-16T10:52:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (6m) 59s ago 15m 77.1M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f22d406dfb0 msgr2=0x7f22d4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f22d406dfb0 0x7f22d4070470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f22e8068d50 tx=0x7f22e400b410 comp rx=0 tx=0).stop 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 msgr2=0x7f22e8196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e8196aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005b40 tx=0x7f22dc00bd70 comp rx=0 tx=0).stop 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 shutdown_connections 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f22d406dfb0 0x7f22d4070470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22e80fedf0 0x7f22e8196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 --2- 172.21.15.2:0/2114207421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22e8101e70 0x7f22e8196aa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 >> 172.21.15.2:0/2114207421 conn(0x7f22e80fa730 msgr2=0x7f22e8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 shutdown_connections 2024-03-16T10:52:36.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:36.987+0000 7f22efe50700 1 -- 172.21.15.2:0/2114207421 wait complete. 2024-03-16T10:52:37.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 -- 172.21.15.2:0/2303480021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8103e90 msgr2=0x7f0ec8106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 --2- 172.21.15.2:0/2303480021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8103e90 0x7f0ec8106280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009910 tx=0x7f0ebc009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:37.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 -- 172.21.15.2:0/2303480021 shutdown_connections 2024-03-16T10:52:37.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 --2- 172.21.15.2:0/2303480021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8103e90 0x7f0ec8106280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 --2- 172.21.15.2:0/2303480021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ec8101560 0x7f0ec8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 -- 172.21.15.2:0/2303480021 >> 172.21.15.2:0/2303480021 conn(0x7f0ec80faf70 msgr2=0x7f0ec80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:37.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 -- 172.21.15.2:0/2303480021 shutdown_connections 2024-03-16T10:52:37.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.091+0000 7f0ece19b700 1 -- 172.21.15.2:0/2303480021 wait complete. 2024-03-16T10:52:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.092+0000 7f0ece19b700 1 Processor -- start 2024-03-16T10:52:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.092+0000 7f0ece19b700 1 -- start start 2024-03-16T10:52:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.092+0000 7f0ece19b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8101560 0x7f0ec8196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.092+0000 7f0ece19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ec8103e90 0x7f0ec81969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.092+0000 7f0ece19b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec8196fd0 con 0x7f0ec8101560 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.092+0000 7f0ece19b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec8197110 con 0x7f0ec8103e90 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8101560 0x7f0ec8196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8101560 0x7f0ec8196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55852/0 (socket says 172.21.15.2:55852) 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec77fe700 1 -- 172.21.15.2:0/318449602 learned_addr learned my addr 172.21.15.2:0/318449602 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec6ffd700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ec8103e90 0x7f0ec81969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec77fe700 1 -- 172.21.15.2:0/318449602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ec8103e90 msgr2=0x7f0ec81969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec77fe700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ec8103e90 0x7f0ec81969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec77fe700 1 -- 172.21.15.2:0/318449602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc0095f0 con 0x7f0ec8101560 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec77fe700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8101560 0x7f0ec8196470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800d8d0 tx=0x7f0eb800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec4ff9700 1 -- 172.21.15.2:0/318449602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb80098a0 con 0x7f0ec8101560 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec819bb70 con 0x7f0ec8101560 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec4ff9700 1 -- 172.21.15.2:0/318449602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0eb8010460 con 0x7f0ec8101560 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ec4ff9700 1 -- 172.21.15.2:0/318449602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb800f5f0 con 0x7f0ec8101560 2024-03-16T10:52:37.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.093+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec819c0c0 con 0x7f0ec8101560 2024-03-16T10:52:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.094+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec804ea80 con 0x7f0ec8101560 2024-03-16T10:52:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.095+0000 7f0ec4ff9700 1 -- 172.21.15.2:0/318449602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0eb8009a00 con 0x7f0ec8101560 2024-03-16T10:52:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.095+0000 7f0ec4ff9700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0eb006dfb0 0x7f0eb0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.095+0000 7f0ec4ff9700 1 -- 172.21.15.2:0/318449602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0eb808dc60 con 0x7f0ec8101560 2024-03-16T10:52:37.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.096+0000 7f0ec6ffd700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0eb006dfb0 0x7f0eb0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.096+0000 7f0ec6ffd700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0eb006dfb0 0x7f0eb0070470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006010 tx=0x7f0ebc00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:37.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.100+0000 7f0ec4ff9700 1 -- 172.21.15.2:0/318449602 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0eb805ab60 con 0x7f0ec8101560 2024-03-16T10:52:37.144 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:36 smithi026 ceph-mon[82600]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-16T10:52:37.144 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:36 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:37.144 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:36 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:52:37.144 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:36 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:36 smithi002 ceph-mon[94239]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-16T10:52:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:36 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:36 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi026", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-16T10:52:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:36 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:37.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.471+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0ec819ca00 con 0x7f0ec8101560 2024-03-16T10:52:37.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.471+0000 7f0ec4ff9700 1 -- 172.21.15.2:0/318449602 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0eb805a6f0 con 0x7f0ec8101560 2024-03-16T10:52:37.472 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:52:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:52:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:52:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:52:37.473 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:52:37.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0eb006dfb0 msgr2=0x7f0eb0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0eb006dfb0 0x7f0eb0070470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006010 tx=0x7f0ebc00b540 comp rx=0 tx=0).stop 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8101560 msgr2=0x7f0ec8196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8101560 0x7f0ec8196470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800d8d0 tx=0x7f0eb800dbe0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 shutdown_connections 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0eb006dfb0 0x7f0eb0070470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ec8101560 0x7f0ec8196470 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 --2- 172.21.15.2:0/318449602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ec8103e90 0x7f0ec81969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 >> 172.21.15.2:0/318449602 conn(0x7f0ec80faf70 msgr2=0x7f0ec80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 shutdown_connections 2024-03-16T10:52:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.474+0000 7f0ece19b700 1 -- 172.21.15.2:0/318449602 wait complete. 2024-03-16T10:52:37.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.578+0000 7efe17e7a700 1 -- 172.21.15.2:0/2657055817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 msgr2=0x7efe101039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.578+0000 7efe17e7a700 1 --2- 172.21.15.2:0/2657055817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe101039b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009910 tx=0x7efe0c009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:37.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.578+0000 7efe17e7a700 1 -- 172.21.15.2:0/2657055817 shutdown_connections 2024-03-16T10:52:37.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.578+0000 7efe17e7a700 1 --2- 172.21.15.2:0/2657055817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe10103ef0 0x7efe101062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.578+0000 7efe17e7a700 1 --2- 172.21.15.2:0/2657055817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe101039b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.578+0000 7efe17e7a700 1 -- 172.21.15.2:0/2657055817 >> 172.21.15.2:0/2657055817 conn(0x7efe100faf70 msgr2=0x7efe100fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:37.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 -- 172.21.15.2:0/2657055817 shutdown_connections 2024-03-16T10:52:37.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 -- 172.21.15.2:0/2657055817 wait complete. 2024-03-16T10:52:37.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 Processor -- start 2024-03-16T10:52:37.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 -- start start 2024-03-16T10:52:37.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe10196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe10103ef0 0x7efe101969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10196ff0 con 0x7efe101015c0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.579+0000 7efe17e7a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10197130 con 0x7efe10103ef0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15c16700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe10196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15c16700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe10196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55874/0 (socket says 172.21.15.2:55874) 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15c16700 1 -- 172.21.15.2:0/1748659279 learned_addr learned my addr 172.21.15.2:0/1748659279 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15415700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe10103ef0 0x7efe101969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15c16700 1 -- 172.21.15.2:0/1748659279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe10103ef0 msgr2=0x7efe101969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15c16700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe10103ef0 0x7efe101969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15c16700 1 -- 172.21.15.2:0/1748659279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0095f0 con 0x7efe101015c0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe15c16700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe10196490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00b800 tx=0x7efe0c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe06ffd700 1 -- 172.21.15.2:0/1748659279 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0c01d070 con 0x7efe101015c0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe06ffd700 1 -- 172.21.15.2:0/1748659279 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe0c0052a0 con 0x7efe101015c0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1019bb30 con 0x7efe101015c0 2024-03-16T10:52:37.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe06ffd700 1 -- 172.21.15.2:0/1748659279 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0c017400 con 0x7efe101015c0 2024-03-16T10:52:37.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.580+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1019c020 con 0x7efe101015c0 2024-03-16T10:52:37.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.581+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1004ea80 con 0x7efe101015c0 2024-03-16T10:52:37.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.582+0000 7efe06ffd700 1 -- 172.21.15.2:0/1748659279 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7efe0c003780 con 0x7efe101015c0 2024-03-16T10:52:37.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.582+0000 7efe06ffd700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efdfc06dfa0 0x7efdfc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.582+0000 7efe06ffd700 1 -- 172.21.15.2:0/1748659279 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7efe0c08f140 con 0x7efe101015c0 2024-03-16T10:52:37.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.582+0000 7efe15415700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efdfc06dfa0 0x7efdfc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.583+0000 7efe15415700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efdfc06dfa0 0x7efdfc070460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efe00009c60 tx=0x7efe00009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:37.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.588+0000 7efe06ffd700 1 -- 172.21.15.2:0/1748659279 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe0c05c0c0 con 0x7efe101015c0 2024-03-16T10:52:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.841+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe1019c350 con 0x7efe101015c0 2024-03-16T10:52:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.842+0000 7efe06ffd700 1 -- 172.21.15.2:0/1748659279 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1929 (secure 0 0 0) 0x7efe0c056e60 con 0x7efe101015c0 2024-03-16T10:52:37.842 INFO:teuthology.orchestra.run.smithi002.stdout:e31 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 31 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:52:21.877610+0000 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:52:37.843 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 86 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=34408} 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:52:37.844 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 34408 members: 34408 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{0:34408} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6828/1125443118,v1:172.21.15.2:6829/1125443118] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{-1:44337} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6824/2953126152,v1:172.21.15.26:6825/2953126152] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:52:37.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:44367} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6826/1298103624,v1:172.21.15.26:6827/1298103624] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.844+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efdfc06dfa0 msgr2=0x7efdfc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.844+0000 7efe17e7a700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efdfc06dfa0 0x7efdfc070460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efe00009c60 tx=0x7efe00009260 comp rx=0 tx=0).stop 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.844+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 msgr2=0x7efe10196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.844+0000 7efe17e7a700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe10196490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00b800 tx=0x7efe0c005f00 comp rx=0 tx=0).stop 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.844+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 shutdown_connections 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.844+0000 7efe17e7a700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7efdfc06dfa0 0x7efdfc070460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.844+0000 7efe17e7a700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe101015c0 0x7efe10196490 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.845+0000 7efe17e7a700 1 --2- 172.21.15.2:0/1748659279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe10103ef0 0x7efe101969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.845+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 >> 172.21.15.2:0/1748659279 conn(0x7efe100faf70 msgr2=0x7efe100fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.845+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 shutdown_connections 2024-03-16T10:52:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.845+0000 7efe17e7a700 1 -- 172.21.15.2:0/1748659279 wait complete. 2024-03-16T10:52:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 31 2024-03-16T10:52:37.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 -- 172.21.15.2:0/862508680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 msgr2=0x7f4f28100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 --2- 172.21.15.2:0/862508680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f28100ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009910 tx=0x7f4f24009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:37.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 -- 172.21.15.2:0/862508680 shutdown_connections 2024-03-16T10:52:37.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 --2- 172.21.15.2:0/862508680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f28101cd0 0x7f4f28102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 --2- 172.21.15.2:0/862508680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f28100ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 -- 172.21.15.2:0/862508680 >> 172.21.15.2:0/862508680 conn(0x7f4f280fc090 msgr2=0x7f4f280fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:37.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 -- 172.21.15.2:0/862508680 shutdown_connections 2024-03-16T10:52:37.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.954+0000 7f4f3066d700 1 -- 172.21.15.2:0/862508680 wait complete. 2024-03-16T10:52:37.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f3066d700 1 Processor -- start 2024-03-16T10:52:37.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f3066d700 1 -- start start 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f3066d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f281964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f3066d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f28101cd0 0x7f4f28196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f3066d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28197050 con 0x7f4f28100ad0 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f3066d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28197190 con 0x7f4f28101cd0 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f2e409700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f281964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f2e409700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f281964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55900/0 (socket says 172.21.15.2:55900) 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f2e409700 1 -- 172.21.15.2:0/530353119 learned_addr learned my addr 172.21.15.2:0/530353119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:37.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.955+0000 7f4f2dc08700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f28101cd0 0x7f4f28196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f2e409700 1 -- 172.21.15.2:0/530353119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f28101cd0 msgr2=0x7f4f28196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f2e409700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f28101cd0 0x7f4f28196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f2e409700 1 -- 172.21.15.2:0/530353119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f240095f0 con 0x7f4f28100ad0 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f2e409700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f281964f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f240059d0 tx=0x7f4f2400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f1f7fe700 1 -- 172.21.15.2:0/530353119 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f2401d070 con 0x7f4f28100ad0 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f1f7fe700 1 -- 172.21.15.2:0/530353119 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f24005dc0 con 0x7f4f28100ad0 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f1f7fe700 1 -- 172.21.15.2:0/530353119 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f24021410 con 0x7f4f28100ad0 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2819bb90 con 0x7f4f28100ad0 2024-03-16T10:52:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.956+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f28073230 con 0x7f4f28100ad0 2024-03-16T10:52:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.957+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f28190650 con 0x7f4f28100ad0 2024-03-16T10:52:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.958+0000 7f4f1f7fe700 1 -- 172.21.15.2:0/530353119 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4f2400f460 con 0x7f4f28100ad0 2024-03-16T10:52:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.959+0000 7f4f1f7fe700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4f140723b0 0x7f4f14074870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.959+0000 7f4f2dc08700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4f140723b0 0x7f4f14074870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:37.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.959+0000 7f4f1f7fe700 1 -- 172.21.15.2:0/530353119 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4f2408ef40 con 0x7f4f28100ad0 2024-03-16T10:52:37.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.959+0000 7f4f2dc08700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4f140723b0 0x7f4f14074870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005950 tx=0x7f4f1800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:37.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:37.963+0000 7f4f1f7fe700 1 -- 172.21.15.2:0/530353119 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f2405be40 con 0x7f4f28100ad0 2024-03-16T10:52:38.075 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:37 smithi026 ceph-mon[82600]: Upgrade: Updating ceph-exporter.smithi026 (2/2) 2024-03-16T10:52:38.075 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:37 smithi026 ceph-mon[82600]: Deploying daemon ceph-exporter.smithi026 on smithi026 2024-03-16T10:52:38.075 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:37 smithi026 ceph-mon[82600]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:38.075 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:37 smithi026 ceph-mon[82600]: from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:38.075 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:37 smithi026 ceph-mon[82600]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:38.075 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:37 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/318449602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:38.075 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:37 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1748659279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:52:38.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.189+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f280611f0 con 0x7f4f140723b0 2024-03-16T10:52:38.190 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:37 smithi002 ceph-mon[94239]: Upgrade: Updating ceph-exporter.smithi026 (2/2) 2024-03-16T10:52:38.190 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:37 smithi002 ceph-mon[94239]: Deploying daemon ceph-exporter.smithi026 on smithi026 2024-03-16T10:52:38.190 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:37 smithi002 ceph-mon[94239]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:38.190 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:37 smithi002 ceph-mon[94239]: from='client.44375 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:38.190 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:37 smithi002 ceph-mon[94239]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:38.190 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:37 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/318449602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:38.191 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:37 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1748659279' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:52:38.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.191+0000 7f4f1f7fe700 1 -- 172.21.15.2:0/530353119 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f4f280611f0 con 0x7f4f140723b0 2024-03-16T10:52:38.191 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:52:38.191 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:52:38.191 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:52:38.191 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:52:38.191 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "osd", 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mds", 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "16/23 daemons upgraded", 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:52:38.192 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4f140723b0 msgr2=0x7f4f14074870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4f140723b0 0x7f4f14074870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005950 tx=0x7f4f1800b410 comp rx=0 tx=0).stop 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 msgr2=0x7f4f281964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f281964f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f240059d0 tx=0x7f4f2400bd70 comp rx=0 tx=0).stop 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 shutdown_connections 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4f140723b0 0x7f4f14074870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f28100ad0 0x7f4f281964f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 --2- 172.21.15.2:0/530353119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f28101cd0 0x7f4f28196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 >> 172.21.15.2:0/530353119 conn(0x7f4f280fc090 msgr2=0x7f4f281052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:38.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 shutdown_connections 2024-03-16T10:52:38.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.193+0000 7f4f3066d700 1 -- 172.21.15.2:0/530353119 wait complete. 2024-03-16T10:52:38.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 -- 172.21.15.2:0/3337960842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c100ae0 msgr2=0x7f0d7c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3337960842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c100ae0 0x7f0d7c100f00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009910 tx=0x7f0d6c009c20 comp rx=0 tx=0).stop 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 -- 172.21.15.2:0/3337960842 shutdown_connections 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3337960842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d7c101ce0 0x7f0d7c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3337960842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c100ae0 0x7f0d7c100f00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 -- 172.21.15.2:0/3337960842 >> 172.21.15.2:0/3337960842 conn(0x7f0d7c0fc0a0 msgr2=0x7f0d7c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 -- 172.21.15.2:0/3337960842 shutdown_connections 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.298+0000 7f0d820f4700 1 -- 172.21.15.2:0/3337960842 wait complete. 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d820f4700 1 Processor -- start 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d820f4700 1 -- start start 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d820f4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d7c100ae0 0x7f0d7c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d820f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c101ce0 0x7f0d7c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d820f4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c197030 con 0x7f0d7c101ce0 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d820f4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c197170 con 0x7f0d7c100ae0 2024-03-16T10:52:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d7affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c101ce0 0x7f0d7c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d7affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c101ce0 0x7f0d7c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55928/0 (socket says 172.21.15.2:55928) 2024-03-16T10:52:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d7affd700 1 -- 172.21.15.2:0/3015344602 learned_addr learned my addr 172.21.15.2:0/3015344602 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:52:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.299+0000 7f0d7b7fe700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d7c100ae0 0x7f0d7c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d7affd700 1 -- 172.21.15.2:0/3015344602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d7c100ae0 msgr2=0x7f0d7c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d7affd700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d7c100ae0 0x7f0d7c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d7affd700 1 -- 172.21.15.2:0/3015344602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c0095f0 con 0x7f0d7c101ce0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d7affd700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c101ce0 0x7f0d7c196a10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d8d0 tx=0x7f0d7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d78ff9700 1 -- 172.21.15.2:0/3015344602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d70009940 con 0x7f0d7c101ce0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d78ff9700 1 -- 172.21.15.2:0/3015344602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d70010460 con 0x7f0d7c101ce0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d78ff9700 1 -- 172.21.15.2:0/3015344602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d7000f5f0 con 0x7f0d7c101ce0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c19bbd0 con 0x7f0d7c101ce0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.300+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c19c1a0 con 0x7f0d7c101ce0 2024-03-16T10:52:38.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.301+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c190660 con 0x7f0d7c101ce0 2024-03-16T10:52:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.302+0000 7f0d78ff9700 1 -- 172.21.15.2:0/3015344602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0d700105d0 con 0x7f0d7c101ce0 2024-03-16T10:52:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.302+0000 7f0d78ff9700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0d6406dfa0 0x7f0d64070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:52:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.302+0000 7f0d78ff9700 1 -- 172.21.15.2:0/3015344602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0d7008dcd0 con 0x7f0d7c101ce0 2024-03-16T10:52:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.302+0000 7f0d7b7fe700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0d6406dfa0 0x7f0d64070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:52:38.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.303+0000 7f0d7b7fe700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0d6406dfa0 0x7f0d64070460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c006010 tx=0x7f0d6c00b580 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:52:38.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.307+0000 7f0d78ff9700 1 -- 172.21.15.2:0/3015344602 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0d7005ac50 con 0x7f0d7c101ce0 2024-03-16T10:52:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.595+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0d7c04ea80 con 0x7f0d7c101ce0 2024-03-16T10:52:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.595+0000 7f0d78ff9700 1 -- 172.21.15.2:0/3015344602 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f0d700200e0 con 0x7f0d7c101ce0 2024-03-16T10:52:38.596 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:52:38.596 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:52:38.596 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:52:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0d6406dfa0 msgr2=0x7f0d64070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0d6406dfa0 0x7f0d64070460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c006010 tx=0x7f0d6c00b580 comp rx=0 tx=0).stop 2024-03-16T10:52:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c101ce0 msgr2=0x7f0d7c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:52:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c101ce0 0x7f0d7c196a10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d8d0 tx=0x7f0d7000dc90 comp rx=0 tx=0).stop 2024-03-16T10:52:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 shutdown_connections 2024-03-16T10:52:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0d6406dfa0 0x7f0d64070460 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d7c100ae0 0x7f0d7c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 --2- 172.21.15.2:0/3015344602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d7c101ce0 0x7f0d7c196a10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:52:38.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.597+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 >> 172.21.15.2:0/3015344602 conn(0x7f0d7c0fc0a0 msgr2=0x7f0d7c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:52:38.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.598+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 shutdown_connections 2024-03-16T10:52:38.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:52:38.598+0000 7f0d820f4700 1 -- 172.21.15.2:0/3015344602 wait complete. 2024-03-16T10:52:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:38 smithi026 ceph-mon[82600]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-03-16T10:52:39.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:38 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3015344602' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:52:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:38 smithi002 ceph-mon[94239]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-03-16T10:52:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:38 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3015344602' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:52:40.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:39 smithi026 ceph-mon[82600]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:40.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:39 smithi002 ceph-mon[94239]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:52:41.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:41 smithi026 ceph-mon[82600]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s wr, 1 op/s 2024-03-16T10:52:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:41 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:41 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:41.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:41 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:41 smithi002 ceph-mon[94239]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s wr, 1 op/s 2024-03-16T10:52:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:41 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:41 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:41.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:41 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:43 smithi002 ceph-mon[94239]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:43.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:43 smithi026 ceph-mon[82600]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:43.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:43.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:44.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:44.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:44 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:44.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:44 smithi002 ceph-mon[94239]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:44.935 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:44.935 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:44 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:44.935 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:44 smithi026 ceph-mon[82600]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:45.806 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:45 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:45.806 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:45 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:46.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:45 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:46.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:45 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:47.169 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:46 smithi026 ceph-mon[82600]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:47.169 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:46 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:47.169 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:46 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:46 smithi002 ceph-mon[94239]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:46 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:46 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:48.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:48 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:48.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:48 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:48.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:48 smithi002 ceph-mon[94239]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:48.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:48 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:48.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:48 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:48.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:48 smithi026 ceph-mon[82600]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:50.696 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.696 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.696 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.697 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all ceph-exporter 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi002"}]: dispatch 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi002"}]': finished 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi026"}]: dispatch 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi026"}]': finished 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all iscsi 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.698 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all nfs 2024-03-16T10:52:50.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: Upgrade: Setting container_image for all nvmeof 2024-03-16T10:52:50.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:50.699 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:50 smithi002 ceph-mon[94239]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:50.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:52:50.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:52:50.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all ceph-exporter 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi002"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi002"}]': finished 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi026"}]: dispatch 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi026"}]': finished 2024-03-16T10:52:50.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all iscsi 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all nfs 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: Upgrade: Setting container_image for all nvmeof 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:52:50.984 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:50 smithi026 ceph-mon[82600]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:53.130 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:52 smithi002 ceph-mon[94239]: Upgrade: Updating node-exporter.smithi002 (1/2) 2024-03-16T10:52:53.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:52 smithi002 ceph-mon[94239]: Deploying daemon node-exporter.smithi002 on smithi002 2024-03-16T10:52:53.131 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:52 smithi002 ceph-mon[94239]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:52 smithi026 ceph-mon[82600]: Upgrade: Updating node-exporter.smithi002 (1/2) 2024-03-16T10:52:53.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:52 smithi026 ceph-mon[82600]: Deploying daemon node-exporter.smithi002 on smithi002 2024-03-16T10:52:53.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:52 smithi026 ceph-mon[82600]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:55.147 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:54 smithi026 ceph-mon[82600]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:55.147 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:55.147 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:54 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:54 smithi002 ceph-mon[94239]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:54 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:57.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:56 smithi002 ceph-mon[94239]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:57.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:56 smithi002 ceph-mon[94239]: Upgrade: Updating node-exporter.smithi026 (2/2) 2024-03-16T10:52:57.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:56 smithi002 ceph-mon[94239]: Deploying daemon node-exporter.smithi026 on smithi026 2024-03-16T10:52:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:56 smithi026 ceph-mon[82600]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:56 smithi026 ceph-mon[82600]: Upgrade: Updating node-exporter.smithi026 (2/2) 2024-03-16T10:52:57.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:56 smithi026 ceph-mon[82600]: Deploying daemon node-exporter.smithi026 on smithi026 2024-03-16T10:52:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:58 smithi026 ceph-mon[82600]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:58 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:58 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:59.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:52:58 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:52:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:58 smithi002 ceph-mon[94239]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:52:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:58 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:58 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:52:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:52:58 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:01.205 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:00 smithi002 ceph-mon[94239]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:00 smithi026 ceph-mon[82600]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:02.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:02.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:01 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:02.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:01 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:03 smithi026 ceph-mon[82600]: pgmap v239: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:03 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:03 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:03.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:03 smithi002 ceph-mon[94239]: pgmap v239: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:03.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:03 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:03.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:03 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:04.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:04.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:04.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:04 smithi026 ceph-mon[82600]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:04.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:04.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:04.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:04 smithi002 ceph-mon[94239]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:05.406 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:05.407 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:05.407 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:06.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:06.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:06.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:06 smithi002 ceph-mon[94239]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:06.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:06.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:06.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:06 smithi026 ceph-mon[82600]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/3888082077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 msgr2=0x7f2bac1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/3888082077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac1010a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009910 tx=0x7f2ba0009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/3888082077 shutdown_connections 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/3888082077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac1010a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/3888082077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bac0fff00 0x7f2bac100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/3888082077 >> 172.21.15.2:0/3888082077 conn(0x7f2bac0fa700 msgr2=0x7f2bac0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/3888082077 shutdown_connections 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/3888082077 wait complete. 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.706+0000 7f2bb1ff8700 1 Processor -- start 2024-03-16T10:53:08.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2bb1ff8700 1 -- start start 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2bb1ff8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bac0fff00 0x7f2bac0ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2bb1ff8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2bb1ff8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac19b910 con 0x7f2bac100c20 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2bb1ff8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac19ba80 con 0x7f2bac0fff00 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2baaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2baaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48538/0 (socket says 172.21.15.2:48538) 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2baaffd700 1 -- 172.21.15.2:0/2436576608 learned_addr learned my addr 172.21.15.2:0/2436576608 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2bab7fe700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bac0fff00 0x7f2bac0ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2baaffd700 1 -- 172.21.15.2:0/2436576608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bac0fff00 msgr2=0x7f2bac0ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2baaffd700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bac0fff00 0x7f2bac0ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.707+0000 7f2baaffd700 1 -- 172.21.15.2:0/2436576608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00095f0 con 0x7f2bac100c20 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.708+0000 7f2baaffd700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac19b3d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005850 tx=0x7f2ba000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.708+0000 7f2ba8ff9700 1 -- 172.21.15.2:0/2436576608 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba001d070 con 0x7f2bac100c20 2024-03-16T10:53:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.708+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac19bd00 con 0x7f2bac100c20 2024-03-16T10:53:08.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.708+0000 7f2ba8ff9700 1 -- 172.21.15.2:0/2436576608 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ba000f460 con 0x7f2bac100c20 2024-03-16T10:53:08.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.708+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac19c220 con 0x7f2bac100c20 2024-03-16T10:53:08.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.708+0000 7f2ba8ff9700 1 -- 172.21.15.2:0/2436576608 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba0017610 con 0x7f2bac100c20 2024-03-16T10:53:08.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.709+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac18f630 con 0x7f2bac100c20 2024-03-16T10:53:08.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.710+0000 7f2ba8ff9700 1 -- 172.21.15.2:0/2436576608 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2ba000f5d0 con 0x7f2bac100c20 2024-03-16T10:53:08.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.711+0000 7f2ba8ff9700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b9406dfa0 0x7f2b94070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:08.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.711+0000 7f2ba8ff9700 1 -- 172.21.15.2:0/2436576608 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2ba008f420 con 0x7f2bac100c20 2024-03-16T10:53:08.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.711+0000 7f2bab7fe700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b9406dfa0 0x7f2b94070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:08.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.711+0000 7f2bab7fe700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b9406dfa0 0x7f2b94070460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006fd0 tx=0x7f2b9c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.716+0000 7f2ba8ff9700 1 -- 172.21.15.2:0/2436576608 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2ba005c320 con 0x7f2bac100c20 2024-03-16T10:53:08.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.978+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bac0611f0 con 0x7f2b9406dfa0 2024-03-16T10:53:08.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.980+0000 7f2ba8ff9700 1 -- 172.21.15.2:0/2436576608 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f2bac0611f0 con 0x7f2b9406dfa0 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b9406dfa0 msgr2=0x7f2b94070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b9406dfa0 0x7f2b94070460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006fd0 tx=0x7f2b9c008040 comp rx=0 tx=0).stop 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 msgr2=0x7f2bac19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac19b3d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0005850 tx=0x7f2ba000be20 comp rx=0 tx=0).stop 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 shutdown_connections 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2b9406dfa0 0x7f2b94070460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bac0fff00 0x7f2bac0ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 --2- 172.21.15.2:0/2436576608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2bac100c20 0x7f2bac19b3d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:08.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 >> 172.21.15.2:0/2436576608 conn(0x7f2bac0fa700 msgr2=0x7f2bac1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:08.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 shutdown_connections 2024-03-16T10:53:08.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:08.982+0000 7f2bb1ff8700 1 -- 172.21.15.2:0/2436576608 wait complete. 2024-03-16T10:53:08.993 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:53:09.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 -- 172.21.15.2:0/4097838086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 msgr2=0x7f53dc101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 --2- 172.21.15.2:0/4097838086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc101030 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009910 tx=0x7f53d0009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:09.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 -- 172.21.15.2:0/4097838086 shutdown_connections 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 --2- 172.21.15.2:0/4097838086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc101030 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 --2- 172.21.15.2:0/4097838086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53dc0ff9d0 0x7f53dc0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 -- 172.21.15.2:0/4097838086 >> 172.21.15.2:0/4097838086 conn(0x7f53dc0faf70 msgr2=0x7f53dc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 -- 172.21.15.2:0/4097838086 shutdown_connections 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.090+0000 7f53e41b0700 1 -- 172.21.15.2:0/4097838086 wait complete. 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e41b0700 1 Processor -- start 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e41b0700 1 -- start start 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e41b0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53dc0ff9d0 0x7f53dc196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e41b0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e41b0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc1970e0 con 0x7f53dc100bd0 2024-03-16T10:53:09.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e41b0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc197220 con 0x7f53dc0ff9d0 2024-03-16T10:53:09.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e174b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e174b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48548/0 (socket says 172.21.15.2:48548) 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e174b700 1 -- 172.21.15.2:0/17807606 learned_addr learned my addr 172.21.15.2:0/17807606 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e1f4c700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53dc0ff9d0 0x7f53dc196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e174b700 1 -- 172.21.15.2:0/17807606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53dc0ff9d0 msgr2=0x7f53dc196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e174b700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53dc0ff9d0 0x7f53dc196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.091+0000 7f53e174b700 1 -- 172.21.15.2:0/17807606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d00095f0 con 0x7f53dc100bd0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.092+0000 7f53e174b700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc196ac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b800 tx=0x7f53d0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.092+0000 7f53ceffd700 1 -- 172.21.15.2:0/17807606 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d001d070 con 0x7f53dc100bd0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.092+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc19bbd0 con 0x7f53dc100bd0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.092+0000 7f53ceffd700 1 -- 172.21.15.2:0/17807606 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53d00052a0 con 0x7f53dc100bd0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.092+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc19c140 con 0x7f53dc100bd0 2024-03-16T10:53:09.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.092+0000 7f53ceffd700 1 -- 172.21.15.2:0/17807606 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d0017400 con 0x7f53dc100bd0 2024-03-16T10:53:09.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.093+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53dc04ea80 con 0x7f53dc100bd0 2024-03-16T10:53:09.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.094+0000 7f53ceffd700 1 -- 172.21.15.2:0/17807606 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f53d0003780 con 0x7f53dc100bd0 2024-03-16T10:53:09.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.094+0000 7f53ceffd700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f53c806dfa0 0x7f53c8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.094+0000 7f53ceffd700 1 -- 172.21.15.2:0/17807606 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f53d008f150 con 0x7f53dc100bd0 2024-03-16T10:53:09.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.094+0000 7f53e1f4c700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f53c806dfa0 0x7f53c8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.095+0000 7f53e1f4c700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f53c806dfa0 0x7f53c8070460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f53dc100a30 tx=0x7f53d8009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:09.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.103+0000 7f53ceffd700 1 -- 172.21.15.2:0/17807606 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f53d005c0d0 con 0x7f53dc100bd0 2024-03-16T10:53:09.235 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:09.235 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:09.235 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:09.235 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:53:09.235 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.236 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.237 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:09 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.339+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53dc068c60 con 0x7f53c806dfa0 2024-03-16T10:53:09.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.341+0000 7f53ceffd700 1 -- 172.21.15.2:0/17807606 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f53dc068c60 con 0x7f53c806dfa0 2024-03-16T10:53:09.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f53c806dfa0 msgr2=0x7f53c8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f53c806dfa0 0x7f53c8070460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f53dc100a30 tx=0x7f53d8009260 comp rx=0 tx=0).stop 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 msgr2=0x7f53dc196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc196ac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53d000b800 tx=0x7f53d0005f00 comp rx=0 tx=0).stop 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 shutdown_connections 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f53c806dfa0 0x7f53c8070460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53dc0ff9d0 0x7f53dc196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 --2- 172.21.15.2:0/17807606 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53dc100bd0 0x7f53dc196ac0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 >> 172.21.15.2:0/17807606 conn(0x7f53dc0faf70 msgr2=0x7f53dc068550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 shutdown_connections 2024-03-16T10:53:09.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.343+0000 7f53e41b0700 1 -- 172.21.15.2:0/17807606 wait complete. 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 -- 172.21.15.2:0/617625506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 msgr2=0x7fa23c101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 --2- 172.21.15.2:0/617625506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa230009910 tx=0x7fa230009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 -- 172.21.15.2:0/617625506 shutdown_connections 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 --2- 172.21.15.2:0/617625506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 --2- 172.21.15.2:0/617625506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa23c0ff9e0 0x7fa23c0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 -- 172.21.15.2:0/617625506 >> 172.21.15.2:0/617625506 conn(0x7fa23c0faf80 msgr2=0x7fa23c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 -- 172.21.15.2:0/617625506 shutdown_connections 2024-03-16T10:53:09.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.452+0000 7fa240ca7700 1 -- 172.21.15.2:0/617625506 wait complete. 2024-03-16T10:53:09.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa240ca7700 1 Processor -- start 2024-03-16T10:53:09.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa240ca7700 1 -- start start 2024-03-16T10:53:09.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa240ca7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa23c0ff9e0 0x7fa23c103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa240ca7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa240ca7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c102500 con 0x7fa23c100be0 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa240ca7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c102640 con 0x7fa23c0ff9e0 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa239d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa239d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48574/0 (socket says 172.21.15.2:48574) 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa239d9b700 1 -- 172.21.15.2:0/4178551747 learned_addr learned my addr 172.21.15.2:0/4178551747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa239d9b700 1 -- 172.21.15.2:0/4178551747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa23c0ff9e0 msgr2=0x7fa23c103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa23a59c700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa23c0ff9e0 0x7fa23c103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa239d9b700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa23c0ff9e0 0x7fa23c103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa239d9b700 1 -- 172.21.15.2:0/4178551747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2300095f0 con 0x7fa23c100be0 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.453+0000 7fa23a59c700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa23c0ff9e0 0x7fa23c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.454+0000 7fa239d9b700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa230006010 tx=0x7fa23000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.454+0000 7fa22b7fe700 1 -- 172.21.15.2:0/4178551747 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa23001d070 con 0x7fa23c100be0 2024-03-16T10:53:09.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.454+0000 7fa22b7fe700 1 -- 172.21.15.2:0/4178551747 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa23000f460 con 0x7fa23c100be0 2024-03-16T10:53:09.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.454+0000 7fa22b7fe700 1 -- 172.21.15.2:0/4178551747 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa230021640 con 0x7fa23c100be0 2024-03-16T10:53:09.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.454+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c1028c0 con 0x7fa23c100be0 2024-03-16T10:53:09.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.454+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c102ca0 con 0x7fa23c100be0 2024-03-16T10:53:09.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.455+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23c04ea80 con 0x7fa23c100be0 2024-03-16T10:53:09.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.455+0000 7fa22b7fe700 1 -- 172.21.15.2:0/4178551747 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa23000fab0 con 0x7fa23c100be0 2024-03-16T10:53:09.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.456+0000 7fa22b7fe700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa22406dfa0 0x7fa224070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.456+0000 7fa23a59c700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa22406dfa0 0x7fa224070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.456+0000 7fa22b7fe700 1 -- 172.21.15.2:0/4178551747 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa23008f080 con 0x7fa23c100be0 2024-03-16T10:53:09.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.457+0000 7fa23a59c700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa22406dfa0 0x7fa224070460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa23c100a40 tx=0x7fa22c009670 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:09.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.461+0000 7fa22b7fe700 1 -- 172.21.15.2:0/4178551747 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa23005c000 con 0x7fa23c100be0 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:09 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:09.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.702+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa23c069730 con 0x7fa22406dfa0 2024-03-16T10:53:09.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.713+0000 7fa22b7fe700 1 -- 172.21.15.2:0/4178551747 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fa23c069730 con 0x7fa22406dfa0 2024-03-16T10:53:09.714 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:53:09.714 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (15m) 6s ago 16m 25.0M - 0.25.0 c8568f914cd2 a86e022888db 2024-03-16T10:53:09.714 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (35s) 6s ago 17m 7931k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 01d0397a345d 2024-03-16T10:53:09.714 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (30s) 8s ago 15m 7541k - 19.0.0-2162-g3d78f5ba be86dc8cb22b e515b4badb59 2024-03-16T10:53:09.714 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (5m) 6s ago 16m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:53:09.714 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (5m) 8s ago 15m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:53:09.714 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (15m) 6s ago 16m 87.1M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (97s) 6s ago 11m 13.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a6b65504c2dc 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (114s) 6s ago 11m 90.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a68e60da4931 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (72s) 8s ago 11m 12.8M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 745c119b7ff5 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (49s) 8s ago 11m 14.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 1cb95b8df6a0 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (8m) 6s ago 18m 510M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (7m) 8s ago 15m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (6m) 6s ago 18m 54.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (6m) 8s ago 15m 42.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (15s) 6s ago 16m 5305k - 1.5.0 0da6a335fe13 80c6dac2c511 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (11s) 8s ago 15m 3347k - 1.5.0 0da6a335fe13 46339cbf80f5 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (5m) 6s ago 14m 92.0M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (4m) 6s ago 14m 99.1M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:53:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (3m) 6s ago 13m 91.4M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:53:09.716 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (3m) 8s ago 13m 111M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:53:09.716 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (2m) 8s ago 13m 88.2M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:53:09.716 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (2m) 8s ago 12m 92.6M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b cf40ac20a977 2024-03-16T10:53:09.716 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (7m) 6s ago 16m 78.9M - 2.43.0 a07b618ecd1d ce0cd1434fea 2024-03-16T10:53:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa22406dfa0 msgr2=0x7fa224070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa22406dfa0 0x7fa224070460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa23c100a40 tx=0x7fa22c009670 comp rx=0 tx=0).stop 2024-03-16T10:53:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 msgr2=0x7fa23c101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa230006010 tx=0x7fa23000be20 comp rx=0 tx=0).stop 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 shutdown_connections 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa22406dfa0 0x7fa224070460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa23c0ff9e0 0x7fa23c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 --2- 172.21.15.2:0/4178551747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa23c100be0 0x7fa23c101fc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 >> 172.21.15.2:0/4178551747 conn(0x7fa23c0faf80 msgr2=0x7fa23c069020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 shutdown_connections 2024-03-16T10:53:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.715+0000 7fa240ca7700 1 -- 172.21.15.2:0/4178551747 wait complete. 2024-03-16T10:53:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 -- 172.21.15.2:0/1114169079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0101d00 msgr2=0x7fecc0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 --2- 172.21.15.2:0/1114169079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0101d00 0x7fecc0102160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009910 tx=0x7fecb4009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 -- 172.21.15.2:0/1114169079 shutdown_connections 2024-03-16T10:53:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 --2- 172.21.15.2:0/1114169079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0101d00 0x7fecc0102160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 --2- 172.21.15.2:0/1114169079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc0100b00 0x7fecc0100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 -- 172.21.15.2:0/1114169079 >> 172.21.15.2:0/1114169079 conn(0x7fecc00fc0a0 msgr2=0x7fecc00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:09.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 -- 172.21.15.2:0/1114169079 shutdown_connections 2024-03-16T10:53:09.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.833+0000 7fecc7537700 1 -- 172.21.15.2:0/1114169079 wait complete. 2024-03-16T10:53:09.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc7537700 1 Processor -- start 2024-03-16T10:53:09.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc7537700 1 -- start start 2024-03-16T10:53:09.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc7537700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0100b00 0x7fecc0196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc7537700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc0101d00 0x7fecc0196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc7537700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc0196ff0 con 0x7fecc0100b00 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc7537700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc0197130 con 0x7fecc0101d00 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc52d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0100b00 0x7fecc0196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc52d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0100b00 0x7fecc0196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48594/0 (socket says 172.21.15.2:48594) 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc52d3700 1 -- 172.21.15.2:0/2447083001 learned_addr learned my addr 172.21.15.2:0/2447083001 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc4ad2700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc0101d00 0x7fecc0196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc52d3700 1 -- 172.21.15.2:0/2447083001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc0101d00 msgr2=0x7fecc0196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc52d3700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc0101d00 0x7fecc0196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc52d3700 1 -- 172.21.15.2:0/2447083001 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb40095f0 con 0x7fecc0100b00 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.834+0000 7fecc52d3700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0100b00 0x7fecc0196520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fecb000cfd0 tx=0x7fecb000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.835+0000 7fecbe7fc700 1 -- 172.21.15.2:0/2447083001 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecb000ad70 con 0x7fecc0100b00 2024-03-16T10:53:09.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.835+0000 7fecbe7fc700 1 -- 172.21.15.2:0/2447083001 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecb0007e50 con 0x7fecc0100b00 2024-03-16T10:53:09.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.835+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc019bb50 con 0x7fecc0100b00 2024-03-16T10:53:09.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.835+0000 7fecbe7fc700 1 -- 172.21.15.2:0/2447083001 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecb0012650 con 0x7fecc0100b00 2024-03-16T10:53:09.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.835+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc019c070 con 0x7fecc0100b00 2024-03-16T10:53:09.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.836+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc0190680 con 0x7fecc0100b00 2024-03-16T10:53:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.836+0000 7fecbe7fc700 1 -- 172.21.15.2:0/2447083001 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fecb0003a00 con 0x7fecc0100b00 2024-03-16T10:53:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.837+0000 7fecbe7fc700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7feca806dfa0 0x7feca8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.837+0000 7fecbe7fc700 1 -- 172.21.15.2:0/2447083001 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fecb008de50 con 0x7fecc0100b00 2024-03-16T10:53:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.837+0000 7fecc4ad2700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7feca806dfa0 0x7feca8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.837+0000 7fecc4ad2700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7feca806dfa0 0x7feca8070460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005090 tx=0x7fecb4005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:09.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:09.844+0000 7fecbe7fc700 1 -- 172.21.15.2:0/2447083001 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fecb005add0 con 0x7fecc0100b00 2024-03-16T10:53:10.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.174+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fecc002cd00 con 0x7fecc0100b00 2024-03-16T10:53:10.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.175+0000 7fecbe7fc700 1 -- 172.21.15.2:0/2447083001 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fecb005a960 con 0x7fecc0100b00 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:53:10.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:10.177 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:53:10.177 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-16T10:53:10.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:53:10.177 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7feca806dfa0 msgr2=0x7feca8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7feca806dfa0 0x7feca8070460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fecb4005090 tx=0x7fecb4005960 comp rx=0 tx=0).stop 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0100b00 msgr2=0x7fecc0196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0100b00 0x7fecc0196520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fecb000cfd0 tx=0x7fecb000a340 comp rx=0 tx=0).stop 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 shutdown_connections 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7feca806dfa0 0x7feca8070460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fecc0100b00 0x7fecc0196520 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 --2- 172.21.15.2:0/2447083001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecc0101d00 0x7fecc0196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 >> 172.21.15.2:0/2447083001 conn(0x7fecc00fc0a0 msgr2=0x7fecc0105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:10.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 shutdown_connections 2024-03-16T10:53:10.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.177+0000 7fecc7537700 1 -- 172.21.15.2:0/2447083001 wait complete. 2024-03-16T10:53:10.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 -- 172.21.15.2:0/3426927848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 msgr2=0x7f5fa00ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/3426927848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa00ff1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009910 tx=0x7f5f9c009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:10.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 -- 172.21.15.2:0/3426927848 shutdown_connections 2024-03-16T10:53:10.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/3426927848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa0101d80 0x7f5fa01021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/3426927848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa00ff1c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 -- 172.21.15.2:0/3426927848 >> 172.21.15.2:0/3426927848 conn(0x7f5fa00fa700 msgr2=0x7f5fa00fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:10.295 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:10 smithi002 ceph-mon[94239]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 -- 172.21.15.2:0/3426927848 shutdown_connections 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 -- 172.21.15.2:0/3426927848 wait complete. 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 Processor -- start 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.294+0000 7f5fa73cc700 1 -- start start 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa73cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa0101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa73cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa0101d80 0x7f5fa00ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa73cc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa0100270 con 0x7f5fa00feda0 2024-03-16T10:53:10.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa73cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa01003b0 con 0x7f5fa0101d80 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa5168700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa0101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa5168700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa0101640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48602/0 (socket says 172.21.15.2:48602) 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa5168700 1 -- 172.21.15.2:0/463773166 learned_addr learned my addr 172.21.15.2:0/463773166 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa4967700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa0101d80 0x7f5fa00ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa5168700 1 -- 172.21.15.2:0/463773166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa0101d80 msgr2=0x7f5fa00ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa5168700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa0101d80 0x7f5fa00ffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa5168700 1 -- 172.21.15.2:0/463773166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0095f0 con 0x7f5fa00feda0 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.295+0000 7f5fa5168700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa0101640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0092e0 tx=0x7f5f9c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.296+0000 7f5f967fc700 1 -- 172.21.15.2:0/463773166 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9c01d070 con 0x7f5fa00feda0 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.296+0000 7f5f967fc700 1 -- 172.21.15.2:0/463773166 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f9c005dc0 con 0x7f5fa00feda0 2024-03-16T10:53:10.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.296+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa0100630 con 0x7f5fa00feda0 2024-03-16T10:53:10.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.296+0000 7f5f967fc700 1 -- 172.21.15.2:0/463773166 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9c017400 con 0x7f5fa00feda0 2024-03-16T10:53:10.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.296+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa0100ba0 con 0x7f5fa00feda0 2024-03-16T10:53:10.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.297+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa0190660 con 0x7f5fa00feda0 2024-03-16T10:53:10.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.298+0000 7f5f967fc700 1 -- 172.21.15.2:0/463773166 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f5f9c00f6f0 con 0x7f5fa00feda0 2024-03-16T10:53:10.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.299+0000 7f5f967fc700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5f8c06dfb0 0x7f5f8c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:10.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.299+0000 7f5f967fc700 1 -- 172.21.15.2:0/463773166 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5f9c08f030 con 0x7f5fa00feda0 2024-03-16T10:53:10.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.299+0000 7f5fa4967700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5f8c06dfb0 0x7f5f8c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:10.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.299+0000 7f5fa4967700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5f8c06dfb0 0x7f5f8c070470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005950 tx=0x7f5f9000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:10.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.305+0000 7f5f967fc700 1 -- 172.21.15.2:0/463773166 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f9c05c070 con 0x7f5fa00feda0 2024-03-16T10:53:10.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:10 smithi026 ceph-mon[82600]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:10.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.590+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5fa0100ba0 con 0x7f5fa00feda0 2024-03-16T10:53:10.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.590+0000 7f5f967fc700 1 -- 172.21.15.2:0/463773166 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1929 (secure 0 0 0) 0x7f5fa0100ba0 con 0x7f5fa00feda0 2024-03-16T10:53:10.591 INFO:teuthology.orchestra.run.smithi002.stdout:e31 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 31 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:52:21.877610+0000 2024-03-16T10:53:10.592 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 86 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:53:10.593 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=34408} 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 34408 members: 34408 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{0:34408} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:10.594 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:10.595 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:10.595 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:53:10.595 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:10.595 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6828/1125443118,v1:172.21.15.2:6829/1125443118] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:10.595 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{-1:44337} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6824/2953126152,v1:172.21.15.26:6825/2953126152] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:10.595 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:44367} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6826/1298103624,v1:172.21.15.26:6827/1298103624] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.592+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5f8c06dfb0 msgr2=0x7f5f8c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.592+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5f8c06dfb0 0x7f5f8c070470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005950 tx=0x7f5f9000b410 comp rx=0 tx=0).stop 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.592+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 msgr2=0x7f5fa0101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.592+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa0101640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0092e0 tx=0x7f5f9c00bd70 comp rx=0 tx=0).stop 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.593+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 shutdown_connections 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.593+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5f8c06dfb0 0x7f5f8c070470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.593+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5fa00feda0 0x7f5fa0101640 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.593+0000 7f5fa73cc700 1 --2- 172.21.15.2:0/463773166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fa0101d80 0x7f5fa00ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.593+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 >> 172.21.15.2:0/463773166 conn(0x7f5fa00fa700 msgr2=0x7f5fa01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:10.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.593+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 shutdown_connections 2024-03-16T10:53:10.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.593+0000 7f5fa73cc700 1 -- 172.21.15.2:0/463773166 wait complete. 2024-03-16T10:53:10.597 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 31 2024-03-16T10:53:10.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.707+0000 7f77fa108700 1 -- 172.21.15.2:0/352341060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f40ff9e0 msgr2=0x7f77f40ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.707+0000 7f77fa108700 1 --2- 172.21.15.2:0/352341060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f40ff9e0 0x7f77f40ffe00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009910 tx=0x7f77e4009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:10.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.707+0000 7f77fa108700 1 -- 172.21.15.2:0/352341060 shutdown_connections 2024-03-16T10:53:10.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.707+0000 7f77fa108700 1 --2- 172.21.15.2:0/352341060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77f4100be0 0x7f77f4101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.707+0000 7f77fa108700 1 --2- 172.21.15.2:0/352341060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f40ff9e0 0x7f77f40ffe00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.707+0000 7f77fa108700 1 -- 172.21.15.2:0/352341060 >> 172.21.15.2:0/352341060 conn(0x7f77f40faf80 msgr2=0x7f77f40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:10.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.708+0000 7f77fa108700 1 -- 172.21.15.2:0/352341060 shutdown_connections 2024-03-16T10:53:10.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.708+0000 7f77fa108700 1 -- 172.21.15.2:0/352341060 wait complete. 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.708+0000 7f77fa108700 1 Processor -- start 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77fa108700 1 -- start start 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77fa108700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77f40ff9e0 0x7f77f4103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77fa108700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f4100be0 0x7f77f4101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77fa108700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4102500 con 0x7f77f4100be0 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77fa108700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4102640 con 0x7f77f40ff9e0 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f4100be0 0x7f77f4101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f4100be0 0x7f77f4101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48612/0 (socket says 172.21.15.2:48612) 2024-03-16T10:53:10.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f2ffd700 1 -- 172.21.15.2:0/3024159952 learned_addr learned my addr 172.21.15.2:0/3024159952 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f37fe700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77f40ff9e0 0x7f77f4103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f2ffd700 1 -- 172.21.15.2:0/3024159952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77f40ff9e0 msgr2=0x7f77f4103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f2ffd700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77f40ff9e0 0x7f77f4103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f2ffd700 1 -- 172.21.15.2:0/3024159952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40095f0 con 0x7f77f4100be0 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.709+0000 7f77f2ffd700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f4100be0 0x7f77f4101fc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c8a0 tx=0x7f77e800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.710+0000 7f77f0ff9700 1 -- 172.21.15.2:0/3024159952 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e80078e0 con 0x7f77f4100be0 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.710+0000 7f77f0ff9700 1 -- 172.21.15.2:0/3024159952 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77e800f450 con 0x7f77f4100be0 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.710+0000 7f77f0ff9700 1 -- 172.21.15.2:0/3024159952 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e800e5e0 con 0x7f77f4100be0 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.710+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f4102920 con 0x7f77f4100be0 2024-03-16T10:53:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.710+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f419e550 con 0x7f77f4100be0 2024-03-16T10:53:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.711+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f404ea80 con 0x7f77f4100be0 2024-03-16T10:53:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.712+0000 7f77f0ff9700 1 -- 172.21.15.2:0/3024159952 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f77e8007a40 con 0x7f77f4100be0 2024-03-16T10:53:10.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.712+0000 7f77f0ff9700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f77dc06dfa0 0x7f77dc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.712+0000 7f77f0ff9700 1 -- 172.21.15.2:0/3024159952 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f77e808dc60 con 0x7f77f4100be0 2024-03-16T10:53:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.712+0000 7f77f37fe700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f77dc06dfa0 0x7f77dc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:10.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.713+0000 7f77f37fe700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f77dc06dfa0 0x7f77dc070460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:10.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.719+0000 7f77f0ff9700 1 -- 172.21.15.2:0/3024159952 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f77e805abe0 con 0x7f77f4100be0 2024-03-16T10:53:10.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.952+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77f419e6e0 con 0x7f77dc06dfa0 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.953+0000 7f77f0ff9700 1 -- 172.21.15.2:0/3024159952 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f77f419e6e0 con 0x7f77dc06dfa0 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph-exporter", 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:53:10.954 INFO:teuthology.orchestra.run.smithi002.stdout: "osd", 2024-03-16T10:53:10.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mds", 2024-03-16T10:53:10.955 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:53:10.955 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:53:10.955 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "18/23 daemons upgraded", 2024-03-16T10:53:10.955 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-16T10:53:10.955 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:53:10.955 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f77dc06dfa0 msgr2=0x7f77dc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f77dc06dfa0 0x7f77dc070460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e400b540 comp rx=0 tx=0).stop 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f4100be0 msgr2=0x7f77f4101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f4100be0 0x7f77f4101fc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77e800c8a0 tx=0x7f77e800cbb0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 shutdown_connections 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f77dc06dfa0 0x7f77dc070460 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77f40ff9e0 0x7f77f4103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 --2- 172.21.15.2:0/3024159952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77f4100be0 0x7f77f4101fc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 >> 172.21.15.2:0/3024159952 conn(0x7f77f40faf80 msgr2=0x7f77f4068fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 shutdown_connections 2024-03-16T10:53:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:10.956+0000 7f77fa108700 1 -- 172.21.15.2:0/3024159952 wait complete. 2024-03-16T10:53:11.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 -- 172.21.15.2:0/667849179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 msgr2=0x7fc994100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/667849179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994100ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc98400b0a0 tx=0x7fc98400b3b0 comp rx=0 tx=0).stop 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 -- 172.21.15.2:0/667849179 shutdown_connections 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/667849179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc994101cd0 0x7fc994102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/667849179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994100ef0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 -- 172.21.15.2:0/667849179 >> 172.21.15.2:0/667849179 conn(0x7fc9940fc090 msgr2=0x7fc9940fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 -- 172.21.15.2:0/667849179 shutdown_connections 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.068+0000 7fc99b9a5700 1 -- 172.21.15.2:0/667849179 wait complete. 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc99b9a5700 1 Processor -- start 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc99b9a5700 1 -- start start 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc99b9a5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc99b9a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc994101cd0 0x7fc994196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc99b9a5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994197080 con 0x7fc994100ad0 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc99b9a5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941971c0 con 0x7fc994101cd0 2024-03-16T10:53:11.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc999741700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc999741700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48634/0 (socket says 172.21.15.2:48634) 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc999741700 1 -- 172.21.15.2:0/2787341029 learned_addr learned my addr 172.21.15.2:0/2787341029 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.069+0000 7fc998f40700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc994101cd0 0x7fc994196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc999741700 1 -- 172.21.15.2:0/2787341029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc994101cd0 msgr2=0x7fc994196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc999741700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc994101cd0 0x7fc994196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc999741700 1 -- 172.21.15.2:0/2787341029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc984009d00 con 0x7fc994100ad0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc999741700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994196520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc98400b070 tx=0x7fc984003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc9927fc700 1 -- 172.21.15.2:0/2787341029 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98400e040 con 0x7fc994100ad0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc9927fc700 1 -- 172.21.15.2:0/2787341029 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9840096e0 con 0x7fc994100ad0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99419bbc0 con 0x7fc994100ad0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc9927fc700 1 -- 172.21.15.2:0/2787341029 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98401d850 con 0x7fc994100ad0 2024-03-16T10:53:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.070+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99419c0b0 con 0x7fc994100ad0 2024-03-16T10:53:11.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.071+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994190650 con 0x7fc994100ad0 2024-03-16T10:53:11.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.072+0000 7fc9927fc700 1 -- 172.21.15.2:0/2787341029 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc984004250 con 0x7fc994100ad0 2024-03-16T10:53:11.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.072+0000 7fc9927fc700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc97c06dfb0 0x7fc97c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:11.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.073+0000 7fc9927fc700 1 -- 172.21.15.2:0/2787341029 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc98408f710 con 0x7fc994100ad0 2024-03-16T10:53:11.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.073+0000 7fc998f40700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc97c06dfb0 0x7fc97c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:11.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.073+0000 7fc998f40700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc97c06dfb0 0x7fc97c070470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc988005950 tx=0x7fc98800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:11.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.080+0000 7fc9927fc700 1 -- 172.21.15.2:0/2787341029 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc98405c590 con 0x7fc994100ad0 2024-03-16T10:53:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:11 smithi002 ceph-mon[94239]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:11 smithi002 ceph-mon[94239]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:11 smithi002 ceph-mon[94239]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:11 smithi002 ceph-mon[94239]: Upgrade: Updating prometheus.smithi002 2024-03-16T10:53:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:11 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2447083001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:11.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:11 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/463773166' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:53:11.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.437+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc99404ea80 con 0x7fc994100ad0 2024-03-16T10:53:11.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.438+0000 7fc9927fc700 1 -- 172.21.15.2:0/2787341029 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc9840074e0 con 0x7fc994100ad0 2024-03-16T10:53:11.439 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:53:11.439 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:53:11.439 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc97c06dfb0 msgr2=0x7fc97c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc97c06dfb0 0x7fc97c070470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc988005950 tx=0x7fc98800b410 comp rx=0 tx=0).stop 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 msgr2=0x7fc994196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994196520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc98400b070 tx=0x7fc984003c30 comp rx=0 tx=0).stop 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 shutdown_connections 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc97c06dfb0 0x7fc97c070470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc994100ad0 0x7fc994196520 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 --2- 172.21.15.2:0/2787341029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc994101cd0 0x7fc994196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 >> 172.21.15.2:0/2787341029 conn(0x7fc9940fc090 msgr2=0x7fc994105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 shutdown_connections 2024-03-16T10:53:11.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:11.440+0000 7fc99b9a5700 1 -- 172.21.15.2:0/2787341029 wait complete. 2024-03-16T10:53:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:11 smithi026 ceph-mon[82600]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:11 smithi026 ceph-mon[82600]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:11 smithi026 ceph-mon[82600]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:11 smithi026 ceph-mon[82600]: Upgrade: Updating prometheus.smithi002 2024-03-16T10:53:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:11 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2447083001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:11.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:11 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/463773166' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:53:12.315 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:12 smithi002 ceph-mon[94239]: Deploying daemon prometheus.smithi002 on smithi002 2024-03-16T10:53:12.316 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:12 smithi002 ceph-mon[94239]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:12.316 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:12 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2787341029' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:53:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:12 smithi026 ceph-mon[82600]: Deploying daemon prometheus.smithi002 on smithi002 2024-03-16T10:53:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:12 smithi026 ceph-mon[82600]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:12 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2787341029' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:53:13.262 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:13 smithi002 ceph-mon[94239]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:13 smithi026 ceph-mon[82600]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:15.297 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:15 smithi002 ceph-mon[94239]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:15.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:15 smithi026 ceph-mon[82600]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:17.404 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:17 smithi002 ceph-mon[94239]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:17.404 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:17.404 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:17.404 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:17 smithi026 ceph-mon[82600]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:17.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:19 smithi002 ceph-mon[94239]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:19.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:19 smithi026 ceph-mon[82600]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:20.219 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:20.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:21.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:21 smithi026 ceph-mon[82600]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:21.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:21 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:21.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:21 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:21.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:21 smithi002 ceph-mon[94239]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:21.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:21 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:21.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:21 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:23 smithi002 ceph-mon[94239]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:23 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:23.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:23 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:23.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:23 smithi026 ceph-mon[82600]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:23.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:23 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:23.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:23 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:25 smithi002 ceph-mon[94239]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:25.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:25 smithi026 ceph-mon[82600]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:26.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:26.983 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:26 smithi026 ceph-mon[82600]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:53:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:53:27.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.049 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:27.050 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:26 smithi002 ceph-mon[94239]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:28 smithi002 ceph-mon[94239]: Upgrade: Updating alertmanager.smithi002 2024-03-16T10:53:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:28 smithi002 ceph-mon[94239]: Deploying daemon alertmanager.smithi002 on smithi002 2024-03-16T10:53:29.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:28 smithi002 ceph-mon[94239]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:28 smithi026 ceph-mon[82600]: Upgrade: Updating alertmanager.smithi002 2024-03-16T10:53:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:28 smithi026 ceph-mon[82600]: Deploying daemon alertmanager.smithi002 on smithi002 2024-03-16T10:53:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:28 smithi026 ceph-mon[82600]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:31.008 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:30 smithi002 ceph-mon[94239]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:30 smithi026 ceph-mon[82600]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:32 smithi026 ceph-mon[82600]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:33.281 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:32 smithi002 ceph-mon[94239]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:34.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:34.799 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:34 smithi002 ceph-mon[94239]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:34.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:34.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:34.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:34.982 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:34 smithi026 ceph-mon[82600]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:35.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:35.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:36.783 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:36 smithi002 ceph-mon[94239]: pgmap v256: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:36.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:36 smithi026 ceph-mon[82600]: pgmap v256: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:38 smithi026 ceph-mon[82600]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:38 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:38 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:38 smithi002 ceph-mon[94239]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:38 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:38 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:40.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:40.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:40 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:40.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:40 smithi002 ceph-mon[94239]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:40.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:40.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:40 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:40.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:40 smithi026 ceph-mon[82600]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.553+0000 7fc436748700 1 -- 172.21.15.2:0/1641303577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 msgr2=0x7fc430102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.553+0000 7fc436748700 1 --2- 172.21.15.2:0/1641303577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430102210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc424009910 tx=0x7fc424009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.553+0000 7fc436748700 1 -- 172.21.15.2:0/1641303577 shutdown_connections 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.553+0000 7fc436748700 1 --2- 172.21.15.2:0/1641303577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430102210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.553+0000 7fc436748700 1 --2- 172.21.15.2:0/1641303577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4300fedd0 0x7fc4300ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.553+0000 7fc436748700 1 -- 172.21.15.2:0/1641303577 >> 172.21.15.2:0/1641303577 conn(0x7fc4300fa6f0 msgr2=0x7fc4300fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.554+0000 7fc436748700 1 -- 172.21.15.2:0/1641303577 shutdown_connections 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.554+0000 7fc436748700 1 -- 172.21.15.2:0/1641303577 wait complete. 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.554+0000 7fc436748700 1 Processor -- start 2024-03-16T10:53:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.554+0000 7fc436748700 1 -- start start 2024-03-16T10:53:41.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc436748700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4300fedd0 0x7fc430196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:41.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc436748700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc436748700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc430197160 con 0x7fc430101db0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc436748700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301972a0 con 0x7fc4300fedd0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34478/0 (socket says 172.21.15.2:34478) 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42f7fe700 1 -- 172.21.15.2:0/1781784956 learned_addr learned my addr 172.21.15.2:0/1781784956 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42ffff700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4300fedd0 0x7fc430196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42f7fe700 1 -- 172.21.15.2:0/1781784956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4300fedd0 msgr2=0x7fc430196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42f7fe700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4300fedd0 0x7fc430196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42f7fe700 1 -- 172.21.15.2:0/1781784956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240095f0 con 0x7fc430101db0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.555+0000 7fc42f7fe700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430196b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc424006010 tx=0x7fc42400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.556+0000 7fc42d7fa700 1 -- 172.21.15.2:0/1781784956 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc42401d070 con 0x7fc430101db0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.556+0000 7fc42d7fa700 1 -- 172.21.15.2:0/1781784956 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc42400f460 con 0x7fc430101db0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.556+0000 7fc42d7fa700 1 -- 172.21.15.2:0/1781784956 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc424021620 con 0x7fc430101db0 2024-03-16T10:53:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.556+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43019bc50 con 0x7fc430101db0 2024-03-16T10:53:41.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.556+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43019c100 con 0x7fc430101db0 2024-03-16T10:53:41.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.557+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc430190730 con 0x7fc430101db0 2024-03-16T10:53:41.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.557+0000 7fc42d7fa700 1 -- 172.21.15.2:0/1781784956 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fc42400f5d0 con 0x7fc430101db0 2024-03-16T10:53:41.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.558+0000 7fc42d7fa700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc41806dfa0 0x7fc418070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:41.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.558+0000 7fc42d7fa700 1 -- 172.21.15.2:0/1781784956 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fc42408f080 con 0x7fc430101db0 2024-03-16T10:53:41.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.558+0000 7fc42ffff700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc41806dfa0 0x7fc418070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:41.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.559+0000 7fc42ffff700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc41806dfa0 0x7fc418070460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc430068d70 tx=0x7fc420009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:41.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.563+0000 7fc42d7fa700 1 -- 172.21.15.2:0/1781784956 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc42405c000 con 0x7fc430101db0 2024-03-16T10:53:41.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.793+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4300611f0 con 0x7fc41806dfa0 2024-03-16T10:53:41.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.794+0000 7fc42d7fa700 1 -- 172.21.15.2:0/1781784956 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7fc4300611f0 con 0x7fc41806dfa0 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc41806dfa0 msgr2=0x7fc418070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc41806dfa0 0x7fc418070460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc430068d70 tx=0x7fc420009190 comp rx=0 tx=0).stop 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 msgr2=0x7fc430196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430196b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc424006010 tx=0x7fc42400bd70 comp rx=0 tx=0).stop 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 shutdown_connections 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fc41806dfa0 0x7fc418070460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4300fedd0 0x7fc430196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 --2- 172.21.15.2:0/1781784956 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc430101db0 0x7fc430196b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 >> 172.21.15.2:0/1781784956 conn(0x7fc4300fa6f0 msgr2=0x7fc430105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 shutdown_connections 2024-03-16T10:53:41.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.797+0000 7fc436748700 1 -- 172.21.15.2:0/1781784956 wait complete. 2024-03-16T10:53:41.808 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:53:41.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1174229194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88101d00 msgr2=0x7f3a88102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:41.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1174229194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88101d00 0x7f3a88102160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009910 tx=0x7f3a78009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1174229194 shutdown_connections 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1174229194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88101d00 0x7f3a88102160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1174229194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88100b00 0x7f3a88100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1174229194 >> 172.21.15.2:0/1174229194 conn(0x7f3a880fc0a0 msgr2=0x7f3a880fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1174229194 shutdown_connections 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.919+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1174229194 wait complete. 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8f99b700 1 Processor -- start 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8f99b700 1 -- start start 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8f99b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88100b00 0x7f3a881986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8f99b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88101d00 0x7f3a88198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8f99b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88199250 con 0x7f3a88100b00 2024-03-16T10:53:41.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8f99b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88199390 con 0x7f3a88101d00 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8cf36700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88101d00 0x7f3a88198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8cf36700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88101d00 0x7f3a88198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.2:46992/0 (socket says 172.21.15.2:46992) 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8cf36700 1 -- 172.21.15.2:0/1948048898 learned_addr learned my addr 172.21.15.2:0/1948048898 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.920+0000 7f3a8d737700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88100b00 0x7f3a881986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a8cf36700 1 -- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88100b00 msgr2=0x7f3a881986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a8cf36700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88100b00 0x7f3a881986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a8cf36700 1 -- 172.21.15.2:0/1948048898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780095f0 con 0x7f3a88101d00 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a8cf36700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88101d00 0x7f3a88198c30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3a78006010 tx=0x7f3a7800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a7e7fc700 1 -- 172.21.15.2:0/1948048898 <== mon.1 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a7801d070 con 0x7f3a88101d00 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8819dd90 con 0x7f3a88101d00 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8819e280 con 0x7f3a88101d00 2024-03-16T10:53:41.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a7e7fc700 1 -- 172.21.15.2:0/1948048898 <== mon.1 v2:172.21.15.26:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a7800f460 con 0x7f3a88101d00 2024-03-16T10:53:41.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.921+0000 7f3a7e7fc700 1 -- 172.21.15.2:0/1948048898 <== mon.1 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a78021620 con 0x7f3a88101d00 2024-03-16T10:53:41.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.922+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c0052f0 con 0x7f3a88101d00 2024-03-16T10:53:41.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.923+0000 7f3a7e7fc700 1 -- 172.21.15.2:0/1948048898 <== mon.1 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3a78003a60 con 0x7f3a88101d00 2024-03-16T10:53:41.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.923+0000 7f3a7e7fc700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a7406dff0 0x7f3a740704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:41.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.923+0000 7f3a7e7fc700 1 -- 172.21.15.2:0/1948048898 <== mon.1 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3a7808ef70 con 0x7f3a88101d00 2024-03-16T10:53:41.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.923+0000 7f3a8d737700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a7406dff0 0x7f3a740704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:41.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.924+0000 7f3a8d737700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a7406dff0 0x7f3a740704b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a88101b60 tx=0x7f3a84007400 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:41.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:41.928+0000 7f3a7e7fc700 1 -- 172.21.15.2:0/1948048898 <== mon.1 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a7805be70 con 0x7f3a88101d00 2024-03-16T10:53:42.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.232+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a6c000bc0 con 0x7f3a7406dff0 2024-03-16T10:53:42.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.234+0000 7f3a7e7fc700 1 -- 172.21.15.2:0/1948048898 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+465 (secure 0 0 0) 0x7f3a6c000bc0 con 0x7f3a7406dff0 2024-03-16T10:53:42.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.237+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a7406dff0 msgr2=0x7f3a740704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.237+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a7406dff0 0x7f3a740704b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a88101b60 tx=0x7f3a84007400 comp rx=0 tx=0).stop 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.237+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88101d00 msgr2=0x7f3a88198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.237+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88101d00 0x7f3a88198c30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3a78006010 tx=0x7f3a7800bd70 comp rx=0 tx=0).stop 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.238+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 shutdown_connections 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.238+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3a7406dff0 0x7f3a740704b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.238+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a88100b00 0x7f3a881986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.238+0000 7f3a8f99b700 1 --2- 172.21.15.2:0/1948048898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a88101d00 0x7f3a88198c30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.238+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 >> 172.21.15.2:0/1948048898 conn(0x7f3a880fc0a0 msgr2=0x7f3a88073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.238+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 shutdown_connections 2024-03-16T10:53:42.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.238+0000 7f3a8f99b700 1 -- 172.21.15.2:0/1948048898 wait complete. 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 -- 172.21.15.2:0/1797923264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270100b00 msgr2=0x7f6270100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 --2- 172.21.15.2:0/1797923264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270100b00 0x7f6270100f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6260009910 tx=0x7f6260009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 -- 172.21.15.2:0/1797923264 shutdown_connections 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 --2- 172.21.15.2:0/1797923264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6270101d00 0x7f6270102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 --2- 172.21.15.2:0/1797923264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270100b00 0x7f6270100f20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 -- 172.21.15.2:0/1797923264 >> 172.21.15.2:0/1797923264 conn(0x7f62700fc0a0 msgr2=0x7f62700fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 -- 172.21.15.2:0/1797923264 shutdown_connections 2024-03-16T10:53:42.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 -- 172.21.15.2:0/1797923264 wait complete. 2024-03-16T10:53:42.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.357+0000 7f6275859700 1 Processor -- start 2024-03-16T10:53:42.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f6275859700 1 -- start start 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f6275859700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6270100b00 0x7f62701964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f6275859700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270101d00 0x7f6270196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f6275859700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6270197050 con 0x7f6270101d00 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f6275859700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6270197190 con 0x7f6270100b00 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270101d00 0x7f6270196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270101d00 0x7f6270196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34520/0 (socket says 172.21.15.2:34520) 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626e7fc700 1 -- 172.21.15.2:0/3480238984 learned_addr learned my addr 172.21.15.2:0/3480238984 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626effd700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6270100b00 0x7f62701964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626e7fc700 1 -- 172.21.15.2:0/3480238984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6270100b00 msgr2=0x7f62701964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626e7fc700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6270100b00 0x7f62701964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626e7fc700 1 -- 172.21.15.2:0/3480238984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600095f0 con 0x7f6270101d00 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.358+0000 7f626e7fc700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270101d00 0x7f6270196a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f626400e920 tx=0x7f626400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.359+0000 7f6274857700 1 -- 172.21.15.2:0/3480238984 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626400c9d0 con 0x7f6270101d00 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.359+0000 7f6274857700 1 -- 172.21.15.2:0/3480238984 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6264010430 con 0x7f6270101d00 2024-03-16T10:53:42.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.359+0000 7f6274857700 1 -- 172.21.15.2:0/3480238984 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6264018630 con 0x7f6270101d00 2024-03-16T10:53:42.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.359+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627019bbf0 con 0x7f6270101d00 2024-03-16T10:53:42.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.359+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627019c1c0 con 0x7f6270101d00 2024-03-16T10:53:42.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.360+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6270190680 con 0x7f6270101d00 2024-03-16T10:53:42.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.360+0000 7f6274857700 1 -- 172.21.15.2:0/3480238984 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f62640105a0 con 0x7f6270101d00 2024-03-16T10:53:42.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.361+0000 7f6274857700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f625806dfa0 0x7f6258070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:42.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.361+0000 7f6274857700 1 -- 172.21.15.2:0/3480238984 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6264014070 con 0x7f6270101d00 2024-03-16T10:53:42.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.361+0000 7f626effd700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f625806dfa0 0x7f6258070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:42.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.361+0000 7f626effd700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f625806dfa0 0x7f6258070460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f626000b800 tx=0x7f6260009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.365+0000 7f6274857700 1 -- 172.21.15.2:0/3480238984 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f626405b470 con 0x7f6270101d00 2024-03-16T10:53:42.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.589+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f62700611f0 con 0x7f625806dfa0 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.601+0000 7f6274857700 1 -- 172.21.15.2:0/3480238984 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f62700611f0 con 0x7f625806dfa0 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (9s) 4s ago 17m 15.3M - 0.25.0 c8568f914cd2 552a20b3c093 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (68s) 4s ago 17m 7977k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 01d0397a345d 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (63s) 41s ago 16m 7541k - 19.0.0-2162-g3d78f5ba be86dc8cb22b e515b4badb59 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (6m) 4s ago 17m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (5m) 41s ago 16m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (15m) 4s ago 17m 87.8M - 9.4.7 954c08fa6188 2bf6cb9e633c 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (2m) 4s ago 11m 13.5M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a6b65504c2dc 2024-03-16T10:53:42.602 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (2m) 4s ago 11m 89.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a68e60da4931 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (105s) 41s ago 11m 12.8M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 745c119b7ff5 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (82s) 41s ago 11m 14.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 1cb95b8df6a0 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (8m) 4s ago 18m 512M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (7m) 41s ago 16m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (7m) 4s ago 19m 55.0M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (7m) 41s ago 16m 42.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (48s) 4s ago 17m 8564k - 1.5.0 0da6a335fe13 80c6dac2c511 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (44s) 41s ago 16m 3347k - 1.5.0 0da6a335fe13 46339cbf80f5 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (5m) 4s ago 15m 92.2M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (5m) 4s ago 14m 99.2M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (4m) 4s ago 14m 91.5M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:53:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (3m) 41s ago 14m 111M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:53:42.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (3m) 41s ago 13m 88.2M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:53:42.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (2m) 41s ago 13m 92.6M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b cf40ac20a977 2024-03-16T10:53:42.604 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (26s) 4s ago 16m 59.4M - 2.43.0 a07b618ecd1d 4e1230eee228 2024-03-16T10:53:42.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f625806dfa0 msgr2=0x7f6258070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f625806dfa0 0x7f6258070460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f626000b800 tx=0x7f6260009f70 comp rx=0 tx=0).stop 2024-03-16T10:53:42.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270101d00 msgr2=0x7f6270196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270101d00 0x7f6270196a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f626400e920 tx=0x7f626400ec30 comp rx=0 tx=0).stop 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 shutdown_connections 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f625806dfa0 0x7f6258070460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6270100b00 0x7f62701964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 --2- 172.21.15.2:0/3480238984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6270101d00 0x7f6270196a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 >> 172.21.15.2:0/3480238984 conn(0x7f62700fc0a0 msgr2=0x7f6270105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 shutdown_connections 2024-03-16T10:53:42.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.604+0000 7f6275859700 1 -- 172.21.15.2:0/3480238984 wait complete. 2024-03-16T10:53:42.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.712+0000 7fcb662f7700 1 -- 172.21.15.2:0/775994247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60103e90 msgr2=0x7fcb60106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.712+0000 7fcb662f7700 1 --2- 172.21.15.2:0/775994247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60103e90 0x7fcb60106280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009910 tx=0x7fcb54009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:42.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.712+0000 7fcb662f7700 1 -- 172.21.15.2:0/775994247 shutdown_connections 2024-03-16T10:53:42.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.712+0000 7fcb662f7700 1 --2- 172.21.15.2:0/775994247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60103e90 0x7fcb60106280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.712+0000 7fcb662f7700 1 --2- 172.21.15.2:0/775994247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60101560 0x7fcb60103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.712+0000 7fcb662f7700 1 -- 172.21.15.2:0/775994247 >> 172.21.15.2:0/775994247 conn(0x7fcb600faf70 msgr2=0x7fcb600fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:42.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.713+0000 7fcb662f7700 1 -- 172.21.15.2:0/775994247 shutdown_connections 2024-03-16T10:53:42.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.713+0000 7fcb662f7700 1 -- 172.21.15.2:0/775994247 wait complete. 2024-03-16T10:53:42.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.713+0000 7fcb662f7700 1 Processor -- start 2024-03-16T10:53:42.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.713+0000 7fcb662f7700 1 -- start start 2024-03-16T10:53:42.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb662f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60101560 0x7fcb60196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:42.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb662f7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60103e90 0x7fcb601969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:42.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb662f7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60196ff0 con 0x7fcb60101560 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb662f7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60197130 con 0x7fcb60103e90 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60101560 0x7fcb60196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60101560 0x7fcb60196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42100/0 (socket says 172.21.15.2:42100) 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5ffff700 1 -- 172.21.15.2:0/3729513263 learned_addr learned my addr 172.21.15.2:0/3729513263 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5f7fe700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60103e90 0x7fcb601969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5ffff700 1 -- 172.21.15.2:0/3729513263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60103e90 msgr2=0x7fcb601969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5ffff700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60103e90 0x7fcb601969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5ffff700 1 -- 172.21.15.2:0/3729513263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540095f0 con 0x7fcb60101560 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.714+0000 7fcb5ffff700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60101560 0x7fcb60196490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c8a0 tx=0x7fcb5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.715+0000 7fcb5d7fa700 1 -- 172.21.15.2:0/3729513263 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb5000cea0 con 0x7fcb60101560 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.715+0000 7fcb5d7fa700 1 -- 172.21.15.2:0/3729513263 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb50004830 con 0x7fcb60101560 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.715+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6019bb90 con 0x7fcb60101560 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.715+0000 7fcb5d7fa700 1 -- 172.21.15.2:0/3729513263 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb500055b0 con 0x7fcb60101560 2024-03-16T10:53:42.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.715+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6019c0e0 con 0x7fcb60101560 2024-03-16T10:53:42.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.716+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6004ea80 con 0x7fcb60101560 2024-03-16T10:53:42.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.716+0000 7fcb5d7fa700 1 -- 172.21.15.2:0/3729513263 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fcb500049a0 con 0x7fcb60101560 2024-03-16T10:53:42.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.717+0000 7fcb5d7fa700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcb4806dfa0 0x7fcb48070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:42.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.717+0000 7fcb5d7fa700 1 -- 172.21.15.2:0/3729513263 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcb5008daa0 con 0x7fcb60101560 2024-03-16T10:53:42.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.717+0000 7fcb5f7fe700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcb4806dfa0 0x7fcb48070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:42.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.717+0000 7fcb5f7fe700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcb4806dfa0 0x7fcb48070460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b5c0 tx=0x7fcb54005160 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:42.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:42.722+0000 7fcb5d7fa700 1 -- 172.21.15.2:0/3729513263 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb5005a9a0 con 0x7fcb60101560 2024-03-16T10:53:43.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.030+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb6019c360 con 0x7fcb60101560 2024-03-16T10:53:43.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.030+0000 7fcb5d7fa700 1 -- 172.21.15.2:0/3729513263 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcb50004c50 con 0x7fcb60101560 2024-03-16T10:53:43.031 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:53:43.031 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-16T10:53:43.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:53:43.033 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:53:43.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.032+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcb4806dfa0 msgr2=0x7fcb48070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.032+0000 7fcb662f7700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcb4806dfa0 0x7fcb48070460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400b5c0 tx=0x7fcb54005160 comp rx=0 tx=0).stop 2024-03-16T10:53:43.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.032+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60101560 msgr2=0x7fcb60196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.032+0000 7fcb662f7700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60101560 0x7fcb60196490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c8a0 tx=0x7fcb5000cc60 comp rx=0 tx=0).stop 2024-03-16T10:53:43.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.033+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 shutdown_connections 2024-03-16T10:53:43.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.033+0000 7fcb662f7700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcb4806dfa0 0x7fcb48070460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.033+0000 7fcb662f7700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb60101560 0x7fcb60196490 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.033+0000 7fcb662f7700 1 --2- 172.21.15.2:0/3729513263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb60103e90 0x7fcb601969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.033+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 >> 172.21.15.2:0/3729513263 conn(0x7fcb600faf70 msgr2=0x7fcb600fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:43.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.033+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 shutdown_connections 2024-03-16T10:53:43.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.033+0000 7fcb662f7700 1 -- 172.21.15.2:0/3729513263 wait complete. 2024-03-16T10:53:43.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:42 smithi002 ceph-mon[94239]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:43.139 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:42 smithi002 ceph-mon[94239]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:43.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 -- 172.21.15.2:0/1205946216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 msgr2=0x7fb07c106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 --2- 172.21.15.2:0/1205946216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c106280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb070009910 tx=0x7fb070009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:43.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 -- 172.21.15.2:0/1205946216 shutdown_connections 2024-03-16T10:53:43.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 --2- 172.21.15.2:0/1205946216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c106280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 --2- 172.21.15.2:0/1205946216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07c101560 0x7fb07c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 -- 172.21.15.2:0/1205946216 >> 172.21.15.2:0/1205946216 conn(0x7fb07c0faf70 msgr2=0x7fb07c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 -- 172.21.15.2:0/1205946216 shutdown_connections 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 -- 172.21.15.2:0/1205946216 wait complete. 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.138+0000 7fb08454f700 1 Processor -- start 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb08454f700 1 -- start start 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb08454f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07c101560 0x7fb07c100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb08454f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c0ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb08454f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0ffa50 con 0x7fb07c103e90 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb08454f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0ffb90 con 0x7fb07c101560 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb081aea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c0ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb0822eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07c101560 0x7fb07c100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb081aea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c0ff510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42128/0 (socket says 172.21.15.2:42128) 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb081aea700 1 -- 172.21.15.2:0/1244145713 learned_addr learned my addr 172.21.15.2:0/1244145713 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:43.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb081aea700 1 -- 172.21.15.2:0/1244145713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07c101560 msgr2=0x7fb07c100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb081aea700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07c101560 0x7fb07c100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb081aea700 1 -- 172.21.15.2:0/1244145713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700095f0 con 0x7fb07c103e90 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.139+0000 7fb081aea700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c0ff510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb07000b800 tx=0x7fb070005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.140+0000 7fb07b7fe700 1 -- 172.21.15.2:0/1244145713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07001d070 con 0x7fb07c103e90 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.140+0000 7fb07b7fe700 1 -- 172.21.15.2:0/1244145713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0700052a0 con 0x7fb07c103e90 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.140+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c0ffe10 con 0x7fb07c103e90 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.140+0000 7fb07b7fe700 1 -- 172.21.15.2:0/1244145713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb070017400 con 0x7fb07c103e90 2024-03-16T10:53:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.140+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c100380 con 0x7fb07c103e90 2024-03-16T10:53:43.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.141+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c04ea80 con 0x7fb07c103e90 2024-03-16T10:53:43.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.141+0000 7fb07b7fe700 1 -- 172.21.15.2:0/1244145713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fb070003780 con 0x7fb07c103e90 2024-03-16T10:53:43.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.142+0000 7fb07b7fe700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb06406dfa0 0x7fb064070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.142+0000 7fb07b7fe700 1 -- 172.21.15.2:0/1244145713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb07008eff0 con 0x7fb07c103e90 2024-03-16T10:53:43.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.142+0000 7fb0822eb700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb06406dfa0 0x7fb064070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.142+0000 7fb0822eb700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb06406dfa0 0x7fb064070460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00f540 tx=0x7fb06c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:43.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.147+0000 7fb07b7fe700 1 -- 172.21.15.2:0/1244145713 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb07005bf70 con 0x7fb07c103e90 2024-03-16T10:53:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:42 smithi026 ceph-mon[82600]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:43.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:42 smithi026 ceph-mon[82600]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.449+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb07c19e530 con 0x7fb07c103e90 2024-03-16T10:53:43.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.449+0000 7fb07b7fe700 1 -- 172.21.15.2:0/1244145713 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1929 (secure 0 0 0) 0x7fb07005bb00 con 0x7fb07c103e90 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:e31 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 31 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:52:21.877610+0000 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:53:43.451 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 86 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=34408} 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:53:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 34408 members: 34408 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{0:34408} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:53:43.453 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6828/1125443118,v1:172.21.15.2:6829/1125443118] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:43.454 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{-1:44337} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6824/2953126152,v1:172.21.15.26:6825/2953126152] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:43.454 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:44367} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6826/1298103624,v1:172.21.15.26:6827/1298103624] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:53:43.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb06406dfa0 msgr2=0x7fb064070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb06406dfa0 0x7fb064070460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00f540 tx=0x7fb06c00b040 comp rx=0 tx=0).stop 2024-03-16T10:53:43.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 msgr2=0x7fb07c0ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c0ff510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb07000b800 tx=0x7fb070005f00 comp rx=0 tx=0).stop 2024-03-16T10:53:43.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 shutdown_connections 2024-03-16T10:53:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb06406dfa0 0x7fb064070460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07c101560 0x7fb07c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 --2- 172.21.15.2:0/1244145713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb07c103e90 0x7fb07c0ff510 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 >> 172.21.15.2:0/1244145713 conn(0x7fb07c0faf70 msgr2=0x7fb07c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 shutdown_connections 2024-03-16T10:53:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.452+0000 7fb08454f700 1 -- 172.21.15.2:0/1244145713 wait complete. 2024-03-16T10:53:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 31 2024-03-16T10:53:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1157026835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28100ae0 msgr2=0x7f7e28100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1157026835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28100ae0 0x7f7e28100f00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009910 tx=0x7f7e18009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1157026835 shutdown_connections 2024-03-16T10:53:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1157026835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28101ce0 0x7f7e28102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1157026835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28100ae0 0x7f7e28100f00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1157026835 >> 172.21.15.2:0/1157026835 conn(0x7f7e280fc0a0 msgr2=0x7f7e280fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1157026835 shutdown_connections 2024-03-16T10:53:43.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.600+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1157026835 wait complete. 2024-03-16T10:53:43.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.601+0000 7f7e2c8a9700 1 Processor -- start 2024-03-16T10:53:43.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.601+0000 7f7e2c8a9700 1 -- start start 2024-03-16T10:53:43.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.601+0000 7f7e2c8a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28100ae0 0x7f7e281964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e2c8a9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28101ce0 0x7f7e28196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e2c8a9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28197030 con 0x7f7e28101ce0 2024-03-16T10:53:43.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e2c8a9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28197170 con 0x7f7e28100ae0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e25d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28101ce0 0x7f7e28196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e25d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28101ce0 0x7f7e28196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42132/0 (socket says 172.21.15.2:42132) 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e25d9b700 1 -- 172.21.15.2:0/1868310858 learned_addr learned my addr 172.21.15.2:0/1868310858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e2659c700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28100ae0 0x7f7e281964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e25d9b700 1 -- 172.21.15.2:0/1868310858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28100ae0 msgr2=0x7f7e281964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e25d9b700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28100ae0 0x7f7e281964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e25d9b700 1 -- 172.21.15.2:0/1868310858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e180095f0 con 0x7f7e28101ce0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e2659c700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28100ae0 0x7f7e281964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e25d9b700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28101ce0 0x7f7e28196a10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00d8d0 tx=0x7f7e1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.602+0000 7f7e177fe700 1 -- 172.21.15.2:0/1868310858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1c009940 con 0x7f7e28101ce0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.603+0000 7f7e177fe700 1 -- 172.21.15.2:0/1868310858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e1c010460 con 0x7f7e28101ce0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.603+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2819bbd0 con 0x7f7e28101ce0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.603+0000 7f7e177fe700 1 -- 172.21.15.2:0/1868310858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1c00f5f0 con 0x7f7e28101ce0 2024-03-16T10:53:43.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.603+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2819c1a0 con 0x7f7e28101ce0 2024-03-16T10:53:43.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.604+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e28190660 con 0x7f7e28101ce0 2024-03-16T10:53:43.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.604+0000 7f7e177fe700 1 -- 172.21.15.2:0/1868310858 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f7e1c0105d0 con 0x7f7e28101ce0 2024-03-16T10:53:43.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.605+0000 7f7e177fe700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7e1006dfb0 0x7f7e10070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.605+0000 7f7e177fe700 1 -- 172.21.15.2:0/1868310858 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7e1c08de30 con 0x7f7e28101ce0 2024-03-16T10:53:43.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.605+0000 7f7e2659c700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7e1006dfb0 0x7f7e10070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.605+0000 7f7e2659c700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7e1006dfb0 0x7f7e10070470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006010 tx=0x7f7e1800b560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:43.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.609+0000 7f7e177fe700 1 -- 172.21.15.2:0/1868310858 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e1c05ade0 con 0x7f7e28101ce0 2024-03-16T10:53:43.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.836+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e280611f0 con 0x7f7e1006dfb0 2024-03-16T10:53:43.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.837+0000 7f7e177fe700 1 -- 172.21.15.2:0/1868310858 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7f7e280611f0 con 0x7f7e1006dfb0 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513", 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": true, 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [ 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph-exporter", 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr", 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "osd", 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "mds", 2024-03-16T10:53:43.838 INFO:teuthology.orchestra.run.smithi002.stdout: "mon" 2024-03-16T10:53:43.839 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-03-16T10:53:43.839 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": "18/23 daemons upgraded", 2024-03-16T10:53:43.839 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "Currently upgrading grafana daemons", 2024-03-16T10:53:43.839 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:53:43.839 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:53:43.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7e1006dfb0 msgr2=0x7f7e10070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7e1006dfb0 0x7f7e10070470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006010 tx=0x7f7e1800b560 comp rx=0 tx=0).stop 2024-03-16T10:53:43.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28101ce0 msgr2=0x7f7e28196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28101ce0 0x7f7e28196a10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00d8d0 tx=0x7f7e1c00dc90 comp rx=0 tx=0).stop 2024-03-16T10:53:43.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 shutdown_connections 2024-03-16T10:53:43.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7e1006dfb0 0x7f7e10070470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e28100ae0 0x7f7e281964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 --2- 172.21.15.2:0/1868310858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e28101ce0 0x7f7e28196a10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 >> 172.21.15.2:0/1868310858 conn(0x7f7e280fc0a0 msgr2=0x7f7e281052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:43.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.839+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 shutdown_connections 2024-03-16T10:53:43.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.840+0000 7f7e2c8a9700 1 -- 172.21.15.2:0/1868310858 wait complete. 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 -- 172.21.15.2:0/3359063633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 msgr2=0x7f7934101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 --2- 172.21.15.2:0/3359063633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f7934101030 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7928009910 tx=0x7f7928009c20 comp rx=0 tx=0).stop 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 -- 172.21.15.2:0/3359063633 shutdown_connections 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 --2- 172.21.15.2:0/3359063633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f7934101030 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 --2- 172.21.15.2:0/3359063633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79340ff9d0 0x7f79340ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 -- 172.21.15.2:0/3359063633 >> 172.21.15.2:0/3359063633 conn(0x7f79340faf70 msgr2=0x7f79340fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 -- 172.21.15.2:0/3359063633 shutdown_connections 2024-03-16T10:53:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.949+0000 7f7938be9700 1 -- 172.21.15.2:0/3359063633 wait complete. 2024-03-16T10:53:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.950+0000 7f7938be9700 1 Processor -- start 2024-03-16T10:53:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.950+0000 7f7938be9700 1 -- start start 2024-03-16T10:53:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.950+0000 7f7938be9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79340ff9d0 0x7f79340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.950+0000 7f7938be9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f793419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.950+0000 7f7938be9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793419b780 con 0x7f7934100bd0 2024-03-16T10:53:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.950+0000 7f7938be9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793419b8c0 con 0x7f79340ff9d0 2024-03-16T10:53:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f7931d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f793419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f7931d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f793419b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42144/0 (socket says 172.21.15.2:42144) 2024-03-16T10:53:43.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f7931d9b700 1 -- 172.21.15.2:0/756996607 learned_addr learned my addr 172.21.15.2:0/756996607 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f793259c700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79340ff9d0 0x7f79340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f7931d9b700 1 -- 172.21.15.2:0/756996607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79340ff9d0 msgr2=0x7f79340697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f7931d9b700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79340ff9d0 0x7f79340697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f7931d9b700 1 -- 172.21.15.2:0/756996607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79280095f0 con 0x7f7934100bd0 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.951+0000 7f793259c700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79340ff9d0 0x7f79340697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.952+0000 7f7931d9b700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f793419b240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7928006010 tx=0x7f792800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.952+0000 7f79237fe700 1 -- 172.21.15.2:0/756996607 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792801d070 con 0x7f7934100bd0 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.952+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793419bb10 con 0x7f7934100bd0 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.952+0000 7f79237fe700 1 -- 172.21.15.2:0/756996607 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f792800f460 con 0x7f7934100bd0 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.952+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793419c080 con 0x7f7934100bd0 2024-03-16T10:53:43.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.952+0000 7f79237fe700 1 -- 172.21.15.2:0/756996607 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7928021640 con 0x7f7934100bd0 2024-03-16T10:53:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.953+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793418f550 con 0x7f7934100bd0 2024-03-16T10:53:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.953+0000 7f79237fe700 1 -- 172.21.15.2:0/756996607 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f792800fab0 con 0x7f7934100bd0 2024-03-16T10:53:43.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.954+0000 7f79237fe700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791c06dff0 0x7f791c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:53:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.954+0000 7f79237fe700 1 -- 172.21.15.2:0/756996607 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f792808f0f0 con 0x7f7934100bd0 2024-03-16T10:53:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.954+0000 7f793259c700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791c06dff0 0x7f791c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:53:43.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.955+0000 7f793259c700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791c06dff0 0x7f791c0704b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7924005950 tx=0x7f79240058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:53:43.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:43.959+0000 7f79237fe700 1 -- 172.21.15.2:0/756996607 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f792805bff0 con 0x7f7934100bd0 2024-03-16T10:53:44.115 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='client.44423 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:44.115 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:44.115 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3729513263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.115 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:44.115 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:44.115 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1244145713' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.116 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.117 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.117 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.117 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.117 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.117 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.117 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:43 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='client.44423 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3729513263' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:53:44.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1244145713' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:43 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:53:44.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.268+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f793404ea80 con 0x7f7934100bd0 2024-03-16T10:53:44.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.269+0000 7f79237fe700 1 -- 172.21.15.2:0/756996607 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f792805bb80 con 0x7f7934100bd0 2024-03-16T10:53:44.269 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:53:44.270 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:53:44.270 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791c06dff0 msgr2=0x7f791c0704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791c06dff0 0x7f791c0704b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7924005950 tx=0x7f79240058e0 comp rx=0 tx=0).stop 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 msgr2=0x7f793419b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f793419b240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7928006010 tx=0x7f792800be20 comp rx=0 tx=0).stop 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 shutdown_connections 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791c06dff0 0x7f791c0704b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79340ff9d0 0x7f79340697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 --2- 172.21.15.2:0/756996607 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7934100bd0 0x7f793419b240 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 >> 172.21.15.2:0/756996607 conn(0x7f79340faf70 msgr2=0x7f7934104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 shutdown_connections 2024-03-16T10:53:44.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:53:44.271+0000 7f7938be9700 1 -- 172.21.15.2:0/756996607 wait complete. 2024-03-16T10:53:45.142 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:44 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:53:45.142 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:44 smithi002 ceph-mon[94239]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:45.142 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:44 smithi002 ceph-mon[94239]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:45.142 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:44 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/756996607' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:53:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:44 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-16T10:53:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:44 smithi026 ceph-mon[82600]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:44 smithi026 ceph-mon[82600]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:53:45.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:44 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/756996607' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:53:47.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:46 smithi026 ceph-mon[82600]: Upgrade: Updating grafana.smithi002 2024-03-16T10:53:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:46 smithi026 ceph-mon[82600]: Deploying daemon grafana.smithi002 on smithi002 2024-03-16T10:53:47.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:46 smithi026 ceph-mon[82600]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:46 smithi002 ceph-mon[94239]: Upgrade: Updating grafana.smithi002 2024-03-16T10:53:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:46 smithi002 ceph-mon[94239]: Deploying daemon grafana.smithi002 on smithi002 2024-03-16T10:53:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:46 smithi002 ceph-mon[94239]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:48 smithi026 ceph-mon[82600]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:48 smithi002 ceph-mon[94239]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:50.181 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:49 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:50.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:49 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:53:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:50 smithi026 ceph-mon[82600]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:50 smithi002 ceph-mon[94239]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:52 smithi026 ceph-mon[82600]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:52 smithi002 ceph-mon[94239]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:55.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:54 smithi026 ceph-mon[82600]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:55.244 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:54 smithi002 ceph-mon[94239]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:56 smithi026 ceph-mon[82600]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:56 smithi002 ceph-mon[94239]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:59.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:53:58 smithi026 ceph-mon[82600]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:53:59.275 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:53:58 smithi002 ceph-mon[94239]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:00.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:00 smithi026 ceph-mon[82600]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:01.211 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:00 smithi002 ceph-mon[94239]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:02.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:02.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:02.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:02 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:02.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:02 smithi002 ceph-mon[94239]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:02.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:02.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:02.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:02 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:02.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:02 smithi026 ceph-mon[82600]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:05.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:04 smithi002 ceph-mon[94239]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:05.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:04 smithi026 ceph-mon[82600]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:07.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:06 smithi026 ceph-mon[82600]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:07.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:07.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:06 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:06 smithi002 ceph-mon[94239]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:06 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:08.731 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:08.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:08 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:08.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:08 smithi026 ceph-mon[82600]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:08.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:08.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:08 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:08.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:08 smithi002 ceph-mon[94239]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:11.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:10 smithi002 ceph-mon[94239]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:11.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:10 smithi026 ceph-mon[82600]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:54:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:54:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.483 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: Upgrade: Finalizing container_image settings 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-16T10:54:12.484 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-16T10:54:12.485 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.486 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: Upgrade: Complete! 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:54:12.487 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.549 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.550 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: Upgrade: Finalizing container_image settings 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-16T10:54:12.551 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.552 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: Upgrade: Complete! 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:54:12.553 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 -- 172.21.15.2:0/579921601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 msgr2=0x7f55c4103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 --2- 172.21.15.2:0/579921601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4103950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009910 tx=0x7f55b4009c20 comp rx=0 tx=0).stop 2024-03-16T10:54:14.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 -- 172.21.15.2:0/579921601 shutdown_connections 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 --2- 172.21.15.2:0/579921601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c4103e90 0x7f55c4106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 --2- 172.21.15.2:0/579921601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4103950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 -- 172.21.15.2:0/579921601 >> 172.21.15.2:0/579921601 conn(0x7f55c40faf70 msgr2=0x7f55c40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 -- 172.21.15.2:0/579921601 shutdown_connections 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 -- 172.21.15.2:0/579921601 wait complete. 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 Processor -- start 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.382+0000 7f55c974a700 1 -- start start 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c974a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c974a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c4103e90 0x7f55c40ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c974a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40ffa50 con 0x7f55c4101560 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c974a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40ffb90 con 0x7f55c4103e90 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39524/0 (socket says 172.21.15.2:39524) 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c2ffd700 1 -- 172.21.15.2:0/2896086392 learned_addr learned my addr 172.21.15.2:0/2896086392 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:54:14.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c27fc700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c4103e90 0x7f55c40ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c2ffd700 1 -- 172.21.15.2:0/2896086392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c4103e90 msgr2=0x7f55c40ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c2ffd700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c4103e90 0x7f55c40ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c2ffd700 1 -- 172.21.15.2:0/2896086392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b40095f0 con 0x7f55c4101560 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c27fc700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c4103e90 0x7f55c40ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c2ffd700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4100e70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55b40092e0 tx=0x7f55b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55abfff700 1 -- 172.21.15.2:0/2896086392 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55b401d070 con 0x7f55c4101560 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55abfff700 1 -- 172.21.15.2:0/2896086392 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55b400f460 con 0x7f55c4101560 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c40ffe10 con 0x7f55c4101560 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.383+0000 7f55abfff700 1 -- 172.21.15.2:0/2896086392 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55b4021640 con 0x7f55c4101560 2024-03-16T10:54:14.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.384+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c4100380 con 0x7f55c4101560 2024-03-16T10:54:14.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.384+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c404ea80 con 0x7f55c4101560 2024-03-16T10:54:14.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.385+0000 7f55abfff700 1 -- 172.21.15.2:0/2896086392 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f55b400fad0 con 0x7f55c4101560 2024-03-16T10:54:14.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.385+0000 7f55abfff700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f55ac06dfa0 0x7f55ac070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:14.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.385+0000 7f55abfff700 1 -- 172.21.15.2:0/2896086392 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f55b408ef20 con 0x7f55c4101560 2024-03-16T10:54:14.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.385+0000 7f55c27fc700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f55ac06dfa0 0x7f55ac070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:14.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.386+0000 7f55c27fc700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f55ac06dfa0 0x7f55ac070460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55b8005fd0 tx=0x7f55b8005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:14.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.392+0000 7f55abfff700 1 -- 172.21.15.2:0/2896086392 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55b405bda0 con 0x7f55c4101560 2024-03-16T10:54:14.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.632+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55c419e5b0 con 0x7f55ac06dfa0 2024-03-16T10:54:14.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.634+0000 7f55abfff700 1 -- 172.21.15.2:0/2896086392 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f55c419e5b0 con 0x7f55ac06dfa0 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f55ac06dfa0 msgr2=0x7f55ac070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f55ac06dfa0 0x7f55ac070460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55b8005fd0 tx=0x7f55b8005ee0 comp rx=0 tx=0).stop 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 msgr2=0x7f55c4100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4100e70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55b40092e0 tx=0x7f55b400be20 comp rx=0 tx=0).stop 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 shutdown_connections 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f55ac06dfa0 0x7f55ac070460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c4101560 0x7f55c4100e70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 --2- 172.21.15.2:0/2896086392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c4103e90 0x7f55c40ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 >> 172.21.15.2:0/2896086392 conn(0x7f55c40faf70 msgr2=0x7f55c40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 shutdown_connections 2024-03-16T10:54:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:14.636+0000 7f55c974a700 1 -- 172.21.15.2:0/2896086392 wait complete. 2024-03-16T10:54:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:14 smithi026 ceph-mon[82600]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:15.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:15.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:14 smithi002 ceph-mon[94239]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:15.274 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:54:15.295 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-16T10:54:15.562 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:54:15.953 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:15 smithi002 ceph-mon[94239]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:54:16.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:15 smithi026 ceph-mon[82600]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:54:16.999 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:16 smithi002 ceph-mon[94239]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:17.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 -- 172.21.15.2:0/388967457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928106a90 msgr2=0x7f7928106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:17.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 --2- 172.21.15.2:0/388967457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928106a90 0x7f7928106e70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7924009910 tx=0x7f7924009c20 comp rx=0 tx=0).stop 2024-03-16T10:54:17.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 -- 172.21.15.2:0/388967457 shutdown_connections 2024-03-16T10:54:17.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 --2- 172.21.15.2:0/388967457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7928100a90 0x7f7928100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:17.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 --2- 172.21.15.2:0/388967457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928106a90 0x7f7928106e70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:17.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 -- 172.21.15.2:0/388967457 >> 172.21.15.2:0/388967457 conn(0x7f79280fc580 msgr2=0x7f79280fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:17.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 -- 172.21.15.2:0/388967457 shutdown_connections 2024-03-16T10:54:17.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.220+0000 7f792f7ec700 1 -- 172.21.15.2:0/388967457 wait complete. 2024-03-16T10:54:17.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792f7ec700 1 Processor -- start 2024-03-16T10:54:17.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792f7ec700 1 -- start start 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792f7ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928100a90 0x7f79280730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792f7ec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7928106a90 0x7f79280735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792f7ec700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928073b30 con 0x7f7928100a90 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792f7ec700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928073c70 con 0x7f7928106a90 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792d588700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928100a90 0x7f79280730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792d588700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928100a90 0x7f79280730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39546/0 (socket says 172.21.15.2:39546) 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792d588700 1 -- 172.21.15.2:0/2440047808 learned_addr learned my addr 172.21.15.2:0/2440047808 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.221+0000 7f792cd87700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7928106a90 0x7f79280735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f792d588700 1 -- 172.21.15.2:0/2440047808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7928106a90 msgr2=0x7f79280735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f792d588700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7928106a90 0x7f79280735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f792d588700 1 -- 172.21.15.2:0/2440047808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240095f0 con 0x7f7928100a90 2024-03-16T10:54:17.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f792cd87700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7928106a90 0x7f79280735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f792d588700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928100a90 0x7f79280730b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7924006010 tx=0x7f792400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f791e7fc700 1 -- 172.21.15.2:0/2440047808 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792401d070 con 0x7f7928100a90 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f791e7fc700 1 -- 172.21.15.2:0/2440047808 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f792400f460 con 0x7f7928100a90 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7928073ef0 con 0x7f7928100a90 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f791e7fc700 1 -- 172.21.15.2:0/2440047808 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7924017630 con 0x7f7928100a90 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.222+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79281a2ef0 con 0x7f7928100a90 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.223+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928109090 con 0x7f7928100a90 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.223+0000 7f791e7fc700 1 -- 172.21.15.2:0/2440047808 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f792400f5d0 con 0x7f7928100a90 2024-03-16T10:54:17.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.224+0000 7f791e7fc700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791406dfa0 0x7f7914070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:17.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.224+0000 7f791e7fc700 1 -- 172.21.15.2:0/2440047808 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f792408f290 con 0x7f7928100a90 2024-03-16T10:54:17.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.224+0000 7f792cd87700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791406dfa0 0x7f7914070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:17.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.225+0000 7f792cd87700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791406dfa0 0x7f7914070460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79280747e0 tx=0x7f7918008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:17.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.229+0000 7f791e7fc700 1 -- 172.21.15.2:0/2440047808 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7924058720 con 0x7f7928100a90 2024-03-16T10:54:17.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:16 smithi026 ceph-mon[82600]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:17.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.442+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79280611f0 con 0x7f791406dfa0 2024-03-16T10:54:17.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.454+0000 7f791e7fc700 1 -- 172.21.15.2:0/2440047808 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f79280611f0 con 0x7f791406dfa0 2024-03-16T10:54:17.454 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (44s) 11s ago 17m 15.3M - 0.25.0 c8568f914cd2 552a20b3c093 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (103s) 11s ago 18m 8019k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 01d0397a345d 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (97s) 76s ago 17m 7541k - 19.0.0-2162-g3d78f5ba be86dc8cb22b e515b4badb59 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (6m) 11s ago 18m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (6m) 76s ago 17m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (16s) 11s ago 17m 59.5M - 9.4.12 a0ea5e3c46d9 70910746629d 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (2m) 11s ago 12m 13.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a6b65504c2dc 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (3m) 11s ago 12m 90.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a68e60da4931 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (2m) 76s ago 12m 12.8M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 745c119b7ff5 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (117s) 76s ago 12m 14.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 1cb95b8df6a0 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (9m) 11s ago 19m 513M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (8m) 76s ago 16m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (8m) 11s ago 19m 54.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (7m) 76s ago 16m 42.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (83s) 11s ago 18m 8833k - 1.5.0 0da6a335fe13 80c6dac2c511 2024-03-16T10:54:17.455 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (79s) 76s ago 16m 3347k - 1.5.0 0da6a335fe13 46339cbf80f5 2024-03-16T10:54:17.456 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (6m) 11s ago 16m 92.4M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:54:17.456 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (5m) 11s ago 15m 99.3M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:54:17.456 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (5m) 11s ago 15m 91.7M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:54:17.456 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (4m) 76s ago 14m 111M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:54:17.456 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (3m) 76s ago 14m 88.2M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:54:17.456 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (3m) 76s ago 13m 92.6M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b cf40ac20a977 2024-03-16T10:54:17.456 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (61s) 11s ago 17m 62.0M - 2.43.0 a07b618ecd1d 4e1230eee228 2024-03-16T10:54:17.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791406dfa0 msgr2=0x7f7914070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:17.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791406dfa0 0x7f7914070460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79280747e0 tx=0x7f7918008040 comp rx=0 tx=0).stop 2024-03-16T10:54:17.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928100a90 msgr2=0x7f79280730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:17.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928100a90 0x7f79280730b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7924006010 tx=0x7f792400be20 comp rx=0 tx=0).stop 2024-03-16T10:54:17.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 shutdown_connections 2024-03-16T10:54:17.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f791406dfa0 0x7f7914070460 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:17.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7928100a90 0x7f79280730b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:17.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 --2- 172.21.15.2:0/2440047808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7928106a90 0x7f79280735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:17.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.456+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 >> 172.21.15.2:0/2440047808 conn(0x7f79280fc580 msgr2=0x7f79280fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:17.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.457+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 shutdown_connections 2024-03-16T10:54:17.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:17.457+0000 7f792f7ec700 1 -- 172.21.15.2:0/2440047808 wait complete. 2024-03-16T10:54:18.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch upgrade status' 2024-03-16T10:54:18.354 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:54:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:18 smithi026 ceph-mon[82600]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:54:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:18 smithi026 ceph-mon[82600]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:18 smithi002 ceph-mon[94239]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:54:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:18 smithi002 ceph-mon[94239]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/149186676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb00ff9a0 msgr2=0x7f3bb00ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/149186676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb00ff9a0 0x7f3bb00ffe20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009910 tx=0x7f3ba4009c20 comp rx=0 tx=0).stop 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/149186676 shutdown_connections 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/149186676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb00ff9a0 0x7f3bb00ffe20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/149186676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bb01059a0 0x7f3bb0105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/149186676 >> 172.21.15.2:0/149186676 conn(0x7f3bb00fb4b0 msgr2=0x7f3bb00fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/149186676 shutdown_connections 2024-03-16T10:54:19.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.962+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/149186676 wait complete. 2024-03-16T10:54:19.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3bb5b8a700 1 Processor -- start 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3bb5b8a700 1 -- start start 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3bb5b8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bb00ff9a0 0x7f3bb0197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3bb5b8a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb01059a0 0x7f3bb0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3bb5b8a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb01982a0 con 0x7f3bb01059a0 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3bb5b8a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb01983e0 con 0x7f3bb00ff9a0 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3baeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb01059a0 0x7f3bb0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3baeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb01059a0 0x7f3bb0197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39556/0 (socket says 172.21.15.2:39556) 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3baeffd700 1 -- 172.21.15.2:0/3882030944 learned_addr learned my addr 172.21.15.2:0/3882030944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3baf7fe700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bb00ff9a0 0x7f3bb0197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3baeffd700 1 -- 172.21.15.2:0/3882030944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bb00ff9a0 msgr2=0x7f3bb0197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3baeffd700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bb00ff9a0 0x7f3bb0197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.963+0000 7f3baeffd700 1 -- 172.21.15.2:0/3882030944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40095f0 con 0x7f3bb01059a0 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.964+0000 7f3baeffd700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb01059a0 0x7f3bb0197d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4006010 tx=0x7f3ba400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:19.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.964+0000 7f3bacff9700 1 -- 172.21.15.2:0/3882030944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba401d070 con 0x7f3bb01059a0 2024-03-16T10:54:19.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.964+0000 7f3bacff9700 1 -- 172.21.15.2:0/3882030944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ba400f460 con 0x7f3bb01059a0 2024-03-16T10:54:19.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.964+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb0198660 con 0x7f3bb01059a0 2024-03-16T10:54:19.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.964+0000 7f3bacff9700 1 -- 172.21.15.2:0/3882030944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba40175f0 con 0x7f3bb01059a0 2024-03-16T10:54:19.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.964+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb01a0b70 con 0x7f3bb01059a0 2024-03-16T10:54:19.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.965+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb0107fc0 con 0x7f3bb01059a0 2024-03-16T10:54:19.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.966+0000 7f3bacff9700 1 -- 172.21.15.2:0/3882030944 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3ba400f5d0 con 0x7f3bb01059a0 2024-03-16T10:54:19.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.966+0000 7f3bacff9700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3b9806dfa0 0x7f3b98070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:19.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.966+0000 7f3bacff9700 1 -- 172.21.15.2:0/3882030944 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3ba408f250 con 0x7f3bb01059a0 2024-03-16T10:54:19.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.966+0000 7f3baf7fe700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3b9806dfa0 0x7f3b98070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:19.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.966+0000 7f3baf7fe700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3b9806dfa0 0x7f3b98070460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000ac50 tx=0x7f3ba000a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:19.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:19.971+0000 7f3bacff9700 1 -- 172.21.15.2:0/3882030944 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ba4058620 con 0x7f3bb01059a0 2024-03-16T10:54:20.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.200+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bb00611f0 con 0x7f3b9806dfa0 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.201+0000 7f3bacff9700 1 -- 172.21.15.2:0/3882030944 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f3bb00611f0 con 0x7f3b9806dfa0 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stdout: "target_image": null, 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stdout: "in_progress": false, 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stdout: "which": "", 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stdout: "services_complete": [], 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stdout: "progress": null, 2024-03-16T10:54:20.202 INFO:teuthology.orchestra.run.smithi002.stdout: "message": "", 2024-03-16T10:54:20.203 INFO:teuthology.orchestra.run.smithi002.stdout: "is_paused": false 2024-03-16T10:54:20.203 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:54:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3b9806dfa0 msgr2=0x7f3b98070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3b9806dfa0 0x7f3b98070460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000ac50 tx=0x7f3ba000a380 comp rx=0 tx=0).stop 2024-03-16T10:54:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb01059a0 msgr2=0x7f3bb0197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb01059a0 0x7f3bb0197d60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4006010 tx=0x7f3ba400bd70 comp rx=0 tx=0).stop 2024-03-16T10:54:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 shutdown_connections 2024-03-16T10:54:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3b9806dfa0 0x7f3b98070460 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bb00ff9a0 0x7f3bb0197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 --2- 172.21.15.2:0/3882030944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bb01059a0 0x7f3bb0197d60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.203+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 >> 172.21.15.2:0/3882030944 conn(0x7f3bb00fb4b0 msgr2=0x7f3bb00fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.204+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 shutdown_connections 2024-03-16T10:54:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:20.204+0000 7f3bb5b8a700 1 -- 172.21.15.2:0/3882030944 wait complete. 2024-03-16T10:54:20.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:20.846 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph health detail' 2024-03-16T10:54:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:54:21.186 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:20 smithi002 ceph-mon[94239]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:20 smithi026 ceph-mon[82600]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:22.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:21 smithi026 ceph-mon[82600]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:54:22.235 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:21 smithi002 ceph-mon[94239]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 -- 172.21.15.2:0/1331107068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 msgr2=0x7fcffc0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 --2- 172.21.15.2:0/1331107068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0ffdf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009910 tx=0x7fcfec009c20 comp rx=0 tx=0).stop 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 -- 172.21.15.2:0/1331107068 shutdown_connections 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 --2- 172.21.15.2:0/1331107068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0ffdf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 --2- 172.21.15.2:0/1331107068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc105970 0x7fcffc105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 -- 172.21.15.2:0/1331107068 >> 172.21.15.2:0/1331107068 conn(0x7fcffc0fb480 msgr2=0x7fcffc0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 -- 172.21.15.2:0/1331107068 shutdown_connections 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 -- 172.21.15.2:0/1331107068 wait complete. 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 Processor -- start 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.761+0000 7fd0046f2700 1 -- start start 2024-03-16T10:54:22.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd0046f2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd0046f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc105970 0x7fcffc0fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd0046f2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0fc6e0 con 0x7fcffc0ff970 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd0046f2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0fc820 con 0x7fcffc105970 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd00248e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd00248e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0fbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50644/0 (socket says 172.21.15.2:50644) 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd00248e700 1 -- 172.21.15.2:0/3132306920 learned_addr learned my addr 172.21.15.2:0/3132306920 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd001c8d700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc105970 0x7fcffc0fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd00248e700 1 -- 172.21.15.2:0/3132306920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc105970 msgr2=0x7fcffc0fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd00248e700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc105970 0x7fcffc0fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd00248e700 1 -- 172.21.15.2:0/3132306920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0095f0 con 0x7fcffc0ff970 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd00248e700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0fbc60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcff800d8d0 tx=0x7fcff800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:22.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fcff37fe700 1 -- 172.21.15.2:0/3132306920 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff80098a0 con 0x7fcffc0ff970 2024-03-16T10:54:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc1a2ba0 con 0x7fcffc0ff970 2024-03-16T10:54:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fcff37fe700 1 -- 172.21.15.2:0/3132306920 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcff8010460 con 0x7fcffc0ff970 2024-03-16T10:54:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fcff37fe700 1 -- 172.21.15.2:0/3132306920 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff800f5f0 con 0x7fcffc0ff970 2024-03-16T10:54:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.762+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc1a2f00 con 0x7fcffc0ff970 2024-03-16T10:54:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.763+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc108fa0 con 0x7fcffc0ff970 2024-03-16T10:54:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.764+0000 7fcff37fe700 1 -- 172.21.15.2:0/3132306920 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fcff8009a00 con 0x7fcffc0ff970 2024-03-16T10:54:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.765+0000 7fcff37fe700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcfe806dfa0 0x7fcfe8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.765+0000 7fd001c8d700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcfe806dfa0 0x7fcfe8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.765+0000 7fcff37fe700 1 -- 172.21.15.2:0/3132306920 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fcff808dcb0 con 0x7fcffc0ff970 2024-03-16T10:54:22.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.765+0000 7fd001c8d700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcfe806dfa0 0x7fcfe8070460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0fd390 tx=0x7fcfec01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:22.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:22.769+0000 7fcff37fe700 1 -- 172.21.15.2:0/3132306920 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcff8057080 con 0x7fcffc0ff970 2024-03-16T10:54:23.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.115+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcffc0fd140 con 0x7fcffc0ff970 2024-03-16T10:54:23.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.115+0000 7fcff37fe700 1 -- 172.21.15.2:0/3132306920 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fcff805a740 con 0x7fcffc0ff970 2024-03-16T10:54:23.116 INFO:teuthology.orchestra.run.smithi002.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T10:54:23.116 INFO:teuthology.orchestra.run.smithi002.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T10:54:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T10:54:23.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcfe806dfa0 msgr2=0x7fcfe8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcfe806dfa0 0x7fcfe8070460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0fd390 tx=0x7fcfec01a040 comp rx=0 tx=0).stop 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 msgr2=0x7fcffc0fbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0fbc60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcff800d8d0 tx=0x7fcff800dbe0 comp rx=0 tx=0).stop 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 shutdown_connections 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fcfe806dfa0 0x7fcfe8070460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcffc0ff970 0x7fcffc0fbc60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 --2- 172.21.15.2:0/3132306920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcffc105970 0x7fcffc0fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 >> 172.21.15.2:0/3132306920 conn(0x7fcffc0fb480 msgr2=0x7fcffc069230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 shutdown_connections 2024-03-16T10:54:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:23.118+0000 7fd0046f2700 1 -- 172.21.15.2:0/3132306920 wait complete. 2024-03-16T10:54:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:22 smithi026 ceph-mon[82600]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:22 smithi002 ceph-mon[94239]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:23.863 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions' 2024-03-16T10:54:24.105 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:54:24.226 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:23 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3132306920' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:54:24.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:23 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3132306920' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-16T10:54:25.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:24 smithi026 ceph-mon[82600]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:24 smithi002 ceph-mon[94239]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:27.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:26 smithi026 ceph-mon[82600]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:26 smithi002 ceph-mon[94239]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:28 smithi026 ceph-mon[82600]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:28 smithi002 ceph-mon[94239]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:31.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:30 smithi002 ceph-mon[94239]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:30 smithi026 ceph-mon[82600]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:32.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.754+0000 7fe9a07b2700 1 -- 172.21.15.2:0/3538453529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 msgr2=0x7fe998106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:32.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.754+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/3538453529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe998106e50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe994009910 tx=0x7fe994009c20 comp rx=0 tx=0).stop 2024-03-16T10:54:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.754+0000 7fe9a07b2700 1 -- 172.21.15.2:0/3538453529 shutdown_connections 2024-03-16T10:54:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.755+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/3538453529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe998100a70 0x7fe998100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.755+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/3538453529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe998106e50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.755+0000 7fe9a07b2700 1 -- 172.21.15.2:0/3538453529 >> 172.21.15.2:0/3538453529 conn(0x7fe9980fc5a0 msgr2=0x7fe9980fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.755+0000 7fe9a07b2700 1 -- 172.21.15.2:0/3538453529 shutdown_connections 2024-03-16T10:54:32.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.755+0000 7fe9a07b2700 1 -- 172.21.15.2:0/3538453529 wait complete. 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.755+0000 7fe9a07b2700 1 Processor -- start 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.755+0000 7fe9a07b2700 1 -- start start 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe9a07b2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe998100a70 0x7fe99819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe9a07b2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe99819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe9a07b2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99819d310 con 0x7fe998106a70 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe9a07b2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9981968f0 con 0x7fe998100a70 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99dd4d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe99819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99dd4d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe99819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52912/0 (socket says 172.21.15.2:52912) 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99dd4d700 1 -- 172.21.15.2:0/1781930530 learned_addr learned my addr 172.21.15.2:0/1781930530 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99e54e700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe998100a70 0x7fe99819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99dd4d700 1 -- 172.21.15.2:0/1781930530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe998100a70 msgr2=0x7fe99819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:32.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99dd4d700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe998100a70 0x7fe99819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99dd4d700 1 -- 172.21.15.2:0/1781930530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9940095f0 con 0x7fe998106a70 2024-03-16T10:54:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe99dd4d700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe99819ccf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b4b0 tx=0x7fe98800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.756+0000 7fe98f7fe700 1 -- 172.21.15.2:0/1781930530 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe988011800 con 0x7fe998106a70 2024-03-16T10:54:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.757+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe998196bd0 con 0x7fe998106a70 2024-03-16T10:54:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.757+0000 7fe98f7fe700 1 -- 172.21.15.2:0/1781930530 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe988011e20 con 0x7fe998106a70 2024-03-16T10:54:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.757+0000 7fe98f7fe700 1 -- 172.21.15.2:0/1781930530 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe98800f2e0 con 0x7fe998106a70 2024-03-16T10:54:32.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.757+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe998197120 con 0x7fe998106a70 2024-03-16T10:54:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.757+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe998109070 con 0x7fe998106a70 2024-03-16T10:54:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.758+0000 7fe98f7fe700 1 -- 172.21.15.2:0/1781930530 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fe988010380 con 0x7fe998106a70 2024-03-16T10:54:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.759+0000 7fe98f7fe700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe98406dfa0 0x7fe984070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:54:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.759+0000 7fe98f7fe700 1 -- 172.21.15.2:0/1781930530 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe98808d950 con 0x7fe998106a70 2024-03-16T10:54:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.759+0000 7fe99e54e700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe98406dfa0 0x7fe984070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:54:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.759+0000 7fe99e54e700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe98406dfa0 0x7fe984070460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe994005f50 tx=0x7fe99401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:54:32.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:32.764+0000 7fe98f7fe700 1 -- 172.21.15.2:0/1781930530 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe98805a940 con 0x7fe998106a70 2024-03-16T10:54:33.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.073+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe9981021b0 con 0x7fe998106a70 2024-03-16T10:54:33.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.074+0000 7fe98f7fe700 1 -- 172.21.15.2:0/1781930530 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe98805a4d0 con 0x7fe998106a70 2024-03-16T10:54:33.074 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:54:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:54:33.075 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:54:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.075+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe98406dfa0 msgr2=0x7fe984070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.075+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe98406dfa0 0x7fe984070460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe994005f50 tx=0x7fe99401a040 comp rx=0 tx=0).stop 2024-03-16T10:54:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 msgr2=0x7fe99819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:54:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe99819ccf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b4b0 tx=0x7fe98800b870 comp rx=0 tx=0).stop 2024-03-16T10:54:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 shutdown_connections 2024-03-16T10:54:33.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe98406dfa0 0x7fe984070460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:33.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe998100a70 0x7fe99819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:33.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 --2- 172.21.15.2:0/1781930530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe998106a70 0x7fe99819ccf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:54:33.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 >> 172.21.15.2:0/1781930530 conn(0x7fe9980fc5a0 msgr2=0x7fe9980fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:54:33.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 shutdown_connections 2024-03-16T10:54:33.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:54:33.076+0000 7fe9a07b2700 1 -- 172.21.15.2:0/1781930530 wait complete. 2024-03-16T10:54:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:32 smithi026 ceph-mon[82600]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:32 smithi002 ceph-mon[94239]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:33.724 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-16T10:54:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:54:34.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:33 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1781930530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:34.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:33 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1781930530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:54:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:34 smithi026 ceph-mon[82600]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:35.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:34 smithi002 ceph-mon[94239]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:35.269 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:35.681 INFO:teuthology.orchestra.run.smithi002.stdout:wait for servicemap items w/ changing names to refresh 2024-03-16T10:54:36.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'sleep 60' 2024-03-16T10:54:36.564 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:54:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:36 smithi026 ceph-mon[82600]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:37.289 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:36 smithi002 ceph-mon[94239]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:38 smithi026 ceph-mon[82600]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:38 smithi002 ceph-mon[94239]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:41.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:40 smithi026 ceph-mon[82600]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:40 smithi002 ceph-mon[94239]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:42 smithi026 ceph-mon[82600]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:42 smithi002 ceph-mon[94239]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:44 smithi026 ceph-mon[82600]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:44 smithi002 ceph-mon[94239]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:47.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:46 smithi026 ceph-mon[82600]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:46 smithi002 ceph-mon[94239]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:48 smithi026 ceph-mon[82600]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:48 smithi002 ceph-mon[94239]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:50.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:49 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:49 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:54:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:50 smithi026 ceph-mon[82600]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:50 smithi002 ceph-mon[94239]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:52 smithi026 ceph-mon[82600]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:52 smithi002 ceph-mon[94239]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:55.234 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:54 smithi026 ceph-mon[82600]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:54 smithi002 ceph-mon[94239]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:56 smithi026 ceph-mon[82600]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:56 smithi002 ceph-mon[94239]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:59.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:54:58 smithi026 ceph-mon[82600]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:54:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:54:58 smithi002 ceph-mon[94239]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:01.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:00 smithi026 ceph-mon[82600]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:00 smithi002 ceph-mon[94239]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:03.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:02 smithi026 ceph-mon[82600]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:02 smithi002 ceph-mon[94239]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:04 smithi026 ceph-mon[82600]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:04 smithi002 ceph-mon[94239]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:07.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:06 smithi026 ceph-mon[82600]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:06 smithi002 ceph-mon[94239]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:09.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:08 smithi026 ceph-mon[82600]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:08 smithi002 ceph-mon[94239]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:11.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:10 smithi026 ceph-mon[82600]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:10 smithi002 ceph-mon[94239]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:12 smithi026 ceph-mon[82600]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:55:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:55:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:55:13.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:12 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:55:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:12 smithi002 ceph-mon[94239]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:55:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:55:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:55:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:12 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:55:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:14 smithi026 ceph-mon[82600]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:14 smithi002 ceph-mon[94239]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:17.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:16 smithi026 ceph-mon[82600]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:16 smithi002 ceph-mon[94239]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:18 smithi026 ceph-mon[82600]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:18 smithi002 ceph-mon[94239]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:20.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:21.240 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:20 smithi002 ceph-mon[94239]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:21.448 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:20 smithi026 ceph-mon[82600]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:22 smithi002 ceph-mon[94239]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:23.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:22 smithi026 ceph-mon[82600]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:24 smithi002 ceph-mon[94239]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:25.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:24 smithi026 ceph-mon[82600]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:26 smithi002 ceph-mon[94239]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:27.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:26 smithi026 ceph-mon[82600]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:28 smithi002 ceph-mon[94239]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:29.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:28 smithi026 ceph-mon[82600]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:30 smithi002 ceph-mon[94239]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:31.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:30 smithi026 ceph-mon[82600]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:32 smithi002 ceph-mon[94239]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:33.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:32 smithi026 ceph-mon[82600]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:34 smithi002 ceph-mon[94239]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:35.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:35 smithi026 ceph-mon[82600]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:35.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:37 smithi002 ceph-mon[94239]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:37 smithi026 ceph-mon[82600]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:38.976 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph orch ps' 2024-03-16T10:55:39.195 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:55:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:39 smithi002 ceph-mon[94239]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:39.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:39 smithi026 ceph-mon[82600]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:40.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.985+0000 7f0369e66700 1 -- 172.21.15.2:0/2058278681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 msgr2=0x7f0364101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.985+0000 7f0369e66700 1 --2- 172.21.15.2:0/2058278681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364101ff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0354009910 tx=0x7f0354009c20 comp rx=0 tx=0).stop 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.985+0000 7f0369e66700 1 -- 172.21.15.2:0/2058278681 shutdown_connections 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.985+0000 7f0369e66700 1 --2- 172.21.15.2:0/2058278681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0364102530 0x7f036410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.985+0000 7f0369e66700 1 --2- 172.21.15.2:0/2058278681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364101ff0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.986+0000 7f0369e66700 1 -- 172.21.15.2:0/2058278681 >> 172.21.15.2:0/2058278681 conn(0x7f0364076640 msgr2=0x7f03640fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.986+0000 7f0369e66700 1 -- 172.21.15.2:0/2058278681 shutdown_connections 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.986+0000 7f0369e66700 1 -- 172.21.15.2:0/2058278681 wait complete. 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.986+0000 7f0369e66700 1 Processor -- start 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.986+0000 7f0369e66700 1 -- start start 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f0369e66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f0369e66700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0364102530 0x7f0364196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f0369e66700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0364197580 con 0x7f0364101c10 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f0369e66700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03641976f0 con 0x7f0364102530 2024-03-16T10:55:40.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f03637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f03637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47486/0 (socket says 172.21.15.2:47486) 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f03637fe700 1 -- 172.21.15.2:0/3510997817 learned_addr learned my addr 172.21.15.2:0/3510997817 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f0362ffd700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0364102530 0x7f0364196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f03637fe700 1 -- 172.21.15.2:0/3510997817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0364102530 msgr2=0x7f0364196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f03637fe700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0364102530 0x7f0364196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.987+0000 7f03637fe700 1 -- 172.21.15.2:0/3510997817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03540095f0 con 0x7f0364101c10 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.988+0000 7f03637fe700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364196a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f035400bd70 tx=0x7f035400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.988+0000 7f0360ff9700 1 -- 172.21.15.2:0/3510997817 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f035401d070 con 0x7f0364101c10 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.988+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036419b450 con 0x7f0364101c10 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.988+0000 7f0360ff9700 1 -- 172.21.15.2:0/3510997817 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f035400f460 con 0x7f0364101c10 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.988+0000 7f0360ff9700 1 -- 172.21.15.2:0/3510997817 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0354017630 con 0x7f0364101c10 2024-03-16T10:55:40.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.988+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036419b940 con 0x7f0364101c10 2024-03-16T10:55:40.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.989+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0364068d60 con 0x7f0364101c10 2024-03-16T10:55:40.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.990+0000 7f0360ff9700 1 -- 172.21.15.2:0/3510997817 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f035400f5d0 con 0x7f0364101c10 2024-03-16T10:55:40.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.990+0000 7f0360ff9700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f034c06dfa0 0x7f034c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:40.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.991+0000 7f0360ff9700 1 -- 172.21.15.2:0/3510997817 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f035408f260 con 0x7f0364101c10 2024-03-16T10:55:40.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.991+0000 7f0362ffd700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f034c06dfa0 0x7f034c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:40.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.991+0000 7f0362ffd700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f034c06dfa0 0x7f034c070460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0358005ea0 tx=0x7f0358005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:40.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:40.996+0000 7f0360ff9700 1 -- 172.21.15.2:0/3510997817 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f035405c1e0 con 0x7f0364101c10 2024-03-16T10:55:41.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.209+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 --> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f036419bc20 con 0x7f034c06dfa0 2024-03-16T10:55:41.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.220+0000 7f0360ff9700 1 -- 172.21.15.2:0/3510997817 <== mgr.34102 v2:172.21.15.2:6800/1520670831 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f036419bc20 con 0x7f034c06dfa0 2024-03-16T10:55:41.221 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T10:55:41.221 INFO:teuthology.orchestra.run.smithi002.stdout:alertmanager.smithi002 smithi002 *:9093,9094 running (2m) 95s ago 19m 15.3M - 0.25.0 c8568f914cd2 552a20b3c093 2024-03-16T10:55:41.221 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi002 smithi002 running (3m) 95s ago 19m 8019k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 01d0397a345d 2024-03-16T10:55:41.221 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-exporter.smithi026 smithi026 running (3m) 2m ago 18m 7541k - 19.0.0-2162-g3d78f5ba be86dc8cb22b e515b4badb59 2024-03-16T10:55:41.221 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi002 smithi002 running (8m) 95s ago 19m 6656k - 19.0.0-2162-g3d78f5ba be86dc8cb22b ff0e3d8c5f3b 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:crash.smithi026 smithi026 running (7m) 2m ago 18m 6669k - 19.0.0-2162-g3d78f5ba be86dc8cb22b 69de7b9c4c8f 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:grafana.smithi002 smithi002 *:3000 running (100s) 95s ago 19m 59.5M - 9.4.12 a0ea5e3c46d9 70910746629d 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.ijcmqs smithi002 running (4m) 95s ago 13m 13.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a6b65504c2dc 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi002.qvbseu smithi002 running (4m) 95s ago 13m 90.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b a68e60da4931 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.dzzagf smithi026 running (3m) 2m ago 13m 12.8M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 745c119b7ff5 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mds.cephfs.smithi026.ojzcsw smithi026 running (3m) 2m ago 13m 14.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 1cb95b8df6a0 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi002.zevrtm smithi002 *:8443,9283,8765 running (10m) 95s ago 20m 513M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0a96aa0b2511 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.smithi026.omjkpo smithi026 *:8443,9283,8765 running (9m) 2m ago 18m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 57c3f4913f4e 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi002 smithi002 running (9m) 95s ago 20m 54.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 89f53efb5b44 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:mon.smithi026 smithi026 running (9m) 2m ago 18m 42.6M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c4945cc8635f 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi002 smithi002 *:9100 running (2m) 95s ago 19m 8833k - 1.5.0 0da6a335fe13 80c6dac2c511 2024-03-16T10:55:41.222 INFO:teuthology.orchestra.run.smithi002.stdout:node-exporter.smithi026 smithi026 *:9100 running (2m) 2m ago 18m 3347k - 1.5.0 0da6a335fe13 46339cbf80f5 2024-03-16T10:55:41.223 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (7m) 95s ago 17m 92.4M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 986db8fe457d 2024-03-16T10:55:41.223 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi002 running (7m) 95s ago 16m 99.3M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b dfff225aa9a3 2024-03-16T10:55:41.223 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi002 running (6m) 95s ago 16m 91.7M 1217M 19.0.0-2162-g3d78f5ba be86dc8cb22b 803c7eeb943b 2024-03-16T10:55:41.223 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi026 running (5m) 2m ago 16m 111M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7a6319cdca6d 2024-03-16T10:55:41.223 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi026 running (5m) 2m ago 15m 88.2M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b ae5f63e37d77 2024-03-16T10:55:41.223 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi026 running (4m) 2m ago 15m 92.6M 2241M 19.0.0-2162-g3d78f5ba be86dc8cb22b cf40ac20a977 2024-03-16T10:55:41.223 INFO:teuthology.orchestra.run.smithi002.stdout:prometheus.smithi002 smithi002 *:9095 running (2m) 95s ago 18m 62.0M - 2.43.0 a07b618ecd1d 4e1230eee228 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f034c06dfa0 msgr2=0x7f034c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f034c06dfa0 0x7f034c070460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0358005ea0 tx=0x7f0358005e30 comp rx=0 tx=0).stop 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 msgr2=0x7f0364196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364196a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f035400bd70 tx=0x7f035400be50 comp rx=0 tx=0).stop 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 shutdown_connections 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f034c06dfa0 0x7f034c070460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0364101c10 0x7f0364196a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 --2- 172.21.15.2:0/3510997817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0364102530 0x7f0364196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.222+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 >> 172.21.15.2:0/3510997817 conn(0x7f0364076640 msgr2=0x7f0364102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.223+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 shutdown_connections 2024-03-16T10:55:41.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:41.223+0000 7f0369e66700 1 -- 172.21.15.2:0/3510997817 wait complete. 2024-03-16T10:55:41.251 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:41 smithi002 ceph-mon[94239]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:41.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:41 smithi026 ceph-mon[82600]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:41.892 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions' 2024-03-16T10:55:42.104 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:55:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:43 smithi002 ceph-mon[94239]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:55:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:43 smithi002 ceph-mon[94239]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:43.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:43 smithi026 ceph-mon[82600]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T10:55:43.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:43 smithi026 ceph-mon[82600]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 -- 172.21.15.2:0/4200868630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 msgr2=0x7ff250106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 --2- 172.21.15.2:0/4200868630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff250106e50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff238009910 tx=0x7ff238009c20 comp rx=0 tx=0).stop 2024-03-16T10:55:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 -- 172.21.15.2:0/4200868630 shutdown_connections 2024-03-16T10:55:43.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 --2- 172.21.15.2:0/4200868630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250100a70 0x7ff250100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:43.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 --2- 172.21.15.2:0/4200868630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff250106e50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:43.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 -- 172.21.15.2:0/4200868630 >> 172.21.15.2:0/4200868630 conn(0x7ff2500fc5a0 msgr2=0x7ff2500fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:43.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 -- 172.21.15.2:0/4200868630 shutdown_connections 2024-03-16T10:55:43.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.884+0000 7ff256676700 1 -- 172.21.15.2:0/4200868630 wait complete. 2024-03-16T10:55:43.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff256676700 1 Processor -- start 2024-03-16T10:55:43.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff256676700 1 -- start start 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff256676700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250100a70 0x7ff2500730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff256676700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff2500735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff256676700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff250073b30 con 0x7ff250106a70 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff256676700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff250073c70 con 0x7ff250100a70 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff24f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff2500735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff24f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff2500735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59988/0 (socket says 172.21.15.2:59988) 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff24f7fe700 1 -- 172.21.15.2:0/866170052 learned_addr learned my addr 172.21.15.2:0/866170052 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.885+0000 7ff24ffff700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250100a70 0x7ff2500730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff24f7fe700 1 -- 172.21.15.2:0/866170052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250100a70 msgr2=0x7ff2500730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff24f7fe700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250100a70 0x7ff2500730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff24f7fe700 1 -- 172.21.15.2:0/866170052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2380095f0 con 0x7ff250106a70 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff24f7fe700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff2500735f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d8d0 tx=0x7ff24000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff24d7fa700 1 -- 172.21.15.2:0/866170052 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2400098a0 con 0x7ff250106a70 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2501a2b70 con 0x7ff250106a70 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2501a2ed0 con 0x7ff250106a70 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff24d7fa700 1 -- 172.21.15.2:0/866170052 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff240010460 con 0x7ff250106a70 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.886+0000 7ff24d7fa700 1 -- 172.21.15.2:0/866170052 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff24000f5f0 con 0x7ff250106a70 2024-03-16T10:55:43.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.887+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff250109070 con 0x7ff250106a70 2024-03-16T10:55:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.888+0000 7ff24d7fa700 1 -- 172.21.15.2:0/866170052 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7ff240009a00 con 0x7ff250106a70 2024-03-16T10:55:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.888+0000 7ff24d7fa700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff23c06dfa0 0x7ff23c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.888+0000 7ff24d7fa700 1 -- 172.21.15.2:0/866170052 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7ff24008dc80 con 0x7ff250106a70 2024-03-16T10:55:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.888+0000 7ff24ffff700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff23c06dfa0 0x7ff23c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.889+0000 7ff24ffff700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff23c06dfa0 0x7ff23c070460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff238006010 tx=0x7ff23801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:43.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:43.893+0000 7ff24d7fa700 1 -- 172.21.15.2:0/866170052 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff240057050 con 0x7ff250106a70 2024-03-16T10:55:44.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.200+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff250073e30 con 0x7ff250106a70 2024-03-16T10:55:44.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.201+0000 7ff24d7fa700 1 -- 172.21.15.2:0/866170052 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff24005a710 con 0x7ff250106a70 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": { 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": { 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 2 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": { 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 6 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": { 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 4 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-03-16T10:55:44.202 INFO:teuthology.orchestra.run.smithi002.stdout: "overall": { 2024-03-16T10:55:44.203 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)": 14 2024-03-16T10:55:44.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-03-16T10:55:44.203 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.203+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff23c06dfa0 msgr2=0x7ff23c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.203+0000 7ff256676700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff23c06dfa0 0x7ff23c070460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff238006010 tx=0x7ff23801a040 comp rx=0 tx=0).stop 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.203+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 msgr2=0x7ff2500735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.203+0000 7ff256676700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff2500735f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d8d0 tx=0x7ff24000dbe0 comp rx=0 tx=0).stop 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.204+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 shutdown_connections 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.204+0000 7ff256676700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7ff23c06dfa0 0x7ff23c070460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.204+0000 7ff256676700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff250100a70 0x7ff2500730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:44.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.204+0000 7ff256676700 1 --2- 172.21.15.2:0/866170052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff250106a70 0x7ff2500735f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:44.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.204+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 >> 172.21.15.2:0/866170052 conn(0x7ff2500fc5a0 msgr2=0x7ff2500fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:44.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.204+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 shutdown_connections 2024-03-16T10:55:44.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:44.204+0000 7ff256676700 1 -- 172.21.15.2:0/866170052 wait complete. 2024-03-16T10:55:44.808 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-16T10:55:45.023 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:55:45.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:45 smithi002 ceph-mon[94239]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:45.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:45 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/866170052' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:55:45.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:45 smithi026 ceph-mon[82600]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:45.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:45 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/866170052' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.730+0000 7fd88716d700 1 -- 172.21.15.2:0/3123532971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801009f0 msgr2=0x7fd880100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.730+0000 7fd88716d700 1 --2- 172.21.15.2:0/3123532971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801009f0 0x7fd880100e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd874009910 tx=0x7fd874009c20 comp rx=0 tx=0).stop 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 -- 172.21.15.2:0/3123532971 shutdown_connections 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 --2- 172.21.15.2:0/3123532971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801009f0 0x7fd880100e70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 --2- 172.21.15.2:0/3123532971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8801069f0 0x7fd880106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 -- 172.21.15.2:0/3123532971 >> 172.21.15.2:0/3123532971 conn(0x7fd8800fc500 msgr2=0x7fd8800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 -- 172.21.15.2:0/3123532971 shutdown_connections 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 -- 172.21.15.2:0/3123532971 wait complete. 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 Processor -- start 2024-03-16T10:55:46.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 -- start start 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8801009f0 0x7fd8800730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.731+0000 7fd88716d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801069f0 0x7fd8800735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd88716d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880073b30 con 0x7fd8801069f0 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd88716d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880073c70 con 0x7fd8801009f0 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801069f0 0x7fd8800735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801069f0 0x7fd8800735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60014/0 (socket says 172.21.15.2:60014) 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87ffff700 1 -- 172.21.15.2:0/822939703 learned_addr learned my addr 172.21.15.2:0/822939703 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd884f09700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8801009f0 0x7fd8800730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87ffff700 1 -- 172.21.15.2:0/822939703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8801009f0 msgr2=0x7fd8800730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87ffff700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8801009f0 0x7fd8800730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87ffff700 1 -- 172.21.15.2:0/822939703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740095f0 con 0x7fd8801069f0 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87ffff700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801069f0 0x7fd8800735f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8740092e0 tx=0x7fd87400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.732+0000 7fd87dffb700 1 -- 172.21.15.2:0/822939703 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd87401d070 con 0x7fd8801069f0 2024-03-16T10:55:46.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.733+0000 7fd87dffb700 1 -- 172.21.15.2:0/822939703 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8740056e0 con 0x7fd8801069f0 2024-03-16T10:55:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.733+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880073ef0 con 0x7fd8801069f0 2024-03-16T10:55:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.733+0000 7fd87dffb700 1 -- 172.21.15.2:0/822939703 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd87400fe10 con 0x7fd8801069f0 2024-03-16T10:55:46.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.733+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8801a2d90 con 0x7fd8801069f0 2024-03-16T10:55:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.734+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880109010 con 0x7fd8801069f0 2024-03-16T10:55:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.734+0000 7fd87dffb700 1 -- 172.21.15.2:0/822939703 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fd87400f680 con 0x7fd8801069f0 2024-03-16T10:55:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.735+0000 7fd87dffb700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd86806dfb0 0x7fd868070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.735+0000 7fd87dffb700 1 -- 172.21.15.2:0/822939703 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fd87408f130 con 0x7fd8801069f0 2024-03-16T10:55:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.735+0000 7fd884f09700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd86806dfb0 0x7fd868070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:46.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.735+0000 7fd884f09700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd86806dfb0 0x7fd868070470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd870005950 tx=0x7fd87000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:46.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:46.740+0000 7fd87dffb700 1 -- 172.21.15.2:0/822939703 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd874058580 con 0x7fd8801069f0 2024-03-16T10:55:47.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.034+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd880102130 con 0x7fd8801069f0 2024-03-16T10:55:47.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.035+0000 7fd87dffb700 1 -- 172.21.15.2:0/822939703 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd87405bc40 con 0x7fd8801069f0 2024-03-16T10:55:47.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd86806dfb0 msgr2=0x7fd868070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:47.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd86806dfb0 0x7fd868070470 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd870005950 tx=0x7fd87000b410 comp rx=0 tx=0).stop 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801069f0 msgr2=0x7fd8800735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801069f0 0x7fd8800735f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8740092e0 tx=0x7fd87400bd70 comp rx=0 tx=0).stop 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 shutdown_connections 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fd86806dfb0 0x7fd868070470 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8801009f0 0x7fd8800730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 --2- 172.21.15.2:0/822939703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8801069f0 0x7fd8800735f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 >> 172.21.15.2:0/822939703 conn(0x7fd8800fc500 msgr2=0x7fd8800fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 shutdown_connections 2024-03-16T10:55:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:47.037+0000 7fd88716d700 1 -- 172.21.15.2:0/822939703 wait complete. 2024-03-16T10:55:47.048 INFO:teuthology.orchestra.run.smithi002.stdout:true 2024-03-16T10:55:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:47 smithi002 ceph-mon[94239]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:47 smithi026 ceph-mon[82600]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:47.613 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -e sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-16T10:55:47.834 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:55:48.271 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:48 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/822939703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:55:48.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:48 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/822939703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:55:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:49 smithi002 ceph-mon[94239]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 -- 172.21.15.2:0/4070737578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0124105970 msgr2=0x7f0124105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 --2- 172.21.15.2:0/4070737578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0124105970 0x7f0124105d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f010c009910 tx=0x7f010c009c20 comp rx=0 tx=0).stop 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 -- 172.21.15.2:0/4070737578 shutdown_connections 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 --2- 172.21.15.2:0/4070737578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01240ff970 0x7f01240ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 --2- 172.21.15.2:0/4070737578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0124105970 0x7f0124105d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 -- 172.21.15.2:0/4070737578 >> 172.21.15.2:0/4070737578 conn(0x7f01240fb480 msgr2=0x7f01240fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 -- 172.21.15.2:0/4070737578 shutdown_connections 2024-03-16T10:55:49.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.417+0000 7f012aa7c700 1 -- 172.21.15.2:0/4070737578 wait complete. 2024-03-16T10:55:49.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f012aa7c700 1 Processor -- start 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f012aa7c700 1 -- start start 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f012aa7c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01240ff970 0x7f01240fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f012aa7c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0124105970 0x7f01240fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f012aa7c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01240fc710 con 0x7f01240ff970 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f012aa7c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01240fc850 con 0x7f0124105970 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f0128818700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01240ff970 0x7f01240fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f0128818700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01240ff970 0x7f01240fbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60020/0 (socket says 172.21.15.2:60020) 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f0128818700 1 -- 172.21.15.2:0/3024241425 learned_addr learned my addr 172.21.15.2:0/3024241425 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f0123fff700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0124105970 0x7f01240fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f0128818700 1 -- 172.21.15.2:0/3024241425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0124105970 msgr2=0x7f01240fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f0128818700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0124105970 0x7f01240fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.418+0000 7f0128818700 1 -- 172.21.15.2:0/3024241425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c0095f0 con 0x7f01240ff970 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.419+0000 7f0128818700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01240ff970 0x7f01240fbc90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f010c006010 tx=0x7f010c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.419+0000 7f0121ffb700 1 -- 172.21.15.2:0/3024241425 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f010c01d070 con 0x7f01240ff970 2024-03-16T10:55:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.419+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01240fcad0 con 0x7f01240ff970 2024-03-16T10:55:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.419+0000 7f0121ffb700 1 -- 172.21.15.2:0/3024241425 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f010c0056e0 con 0x7f01240ff970 2024-03-16T10:55:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.419+0000 7f0121ffb700 1 -- 172.21.15.2:0/3024241425 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f010c017400 con 0x7f01240ff970 2024-03-16T10:55:49.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.419+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241a2eb0 con 0x7f01240ff970 2024-03-16T10:55:49.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.420+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0124107f70 con 0x7f01240ff970 2024-03-16T10:55:49.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.421+0000 7f0121ffb700 1 -- 172.21.15.2:0/3024241425 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f010c00f860 con 0x7f01240ff970 2024-03-16T10:55:49.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.421+0000 7f0121ffb700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f011006dfa0 0x7f0110070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:49.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.421+0000 7f0121ffb700 1 -- 172.21.15.2:0/3024241425 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f010c08f170 con 0x7f01240ff970 2024-03-16T10:55:49.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.421+0000 7f0123fff700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f011006dfa0 0x7f0110070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:49.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.421+0000 7f0123fff700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f011006dfa0 0x7f0110070460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0114005950 tx=0x7f01140058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:49.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.426+0000 7f0121ffb700 1 -- 172.21.15.2:0/3024241425 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f010c058600 con 0x7f01240ff970 2024-03-16T10:55:49.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:49 smithi026 ceph-mon[82600]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:49.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.719+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01240fd170 con 0x7f01240ff970 2024-03-16T10:55:49.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.720+0000 7f0121ffb700 1 -- 172.21.15.2:0/3024241425 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f010c05bcc0 con 0x7f01240ff970 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f011006dfa0 msgr2=0x7f0110070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f011006dfa0 0x7f0110070460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0114005950 tx=0x7f01140058e0 comp rx=0 tx=0).stop 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01240ff970 msgr2=0x7f01240fbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01240ff970 0x7f01240fbc90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f010c006010 tx=0x7f010c00be20 comp rx=0 tx=0).stop 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 shutdown_connections 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f011006dfa0 0x7f0110070460 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01240ff970 0x7f01240fbc90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 --2- 172.21.15.2:0/3024241425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0124105970 0x7f01240fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 >> 172.21.15.2:0/3024241425 conn(0x7f01240fb480 msgr2=0x7f0124109d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 shutdown_connections 2024-03-16T10:55:49.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:49.722+0000 7f012aa7c700 1 -- 172.21.15.2:0/3024241425 wait complete. 2024-03-16T10:55:49.733 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)" 2024-03-16T10:55:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:50 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3024241425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:55:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:50.411 DEBUG:teuthology.parallel:result is None 2024-03-16T10:55:50.411 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T10:55:50.422 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T10:55:50.422 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T10:55:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:50 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3024241425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-16T10:55:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:55:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/mon.smithi002/config 2024-03-16T10:55:51.282 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:51 smithi002 ceph-mon[94239]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:51.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:51 smithi026 ceph-mon[82600]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:52.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 -- 172.21.15.2:0/4046030939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 msgr2=0x7f3ce010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 --2- 172.21.15.2:0/4046030939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce010aa60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009910 tx=0x7f3cc8009c20 comp rx=0 tx=0).stop 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 -- 172.21.15.2:0/4046030939 shutdown_connections 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 --2- 172.21.15.2:0/4046030939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce010aa60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 --2- 172.21.15.2:0/4046030939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce0101c50 0x7f3ce0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 -- 172.21.15.2:0/4046030939 >> 172.21.15.2:0/4046030939 conn(0x7f3ce0076640 msgr2=0x7f3ce00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 -- 172.21.15.2:0/4046030939 shutdown_connections 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.221+0000 7f3ce632e700 1 -- 172.21.15.2:0/4046030939 wait complete. 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.222+0000 7f3ce632e700 1 Processor -- start 2024-03-16T10:55:52.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.222+0000 7f3ce632e700 1 -- start start 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.222+0000 7f3ce632e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce0101c50 0x7f3ce019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.222+0000 7f3ce632e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.222+0000 7f3ce632e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce019d290 con 0x7f3ce0102570 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.222+0000 7f3ce632e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0196800 con 0x7f3ce0101c50 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60040/0 (socket says 172.21.15.2:60040) 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdf7fe700 1 -- 172.21.15.2:0/3283531409 learned_addr learned my addr 172.21.15.2:0/3283531409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdffff700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce0101c50 0x7f3ce019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdf7fe700 1 -- 172.21.15.2:0/3283531409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce0101c50 msgr2=0x7f3ce019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:52.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdf7fe700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce0101c50 0x7f3ce019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdf7fe700 1 -- 172.21.15.2:0/3283531409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80095f0 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdf7fe700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce019cc00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800bd70 tx=0x7f3cc800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdd7fa700 1 -- 172.21.15.2:0/3283531409 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cc801d070 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdd7fa700 1 -- 172.21.15.2:0/3283531409 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cc800f460 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce0196a80 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce0196f70 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.223+0000 7f3cdd7fa700 1 -- 172.21.15.2:0/3283531409 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cc8017630 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.224+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce0105370 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.225+0000 7f3cdd7fa700 1 -- 172.21.15.2:0/3283531409 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3cc800f5d0 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.225+0000 7f3cdd7fa700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ccc06dfa0 0x7f3ccc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.225+0000 7f3cdd7fa700 1 -- 172.21.15.2:0/3283531409 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3cc808f250 con 0x7f3ce0102570 2024-03-16T10:55:52.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.225+0000 7f3cdffff700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ccc06dfa0 0x7f3ccc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:52.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.226+0000 7f3cdffff700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ccc06dfa0 0x7f3ccc070460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0005fd0 tx=0x7f3cd000a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:52.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.230+0000 7f3cdd7fa700 1 -- 172.21.15.2:0/3283531409 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3cc805c150 con 0x7f3ce0102570 2024-03-16T10:55:52.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.499+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3ce0104d50 con 0x7f3ce0102570 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.499+0000 7f3cdd7fa700 1 -- 172.21.15.2:0/3283531409 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1929 (secure 0 0 0) 0x7f3cc8005740 con 0x7f3ce0102570 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:e31 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 31 2024-03-16T10:55:52.500 INFO:teuthology.orchestra.run.smithi002.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-03-16T10:41:50.343374+0000 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-03-16T10:52:21.877610+0000 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 60 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 86 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 1 2024-03-16T10:55:52.501 INFO:teuthology.orchestra.run.smithi002.stdout:in 0 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=34408} 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data enabled 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 1 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 34408 members: 34408 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.qvbseu{0:34408} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.2:6826/918911081,v1:172.21.15.2:6827/918911081] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:55:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:52.503 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-03-16T10:55:52.503 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:52.503 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi002.ijcmqs{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.2:6828/1125443118,v1:172.21.15.2:6829/1125443118] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:55:52.503 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.dzzagf{-1:44337} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6824/2953126152,v1:172.21.15.26:6825/2953126152] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:55:52.503 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.cephfs.smithi026.ojzcsw{-1:44367} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.26:6826/1298103624,v1:172.21.15.26:6827/1298103624] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ccc06dfa0 msgr2=0x7f3ccc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ccc06dfa0 0x7f3ccc070460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0005fd0 tx=0x7f3cd000a560 comp rx=0 tx=0).stop 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 msgr2=0x7f3ce019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce019cc00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800bd70 tx=0x7f3cc800be50 comp rx=0 tx=0).stop 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 shutdown_connections 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ccc06dfa0 0x7f3ccc070460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ce0101c50 0x7f3ce019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 --2- 172.21.15.2:0/3283531409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ce0102570 0x7f3ce019cc00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.501+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 >> 172.21.15.2:0/3283531409 conn(0x7f3ce0076640 msgr2=0x7f3ce0102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.502+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 shutdown_connections 2024-03-16T10:55:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:52.502+0000 7f3ce632e700 1 -- 172.21.15.2:0/3283531409 wait complete. 2024-03-16T10:55:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 31 2024-03-16T10:55:53.147 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:53 smithi002 ceph-mon[94239]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:53.147 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:53 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3283531409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:55:53.168 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-16T10:55:53.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:55:53.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3705092522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 msgr2=0x7f6cf00fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3705092522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf00fed20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009880 tx=0x7f6ce002bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3705092522 shutdown_connections 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3705092522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf00ff2f0 0x7f6cf0104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3705092522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf00fed20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3705092522 >> 172.21.15.2:0/3705092522 conn(0x7f6cf0074050 msgr2=0x7f6cf0074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3705092522 shutdown_connections 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.283+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3705092522 wait complete. 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cf53d2640 1 Processor -- start 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cf53d2640 1 -- start start 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cf53d2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf0196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cf53d2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf0196570 0x7f6cf019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cf53d2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0196b70 con 0x7f6cf00fe940 2024-03-16T10:55:53.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cf53d2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0196ce0 con 0x7f6cf0196570 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6ceeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf0196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6ceeffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf0196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51630/0 (socket says 172.21.15.2:51630) 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6ceeffd640 1 -- 172.21.15.2:0/3396035524 learned_addr learned my addr 172.21.15.2:0/3396035524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cee7fc640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf0196570 0x7f6cf019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6ceeffd640 1 -- 172.21.15.2:0/3396035524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf0196570 msgr2=0x7f6cf019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6ceeffd640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf0196570 0x7f6cf019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6ceeffd640 1 -- 172.21.15.2:0/3396035524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0009560 con 0x7f6cf00fe940 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.284+0000 7f6cee7fc640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf0196570 0x7f6cf019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.285+0000 7f6ceeffd640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf0196030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006010 tx=0x7f6ce0004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.285+0000 7f6cd7fff640 1 -- 172.21.15.2:0/3396035524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce0004420 con 0x7f6cf00fe940 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.285+0000 7f6cd7fff640 1 -- 172.21.15.2:0/3396035524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ce0003ab0 con 0x7f6cf00fe940 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.285+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf019aeb0 con 0x7f6cf00fe940 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.285+0000 7f6cd7fff640 1 -- 172.21.15.2:0/3396035524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ce0005520 con 0x7f6cf00fe940 2024-03-16T10:55:53.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.285+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf019b410 con 0x7f6cf00fe940 2024-03-16T10:55:53.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.286+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf00680c0 con 0x7f6cf00fe940 2024-03-16T10:55:53.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.286+0000 7f6cd7fff640 1 -- 172.21.15.2:0/3396035524 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f6ce0003c20 con 0x7f6cf00fe940 2024-03-16T10:55:53.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.287+0000 7f6cd7fff640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6cd806df10 0x7f6cd80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.287+0000 7f6cd7fff640 1 -- 172.21.15.2:0/3396035524 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6ce00b0090 con 0x7f6cf00fe940 2024-03-16T10:55:53.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.287+0000 7f6cee7fc640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6cd806df10 0x7f6cd80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.287+0000 7f6cee7fc640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6cd806df10 0x7f6cd80703d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0069c40 tx=0x7f6ce4005f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:53.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.292+0000 7f6cd7fff640 1 -- 172.21.15.2:0/3396035524 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ce0079460 con 0x7f6cf00fe940 2024-03-16T10:55:53.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.479+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cf004e030 con 0x7f6cf00fe940 2024-03-16T10:55:53.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:53 smithi026 ceph-mon[82600]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:53.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:53 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3283531409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T10:55:53.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.481+0000 7f6cd7fff640 1 -- 172.21.15.2:0/3396035524 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16795 (secure 0 0 0) 0x7f6ce007cb20 con 0x7f6cf00fe940 2024-03-16T10:55:53.482 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:53.482 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":86,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","created":"2024-03-16T10:34:41.302807+0000","modified":"2024-03-16T10:52:09.467585+0000","last_up_change":"2024-03-16T10:51:06.706637+0000","last_in_change":"2024-03-16T10:40:14.935894+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:39:18.453991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T10:41:48.797183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T10:41:49.583833+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4befd156-b58c-44e4-bc1e-f16112b3df27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":80,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6803","nonce":1147005518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6805","nonce":1147005518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6809","nonce":1147005518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6807","nonce":1147005518}]},"public_addr":"172.21.15.2:6803/1147005518","cluster_addr":"172.21.15.2:6805/1147005518","heartbeat_back_addr":"172.21.15.2:6809/1147005518","heartbeat_front_addr":"172.21.15.2:6807/1147005518","state":["exists","up"]},{"osd":1,"uuid":"95448bdc-721b-45c0-8c58-0b416e188aa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":60,"up_thru":80,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6811","nonce":648077700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6813","nonce":648077700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6817","nonce":648077700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6815","nonce":648077700}]},"public_addr":"172.21.15.2:6811/648077700","cluster_addr":"172.21.15.2:6813/648077700","heartbeat_back_addr":"172.21.15.2:6817/648077700","heartbeat_front_addr":"172.21.15.2:6815/648077700","state":["exists","up"]},{"osd":2,"uuid":"ce6a7a3f-db89-426c-bbb4-1339c5adac81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":61,"up_from":65,"up_thru":80,"down_at":62,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6819","nonce":1082105549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6821","nonce":1082105549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6825","nonce":1082105549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6823","nonce":1082105549}]},"public_addr":"172.21.15.2:6819/1082105549","cluster_addr":"172.21.15.2:6821/1082105549","heartbeat_back_addr":"172.21.15.2:6825/1082105549","heartbeat_front_addr":"172.21.15.2:6823/1082105549","state":["exists","up"]},{"osd":3,"uuid":"8d760311-5fda-425b-ac3c-5045ecb1cbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":66,"up_from":70,"up_thru":80,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6801","nonce":3946506757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6803","nonce":3946506757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6807","nonce":3946506757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6805","nonce":3946506757}]},"public_addr":"172.21.15.26:6801/3946506757","cluster_addr":"172.21.15.26:6803/3946506757","heartbeat_back_addr":"172.21.15.26:6807/3946506757","heartbeat_front_addr":"172.21.15.26:6805/3946506757","state":["exists","up"]},{"osd":4,"uuid":"250a3093-6818-4f4f-85d5-d1f2e4381afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":71,"up_from":75,"up_thru":80,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6809","nonce":2367920652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6811","nonce":2367920652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6815","nonce":2367920652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6813","nonce":2367920652}]},"public_addr":"172.21.15.26:6809/2367920652","cluster_addr":"172.21.15.26:6811/2367920652","heartbeat_back_addr":"172.21.15.26:6815/2367920652","heartbeat_front_addr":"172.21.15.26:6813/2367920652","state":["exists","up"]},{"osd":5,"uuid":"b16f392a-1d2f-47b5-b356-cab94bc834cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":76,"up_from":80,"up_thru":80,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6817","nonce":2348241985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6819","nonce":2348241985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6823","nonce":2348241985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6821","nonce":2348241985}]},"public_addr":"172.21.15.26:6817/2348241985","cluster_addr":"172.21.15.26:6819/2348241985","heartbeat_back_addr":"172.21.15.26:6823/2348241985","heartbeat_front_addr":"172.21.15.26:6821/2348241985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:18.975873+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:47.618973+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:15.584615+0000","dead_epoch":61},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:39.921213+0000","dead_epoch":66},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:06.286712+0000","dead_epoch":71},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:32.567112+0000","dead_epoch":76}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6826/2911421579":"2024-03-17T10:52:09.467533+0000","172.21.15.2:0/3116141534":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/899962313":"2024-03-17T10:46:19.722815+0000","172.21.15.26:0/2161833313":"2024-03-17T10:45:15.531471+0000","172.21.15.26:0/2320376866":"2024-03-17T10:45:15.531471+0000","172.21.15.2:6801/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1498322259":"2024-03-17T10:37:00.118083+0000","172.21.15.26:0/1307372554":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/4132065053":"2024-03-17T10:46:19.722815+0000","172.21.15.26:6825/2074466615":"2024-03-17T10:51:46.663682+0000","172.21.15.2:0/2565089320":"2024-03-17T10:35:17.551709+0000","172.21.15.26:6824/2074466615":"2024-03-17T10:51:46.663682+0000","172.21.15.2:0/567924185":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/100368483":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/3514263595":"2024-03-17T10:43:58.039282+0000","172.21.15.26:6827/2911421579":"2024-03-17T10:52:09.467533+0000","172.21.15.2:6801/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3421178356":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/2322904669":"2024-03-17T10:35:17.551709+0000","172.21.15.26:0/2024487458":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/401513378":"2024-03-17T10:43:58.039282+0000","172.21.15.26:0/2419254018":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/1971862470":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/730693462":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/2563315045":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/1789736475":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/730693462":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/4201560868":"2024-03-17T10:43:58.039282+0000","172.21.15.2:0/1144469616":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6826/2835000958":"2024-03-17T10:42:06.208262+0000","172.21.15.2:0/3462160730":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6827/2835000958":"2024-03-17T10:42:06.208262+0000","172.21.15.2:6800/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.26:6829/274393035":"2024-03-17T10:45:15.531471+0000","172.21.15.2:6800/3195213211":"2024-03-17T10:43:58.039282+0000","172.21.15.2:6801/3195213211":"2024-03-17T10:43:58.039282+0000","172.21.15.2:0/3582566838":"2024-03-17T10:43:58.039282+0000","172.21.15.26:6828/274393035":"2024-03-17T10:45:15.531471+0000","172.21.15.26:0/1025795450":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/2348229359":"2024-03-17T10:35:51.598930+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:55:53.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6cd806df10 msgr2=0x7f6cd80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6cd806df10 0x7f6cd80703d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0069c40 tx=0x7f6ce4005f90 comp rx=0 tx=0).stop 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 msgr2=0x7f6cf0196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf0196030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006010 tx=0x7f6ce0004a10 comp rx=0 tx=0).stop 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 shutdown_connections 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6cd806df10 0x7f6cd80703d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cf0196570 0x7f6cf019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 --2- 172.21.15.2:0/3396035524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cf00fe940 0x7f6cf0196030 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 >> 172.21.15.2:0/3396035524 conn(0x7f6cf0074050 msgr2=0x7f6cf00fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 shutdown_connections 2024-03-16T10:55:53.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.483+0000 7f6cf53d2640 1 -- 172.21.15.2:0/3396035524 wait complete. 2024-03-16T10:55:53.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:39:18.453991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T10:41:48.797183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T10:41:49.583833+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:55:53.495 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 -- 172.21.15.2:0/505732370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1900ffa70 msgr2=0x7fe190103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 --2- 172.21.15.2:0/505732370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1900ffa70 0x7fe190103ab0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009c20 tx=0x7fe18c02c1b0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 -- 172.21.15.2:0/505732370 shutdown_connections 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 --2- 172.21.15.2:0/505732370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1900ffa70 0x7fe190103ab0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 --2- 172.21.15.2:0/505732370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1900ff0c0 0x7fe1900ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 -- 172.21.15.2:0/505732370 >> 172.21.15.2:0/505732370 conn(0x7fe1900fa940 msgr2=0x7fe1900fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 -- 172.21.15.2:0/505732370 shutdown_connections 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.602+0000 7fe198342640 1 -- 172.21.15.2:0/505732370 wait complete. 2024-03-16T10:55:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe198342640 1 Processor -- start 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe198342640 1 -- start start 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe198342640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1900ff0c0 0x7fe1901960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe198342640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe190196620 0x7fe19019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe198342640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190196c20 con 0x7fe190196620 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe198342640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190196d90 con 0x7fe1900ff0c0 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1958dd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe190196620 0x7fe19019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1958dd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe190196620 0x7fe19019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51658/0 (socket says 172.21.15.2:51658) 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1958dd640 1 -- 172.21.15.2:0/1717628825 learned_addr learned my addr 172.21.15.2:0/1717628825 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1958dd640 1 -- 172.21.15.2:0/1717628825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1900ff0c0 msgr2=0x7fe1901960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1960de640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1900ff0c0 0x7fe1901960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1958dd640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1900ff0c0 0x7fe1901960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1958dd640 1 -- 172.21.15.2:0/1717628825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c009900 con 0x7fe190196620 2024-03-16T10:55:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.603+0000 7fe1958dd640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe190196620 0x7fe19019a9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe18c000f80 tx=0x7fe18c032570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.604+0000 7fe1877fe640 1 -- 172.21.15.2:0/1717628825 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe18c034820 con 0x7fe190196620 2024-03-16T10:55:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.604+0000 7fe1877fe640 1 -- 172.21.15.2:0/1717628825 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe18c003e30 con 0x7fe190196620 2024-03-16T10:55:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.604+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19019af60 con 0x7fe190196620 2024-03-16T10:55:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.604+0000 7fe1877fe640 1 -- 172.21.15.2:0/1717628825 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe18c03f440 con 0x7fe190196620 2024-03-16T10:55:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.604+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19019b4c0 con 0x7fe190196620 2024-03-16T10:55:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.605+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe190100590 con 0x7fe190196620 2024-03-16T10:55:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.605+0000 7fe1877fe640 1 -- 172.21.15.2:0/1717628825 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fe18c03b040 con 0x7fe190196620 2024-03-16T10:55:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.606+0000 7fe1877fe640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe17c06df20 0x7fe17c0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.606+0000 7fe1877fe640 1 -- 172.21.15.2:0/1717628825 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe18c0b0bc0 con 0x7fe190196620 2024-03-16T10:55:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.606+0000 7fe1960de640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe17c06df20 0x7fe17c0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.606+0000 7fe1960de640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe17c06df20 0x7fe17c0703e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe18000a810 tx=0x7fe180008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.615+0000 7fe1877fe640 1 -- 172.21.15.2:0/1717628825 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe18c079f90 con 0x7fe190196620 2024-03-16T10:55:53.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.796+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1901973e0 con 0x7fe190196620 2024-03-16T10:55:53.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.796+0000 7fe1877fe640 1 -- 172.21.15.2:0/1717628825 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe18c07d650 con 0x7fe190196620 2024-03-16T10:55:53.797 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-16T10:55:53.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe17c06df20 msgr2=0x7fe17c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe17c06df20 0x7fe17c0703e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe18000a810 tx=0x7fe180008040 comp rx=0 tx=0).stop 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe190196620 msgr2=0x7fe19019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe190196620 0x7fe19019a9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe18c000f80 tx=0x7fe18c032570 comp rx=0 tx=0).stop 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 shutdown_connections 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe17c06df20 0x7fe17c0703e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe190196620 0x7fe19019a9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 --2- 172.21.15.2:0/1717628825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1900ff0c0 0x7fe1901960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 >> 172.21.15.2:0/1717628825 conn(0x7fe1900fa940 msgr2=0x7fe1900681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 shutdown_connections 2024-03-16T10:55:53.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.798+0000 7fe198342640 1 -- 172.21.15.2:0/1717628825 wait complete. 2024-03-16T10:55:53.811 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T10:55:53.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 -- 172.21.15.2:0/343059307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 msgr2=0x7f7c100ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 --2- 172.21.15.2:0/343059307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c100ff490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009880 tx=0x7f7c0402bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 -- 172.21.15.2:0/343059307 shutdown_connections 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 --2- 172.21.15.2:0/343059307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100ffa60 0x7f7c10103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 --2- 172.21.15.2:0/343059307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c100ff490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 -- 172.21.15.2:0/343059307 >> 172.21.15.2:0/343059307 conn(0x7f7c100fa930 msgr2=0x7f7c100fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 -- 172.21.15.2:0/343059307 shutdown_connections 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 -- 172.21.15.2:0/343059307 wait complete. 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 Processor -- start 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.917+0000 7f7c183b1640 1 -- start start 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c183b1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c10196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c183b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c101965a0 0x7f7c1019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c183b1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c10196ba0 con 0x7f7c100ff0b0 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c183b1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c10196d10 con 0x7f7c101965a0 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1614d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c10196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1614d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c10196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51684/0 (socket says 172.21.15.2:51684) 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1614d640 1 -- 172.21.15.2:0/805028560 learned_addr learned my addr 172.21.15.2:0/805028560 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1594c640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c101965a0 0x7f7c1019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1614d640 1 -- 172.21.15.2:0/805028560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c101965a0 msgr2=0x7f7c1019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1614d640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c101965a0 0x7f7c1019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1614d640 1 -- 172.21.15.2:0/805028560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c04009560 con 0x7f7c100ff0b0 2024-03-16T10:55:53.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c1614d640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c10196060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c04036040 tx=0x7f7c04004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:53.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c037fe640 1 -- 172.21.15.2:0/805028560 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c040035f0 con 0x7f7c100ff0b0 2024-03-16T10:55:53.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1019aee0 con 0x7f7c100ff0b0 2024-03-16T10:55:53.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c037fe640 1 -- 172.21.15.2:0/805028560 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c04003c10 con 0x7f7c100ff0b0 2024-03-16T10:55:53.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c037fe640 1 -- 172.21.15.2:0/805028560 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c04034440 con 0x7f7c100ff0b0 2024-03-16T10:55:53.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.918+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1019b440 con 0x7f7c100ff0b0 2024-03-16T10:55:53.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.919+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c10100580 con 0x7f7c100ff0b0 2024-03-16T10:55:53.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.920+0000 7f7c037fe640 1 -- 172.21.15.2:0/805028560 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f7c04004f60 con 0x7f7c100ff0b0 2024-03-16T10:55:53.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.920+0000 7f7c037fe640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7bfc06df10 0x7f7bfc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:53.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.920+0000 7f7c037fe640 1 -- 172.21.15.2:0/805028560 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f7c040b0240 con 0x7f7c100ff0b0 2024-03-16T10:55:53.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.920+0000 7f7c1594c640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7bfc06df10 0x7f7bfc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:53.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.921+0000 7f7c1594c640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7bfc06df10 0x7f7bfc0703d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7c10069620 tx=0x7f7c0c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:53.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:53.925+0000 7f7c037fe640 1 -- 172.21.15.2:0/805028560 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c04079610 con 0x7f7c100ff0b0 2024-03-16T10:55:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.109+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f7c10197360 con 0x7f7c100ff0b0 2024-03-16T10:55:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.110+0000 7f7c037fe640 1 -- 172.21.15.2:0/805028560 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7c0407ccd0 con 0x7f7c100ff0b0 2024-03-16T10:55:54.110 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:55:54.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.111+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7bfc06df10 msgr2=0x7f7bfc0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.111+0000 7f7c183b1640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7bfc06df10 0x7f7bfc0703d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7c10069620 tx=0x7f7c0c008040 comp rx=0 tx=0).stop 2024-03-16T10:55:54.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.111+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 msgr2=0x7f7c10196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.111+0000 7f7c183b1640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c10196060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c04036040 tx=0x7f7c04004040 comp rx=0 tx=0).stop 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.111+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 shutdown_connections 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.111+0000 7f7c183b1640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f7bfc06df10 0x7f7bfc0703d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.112+0000 7f7c183b1640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c101965a0 0x7f7c1019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.112+0000 7f7c183b1640 1 --2- 172.21.15.2:0/805028560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c100ff0b0 0x7f7c10196060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.112+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 >> 172.21.15.2:0/805028560 conn(0x7f7c100fa930 msgr2=0x7f7c100680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.112+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 shutdown_connections 2024-03-16T10:55:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.112+0000 7f7c183b1640 1 -- 172.21.15.2:0/805028560 wait complete. 2024-03-16T10:55:54.125 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:54 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3396035524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:55:54.125 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:54 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1717628825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:55:54.127 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T10:55:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 -- 172.21.15.2:0/4171001665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 msgr2=0x7f4cfc109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 --2- 172.21.15.2:0/4171001665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc109f40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00098b0 tx=0x7f4cf002bd60 comp rx=0 tx=0).stop 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 -- 172.21.15.2:0/4171001665 shutdown_connections 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 --2- 172.21.15.2:0/4171001665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc109f40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 --2- 172.21.15.2:0/4171001665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc101140 0x7f4cfc101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 -- 172.21.15.2:0/4171001665 >> 172.21.15.2:0/4171001665 conn(0x7f4cfc0fa8e0 msgr2=0x7f4cfc0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 -- 172.21.15.2:0/4171001665 shutdown_connections 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.232+0000 7f4d03df0640 1 -- 172.21.15.2:0/4171001665 wait complete. 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d03df0640 1 Processor -- start 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d03df0640 1 -- start start 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d03df0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc101140 0x7f4cfc0fed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d03df0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d03df0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0ff810 con 0x7f4cfc101a60 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d03df0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0ff980 con 0x7f4cfc101140 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d0138b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d0138b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51710/0 (socket says 172.21.15.2:51710) 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d0138b640 1 -- 172.21.15.2:0/2236540249 learned_addr learned my addr 172.21.15.2:0/2236540249 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d01b8c640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc101140 0x7f4cfc0fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d0138b640 1 -- 172.21.15.2:0/2236540249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc101140 msgr2=0x7f4cfc0fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d0138b640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc101140 0x7f4cfc0fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d0138b640 1 -- 172.21.15.2:0/2236540249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf0009560 con 0x7f4cfc101a60 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4d0138b640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc0ff2d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009430 tx=0x7f4cf002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.233+0000 7f4cfaffd640 1 -- 172.21.15.2:0/2236540249 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf00035f0 con 0x7f4cfc101a60 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.234+0000 7f4cfaffd640 1 -- 172.21.15.2:0/2236540249 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cf0003c10 con 0x7f4cfc101a60 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.234+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc0ffc00 con 0x7f4cfc101a60 2024-03-16T10:55:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.234+0000 7f4cfaffd640 1 -- 172.21.15.2:0/2236540249 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf0005690 con 0x7f4cfc101a60 2024-03-16T10:55:54.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.234+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc1a2450 con 0x7f4cfc101a60 2024-03-16T10:55:54.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.235+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc068ca0 con 0x7f4cfc101a60 2024-03-16T10:55:54.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.235+0000 7f4cfaffd640 1 -- 172.21.15.2:0/2236540249 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4cf0004ed0 con 0x7f4cfc101a60 2024-03-16T10:55:54.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.236+0000 7f4cfaffd640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ce406dec0 0x7f4ce4070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.236+0000 7f4cfaffd640 1 -- 172.21.15.2:0/2236540249 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4cf00b0890 con 0x7f4cfc101a60 2024-03-16T10:55:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.236+0000 7f4d01b8c640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ce406dec0 0x7f4ce4070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.236+0000 7f4d01b8c640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ce406dec0 0x7f4ce4070380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4cec005c40 tx=0x7f4cec00a780 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.240+0000 7f4cfaffd640 1 -- 172.21.15.2:0/2236540249 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4cf0079c60 con 0x7f4cfc101a60 2024-03-16T10:55:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.453+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f4cfc0ffbe0 con 0x7f4cfc101a60 2024-03-16T10:55:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.454+0000 7f4cfaffd640 1 -- 172.21.15.2:0/2236540249 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4cf007d320 con 0x7f4cfc101a60 2024-03-16T10:55:54.455 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ce406dec0 msgr2=0x7f4ce4070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ce406dec0 0x7f4ce4070380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4cec005c40 tx=0x7f4cec00a780 comp rx=0 tx=0).stop 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 msgr2=0x7f4cfc0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc0ff2d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009430 tx=0x7f4cf002c8d0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 shutdown_connections 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ce406dec0 0x7f4ce4070380 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4cfc101a60 0x7f4cfc0ff2d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 --2- 172.21.15.2:0/2236540249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc101140 0x7f4cfc0fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 >> 172.21.15.2:0/2236540249 conn(0x7f4cfc0fa8e0 msgr2=0x7f4cfc102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 shutdown_connections 2024-03-16T10:55:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.456+0000 7f4d03df0640 1 -- 172.21.15.2:0/2236540249 wait complete. 2024-03-16T10:55:54.468 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:55:54.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:54 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3396035524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:55:54.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:54 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1717628825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 -- 172.21.15.2:0/2923973048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f200ffa70 msgr2=0x7f1f20103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 --2- 172.21.15.2:0/2923973048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f200ffa70 0x7f1f20103ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009880 tx=0x7f1f1402bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 -- 172.21.15.2:0/2923973048 shutdown_connections 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 --2- 172.21.15.2:0/2923973048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f200ffa70 0x7f1f20103ab0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 --2- 172.21.15.2:0/2923973048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200ff0c0 0x7f1f200ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 -- 172.21.15.2:0/2923973048 >> 172.21.15.2:0/2923973048 conn(0x7f1f200fa940 msgr2=0x7f1f200fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 -- 172.21.15.2:0/2923973048 shutdown_connections 2024-03-16T10:55:54.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.569+0000 7f1f25215640 1 -- 172.21.15.2:0/2923973048 wait complete. 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f25215640 1 Processor -- start 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f25215640 1 -- start start 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f25215640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200ff0c0 0x7f1f20195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f25215640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f20196500 0x7f1f2019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f25215640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20196a70 con 0x7f1f20196500 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f25215640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20196be0 con 0x7f1f200ff0c0 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f1e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f20196500 0x7f1f2019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f1e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f20196500 0x7f1f2019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51730/0 (socket says 172.21.15.2:51730) 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f1e59c640 1 -- 172.21.15.2:0/4249420848 learned_addr learned my addr 172.21.15.2:0/4249420848 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.570+0000 7f1f1ed9d640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200ff0c0 0x7f1f20195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f1e59c640 1 -- 172.21.15.2:0/4249420848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200ff0c0 msgr2=0x7f1f20195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f1e59c640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200ff0c0 0x7f1f20195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f1e59c640 1 -- 172.21.15.2:0/4249420848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14009560 con 0x7f1f20196500 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f1ed9d640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200ff0c0 0x7f1f20195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f1e59c640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f20196500 0x7f1f2019a8a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1f1402c220 tx=0x7f1f14004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f07fff640 1 -- 172.21.15.2:0/4249420848 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f140035f0 con 0x7f1f20196500 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2019ae40 con 0x7f1f20196500 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f07fff640 1 -- 172.21.15.2:0/4249420848 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f14003c10 con 0x7f1f20196500 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f07fff640 1 -- 172.21.15.2:0/4249420848 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f14034440 con 0x7f1f20196500 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.571+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2019b350 con 0x7f1f20196500 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.572+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f20100590 con 0x7f1f20196500 2024-03-16T10:55:54.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.572+0000 7f1f07fff640 1 -- 172.21.15.2:0/4249420848 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f1f14004f60 con 0x7f1f20196500 2024-03-16T10:55:54.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.573+0000 7f1f07fff640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1f0806df10 0x7f1f080703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:54.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.573+0000 7f1f1ed9d640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1f0806df10 0x7f1f080703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:54.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.573+0000 7f1f07fff640 1 -- 172.21.15.2:0/4249420848 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1f140b01e0 con 0x7f1f20196500 2024-03-16T10:55:54.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.573+0000 7f1f1ed9d640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1f0806df10 0x7f1f080703d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000baa0 tx=0x7f1f10005d90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:54.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.577+0000 7f1f07fff640 1 -- 172.21.15.2:0/4249420848 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f140795b0 con 0x7f1f20196500 2024-03-16T10:55:54.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.883+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f2004e030 con 0x7f1f20196500 2024-03-16T10:55:54.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.884+0000 7f1f07fff640 1 -- 172.21.15.2:0/4249420848 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5198 (secure 0 0 0) 0x7f1f1407cc70 con 0x7f1f20196500 2024-03-16T10:55:54.884 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:54.884 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44337,"name":"cephfs.smithi026.dzzagf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6825/2953126152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2953126152},{"type":"v1","addr":"172.21.15.26:6825","nonce":2953126152}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44367,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6827/1298103624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1298103624},{"type":"v1","addr":"172.21.15.26:6827","nonce":1298103624}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:52:21.877610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":0,"incarnation":28,"state":"up:active","state_seq":19,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34408,"qdb_cluster":[34408]},"id":1}]} 2024-03-16T10:55:54.885 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 31 2024-03-16T10:55:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1f0806df10 msgr2=0x7f1f080703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1f0806df10 0x7f1f080703d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000baa0 tx=0x7f1f10005d90 comp rx=0 tx=0).stop 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f20196500 msgr2=0x7f1f2019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f20196500 0x7f1f2019a8a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1f1402c220 tx=0x7f1f14004040 comp rx=0 tx=0).stop 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 shutdown_connections 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1f0806df10 0x7f1f080703d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f20196500 0x7f1f2019a8a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 --2- 172.21.15.2:0/4249420848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200ff0c0 0x7f1f20195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 >> 172.21.15.2:0/4249420848 conn(0x7f1f200fa940 msgr2=0x7f1f200681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 shutdown_connections 2024-03-16T10:55:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:54.886+0000 7f1f25215640 1 -- 172.21.15.2:0/4249420848 wait complete. 2024-03-16T10:55:54.897 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 1, 'flags': 18} 2024-03-16T10:55:54.898 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-16T10:55:55.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/3236131419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f040ff0a0 msgr2=0x7f3f040ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/3236131419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f040ff0a0 0x7f3f040ff480 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009880 tx=0x7f3f0002bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/3236131419 shutdown_connections 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/3236131419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040ffa50 0x7f3f04103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/3236131419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f040ff0a0 0x7f3f040ff480 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/3236131419 >> 172.21.15.2:0/3236131419 conn(0x7f3f04075b10 msgr2=0x7f3f040fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/3236131419 shutdown_connections 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.000+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/3236131419 wait complete. 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f0bdfb640 1 Processor -- start 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f0bdfb640 1 -- start start 2024-03-16T10:55:55.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f0bdfb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040ff0a0 0x7f3f04195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f0bdfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f041964f0 0x7f3f0419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f0bdfb640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04196af0 con 0x7f3f041964f0 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f0bdfb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04196c60 con 0x7f3f040ff0a0 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f09396640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f041964f0 0x7f3f0419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f09396640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f041964f0 0x7f3f0419a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51748/0 (socket says 172.21.15.2:51748) 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f09396640 1 -- 172.21.15.2:0/493336736 learned_addr learned my addr 172.21.15.2:0/493336736 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f09b97640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040ff0a0 0x7f3f04195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f09396640 1 -- 172.21.15.2:0/493336736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040ff0a0 msgr2=0x7f3f04195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f09396640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040ff0a0 0x7f3f04195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.001+0000 7f3f09396640 1 -- 172.21.15.2:0/493336736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00009560 con 0x7f3f041964f0 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.002+0000 7f3f09396640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f041964f0 0x7f3f0419a890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e9a0 tx=0x7f3ef400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.002+0000 7f3efaffd640 1 -- 172.21.15.2:0/493336736 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ef400cc90 con 0x7f3f041964f0 2024-03-16T10:55:55.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.002+0000 7f3efaffd640 1 -- 172.21.15.2:0/493336736 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ef400cdf0 con 0x7f3f041964f0 2024-03-16T10:55:55.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.002+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0419ae90 con 0x7f3f041964f0 2024-03-16T10:55:55.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.002+0000 7f3efaffd640 1 -- 172.21.15.2:0/493336736 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ef4010430 con 0x7f3f041964f0 2024-03-16T10:55:55.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.002+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0419b3f0 con 0x7f3f041964f0 2024-03-16T10:55:55.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.003+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f04100570 con 0x7f3f041964f0 2024-03-16T10:55:55.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.003+0000 7f3efaffd640 1 -- 172.21.15.2:0/493336736 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3ef4005020 con 0x7f3f041964f0 2024-03-16T10:55:55.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.004+0000 7f3efaffd640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ef006df10 0x7f3ef00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.004+0000 7f3f09b97640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ef006df10 0x7f3ef00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.004+0000 7f3efaffd640 1 -- 172.21.15.2:0/493336736 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3ef4014070 con 0x7f3f041964f0 2024-03-16T10:55:55.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.004+0000 7f3f09b97640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ef006df10 0x7f3ef00703d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3f00036040 tx=0x7f3f00000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:55.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.009+0000 7f3efaffd640 1 -- 172.21.15.2:0/493336736 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ef4057710 con 0x7f3f041964f0 2024-03-16T10:55:55.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.236+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f3f041972e0 con 0x7f3f041964f0 2024-03-16T10:55:55.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.237+0000 7f3efaffd640 1 -- 172.21.15.2:0/493336736 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v31) v1 ==== 107+0+4851 (secure 0 0 0) 0x7f3ef405add0 con 0x7f3f041964f0 2024-03-16T10:55:55.237 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:55.238 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14492,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.2:6829/2260596709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2260596709},{"type":"v1","addr":"172.21.15.2:6829","nonce":2260596709}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":14504,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/1847480131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1847480131},{"type":"v1","addr":"172.21.15.2:6827","nonce":1847480131}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:42:09.214374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi026.dzzagf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6825/2074466615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2074466615},{"type":"v1","addr":"172.21.15.26:6825","nonce":2074466615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:55.238 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2024-03-16T10:55:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ef006df10 msgr2=0x7f3ef00703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ef006df10 0x7f3ef00703d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3f00036040 tx=0x7f3f00000bc0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f041964f0 msgr2=0x7f3f0419a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f041964f0 0x7f3f0419a890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e9a0 tx=0x7f3ef400ee60 comp rx=0 tx=0).stop 2024-03-16T10:55:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 shutdown_connections 2024-03-16T10:55:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3ef006df10 0x7f3ef00703d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f041964f0 0x7f3f0419a890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 --2- 172.21.15.2:0/493336736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040ff0a0 0x7f3f04195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 >> 172.21.15.2:0/493336736 conn(0x7f3f04075b10 msgr2=0x7f3f04067520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 shutdown_connections 2024-03-16T10:55:55.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.239+0000 7f3f0bdfb640 1 -- 172.21.15.2:0/493336736 wait complete. 2024-03-16T10:55:55.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:55 smithi002 ceph-mon[94239]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:55.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:55 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/805028560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:55:55.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:55 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2236540249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:55:55.250 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:55 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/4249420848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:55:55.251 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-16T10:55:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 -- 172.21.15.2:0/208026883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101100 msgr2=0x7f1a6c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 --2- 172.21.15.2:0/208026883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101100 0x7f1a6c1014e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009880 tx=0x7f1a5c02bd70 comp rx=0 tx=0).stop 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 -- 172.21.15.2:0/208026883 shutdown_connections 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 --2- 172.21.15.2:0/208026883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c101a20 0x7f1a6c109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 --2- 172.21.15.2:0/208026883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101100 0x7f1a6c1014e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 -- 172.21.15.2:0/208026883 >> 172.21.15.2:0/208026883 conn(0x7f1a6c0fa940 msgr2=0x7f1a6c0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 -- 172.21.15.2:0/208026883 shutdown_connections 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 -- 172.21.15.2:0/208026883 wait complete. 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.374+0000 7f1a714d5640 1 Processor -- start 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a714d5640 1 -- start start 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a714d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c101100 0x7f1a6c195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a714d5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101a20 0x7f1a6c196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a714d5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c196bc0 con 0x7f1a6c101a20 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a714d5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c19a880 con 0x7f1a6c101100 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101a20 0x7f1a6c196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101a20 0x7f1a6c196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51772/0 (socket says 172.21.15.2:51772) 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a63fff640 1 -- 172.21.15.2:0/1380370140 learned_addr learned my addr 172.21.15.2:0/1380370140 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a6affd640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c101100 0x7f1a6c195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a63fff640 1 -- 172.21.15.2:0/1380370140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c101100 msgr2=0x7f1a6c195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a63fff640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c101100 0x7f1a6c195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a63fff640 1 -- 172.21.15.2:0/1380370140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c009560 con 0x7f1a6c101a20 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a63fff640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101a20 0x7f1a6c196530 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400d6f0 tx=0x7f1a5400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a68ff9640 1 -- 172.21.15.2:0/1380370140 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a54009a30 con 0x7f1a6c101a20 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a68ff9640 1 -- 172.21.15.2:0/1380370140 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a540054e0 con 0x7f1a6c101a20 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c19ab60 con 0x7f1a6c101a20 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.375+0000 7f1a68ff9640 1 -- 172.21.15.2:0/1380370140 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a54010460 con 0x7f1a6c101a20 2024-03-16T10:55:55.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.376+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c19b0c0 con 0x7f1a6c101a20 2024-03-16T10:55:55.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.377+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c068cc0 con 0x7f1a6c101a20 2024-03-16T10:55:55.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.377+0000 7f1a68ff9640 1 -- 172.21.15.2:0/1380370140 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f1a54009b90 con 0x7f1a6c101a20 2024-03-16T10:55:55.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.377+0000 7f1a68ff9640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a5806df10 0x7f1a580703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.378+0000 7f1a6affd640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a5806df10 0x7f1a580703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.378+0000 7f1a68ff9640 1 -- 172.21.15.2:0/1380370140 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f1a5408d7e0 con 0x7f1a6c101a20 2024-03-16T10:55:55.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.378+0000 7f1a6affd640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a5806df10 0x7f1a580703d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c006010 tx=0x7f1a5c005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.382+0000 7f1a68ff9640 1 -- 172.21.15.2:0/1380370140 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a54056ce0 con 0x7f1a6c101a20 2024-03-16T10:55:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:55 smithi026 ceph-mon[82600]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:55 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/805028560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:55:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:55 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2236540249' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:55:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:55 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/4249420848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T10:55:55.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.669+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f1a6c104770 con 0x7f1a6c101a20 2024-03-16T10:55:55.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.670+0000 7f1a68ff9640 1 -- 172.21.15.2:0/1380370140 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v31) v1 ==== 107+0+4075 (secure 0 0 0) 0x7f1a5405a3a0 con 0x7f1a6c101a20 2024-03-16T10:55:55.671 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:55.671 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14492,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.2:6829/2260596709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2260596709},{"type":"v1","addr":"172.21.15.2:6829","nonce":2260596709}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:42:09.214374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi026.dzzagf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6825/2074466615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2074466615},{"type":"v1","addr":"172.21.15.26:6825","nonce":2074466615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:55.671 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 18 2024-03-16T10:55:55.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a5806df10 msgr2=0x7f1a580703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a5806df10 0x7f1a580703d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c006010 tx=0x7f1a5c005d50 comp rx=0 tx=0).stop 2024-03-16T10:55:55.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101a20 msgr2=0x7f1a6c196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101a20 0x7f1a6c196530 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400d6f0 tx=0x7f1a5400dbb0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 shutdown_connections 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f1a5806df10 0x7f1a580703d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a6c101a20 0x7f1a6c196530 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 --2- 172.21.15.2:0/1380370140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a6c101100 0x7f1a6c195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 >> 172.21.15.2:0/1380370140 conn(0x7f1a6c0fa940 msgr2=0x7f1a6c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 shutdown_connections 2024-03-16T10:55:55.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.673+0000 7f1a714d5640 1 -- 172.21.15.2:0/1380370140 wait complete. 2024-03-16T10:55:55.686 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-16T10:55:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 -- 172.21.15.2:0/3187207812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 msgr2=0x7f34f8109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 --2- 172.21.15.2:0/3187207812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f8109f20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009880 tx=0x7f34e002bd70 comp rx=0 tx=0).stop 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 -- 172.21.15.2:0/3187207812 shutdown_connections 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 --2- 172.21.15.2:0/3187207812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f8109f20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 --2- 172.21.15.2:0/3187207812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8101120 0x7f34f8101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 -- 172.21.15.2:0/3187207812 >> 172.21.15.2:0/3187207812 conn(0x7f34f80fa940 msgr2=0x7f34f80fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 -- 172.21.15.2:0/3187207812 shutdown_connections 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 -- 172.21.15.2:0/3187207812 wait complete. 2024-03-16T10:55:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.792+0000 7f34fd142640 1 Processor -- start 2024-03-16T10:55:55.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34fd142640 1 -- start start 2024-03-16T10:55:55.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34fd142640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8101120 0x7f34f8195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34fd142640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34fd142640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8196b50 con 0x7f34f8101a40 2024-03-16T10:55:55.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34fd142640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f819a7f0 con 0x7f34f8101120 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f81964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51796/0 (socket says 172.21.15.2:51796) 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f659c640 1 -- 172.21.15.2:0/3530343533 learned_addr learned my addr 172.21.15.2:0/3530343533 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f6d9d640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8101120 0x7f34f8195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f659c640 1 -- 172.21.15.2:0/3530343533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8101120 msgr2=0x7f34f8195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f659c640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8101120 0x7f34f8195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f659c640 1 -- 172.21.15.2:0/3530343533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0009560 con 0x7f34f8101a40 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f6d9d640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8101120 0x7f34f8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.793+0000 7f34f659c640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f81964c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34e002c2b0 tx=0x7f34e0004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.794+0000 7f34effff640 1 -- 172.21.15.2:0/3530343533 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34e0004420 con 0x7f34f8101a40 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.794+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f819aa70 con 0x7f34f8101a40 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.794+0000 7f34effff640 1 -- 172.21.15.2:0/3530343533 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34e0003ab0 con 0x7f34f8101a40 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.794+0000 7f34effff640 1 -- 172.21.15.2:0/3530343533 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34e0005580 con 0x7f34f8101a40 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.794+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f819afd0 con 0x7f34f8101a40 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.795+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f80686f0 con 0x7f34f8101a40 2024-03-16T10:55:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.795+0000 7f34effff640 1 -- 172.21.15.2:0/3530343533 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f34e00035f0 con 0x7f34f8101a40 2024-03-16T10:55:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.795+0000 7f34effff640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34e406df10 0x7f34e40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.795+0000 7f34effff640 1 -- 172.21.15.2:0/3530343533 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f34e00b0940 con 0x7f34f8101a40 2024-03-16T10:55:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.796+0000 7f34f6d9d640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34e406df10 0x7f34e40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.796+0000 7f34f6d9d640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34e406df10 0x7f34e40703d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f34e8005fd0 tx=0x7f34e8009970 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:55.804+0000 7f34effff640 1 -- 172.21.15.2:0/3530343533 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f34e0079e30 con 0x7f34f8101a40 2024-03-16T10:55:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.049+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f34f804e030 con 0x7f34f8101a40 2024-03-16T10:55:56.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.049+0000 7f34effff640 1 -- 172.21.15.2:0/3530343533 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v31) v1 ==== 107+0+4916 (secure 0 0 0) 0x7f34e007d4f0 con 0x7f34f8101a40 2024-03-16T10:55:56.050 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:56.050 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14492,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.2:6829/2260596709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2260596709},{"type":"v1","addr":"172.21.15.2:6829","nonce":2260596709}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:42:09.214374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi026.dzzagf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6825/2074466615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2074466615},{"type":"v1","addr":"172.21.15.26:6825","nonce":2074466615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:56.051 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 19 2024-03-16T10:55:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34e406df10 msgr2=0x7f34e40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34e406df10 0x7f34e40703d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f34e8005fd0 tx=0x7f34e8009970 comp rx=0 tx=0).stop 2024-03-16T10:55:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 msgr2=0x7f34f81964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f81964c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f34e002c2b0 tx=0x7f34e0004a10 comp rx=0 tx=0).stop 2024-03-16T10:55:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 shutdown_connections 2024-03-16T10:55:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34e406df10 0x7f34e40703d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34f8101a40 0x7f34f81964c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 --2- 172.21.15.2:0/3530343533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f8101120 0x7f34f8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 >> 172.21.15.2:0/3530343533 conn(0x7f34f80fa940 msgr2=0x7f34f8104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 shutdown_connections 2024-03-16T10:55:56.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.052+0000 7f34fd142640 1 -- 172.21.15.2:0/3530343533 wait complete. 2024-03-16T10:55:56.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-16T10:55:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 -- 172.21.15.2:0/738879827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac0ff0a0 msgr2=0x7f93ac0ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 --2- 172.21.15.2:0/738879827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac0ff0a0 0x7f93ac0ff480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f939c009880 tx=0x7f939c02bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 -- 172.21.15.2:0/738879827 shutdown_connections 2024-03-16T10:55:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 --2- 172.21.15.2:0/738879827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93ac0ffa50 0x7f93ac103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 --2- 172.21.15.2:0/738879827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac0ff0a0 0x7f93ac0ff480 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 -- 172.21.15.2:0/738879827 >> 172.21.15.2:0/738879827 conn(0x7f93ac075b10 msgr2=0x7f93ac0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:56.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 -- 172.21.15.2:0/738879827 shutdown_connections 2024-03-16T10:55:56.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 -- 172.21.15.2:0/738879827 wait complete. 2024-03-16T10:55:56.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 Processor -- start 2024-03-16T10:55:56.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.167+0000 7f93b28e7640 1 -- start start 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93b28e7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93ac0ff0a0 0x7f93ac194ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93b28e7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac195410 0x7f93ac1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93b28e7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac195a10 con 0x7f93ac195410 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93b28e7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac195b80 con 0x7f93ac0ff0a0 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93ab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac195410 0x7f93ac1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93ab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac195410 0x7f93ac1997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51804/0 (socket says 172.21.15.2:51804) 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93ab7fe640 1 -- 172.21.15.2:0/610738843 learned_addr learned my addr 172.21.15.2:0/610738843 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93abfff640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93ac0ff0a0 0x7f93ac194ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93ab7fe640 1 -- 172.21.15.2:0/610738843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93ac0ff0a0 msgr2=0x7f93ac194ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93ab7fe640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93ac0ff0a0 0x7f93ac194ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93ab7fe640 1 -- 172.21.15.2:0/610738843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c009560 con 0x7f93ac195410 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.168+0000 7f93ab7fe640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac195410 0x7f93ac1997b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f93a000d6f0 tx=0x7f93a000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:56.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.169+0000 7f93a97fa640 1 -- 172.21.15.2:0/610738843 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a00051f0 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.169+0000 7f93a97fa640 1 -- 172.21.15.2:0/610738843 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93a0009ce0 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.169+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac199db0 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.169+0000 7f93a97fa640 1 -- 172.21.15.2:0/610738843 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a0004e80 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.169+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac19a310 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.170+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac19a5e0 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.170+0000 7f93a97fa640 1 -- 172.21.15.2:0/610738843 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f93a0009820 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.170+0000 7f93a97fa640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f939406deb0 0x7f9394070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.170+0000 7f93a97fa640 1 -- 172.21.15.2:0/610738843 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f93a008d870 con 0x7f93ac195410 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.170+0000 7f93abfff640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f939406deb0 0x7f9394070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.171+0000 7f93abfff640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f939406deb0 0x7f9394070370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f939c000c00 tx=0x7f939c02c820 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:56.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.175+0000 7f93a97fa640 1 -- 172.21.15.2:0/610738843 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f93a0056c40 con 0x7f93ac195410 2024-03-16T10:55:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:56 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/493336736' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-16T10:55:56.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:56 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1380370140' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-16T10:55:56.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.442+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f93ac100570 con 0x7f93ac195410 2024-03-16T10:55:56.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.443+0000 7f93a97fa640 1 -- 172.21.15.2:0/610738843 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v31) v1 ==== 107+0+4140 (secure 0 0 0) 0x7f93a005a300 con 0x7f93ac195410 2024-03-16T10:55:56.443 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:56.444 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:42:09.214374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi026.dzzagf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6825/2074466615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2074466615},{"type":"v1","addr":"172.21.15.26:6825","nonce":2074466615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:56.444 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 20 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f939406deb0 msgr2=0x7f9394070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f939406deb0 0x7f9394070370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f939c000c00 tx=0x7f939c02c820 comp rx=0 tx=0).stop 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac195410 msgr2=0x7f93ac1997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac195410 0x7f93ac1997b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f93a000d6f0 tx=0x7f93a000dbb0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 shutdown_connections 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f939406deb0 0x7f9394070370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93ac195410 0x7f93ac1997b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 --2- 172.21.15.2:0/610738843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93ac0ff0a0 0x7f93ac194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 >> 172.21.15.2:0/610738843 conn(0x7f93ac075b10 msgr2=0x7f93ac0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 shutdown_connections 2024-03-16T10:55:56.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.445+0000 7f93b28e7640 1 -- 172.21.15.2:0/610738843 wait complete. 2024-03-16T10:55:56.457 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-16T10:55:56.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:56 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/493336736' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-16T10:55:56.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:56 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1380370140' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/99485400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ffa70 msgr2=0x7f3fac103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/99485400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ffa70 0x7f3fac103ab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009880 tx=0x7f3fa002bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/99485400 shutdown_connections 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/99485400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ffa70 0x7f3fac103ab0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/99485400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac0ff0c0 0x7f3fac0ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/99485400 >> 172.21.15.2:0/99485400 conn(0x7f3fac0fa980 msgr2=0x7f3fac0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/99485400 shutdown_connections 2024-03-16T10:55:56.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.595+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/99485400 wait complete. 2024-03-16T10:55:56.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fb1e2f640 1 Processor -- start 2024-03-16T10:55:56.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fb1e2f640 1 -- start start 2024-03-16T10:55:56.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fb1e2f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ff0c0 0x7f3fac196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fb1e2f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac196570 0x7f3fac19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fb1e2f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac196b70 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fb1e2f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac196ce0 con 0x7f3fac196570 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ff0c0 0x7f3fac196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fab7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ff0c0 0x7f3fac196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51826/0 (socket says 172.21.15.2:51826) 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3fab7fe640 1 -- 172.21.15.2:0/4171863126 learned_addr learned my addr 172.21.15.2:0/4171863126 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.596+0000 7f3faaffd640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac196570 0x7f3fac19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fab7fe640 1 -- 172.21.15.2:0/4171863126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac196570 msgr2=0x7f3fac19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac196570 0x7f3fac19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fab7fe640 1 -- 172.21.15.2:0/4171863126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0009560 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fab7fe640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ff0c0 0x7f3fac196030 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e9a0 tx=0x7f3f9c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/4171863126 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c00cd50 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac19af10 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/4171863126 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9c00ceb0 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/4171863126 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9c010660 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.597+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac19b470 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.598+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac19b750 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.598+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/4171863126 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f3f9c0048a0 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.599+0000 7f3fa8ff9640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f9406df10 0x7f3f940703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.599+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/4171863126 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f3f9c014070 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.599+0000 7f3faaffd640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f9406df10 0x7f3f940703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.599+0000 7f3faaffd640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f9406df10 0x7f3f940703d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009230 tx=0x7f3fa0005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:56.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.605+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/4171863126 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f9c05b250 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.870+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f3fac100590 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.871+0000 7f3fa8ff9640 1 -- 172.21.15.2:0/4171863126 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v31) v1 ==== 107+0+4984 (secure 0 0 0) 0x7f3f9c05ade0 con 0x7f3fac0ff0c0 2024-03-16T10:55:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:56.872 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:42:09.214374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi026.dzzagf","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6825/2074466615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2074466615},{"type":"v1","addr":"172.21.15.26:6825","nonce":2074466615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:56.872 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 21 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f9406df10 msgr2=0x7f3f940703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f9406df10 0x7f3f940703d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009230 tx=0x7f3fa0005fd0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ff0c0 msgr2=0x7f3fac196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ff0c0 0x7f3fac196030 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00e9a0 tx=0x7f3f9c00eda0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 shutdown_connections 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f3f9406df10 0x7f3f940703d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fac196570 0x7f3fac19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 --2- 172.21.15.2:0/4171863126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3fac0ff0c0 0x7f3fac196030 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 >> 172.21.15.2:0/4171863126 conn(0x7f3fac0fa980 msgr2=0x7f3fac1091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:56.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 shutdown_connections 2024-03-16T10:55:56.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.873+0000 7f3fb1e2f640 1 -- 172.21.15.2:0/4171863126 wait complete. 2024-03-16T10:55:56.885 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-16T10:55:56.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.987+0000 7f275a826640 1 -- 172.21.15.2:0/233572119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ffa70 msgr2=0x7f2754103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.987+0000 7f275a826640 1 --2- 172.21.15.2:0/233572119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ffa70 0x7f2754103ab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2748009880 tx=0x7f274802bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 -- 172.21.15.2:0/233572119 shutdown_connections 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 --2- 172.21.15.2:0/233572119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ffa70 0x7f2754103ab0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 --2- 172.21.15.2:0/233572119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27540ff0c0 0x7f27540ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 -- 172.21.15.2:0/233572119 >> 172.21.15.2:0/233572119 conn(0x7f27540fa940 msgr2=0x7f27540fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 -- 172.21.15.2:0/233572119 shutdown_connections 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 -- 172.21.15.2:0/233572119 wait complete. 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 Processor -- start 2024-03-16T10:55:56.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 -- start start 2024-03-16T10:55:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ff0c0 0x7f2754100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754101090 0x7f27541014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2754102ab0 con 0x7f27540ff0c0 2024-03-16T10:55:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.988+0000 7f275a826640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27541a2150 con 0x7f2754101090 2024-03-16T10:55:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f2753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ff0c0 0x7f2754100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f2753fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ff0c0 0x7f2754100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51848/0 (socket says 172.21.15.2:51848) 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f2753fff640 1 -- 172.21.15.2:0/3697024097 learned_addr learned my addr 172.21.15.2:0/3697024097 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f27537fe640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754101090 0x7f27541014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f2753fff640 1 -- 172.21.15.2:0/3697024097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754101090 msgr2=0x7f27541014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f2753fff640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754101090 0x7f27541014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f2753fff640 1 -- 172.21.15.2:0/3697024097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009560 con 0x7f27540ff0c0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f2753fff640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ff0c0 0x7f2754100b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f274400d6f0 tx=0x7f274400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f27517fa640 1 -- 172.21.15.2:0/3697024097 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2744009a30 con 0x7f27540ff0c0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f27517fa640 1 -- 172.21.15.2:0/3697024097 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27440054e0 con 0x7f27540ff0c0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27541a2350 con 0x7f27540ff0c0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f27517fa640 1 -- 172.21.15.2:0/3697024097 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2744010460 con 0x7f27540ff0c0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.989+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27541a2880 con 0x7f27540ff0c0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.990+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2754100590 con 0x7f27540ff0c0 2024-03-16T10:55:56.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.991+0000 7f27517fa640 1 -- 172.21.15.2:0/3697024097 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2744009b90 con 0x7f27540ff0c0 2024-03-16T10:55:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.991+0000 7f27517fa640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f273c06df10 0x7f273c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.991+0000 7f27517fa640 1 -- 172.21.15.2:0/3697024097 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f274408d920 con 0x7f27540ff0c0 2024-03-16T10:55:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.991+0000 7f27537fe640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f273c06df10 0x7f273c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:56.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.992+0000 7f27537fe640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f273c06df10 0x7f273c0703d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2748036040 tx=0x7f2748000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:56.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:56.996+0000 7f27517fa640 1 -- 172.21.15.2:0/3697024097 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2744056cf0 con 0x7f27540ff0c0 2024-03-16T10:55:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:57 smithi026 ceph-mon[82600]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:57 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3530343533' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-16T10:55:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:57 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/610738843' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-16T10:55:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:57 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/4171863126' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-16T10:55:57.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.239+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f275404e030 con 0x7f27540ff0c0 2024-03-16T10:55:57.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.240+0000 7f27517fa640 1 -- 172.21.15.2:0/3697024097 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v31) v1 ==== 107+0+4194 (secure 0 0 0) 0x7f274405a3b0 con 0x7f27540ff0c0 2024-03-16T10:55:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:57.240 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:51:46.663700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:57.241 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2024-03-16T10:55:57.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f273c06df10 msgr2=0x7f273c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f273c06df10 0x7f273c0703d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2748036040 tx=0x7f2748000bc0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ff0c0 msgr2=0x7f2754100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ff0c0 0x7f2754100b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f274400d6f0 tx=0x7f274400dbb0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 shutdown_connections 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f273c06df10 0x7f273c0703d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754101090 0x7f27541014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 --2- 172.21.15.2:0/3697024097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27540ff0c0 0x7f2754100b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 >> 172.21.15.2:0/3697024097 conn(0x7f27540fa940 msgr2=0x7f27540fad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 shutdown_connections 2024-03-16T10:55:57.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.242+0000 7f275a826640 1 -- 172.21.15.2:0/3697024097 wait complete. 2024-03-16T10:55:57.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:57 smithi002 ceph-mon[94239]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:57.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:57 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3530343533' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-16T10:55:57.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:57 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/610738843' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-16T10:55:57.253 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:57 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/4171863126' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-16T10:55:57.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 -- 172.21.15.2:0/2010491387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 msgr2=0x7f07fc100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 --2- 172.21.15.2:0/2010491387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc100570 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009880 tx=0x7f07e402bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 -- 172.21.15.2:0/2010491387 shutdown_connections 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 --2- 172.21.15.2:0/2010491387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc100b40 0x7f07fc104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 --2- 172.21.15.2:0/2010491387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc100570 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 -- 172.21.15.2:0/2010491387 >> 172.21.15.2:0/2010491387 conn(0x7f07fc0fba30 msgr2=0x7f07fc0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 -- 172.21.15.2:0/2010491387 shutdown_connections 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.360+0000 7f0802df0640 1 -- 172.21.15.2:0/2010491387 wait complete. 2024-03-16T10:55:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0802df0640 1 Processor -- start 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0802df0640 1 -- start start 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0802df0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0802df0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc1964e0 0x7f07fc19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0802df0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc196a50 con 0x7f07fc100190 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0802df0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc196bc0 con 0x7f07fc1964e0 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0800b8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0800b8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51878/0 (socket says 172.21.15.2:51878) 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0800b8c640 1 -- 172.21.15.2:0/1478372879 learned_addr learned my addr 172.21.15.2:0/1478372879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f07fbfff640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc1964e0 0x7f07fc19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0800b8c640 1 -- 172.21.15.2:0/1478372879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc1964e0 msgr2=0x7f07fc19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0800b8c640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc1964e0 0x7f07fc19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0800b8c640 1 -- 172.21.15.2:0/1478372879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4009560 con 0x7f07fc100190 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f07fbfff640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc1964e0 0x7f07fc19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:55:57.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.361+0000 7f0800b8c640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc195fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07e402c220 tx=0x7f07e4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:57.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.362+0000 7f07f9ffb640 1 -- 172.21.15.2:0/1478372879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e40035f0 con 0x7f07fc100190 2024-03-16T10:55:57.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.362+0000 7f07f9ffb640 1 -- 172.21.15.2:0/1478372879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07e4003c10 con 0x7f07fc100190 2024-03-16T10:55:57.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.362+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc19ae20 con 0x7f07fc100190 2024-03-16T10:55:57.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.362+0000 7f07f9ffb640 1 -- 172.21.15.2:0/1478372879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e4034440 con 0x7f07fc100190 2024-03-16T10:55:57.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.362+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc19b330 con 0x7f07fc100190 2024-03-16T10:55:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.363+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc101660 con 0x7f07fc100190 2024-03-16T10:55:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.364+0000 7f07f9ffb640 1 -- 172.21.15.2:0/1478372879 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f07e4004f60 con 0x7f07fc100190 2024-03-16T10:55:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.364+0000 7f07f9ffb640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f07e806df10 0x7f07e80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.364+0000 7f07f9ffb640 1 -- 172.21.15.2:0/1478372879 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f07e40b0200 con 0x7f07fc100190 2024-03-16T10:55:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.364+0000 7f07fbfff640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f07e806df10 0x7f07e80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:57.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.364+0000 7f07fbfff640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f07e806df10 0x7f07e80703d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0694c0 tx=0x7f07ec007850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:57.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.372+0000 7f07f9ffb640 1 -- 172.21.15.2:0/1478372879 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f07e40795d0 con 0x7f07fc100190 2024-03-16T10:55:57.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.637+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f07fc197300 con 0x7f07fc100190 2024-03-16T10:55:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.638+0000 7f07f9ffb640 1 -- 172.21.15.2:0/1478372879 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v31) v1 ==== 107+0+4205 (secure 0 0 0) 0x7f07e407cc90 con 0x7f07fc100190 2024-03-16T10:55:57.639 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:57.639 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:51:46.669165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":0,"incarnation":23,"state":"up:replay","state_seq":2,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 23 2024-03-16T10:55:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f07e806df10 msgr2=0x7f07e80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f07e806df10 0x7f07e80703d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0694c0 tx=0x7f07ec007850 comp rx=0 tx=0).stop 2024-03-16T10:55:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 msgr2=0x7f07fc195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc195fa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07e402c220 tx=0x7f07e4004040 comp rx=0 tx=0).stop 2024-03-16T10:55:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 shutdown_connections 2024-03-16T10:55:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f07e806df10 0x7f07e80703d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc1964e0 0x7f07fc19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 --2- 172.21.15.2:0/1478372879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07fc100190 0x7f07fc195fa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 >> 172.21.15.2:0/1478372879 conn(0x7f07fc0fba30 msgr2=0x7f07fc0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 shutdown_connections 2024-03-16T10:55:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.640+0000 7f0802df0640 1 -- 172.21.15.2:0/1478372879 wait complete. 2024-03-16T10:55:57.652 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.754+0000 7fa16fedf640 1 -- 172.21.15.2:0/3942807075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1680ffa50 msgr2=0x7fa168103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.754+0000 7fa16fedf640 1 --2- 172.21.15.2:0/3942807075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1680ffa50 0x7fa168103a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa158009880 tx=0x7fa15802bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 -- 172.21.15.2:0/3942807075 shutdown_connections 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 --2- 172.21.15.2:0/3942807075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1680ffa50 0x7fa168103a90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 --2- 172.21.15.2:0/3942807075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1680ff0a0 0x7fa1680ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 -- 172.21.15.2:0/3942807075 >> 172.21.15.2:0/3942807075 conn(0x7fa168075b10 msgr2=0x7fa1680fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 -- 172.21.15.2:0/3942807075 shutdown_connections 2024-03-16T10:55:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 -- 172.21.15.2:0/3942807075 wait complete. 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 Processor -- start 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.755+0000 7fa16fedf640 1 -- start start 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16fedf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1680ff0a0 0x7fa168196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16fedf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa168197300 0x7fa168197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16fedf640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16819fe30 con 0x7fa168197300 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16fedf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16819ff70 con 0x7fa1680ff0a0 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16d47a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa168197300 0x7fa168197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16d47a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa168197300 0x7fa168197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51892/0 (socket says 172.21.15.2:51892) 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16d47a640 1 -- 172.21.15.2:0/4180233676 learned_addr learned my addr 172.21.15.2:0/4180233676 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16dc7b640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1680ff0a0 0x7fa168196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16d47a640 1 -- 172.21.15.2:0/4180233676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1680ff0a0 msgr2=0x7fa168196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16d47a640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1680ff0a0 0x7fa168196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16d47a640 1 -- 172.21.15.2:0/4180233676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158009560 con 0x7fa168197300 2024-03-16T10:55:57.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16d47a640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa168197300 0x7fa168197760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa158006010 tx=0x7fa15802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:57.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa15effd640 1 -- 172.21.15.2:0/4180233676 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1580044a0 con 0x7fa168197300 2024-03-16T10:55:57.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa15effd640 1 -- 172.21.15.2:0/4180233676 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa158003ab0 con 0x7fa168197300 2024-03-16T10:55:57.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1681a00b0 con 0x7fa168197300 2024-03-16T10:55:57.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa15effd640 1 -- 172.21.15.2:0/4180233676 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa158005590 con 0x7fa168197300 2024-03-16T10:55:57.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.756+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1681a0610 con 0x7fa168197300 2024-03-16T10:55:57.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.757+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168100570 con 0x7fa168197300 2024-03-16T10:55:57.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.758+0000 7fa15effd640 1 -- 172.21.15.2:0/4180233676 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fa158004dd0 con 0x7fa168197300 2024-03-16T10:55:57.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.758+0000 7fa15effd640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa15406df10 0x7fa1540703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:57.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.758+0000 7fa15effd640 1 -- 172.21.15.2:0/4180233676 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fa1580b00e0 con 0x7fa168197300 2024-03-16T10:55:57.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.758+0000 7fa16dc7b640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa15406df10 0x7fa1540703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:57.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.759+0000 7fa16dc7b640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa15406df10 0x7fa1540703d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa164005fd0 tx=0x7fa164005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:57.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.763+0000 7fa15effd640 1 -- 172.21.15.2:0/4180233676 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa1580794b0 con 0x7fa168197300 2024-03-16T10:55:57.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.988+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fa16804e030 con 0x7fa168197300 2024-03-16T10:55:57.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.989+0000 7fa15effd640 1 -- 172.21.15.2:0/4180233676 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v31) v1 ==== 107+0+4210 (secure 0 0 0) 0x7fa15807cb70 con 0x7fa168197300 2024-03-16T10:55:57.989 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:57.989 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:51:56.526931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":0,"incarnation":23,"state":"up:reconnect","state_seq":149,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:57.990 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa15406df10 msgr2=0x7fa1540703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa15406df10 0x7fa1540703d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa164005fd0 tx=0x7fa164005850 comp rx=0 tx=0).stop 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa168197300 msgr2=0x7fa168197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa168197300 0x7fa168197760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa158006010 tx=0x7fa15802cea0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 shutdown_connections 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fa15406df10 0x7fa1540703d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa168197300 0x7fa168197760 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 --2- 172.21.15.2:0/4180233676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1680ff0a0 0x7fa168196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 >> 172.21.15.2:0/4180233676 conn(0x7fa168075b10 msgr2=0x7fa1680fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 shutdown_connections 2024-03-16T10:55:57.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:57.991+0000 7fa16fedf640 1 -- 172.21.15.2:0/4180233676 wait complete. 2024-03-16T10:55:58.003 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 -- 172.21.15.2:0/734210025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101a80 msgr2=0x7f6b74109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 --2- 172.21.15.2:0/734210025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101a80 0x7f6b74109f60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6b640098b0 tx=0x7f6b6402bd60 comp rx=0 tx=0).stop 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 -- 172.21.15.2:0/734210025 shutdown_connections 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 --2- 172.21.15.2:0/734210025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101a80 0x7f6b74109f60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 --2- 172.21.15.2:0/734210025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b74101160 0x7f6b74101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 -- 172.21.15.2:0/734210025 >> 172.21.15.2:0/734210025 conn(0x7f6b74075b10 msgr2=0x7f6b740fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 -- 172.21.15.2:0/734210025 shutdown_connections 2024-03-16T10:55:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.106+0000 7f6b7c277640 1 -- 172.21.15.2:0/734210025 wait complete. 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7c277640 1 Processor -- start 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7c277640 1 -- start start 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7c277640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101160 0x7f6b74195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7c277640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b74101a80 0x7f6b74196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7c277640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74196b90 con 0x7f6b74101160 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7c277640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7419a8a0 con 0x7f6b74101a80 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7a013640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101160 0x7f6b74195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7a013640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101160 0x7f6b74195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51906/0 (socket says 172.21.15.2:51906) 2024-03-16T10:55:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7a013640 1 -- 172.21.15.2:0/4268634342 learned_addr learned my addr 172.21.15.2:0/4268634342 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b79812640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b74101a80 0x7f6b74196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7a013640 1 -- 172.21.15.2:0/4268634342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b74101a80 msgr2=0x7f6b74196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7a013640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b74101a80 0x7f6b74196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.107+0000 7f6b7a013640 1 -- 172.21.15.2:0/4268634342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64009560 con 0x7f6b74101160 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.108+0000 7f6b7a013640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101160 0x7f6b74195fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c810 tx=0x7f6b7000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.108+0000 7f6b6b7fe640 1 -- 172.21.15.2:0/4268634342 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b700041f0 con 0x7f6b74101160 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.108+0000 7f6b6b7fe640 1 -- 172.21.15.2:0/4268634342 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b70007e70 con 0x7f6b74101160 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.108+0000 7f6b6b7fe640 1 -- 172.21.15.2:0/4268634342 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b700049d0 con 0x7f6b74101160 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.108+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7419ab80 con 0x7f6b74101160 2024-03-16T10:55:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.108+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7419b0e0 con 0x7f6b74101160 2024-03-16T10:55:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.109+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b74067ac0 con 0x7f6b74101160 2024-03-16T10:55:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.109+0000 7f6b6b7fe640 1 -- 172.21.15.2:0/4268634342 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f6b700079b0 con 0x7f6b74101160 2024-03-16T10:55:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.110+0000 7f6b6b7fe640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b6006df10 0x7f6b600703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.110+0000 7f6b6b7fe640 1 -- 172.21.15.2:0/4268634342 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f6b7008dbd0 con 0x7f6b74101160 2024-03-16T10:55:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.110+0000 7f6b79812640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b6006df10 0x7f6b600703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.110+0000 7f6b79812640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b6006df10 0x7f6b600703d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6b64006010 tx=0x7f6b64000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:58.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.115+0000 7f6b6b7fe640 1 -- 172.21.15.2:0/4268634342 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b70056fa0 con 0x7f6b74101160 2024-03-16T10:55:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:58 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3697024097' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-16T10:55:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:58 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1478372879' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-16T10:55:58.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:58 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/4180233676' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-16T10:55:58.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.335+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f6b7404e030 con 0x7f6b74101160 2024-03-16T10:55:58.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.336+0000 7f6b6b7fe640 1 -- 172.21.15.2:0/4268634342 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v31) v1 ==== 107+0+5054 (secure 0 0 0) 0x7f6b7001d090 con 0x7f6b74101160 2024-03-16T10:55:58.337 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:58.337 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44337,"name":"cephfs.smithi026.dzzagf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6825/2953126152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2953126152},{"type":"v1","addr":"172.21.15.26:6825","nonce":2953126152}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:51:56.829513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":0,"incarnation":23,"state":"up:rejoin","state_seq":150,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:58.337 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b6006df10 msgr2=0x7f6b600703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b6006df10 0x7f6b600703d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6b64006010 tx=0x7f6b64000bc0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101160 msgr2=0x7f6b74195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101160 0x7f6b74195fc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c810 tx=0x7f6b7000ccd0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 shutdown_connections 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f6b6006df10 0x7f6b600703d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b74101a80 0x7f6b74196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 --2- 172.21.15.2:0/4268634342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b74101160 0x7f6b74195fc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 >> 172.21.15.2:0/4268634342 conn(0x7f6b74075b10 msgr2=0x7f6b74102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 shutdown_connections 2024-03-16T10:55:58.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.338+0000 7f6b7c277640 1 -- 172.21.15.2:0/4268634342 wait complete. 2024-03-16T10:55:58.349 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-16T10:55:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 -- 172.21.15.2:0/3003060273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 msgr2=0x7f4c14100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 --2- 172.21.15.2:0/3003060273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c14100590 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009880 tx=0x7f4c0402bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 -- 172.21.15.2:0/3003060273 shutdown_connections 2024-03-16T10:55:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 --2- 172.21.15.2:0/3003060273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c14100b60 0x7f4c14104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 --2- 172.21.15.2:0/3003060273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c14100590 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 -- 172.21.15.2:0/3003060273 >> 172.21.15.2:0/3003060273 conn(0x7f4c140fba10 msgr2=0x7f4c140fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 -- 172.21.15.2:0/3003060273 shutdown_connections 2024-03-16T10:55:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.452+0000 7f4c1a549640 1 -- 172.21.15.2:0/3003060273 wait complete. 2024-03-16T10:55:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c1a549640 1 Processor -- start 2024-03-16T10:55:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c1a549640 1 -- start start 2024-03-16T10:55:58.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c1a549640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c140725c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c1a549640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c14072b00 0x7f4c14072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c1a549640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14074520 con 0x7f4c141001b0 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c1a549640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c141a2160 con 0x7f4c14072b00 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c140725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c140725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51916/0 (socket says 172.21.15.2:51916) 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c13fff640 1 -- 172.21.15.2:0/228549961 learned_addr learned my addr 172.21.15.2:0/228549961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c13fff640 1 -- 172.21.15.2:0/228549961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c14072b00 msgr2=0x7f4c14072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c13fff640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c14072b00 0x7f4c14072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.453+0000 7f4c13fff640 1 -- 172.21.15.2:0/228549961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04009560 con 0x7f4c141001b0 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.454+0000 7f4c13fff640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c140725c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4c0402c220 tx=0x7f4c04004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.454+0000 7f4c117fa640 1 -- 172.21.15.2:0/228549961 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c040035f0 con 0x7f4c141001b0 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.454+0000 7f4c117fa640 1 -- 172.21.15.2:0/228549961 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c04003c10 con 0x7f4c141001b0 2024-03-16T10:55:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.454+0000 7f4c117fa640 1 -- 172.21.15.2:0/228549961 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c04034440 con 0x7f4c141001b0 2024-03-16T10:55:58.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.454+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c141a2300 con 0x7f4c141001b0 2024-03-16T10:55:58.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.454+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c141a2830 con 0x7f4c141001b0 2024-03-16T10:55:58.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.455+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c14101680 con 0x7f4c141001b0 2024-03-16T10:55:58.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.455+0000 7f4c117fa640 1 -- 172.21.15.2:0/228549961 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4c04004f60 con 0x7f4c141001b0 2024-03-16T10:55:58.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.456+0000 7f4c117fa640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4c0006df10 0x7f4c000703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.456+0000 7f4c117fa640 1 -- 172.21.15.2:0/228549961 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4c040b01e0 con 0x7f4c141001b0 2024-03-16T10:55:58.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.456+0000 7f4c137fe640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4c0006df10 0x7f4c000703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.456+0000 7f4c137fe640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4c0006df10 0x7f4c000703d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c140693f0 tx=0x7f4bfc005bf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.461+0000 7f4c117fa640 1 -- 172.21.15.2:0/228549961 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c040795b0 con 0x7f4c141001b0 2024-03-16T10:55:58.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:58 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3697024097' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-16T10:55:58.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:58 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1478372879' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-16T10:55:58.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:58 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/4180233676' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-16T10:55:58.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.791+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f4c141084b0 con 0x7f4c141001b0 2024-03-16T10:55:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.791+0000 7f4c117fa640 1 -- 172.21.15.2:0/228549961 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v31) v1 ==== 107+0+5063 (secure 0 0 0) 0x7f4c0407cc70 con 0x7f4c141001b0 2024-03-16T10:55:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:58.792 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44337,"name":"cephfs.smithi026.dzzagf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6825/2953126152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2953126152},{"type":"v1","addr":"172.21.15.26:6825","nonce":2953126152}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:51:58.832187+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24267":{"gid":24267,"name":"cephfs.smithi026.ojzcsw","rank":0,"incarnation":23,"state":"up:active","state_seq":151,"addr":"172.21.15.26:6827/2911421579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2911421579},{"type":"v1","addr":"172.21.15.26:6827","nonce":2911421579}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24267,"qdb_cluster":[24267]},"id":1}]} 2024-03-16T10:55:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 26 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.793+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4c0006df10 msgr2=0x7f4c000703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.793+0000 7f4c1a549640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4c0006df10 0x7f4c000703d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c140693f0 tx=0x7f4bfc005bf0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.793+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 msgr2=0x7f4c140725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.793+0000 7f4c1a549640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c140725c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4c0402c220 tx=0x7f4c04004040 comp rx=0 tx=0).stop 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.794+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 shutdown_connections 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.794+0000 7f4c1a549640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4c0006df10 0x7f4c000703d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.794+0000 7f4c1a549640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c14072b00 0x7f4c14072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.794+0000 7f4c1a549640 1 --2- 172.21.15.2:0/228549961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c141001b0 0x7f4c140725c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.794+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 >> 172.21.15.2:0/228549961 conn(0x7f4c140fba10 msgr2=0x7f4c140fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.794+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 shutdown_connections 2024-03-16T10:55:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.794+0000 7f4c1a549640 1 -- 172.21.15.2:0/228549961 wait complete. 2024-03-16T10:55:58.805 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 -- 172.21.15.2:0/2654698913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae01001d0 msgr2=0x7f0ae01005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/2654698913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae01001d0 0x7f0ae01005b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009880 tx=0x7f0adc02bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 -- 172.21.15.2:0/2654698913 shutdown_connections 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/2654698913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ae0100b80 0x7f0ae0104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/2654698913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae01001d0 0x7f0ae01005b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 -- 172.21.15.2:0/2654698913 >> 172.21.15.2:0/2654698913 conn(0x7f0ae00fba50 msgr2=0x7f0ae00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 -- 172.21.15.2:0/2654698913 shutdown_connections 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.907+0000 7f0ae82dc640 1 -- 172.21.15.2:0/2654698913 wait complete. 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae82dc640 1 Processor -- start 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae82dc640 1 -- start start 2024-03-16T10:55:58.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae82dc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ae01001d0 0x7f0ae0195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae82dc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae0196500 0x7f0ae019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae82dc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0196b00 con 0x7f0ae0196500 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae82dc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0196c70 con 0x7f0ae01001d0 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae5877640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae0196500 0x7f0ae019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae5877640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae0196500 0x7f0ae019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51926/0 (socket says 172.21.15.2:51926) 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae5877640 1 -- 172.21.15.2:0/104346244 learned_addr learned my addr 172.21.15.2:0/104346244 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.908+0000 7f0ae6078640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ae01001d0 0x7f0ae0195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ae5877640 1 -- 172.21.15.2:0/104346244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ae01001d0 msgr2=0x7f0ae0195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ae5877640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ae01001d0 0x7f0ae0195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ae5877640 1 -- 172.21.15.2:0/104346244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc009560 con 0x7f0ae0196500 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ae5877640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae0196500 0x7f0ae019a8a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d6f0 tx=0x7f0ad000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ad77fe640 1 -- 172.21.15.2:0/104346244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad00051f0 con 0x7f0ae0196500 2024-03-16T10:55:58.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ad77fe640 1 -- 172.21.15.2:0/104346244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ad0009ce0 con 0x7f0ae0196500 2024-03-16T10:55:58.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae019aea0 con 0x7f0ae0196500 2024-03-16T10:55:58.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ad77fe640 1 -- 172.21.15.2:0/104346244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad0004e80 con 0x7f0ae0196500 2024-03-16T10:55:58.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.909+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae019b400 con 0x7f0ae0196500 2024-03-16T10:55:58.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.910+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae019b6d0 con 0x7f0ae0196500 2024-03-16T10:55:58.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.911+0000 7f0ad77fe640 1 -- 172.21.15.2:0/104346244 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0ad00046c0 con 0x7f0ae0196500 2024-03-16T10:55:58.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.911+0000 7f0ad77fe640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0acc06deb0 0x7f0acc070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:58.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.911+0000 7f0ae6078640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0acc06deb0 0x7f0acc070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:58.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.911+0000 7f0ad77fe640 1 -- 172.21.15.2:0/104346244 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f0ad008d7c0 con 0x7f0ae0196500 2024-03-16T10:55:58.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.912+0000 7f0ae6078640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0acc06deb0 0x7f0acc070370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0adc006010 tx=0x7f0adc005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:58.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:58.916+0000 7f0ad77fe640 1 -- 172.21.15.2:0/104346244 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ad0056c40 con 0x7f0ae0196500 2024-03-16T10:55:59.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.137+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f0ae01016a0 con 0x7f0ae0196500 2024-03-16T10:55:59.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.138+0000 7f0ad77fe640 1 -- 172.21.15.2:0/104346244 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v31) v1 ==== 107+0+4262 (secure 0 0 0) 0x7f0ad005a300 con 0x7f0ae0196500 2024-03-16T10:55:59.139 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:59.139 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44337,"name":"cephfs.smithi026.dzzagf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6825/2953126152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2953126152},{"type":"v1","addr":"172.21.15.26:6825","nonce":2953126152}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:52:09.467916+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:59.139 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 27 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0acc06deb0 msgr2=0x7f0acc070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0acc06deb0 0x7f0acc070370 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0adc006010 tx=0x7f0adc005d50 comp rx=0 tx=0).stop 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae0196500 msgr2=0x7f0ae019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae0196500 0x7f0ae019a8a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d6f0 tx=0x7f0ad000dbb0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 shutdown_connections 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f0acc06deb0 0x7f0acc070370 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ae0196500 0x7f0ae019a8a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 --2- 172.21.15.2:0/104346244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ae01001d0 0x7f0ae0195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 >> 172.21.15.2:0/104346244 conn(0x7f0ae00fba50 msgr2=0x7f0ae00fbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:59.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 shutdown_connections 2024-03-16T10:55:59.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.140+0000 7f0ae82dc640 1 -- 172.21.15.2:0/104346244 wait complete. 2024-03-16T10:55:59.153 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:59 smithi002 ceph-mon[94239]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:59.153 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:59 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/4268634342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-16T10:55:59.153 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:55:59 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/228549961' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-16T10:55:59.154 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-16T10:55:59.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/3076342021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101100 msgr2=0x7f4bb81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/3076342021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101100 0x7f4bb81014e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009880 tx=0x7f4bac02bd70 comp rx=0 tx=0).stop 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/3076342021 shutdown_connections 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/3076342021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb8101a20 0x7f4bb8109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/3076342021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101100 0x7f4bb81014e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/3076342021 >> 172.21.15.2:0/3076342021 conn(0x7f4bb8075b10 msgr2=0x7f4bb80fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/3076342021 shutdown_connections 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.257+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/3076342021 wait complete. 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.258+0000 7f4bbdcfc640 1 Processor -- start 2024-03-16T10:55:59.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.258+0000 7f4bbdcfc640 1 -- start start 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bbdcfc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb8101100 0x7f4bb8195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bbdcfc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101a20 0x7f4bb8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bbdcfc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb8196b90 con 0x7f4bb8101a20 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bbdcfc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb819a8a0 con 0x7f4bb8101100 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101a20 0x7f4bb8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101a20 0x7f4bb8196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51942/0 (socket says 172.21.15.2:51942) 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb6ffd640 1 -- 172.21.15.2:0/2153281563 learned_addr learned my addr 172.21.15.2:0/2153281563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb77fe640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb8101100 0x7f4bb8195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb6ffd640 1 -- 172.21.15.2:0/2153281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb8101100 msgr2=0x7f4bb8195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb6ffd640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb8101100 0x7f4bb8195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb6ffd640 1 -- 172.21.15.2:0/2153281563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac009560 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb77fe640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb8101100 0x7f4bb8195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb6ffd640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101a20 0x7f4bb8196500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800e970 tx=0x7f4ba800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb4ff9640 1 -- 172.21.15.2:0/2153281563 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ba800cc90 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb4ff9640 1 -- 172.21.15.2:0/2153281563 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ba800cdf0 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb819ab80 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bb4ff9640 1 -- 172.21.15.2:0/2153281563 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ba8010430 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.259+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb819b0e0 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.260+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb8067b20 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.261+0000 7f4bb4ff9640 1 -- 172.21.15.2:0/2153281563 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f4ba8004f20 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.261+0000 7f4bb4ff9640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ba006df10 0x7f4ba00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.261+0000 7f4bb4ff9640 1 -- 172.21.15.2:0/2153281563 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f4ba8014070 con 0x7f4bb8101a20 2024-03-16T10:55:59.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.261+0000 7f4bb77fe640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ba006df10 0x7f4ba00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.262+0000 7f4bb77fe640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ba006df10 0x7f4ba00703d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005ab0 tx=0x7f4bac005a20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:59.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.270+0000 7f4bb4ff9640 1 -- 172.21.15.2:0/2153281563 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ba8057740 con 0x7f4bb8101a20 2024-03-16T10:55:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:59 smithi026 ceph-mon[82600]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:55:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:59 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/4268634342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-16T10:55:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:55:59 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/228549961' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-16T10:55:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.490+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f4bb804e030 con 0x7f4bb8101a20 2024-03-16T10:55:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.491+0000 7f4bb4ff9640 1 -- 172.21.15.2:0/2153281563 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v31) v1 ==== 107+0+4341 (secure 0 0 0) 0x7f4ba805ae00 con 0x7f4bb8101a20 2024-03-16T10:55:59.492 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:59.492 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44337,"name":"cephfs.smithi026.dzzagf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6825/2953126152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2953126152},{"type":"v1","addr":"172.21.15.26:6825","nonce":2953126152}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:52:09.472912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-03-16T10:55:59.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ba006df10 msgr2=0x7f4ba00703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ba006df10 0x7f4ba00703d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005ab0 tx=0x7f4bac005a20 comp rx=0 tx=0).stop 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101a20 msgr2=0x7f4bb8196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101a20 0x7f4bb8196500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800e970 tx=0x7f4ba800ee30 comp rx=0 tx=0).stop 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 shutdown_connections 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f4ba006df10 0x7f4ba00703d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bb8101a20 0x7f4bb8196500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 --2- 172.21.15.2:0/2153281563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bb8101100 0x7f4bb8195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 >> 172.21.15.2:0/2153281563 conn(0x7f4bb8075b10 msgr2=0x7f4bb81023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 shutdown_connections 2024-03-16T10:55:59.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.494+0000 7f4bbdcfc640 1 -- 172.21.15.2:0/2153281563 wait complete. 2024-03-16T10:55:59.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 -- 172.21.15.2:0/1756610242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101100 msgr2=0x7f85d01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 --2- 172.21.15.2:0/1756610242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101100 0x7f85d01014e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009880 tx=0x7f85cc02bd70 comp rx=0 tx=0).stop 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 -- 172.21.15.2:0/1756610242 shutdown_connections 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 --2- 172.21.15.2:0/1756610242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d0101a20 0x7f85d0109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 --2- 172.21.15.2:0/1756610242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101100 0x7f85d01014e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 -- 172.21.15.2:0/1756610242 >> 172.21.15.2:0/1756610242 conn(0x7f85d0075b10 msgr2=0x7f85d00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 -- 172.21.15.2:0/1756610242 shutdown_connections 2024-03-16T10:55:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.611+0000 7f85d8221640 1 -- 172.21.15.2:0/1756610242 wait complete. 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d8221640 1 Processor -- start 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d8221640 1 -- start start 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d8221640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d0101100 0x7f85d0195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d8221640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101a20 0x7f85d0196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d8221640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0196b90 con 0x7f85d0101a20 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d8221640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d019a8a0 con 0x7f85d0101100 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d57bc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101a20 0x7f85d0196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d57bc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101a20 0x7f85d0196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51962/0 (socket says 172.21.15.2:51962) 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.612+0000 7f85d57bc640 1 -- 172.21.15.2:0/1243900623 learned_addr learned my addr 172.21.15.2:0/1243900623 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85d5fbd640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d0101100 0x7f85d0195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85d57bc640 1 -- 172.21.15.2:0/1243900623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d0101100 msgr2=0x7f85d0195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85d57bc640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d0101100 0x7f85d0195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85d57bc640 1 -- 172.21.15.2:0/1243900623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc009560 con 0x7f85d0101a20 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85d57bc640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101a20 0x7f85d0196500 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d6d0 tx=0x7f85c000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85c6ffd640 1 -- 172.21.15.2:0/1243900623 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c00051f0 con 0x7f85d0101a20 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85c6ffd640 1 -- 172.21.15.2:0/1243900623 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85c0009ce0 con 0x7f85d0101a20 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d019ab80 con 0x7f85d0101a20 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85c6ffd640 1 -- 172.21.15.2:0/1243900623 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c0004db0 con 0x7f85d0101a20 2024-03-16T10:55:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.613+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d019b0e0 con 0x7f85d0101a20 2024-03-16T10:55:59.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.614+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d0067b20 con 0x7f85d0101a20 2024-03-16T10:55:59.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.615+0000 7f85c6ffd640 1 -- 172.21.15.2:0/1243900623 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f85c0009820 con 0x7f85d0101a20 2024-03-16T10:55:59.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.615+0000 7f85c6ffd640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f85bc06deb0 0x7f85bc070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.615+0000 7f85c6ffd640 1 -- 172.21.15.2:0/1243900623 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f85c008d830 con 0x7f85d0101a20 2024-03-16T10:55:59.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.615+0000 7f85d5fbd640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f85bc06deb0 0x7f85bc070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.616+0000 7f85d5fbd640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f85bc06deb0 0x7f85bc070370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000c00 tx=0x7f85cc005b90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:59.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.620+0000 7f85c6ffd640 1 -- 172.21.15.2:0/1243900623 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f85c0056c00 con 0x7f85d0101a20 2024-03-16T10:55:59.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.842+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f85d004e030 con 0x7f85d0101a20 2024-03-16T10:55:59.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.843+0000 7f85c6ffd640 1 -- 172.21.15.2:0/1243900623 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v31) v1 ==== 107+0+4345 (secure 0 0 0) 0x7f85c005a2c0 con 0x7f85d0101a20 2024-03-16T10:55:59.844 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:55:59.844 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44337,"name":"cephfs.smithi026.dzzagf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6825/2953126152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2953126152},{"type":"v1","addr":"172.21.15.26:6825","nonce":2953126152}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:52:19.089389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:55:59.844 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 29 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f85bc06deb0 msgr2=0x7f85bc070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f85bc06deb0 0x7f85bc070370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000c00 tx=0x7f85cc005b90 comp rx=0 tx=0).stop 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101a20 msgr2=0x7f85d0196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101a20 0x7f85d0196500 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d6d0 tx=0x7f85c000db90 comp rx=0 tx=0).stop 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 shutdown_connections 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f85bc06deb0 0x7f85bc070370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85d0101a20 0x7f85d0196500 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 --2- 172.21.15.2:0/1243900623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d0101100 0x7f85d0195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 >> 172.21.15.2:0/1243900623 conn(0x7f85d0075b10 msgr2=0x7f85d01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:59.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 shutdown_connections 2024-03-16T10:55:59.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.845+0000 7f85d8221640 1 -- 172.21.15.2:0/1243900623 wait complete. 2024-03-16T10:55:59.857 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 -- 172.21.15.2:0/1139449935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e00ff0a0 msgr2=0x7f90e00ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 --2- 172.21.15.2:0/1139449935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e00ff0a0 0x7f90e00ff480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009880 tx=0x7f90d002bce0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 -- 172.21.15.2:0/1139449935 shutdown_connections 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 --2- 172.21.15.2:0/1139449935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e00ffa50 0x7f90e0103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 --2- 172.21.15.2:0/1139449935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e00ff0a0 0x7f90e00ff480 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 -- 172.21.15.2:0/1139449935 >> 172.21.15.2:0/1139449935 conn(0x7f90e0075b10 msgr2=0x7f90e00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 -- 172.21.15.2:0/1139449935 shutdown_connections 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 -- 172.21.15.2:0/1139449935 wait complete. 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 Processor -- start 2024-03-16T10:55:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.960+0000 7f90e681c640 1 -- start start 2024-03-16T10:55:59.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90e681c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e00ff0a0 0x7f90e0196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90e681c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0197310 0x7f90e0197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90e681c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e019fe40 con 0x7f90e0197310 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90e681c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e019ff80 con 0x7f90e00ff0a0 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0197310 0x7f90e0197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90df7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0197310 0x7f90e0197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51982/0 (socket says 172.21.15.2:51982) 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90df7fe640 1 -- 172.21.15.2:0/3709032262 learned_addr learned my addr 172.21.15.2:0/3709032262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90dffff640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e00ff0a0 0x7f90e0196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90df7fe640 1 -- 172.21.15.2:0/3709032262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e00ff0a0 msgr2=0x7f90e0196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90df7fe640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e00ff0a0 0x7f90e0196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90df7fe640 1 -- 172.21.15.2:0/3709032262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0009560 con 0x7f90e0197310 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90dffff640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e00ff0a0 0x7f90e0196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90df7fe640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0197310 0x7f90e0197770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f90d400e9a0 tx=0x7f90d400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.961+0000 7f90dd7fa640 1 -- 172.21.15.2:0/3709032262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90d400cc90 con 0x7f90e0197310 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.962+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e01a0180 con 0x7f90e0197310 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.962+0000 7f90dd7fa640 1 -- 172.21.15.2:0/3709032262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90d400cdf0 con 0x7f90e0197310 2024-03-16T10:55:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.962+0000 7f90dd7fa640 1 -- 172.21.15.2:0/3709032262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90d4010430 con 0x7f90e0197310 2024-03-16T10:55:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.962+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e01a06b0 con 0x7f90e0197310 2024-03-16T10:55:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.963+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e0100570 con 0x7f90e0197310 2024-03-16T10:55:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.963+0000 7f90dd7fa640 1 -- 172.21.15.2:0/3709032262 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f90d4005020 con 0x7f90e0197310 2024-03-16T10:55:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.963+0000 7f90dd7fa640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f90c806df10 0x7f90c80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:55:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.964+0000 7f90dd7fa640 1 -- 172.21.15.2:0/3709032262 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f90d4014070 con 0x7f90e0197310 2024-03-16T10:55:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.964+0000 7f90dffff640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f90c806df10 0x7f90c80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:55:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.964+0000 7f90dffff640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f90c806df10 0x7f90c80703d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005ab0 tx=0x7f90d0005a20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:55:59.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:55:59.970+0000 7f90dd7fa640 1 -- 172.21.15.2:0/3709032262 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f90d40576e0 con 0x7f90e0197310 2024-03-16T10:56:00.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.200+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f90e004e030 con 0x7f90e0197310 2024-03-16T10:56:00.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.200+0000 7f90dd7fa640 1 -- 172.21.15.2:0/3709032262 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v31) v1 ==== 107+0+5189 (secure 0 0 0) 0x7f90d405ada0 con 0x7f90e0197310 2024-03-16T10:56:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:56:00.201 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi002.ijcmqs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6829/1125443118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1125443118},{"type":"v1","addr":"172.21.15.2:6829","nonce":1125443118}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44337,"name":"cephfs.smithi026.dzzagf","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6825/2953126152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2953126152},{"type":"v1","addr":"172.21.15.26:6825","nonce":2953126152}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44367,"name":"cephfs.smithi026.ojzcsw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6827/1298103624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1298103624},{"type":"v1","addr":"172.21.15.26:6827","nonce":1298103624}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:41:50.343374+0000","modified":"2024-03-16T10:52:19.879210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34408":{"gid":34408,"name":"cephfs.smithi002.qvbseu","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.2:6827/918911081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":918911081},{"type":"v1","addr":"172.21.15.2:6827","nonce":918911081}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-16T10:56:00.201 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 30 2024-03-16T10:56:00.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f90c806df10 msgr2=0x7f90c80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f90c806df10 0x7f90c80703d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005ab0 tx=0x7f90d0005a20 comp rx=0 tx=0).stop 2024-03-16T10:56:00.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0197310 msgr2=0x7f90e0197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0197310 0x7f90e0197770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f90d400e9a0 tx=0x7f90d400ee60 comp rx=0 tx=0).stop 2024-03-16T10:56:00.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 shutdown_connections 2024-03-16T10:56:00.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f90c806df10 0x7f90c80703d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90e0197310 0x7f90e0197770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 --2- 172.21.15.2:0/3709032262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e00ff0a0 0x7f90e0196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.202+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 >> 172.21.15.2:0/3709032262 conn(0x7f90e0075b10 msgr2=0x7f90e00fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.203+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 shutdown_connections 2024-03-16T10:56:00.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.203+0000 7f90e681c640 1 -- 172.21.15.2:0/3709032262 wait complete. 2024-03-16T10:56:00.213 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:00 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/104346244' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-16T10:56:00.213 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:00 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2153281563' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-16T10:56:00.213 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:00 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/1243900623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-16T10:56:00.214 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-16T10:56:00.224 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-16T10:56:00.225 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:56:00.225 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:56:00.240 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:56:00.241 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:56:00.296 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:56:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 -- 172.21.15.2:0/2303660903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb388101ac0 msgr2=0x7fb388109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 --2- 172.21.15.2:0/2303660903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb388101ac0 0x7fb388109fa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009880 tx=0x7fb37c02bd70 comp rx=0 tx=0).stop 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 -- 172.21.15.2:0/2303660903 shutdown_connections 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 --2- 172.21.15.2:0/2303660903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb388101ac0 0x7fb388109fa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 --2- 172.21.15.2:0/2303660903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3881011a0 0x7fb388101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 -- 172.21.15.2:0/2303660903 >> 172.21.15.2:0/2303660903 conn(0x7fb3880fa940 msgr2=0x7fb3880fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 -- 172.21.15.2:0/2303660903 shutdown_connections 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 -- 172.21.15.2:0/2303660903 wait complete. 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.438+0000 7fb38d677640 1 Processor -- start 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb38d677640 1 -- start start 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb38d677640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3881011a0 0x7fb388196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb38d677640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388101ac0 0x7fb388196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb38d677640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb388196c10 con 0x7fb3881011a0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb38d677640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38819a920 con 0x7fb388101ac0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb386ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3881011a0 0x7fb388196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb386ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3881011a0 0x7fb388196040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51996/0 (socket says 172.21.15.2:51996) 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb386ffd640 1 -- 172.21.15.2:0/2707851635 learned_addr learned my addr 172.21.15.2:0/2707851635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb3867fc640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388101ac0 0x7fb388196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb386ffd640 1 -- 172.21.15.2:0/2707851635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388101ac0 msgr2=0x7fb388196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb386ffd640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388101ac0 0x7fb388196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb386ffd640 1 -- 172.21.15.2:0/2707851635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c009560 con 0x7fb3881011a0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.439+0000 7fb386ffd640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3881011a0 0x7fb388196040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c810 tx=0x7fb37800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.440+0000 7fb36ffff640 1 -- 172.21.15.2:0/2707851635 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3780041f0 con 0x7fb3881011a0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.440+0000 7fb36ffff640 1 -- 172.21.15.2:0/2707851635 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb378007e70 con 0x7fb3881011a0 2024-03-16T10:56:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.440+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38819ac00 con 0x7fb3881011a0 2024-03-16T10:56:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.440+0000 7fb36ffff640 1 -- 172.21.15.2:0/2707851635 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb37800f450 con 0x7fb3881011a0 2024-03-16T10:56:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.440+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38819b160 con 0x7fb3881011a0 2024-03-16T10:56:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.441+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb388068c20 con 0x7fb3881011a0 2024-03-16T10:56:00.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.441+0000 7fb36ffff640 1 -- 172.21.15.2:0/2707851635 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fb3780079b0 con 0x7fb3881011a0 2024-03-16T10:56:00.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.442+0000 7fb36ffff640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb37006df10 0x7fb3700703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:00.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.442+0000 7fb36ffff640 1 -- 172.21.15.2:0/2707851635 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fb37808d9c0 con 0x7fb3881011a0 2024-03-16T10:56:00.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.442+0000 7fb3867fc640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb37006df10 0x7fb3700703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:00.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.442+0000 7fb3867fc640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb37006df10 0x7fb3700703d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb37c036040 tx=0x7fb37c000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:00.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.450+0000 7fb36ffff640 1 -- 172.21.15.2:0/2707851635 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb378056d10 con 0x7fb3881011a0 2024-03-16T10:56:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:00 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/104346244' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-16T10:56:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:00 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2153281563' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-16T10:56:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:00 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/1243900623' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-16T10:56:00.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.633+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3881046d0 con 0x7fb3881011a0 2024-03-16T10:56:00.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.635+0000 7fb36ffff640 1 -- 172.21.15.2:0/2707851635 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16795 (secure 0 0 0) 0x7fb37805a3d0 con 0x7fb3881011a0 2024-03-16T10:56:00.635 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:56:00.636 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":86,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","created":"2024-03-16T10:34:41.302807+0000","modified":"2024-03-16T10:52:09.467585+0000","last_up_change":"2024-03-16T10:51:06.706637+0000","last_in_change":"2024-03-16T10:40:14.935894+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:39:18.453991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T10:41:48.797183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T10:41:49.583833+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4befd156-b58c-44e4-bc1e-f16112b3df27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":80,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6803","nonce":1147005518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6805","nonce":1147005518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6809","nonce":1147005518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6807","nonce":1147005518}]},"public_addr":"172.21.15.2:6803/1147005518","cluster_addr":"172.21.15.2:6805/1147005518","heartbeat_back_addr":"172.21.15.2:6809/1147005518","heartbeat_front_addr":"172.21.15.2:6807/1147005518","state":["exists","up"]},{"osd":1,"uuid":"95448bdc-721b-45c0-8c58-0b416e188aa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":60,"up_thru":80,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6811","nonce":648077700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6813","nonce":648077700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6817","nonce":648077700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6815","nonce":648077700}]},"public_addr":"172.21.15.2:6811/648077700","cluster_addr":"172.21.15.2:6813/648077700","heartbeat_back_addr":"172.21.15.2:6817/648077700","heartbeat_front_addr":"172.21.15.2:6815/648077700","state":["exists","up"]},{"osd":2,"uuid":"ce6a7a3f-db89-426c-bbb4-1339c5adac81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":61,"up_from":65,"up_thru":80,"down_at":62,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6819","nonce":1082105549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6821","nonce":1082105549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6825","nonce":1082105549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6823","nonce":1082105549}]},"public_addr":"172.21.15.2:6819/1082105549","cluster_addr":"172.21.15.2:6821/1082105549","heartbeat_back_addr":"172.21.15.2:6825/1082105549","heartbeat_front_addr":"172.21.15.2:6823/1082105549","state":["exists","up"]},{"osd":3,"uuid":"8d760311-5fda-425b-ac3c-5045ecb1cbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":66,"up_from":70,"up_thru":80,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6801","nonce":3946506757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6803","nonce":3946506757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6807","nonce":3946506757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6805","nonce":3946506757}]},"public_addr":"172.21.15.26:6801/3946506757","cluster_addr":"172.21.15.26:6803/3946506757","heartbeat_back_addr":"172.21.15.26:6807/3946506757","heartbeat_front_addr":"172.21.15.26:6805/3946506757","state":["exists","up"]},{"osd":4,"uuid":"250a3093-6818-4f4f-85d5-d1f2e4381afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":71,"up_from":75,"up_thru":80,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6809","nonce":2367920652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6811","nonce":2367920652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6815","nonce":2367920652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6813","nonce":2367920652}]},"public_addr":"172.21.15.26:6809/2367920652","cluster_addr":"172.21.15.26:6811/2367920652","heartbeat_back_addr":"172.21.15.26:6815/2367920652","heartbeat_front_addr":"172.21.15.26:6813/2367920652","state":["exists","up"]},{"osd":5,"uuid":"b16f392a-1d2f-47b5-b356-cab94bc834cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":76,"up_from":80,"up_thru":80,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6817","nonce":2348241985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6819","nonce":2348241985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6823","nonce":2348241985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6821","nonce":2348241985}]},"public_addr":"172.21.15.26:6817/2348241985","cluster_addr":"172.21.15.26:6819/2348241985","heartbeat_back_addr":"172.21.15.26:6823/2348241985","heartbeat_front_addr":"172.21.15.26:6821/2348241985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:18.975873+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:47.618973+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:15.584615+0000","dead_epoch":61},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:39.921213+0000","dead_epoch":66},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:06.286712+0000","dead_epoch":71},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:32.567112+0000","dead_epoch":76}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6826/2911421579":"2024-03-17T10:52:09.467533+0000","172.21.15.2:0/3116141534":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/899962313":"2024-03-17T10:46:19.722815+0000","172.21.15.26:0/2161833313":"2024-03-17T10:45:15.531471+0000","172.21.15.26:0/2320376866":"2024-03-17T10:45:15.531471+0000","172.21.15.2:6801/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1498322259":"2024-03-17T10:37:00.118083+0000","172.21.15.26:0/1307372554":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/4132065053":"2024-03-17T10:46:19.722815+0000","172.21.15.26:6825/2074466615":"2024-03-17T10:51:46.663682+0000","172.21.15.2:0/2565089320":"2024-03-17T10:35:17.551709+0000","172.21.15.26:6824/2074466615":"2024-03-17T10:51:46.663682+0000","172.21.15.2:0/567924185":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/100368483":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/3514263595":"2024-03-17T10:43:58.039282+0000","172.21.15.26:6827/2911421579":"2024-03-17T10:52:09.467533+0000","172.21.15.2:6801/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3421178356":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/2322904669":"2024-03-17T10:35:17.551709+0000","172.21.15.26:0/2024487458":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/401513378":"2024-03-17T10:43:58.039282+0000","172.21.15.26:0/2419254018":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/1971862470":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/730693462":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/2563315045":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/1789736475":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/730693462":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/4201560868":"2024-03-17T10:43:58.039282+0000","172.21.15.2:0/1144469616":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6826/2835000958":"2024-03-17T10:42:06.208262+0000","172.21.15.2:0/3462160730":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6827/2835000958":"2024-03-17T10:42:06.208262+0000","172.21.15.2:6800/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.26:6829/274393035":"2024-03-17T10:45:15.531471+0000","172.21.15.2:6800/3195213211":"2024-03-17T10:43:58.039282+0000","172.21.15.2:6801/3195213211":"2024-03-17T10:43:58.039282+0000","172.21.15.2:0/3582566838":"2024-03-17T10:43:58.039282+0000","172.21.15.26:6828/274393035":"2024-03-17T10:45:15.531471+0000","172.21.15.26:0/1025795450":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/2348229359":"2024-03-17T10:35:51.598930+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:56:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb37006df10 msgr2=0x7fb3700703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb37006df10 0x7fb3700703d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb37c036040 tx=0x7fb37c000bc0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3881011a0 msgr2=0x7fb388196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3881011a0 0x7fb388196040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c810 tx=0x7fb37800ccd0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 shutdown_connections 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fb37006df10 0x7fb3700703d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb388101ac0 0x7fb388196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 --2- 172.21.15.2:0/2707851635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3881011a0 0x7fb388196040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 >> 172.21.15.2:0/2707851635 conn(0x7fb3880fa940 msgr2=0x7fb3881024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 shutdown_connections 2024-03-16T10:56:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.637+0000 7fb38d677640 1 -- 172.21.15.2:0/2707851635 wait complete. 2024-03-16T10:56:00.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:39:18.453991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T10:41:48.797183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T10:41:49.583833+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:56:00.649 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:56:00.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 -- 172.21.15.2:0/475354619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101a20 msgr2=0x7f34bc109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 --2- 172.21.15.2:0/475354619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101a20 0x7f34bc109f00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009880 tx=0x7f34b002bd70 comp rx=0 tx=0).stop 2024-03-16T10:56:00.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 -- 172.21.15.2:0/475354619 shutdown_connections 2024-03-16T10:56:00.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 --2- 172.21.15.2:0/475354619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101a20 0x7f34bc109f00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 --2- 172.21.15.2:0/475354619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc101100 0x7f34bc1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 -- 172.21.15.2:0/475354619 >> 172.21.15.2:0/475354619 conn(0x7f34bc075b10 msgr2=0x7f34bc0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:00.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 -- 172.21.15.2:0/475354619 shutdown_connections 2024-03-16T10:56:00.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 -- 172.21.15.2:0/475354619 wait complete. 2024-03-16T10:56:00.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 Processor -- start 2024-03-16T10:56:00.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.755+0000 7f34c0fae640 1 -- start start 2024-03-16T10:56:00.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34c0fae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101100 0x7f34bc0fed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:00.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34c0fae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc101a20 0x7f34bc0ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:00.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34c0fae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0ff7d0 con 0x7f34bc101100 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34c0fae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0ff940 con 0x7f34bc101a20 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ba59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101100 0x7f34bc0fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ba59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101100 0x7f34bc0fed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52012/0 (socket says 172.21.15.2:52012) 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ba59c640 1 -- 172.21.15.2:0/2341207657 learned_addr learned my addr 172.21.15.2:0/2341207657 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34b9d9b640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc101a20 0x7f34bc0ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ba59c640 1 -- 172.21.15.2:0/2341207657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc101a20 msgr2=0x7f34bc0ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ba59c640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc101a20 0x7f34bc0ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ba59c640 1 -- 172.21.15.2:0/2341207657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b0009560 con 0x7f34bc101100 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ba59c640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101100 0x7f34bc0fed50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e970 tx=0x7f34ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ab7fe640 1 -- 172.21.15.2:0/2341207657 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34ac00cc90 con 0x7f34bc101100 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc1a2180 con 0x7f34bc101100 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ab7fe640 1 -- 172.21.15.2:0/2341207657 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34ac00cdf0 con 0x7f34bc101100 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34ab7fe640 1 -- 172.21.15.2:0/2341207657 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34ac010430 con 0x7f34bc101100 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.756+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc1a24f0 con 0x7f34bc101100 2024-03-16T10:56:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.757+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34bc067b20 con 0x7f34bc101100 2024-03-16T10:56:00.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.758+0000 7f34ab7fe640 1 -- 172.21.15.2:0/2341207657 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f34ac004f70 con 0x7f34bc101100 2024-03-16T10:56:00.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.758+0000 7f34ab7fe640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34a406df20 0x7f34a40703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:00.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.758+0000 7f34ab7fe640 1 -- 172.21.15.2:0/2341207657 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f34ac014070 con 0x7f34bc101100 2024-03-16T10:56:00.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.758+0000 7f34b9d9b640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34a406df20 0x7f34a40703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:00.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.759+0000 7f34b9d9b640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34a406df20 0x7f34a40703e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f34bc100480 tx=0x7f34b002caf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:00.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.763+0000 7f34ab7fe640 1 -- 172.21.15.2:0/2341207657 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f34ac0577e0 con 0x7f34bc101100 2024-03-16T10:56:00.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.949+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34bc04e030 con 0x7f34bc101100 2024-03-16T10:56:00.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.949+0000 7f34ab7fe640 1 -- 172.21.15.2:0/2341207657 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34ac05aea0 con 0x7f34bc101100 2024-03-16T10:56:00.950 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-16T10:56:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34a406df20 msgr2=0x7f34a40703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34a406df20 0x7f34a40703e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f34bc100480 tx=0x7f34b002caf0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101100 msgr2=0x7f34bc0fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101100 0x7f34bc0fed50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e970 tx=0x7f34ac00ed70 comp rx=0 tx=0).stop 2024-03-16T10:56:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 shutdown_connections 2024-03-16T10:56:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f34a406df20 0x7f34a40703e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34bc101a20 0x7f34bc0ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 --2- 172.21.15.2:0/2341207657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34bc101100 0x7f34bc0fed50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.951+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 >> 172.21.15.2:0/2341207657 conn(0x7f34bc075b10 msgr2=0x7f34bc1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.952+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 shutdown_connections 2024-03-16T10:56:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:00.952+0000 7f34c0fae640 1 -- 172.21.15.2:0/2341207657 wait complete. 2024-03-16T10:56:00.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T10:56:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 -- 172.21.15.2:0/2129446247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8067e60 msgr2=0x7f02c8104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 --2- 172.21.15.2:0/2129446247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8067e60 0x7f02c8104ba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009880 tx=0x7f02b802bce0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 -- 172.21.15.2:0/2129446247 shutdown_connections 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 --2- 172.21.15.2:0/2129446247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8067e60 0x7f02c8104ba0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 --2- 172.21.15.2:0/2129446247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c80674b0 0x7f02c8067890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 -- 172.21.15.2:0/2129446247 >> 172.21.15.2:0/2129446247 conn(0x7f02c8074960 msgr2=0x7f02c8074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 -- 172.21.15.2:0/2129446247 shutdown_connections 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.070+0000 7f02cf481640 1 -- 172.21.15.2:0/2129446247 wait complete. 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cf481640 1 Processor -- start 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cf481640 1 -- start start 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cf481640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c80674b0 0x7f02c8196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cf481640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c8196550 0x7f02c819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cf481640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8196b50 con 0x7f02c80674b0 2024-03-16T10:56:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cf481640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8196cc0 con 0x7f02c8196550 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cd21d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c80674b0 0x7f02c8196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cd21d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c80674b0 0x7f02c8196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52038/0 (socket says 172.21.15.2:52038) 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cd21d640 1 -- 172.21.15.2:0/11566536 learned_addr learned my addr 172.21.15.2:0/11566536 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cca1c640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c8196550 0x7f02c819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cd21d640 1 -- 172.21.15.2:0/11566536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c8196550 msgr2=0x7f02c819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cd21d640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c8196550 0x7f02c819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cd21d640 1 -- 172.21.15.2:0/11566536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b8009560 con 0x7f02c80674b0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.071+0000 7f02cd21d640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c80674b0 0x7f02c8196010 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f02c400d900 tx=0x7f02c400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.072+0000 7f02be7fc640 1 -- 172.21.15.2:0/11566536 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c4009c00 con 0x7f02c80674b0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.072+0000 7f02be7fc640 1 -- 172.21.15.2:0/11566536 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02c4009d60 con 0x7f02c80674b0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.072+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c819aef0 con 0x7f02c80674b0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.072+0000 7f02be7fc640 1 -- 172.21.15.2:0/11566536 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c400f710 con 0x7f02c80674b0 2024-03-16T10:56:01.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.072+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c819b450 con 0x7f02c80674b0 2024-03-16T10:56:01.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.073+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c8101680 con 0x7f02c80674b0 2024-03-16T10:56:01.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.073+0000 7f02be7fc640 1 -- 172.21.15.2:0/11566536 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f02c4010460 con 0x7f02c80674b0 2024-03-16T10:56:01.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.074+0000 7f02be7fc640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f02b406df10 0x7f02b40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.074+0000 7f02cca1c640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f02b406df10 0x7f02b40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.074+0000 7f02be7fc640 1 -- 172.21.15.2:0/11566536 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f02c408dbd0 con 0x7f02c80674b0 2024-03-16T10:56:01.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.074+0000 7f02cca1c640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f02b406df10 0x7f02b40703d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009230 tx=0x7f02b802c7e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:01.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.079+0000 7f02be7fc640 1 -- 172.21.15.2:0/11566536 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02c4056fa0 con 0x7f02c80674b0 2024-03-16T10:56:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:01 smithi002 ceph-mon[94239]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:01 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3709032262' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-16T10:56:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:01 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2707851635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:56:01.299 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:01 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/2341207657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:56:01.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.315+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f02c819b760 con 0x7f02c80674b0 2024-03-16T10:56:01.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.315+0000 7f02be7fc640 1 -- 172.21.15.2:0/11566536 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f02c819b760 con 0x7f02c80674b0 2024-03-16T10:56:01.316 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f02b406df10 msgr2=0x7f02b40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f02b406df10 0x7f02b40703d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009230 tx=0x7f02b802c7e0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c80674b0 msgr2=0x7f02c8196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c80674b0 0x7f02c8196010 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f02c400d900 tx=0x7f02c400ddc0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 shutdown_connections 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f02b406df10 0x7f02b40703d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c8196550 0x7f02c819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 --2- 172.21.15.2:0/11566536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c80674b0 0x7f02c8196010 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 >> 172.21.15.2:0/11566536 conn(0x7f02c8074960 msgr2=0x7f02c8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 shutdown_connections 2024-03-16T10:56:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.318+0000 7f02cf481640 1 -- 172.21.15.2:0/11566536 wait complete. 2024-03-16T10:56:01.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T10:56:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 -- 172.21.15.2:0/4107748693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0fe6b0 msgr2=0x7fe40c0fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 --2- 172.21.15.2:0/4107748693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0fe6b0 0x7fe40c0fea90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe404009880 tx=0x7fe40402bce0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 -- 172.21.15.2:0/4107748693 shutdown_connections 2024-03-16T10:56:01.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 --2- 172.21.15.2:0/4107748693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40c0ff060 0x7fe40c104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 --2- 172.21.15.2:0/4107748693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0fe6b0 0x7fe40c0fea90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 -- 172.21.15.2:0/4107748693 >> 172.21.15.2:0/4107748693 conn(0x7fe40c0fa540 msgr2=0x7fe40c0fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 -- 172.21.15.2:0/4107748693 shutdown_connections 2024-03-16T10:56:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.461+0000 7fe411167640 1 -- 172.21.15.2:0/4107748693 wait complete. 2024-03-16T10:56:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.462+0000 7fe411167640 1 Processor -- start 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.462+0000 7fe411167640 1 -- start start 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.462+0000 7fe411167640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40c0fe6b0 0x7fe40c0ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.462+0000 7fe411167640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0ffb60 0x7fe40c0fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe411167640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c101580 con 0x7fe40c0ffb60 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe411167640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c1a21f0 con 0x7fe40c0fe6b0 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0ffb60 0x7fe40c0fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0ffb60 0x7fe40c0fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52052/0 (socket says 172.21.15.2:52052) 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40a59c640 1 -- 172.21.15.2:0/3014370543 learned_addr learned my addr 172.21.15.2:0/3014370543 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40ad9d640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40c0fe6b0 0x7fe40c0ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40a59c640 1 -- 172.21.15.2:0/3014370543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40c0fe6b0 msgr2=0x7fe40c0ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40a59c640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40c0fe6b0 0x7fe40c0ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40a59c640 1 -- 172.21.15.2:0/3014370543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe404009560 con 0x7fe40c0ffb60 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40ad9d640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40c0fe6b0 0x7fe40c0ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe40a59c640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0ffb60 0x7fe40c0fffc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400e970 tx=0x7fe3f400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe403fff640 1 -- 172.21.15.2:0/3014370543 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f400cc90 con 0x7fe40c0ffb60 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c1a23f0 con 0x7fe40c0ffb60 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe403fff640 1 -- 172.21.15.2:0/3014370543 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3f400cdf0 con 0x7fe40c0ffb60 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe403fff640 1 -- 172.21.15.2:0/3014370543 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f4018770 con 0x7fe40c0ffb60 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.463+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c1a2920 con 0x7fe40c0ffb60 2024-03-16T10:56:01.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.464+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c04e030 con 0x7fe40c0ffb60 2024-03-16T10:56:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.465+0000 7fe403fff640 1 -- 172.21.15.2:0/3014370543 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fe3f4010b80 con 0x7fe40c0ffb60 2024-03-16T10:56:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.465+0000 7fe403fff640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe3f806df10 0x7fe3f80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.466+0000 7fe40ad9d640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe3f806df10 0x7fe3f80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.466+0000 7fe403fff640 1 -- 172.21.15.2:0/3014370543 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fe3f4014070 con 0x7fe40c0ffb60 2024-03-16T10:56:01.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.466+0000 7fe40ad9d640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe3f806df10 0x7fe3f80703d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe404036040 tx=0x7fe404000bc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:01.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.470+0000 7fe403fff640 1 -- 172.21.15.2:0/3014370543 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3f4053740 con 0x7fe40c0ffb60 2024-03-16T10:56:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:01 smithi026 ceph-mon[82600]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:01 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3709032262' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-16T10:56:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:01 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2707851635' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:56:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:01 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/2341207657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:56:01.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.730+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe40c100b00 con 0x7fe40c0ffb60 2024-03-16T10:56:01.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.730+0000 7fe403fff640 1 -- 172.21.15.2:0/3014370543 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe3f4057790 con 0x7fe40c0ffb60 2024-03-16T10:56:01.731 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe3f806df10 msgr2=0x7fe3f80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe3f806df10 0x7fe3f80703d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe404036040 tx=0x7fe404000bc0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0ffb60 msgr2=0x7fe40c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0ffb60 0x7fe40c0fffc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400e970 tx=0x7fe3f400ee30 comp rx=0 tx=0).stop 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 shutdown_connections 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fe3f806df10 0x7fe3f80703d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe40c0ffb60 0x7fe40c0fffc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 --2- 172.21.15.2:0/3014370543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40c0fe6b0 0x7fe40c0ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 >> 172.21.15.2:0/3014370543 conn(0x7fe40c0fa540 msgr2=0x7fe40c103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:01.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 shutdown_connections 2024-03-16T10:56:01.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.732+0000 7fe411167640 1 -- 172.21.15.2:0/3014370543 wait complete. 2024-03-16T10:56:01.744 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 -- 172.21.15.2:0/3397254925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce40ff090 msgr2=0x7fcce40ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/3397254925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce40ff090 0x7fcce40ff470 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009880 tx=0x7fcce002bce0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 -- 172.21.15.2:0/3397254925 shutdown_connections 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/3397254925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce40ffa40 0x7fcce4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/3397254925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce40ff090 0x7fcce40ff470 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 -- 172.21.15.2:0/3397254925 >> 172.21.15.2:0/3397254925 conn(0x7fcce40fa8f0 msgr2=0x7fcce40fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 -- 172.21.15.2:0/3397254925 shutdown_connections 2024-03-16T10:56:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.851+0000 7fcceb9dd640 1 -- 172.21.15.2:0/3397254925 wait complete. 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcceb9dd640 1 Processor -- start 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcceb9dd640 1 -- start start 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcceb9dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce40ff090 0x7fcce4196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcceb9dd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4197300 0x7fcce4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcceb9dd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419fe30 con 0x7fcce4197300 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcceb9dd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce419ff70 con 0x7fcce40ff090 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcce8f78640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4197300 0x7fcce4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcce8f78640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4197300 0x7fcce4197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52070/0 (socket says 172.21.15.2:52070) 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcce8f78640 1 -- 172.21.15.2:0/201011534 learned_addr learned my addr 172.21.15.2:0/201011534 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.852+0000 7fcce9779640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce40ff090 0x7fcce4196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fcce8f78640 1 -- 172.21.15.2:0/201011534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce40ff090 msgr2=0x7fcce4196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fcce8f78640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce40ff090 0x7fcce4196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fcce8f78640 1 -- 172.21.15.2:0/201011534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0009560 con 0x7fcce4197300 2024-03-16T10:56:01.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fcce8f78640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4197300 0x7fcce4197760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fccd400d8d0 tx=0x7fccd400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:01.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fccda7fc640 1 -- 172.21.15.2:0/201011534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccd400bea0 con 0x7fcce4197300 2024-03-16T10:56:01.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fccda7fc640 1 -- 172.21.15.2:0/201011534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccd4009c00 con 0x7fcce4197300 2024-03-16T10:56:01.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce41a0110 con 0x7fcce4197300 2024-03-16T10:56:01.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fccda7fc640 1 -- 172.21.15.2:0/201011534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccd4010460 con 0x7fcce4197300 2024-03-16T10:56:01.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.853+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce41a0670 con 0x7fcce4197300 2024-03-16T10:56:01.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.854+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce4100560 con 0x7fcce4197300 2024-03-16T10:56:01.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.854+0000 7fccda7fc640 1 -- 172.21.15.2:0/201011534 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7fccd4003f10 con 0x7fcce4197300 2024-03-16T10:56:01.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.855+0000 7fccda7fc640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fccd006df10 0x7fccd00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:01.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.855+0000 7fcce9779640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fccd006df10 0x7fccd00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:01.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.855+0000 7fccda7fc640 1 -- 172.21.15.2:0/201011534 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7fccd408dac0 con 0x7fcce4197300 2024-03-16T10:56:01.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.856+0000 7fcce9779640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fccd006df10 0x7fccd00703d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006010 tx=0x7fcce0005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:01.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:01.860+0000 7fccda7fc640 1 -- 172.21.15.2:0/201011534 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fccd4056f10 con 0x7fcce4197300 2024-03-16T10:56:02.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.055+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcce4107390 con 0x7fcce4197300 2024-03-16T10:56:02.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.056+0000 7fccda7fc640 1 -- 172.21.15.2:0/201011534 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v86) v1 ==== 81+0+2507 (secure 0 0 0) 0x7fccd40179e0 con 0x7fcce4197300 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6826/2911421579 2024-03-17T10:52:09.467533+0000 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3116141534 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/899962313 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2161833313 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2320376866 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/486800969 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/486800969 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:02.057 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1498322259 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/1307372554 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4132065053 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6825/2074466615 2024-03-17T10:51:46.663682+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2565089320 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6824/2074466615 2024-03-17T10:51:46.663682+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/567924185 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/100368483 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3514263595 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6827/2911421579 2024-03-17T10:52:09.467533+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3464791333 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3421178356 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2322904669 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2024487458 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/401513378 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2419254018 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1971862470 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/730693462 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2563315045 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1789736475 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:02.059 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3739300318 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3464791333 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/730693462 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4201560868 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1144469616 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6826/2835000958 2024-03-17T10:42:06.208262+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3462160730 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6827/2835000958 2024-03-17T10:42:06.208262+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3739300318 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6829/274393035 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3195213211 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3195213211 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:02.060 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3582566838 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:02.061 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6828/274393035 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.061 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/1025795450 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:02.061 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2348229359 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:02.062 INFO:teuthology.orchestra.run.smithi002.stderr:listed 43 entries 2024-03-16T10:56:02.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fccd006df10 msgr2=0x7fccd00703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fccd006df10 0x7fccd00703d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006010 tx=0x7fcce0005d50 comp rx=0 tx=0).stop 2024-03-16T10:56:02.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4197300 msgr2=0x7fcce4197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4197300 0x7fcce4197760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fccd400d8d0 tx=0x7fccd400dd90 comp rx=0 tx=0).stop 2024-03-16T10:56:02.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 shutdown_connections 2024-03-16T10:56:02.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7fccd006df10 0x7fccd00703d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcce4197300 0x7fcce4197760 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 --2- 172.21.15.2:0/201011534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce40ff090 0x7fcce4196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 >> 172.21.15.2:0/201011534 conn(0x7fcce40fa8f0 msgr2=0x7fcce40fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:02.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 shutdown_connections 2024-03-16T10:56:02.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.059+0000 7fcceb9dd640 1 -- 172.21.15.2:0/201011534 wait complete. 2024-03-16T10:56:02.071 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-03-16T10:56:02.071 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:56:02.088 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:56:02.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:02 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/11566536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:56:02.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:02 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3014370543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:56:02.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 -- 172.21.15.2:0/2454981334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 msgr2=0x7f07980feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 --2- 172.21.15.2:0/2454981334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f07980feb50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f078c009880 tx=0x7f078c02bce0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 -- 172.21.15.2:0/2454981334 shutdown_connections 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 --2- 172.21.15.2:0/2454981334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f07980feb50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 --2- 172.21.15.2:0/2454981334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0798100d80 0x7f0798101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 -- 172.21.15.2:0/2454981334 >> 172.21.15.2:0/2454981334 conn(0x7f0798074050 msgr2=0x7f0798074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 -- 172.21.15.2:0/2454981334 shutdown_connections 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.235+0000 7f079e50a640 1 -- 172.21.15.2:0/2454981334 wait complete. 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f079e50a640 1 Processor -- start 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f079e50a640 1 -- start start 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f079e50a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f0798198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f079e50a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0798100d80 0x7f07981986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f079e50a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0798198d50 con 0x7f07980fe6d0 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f079e50a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079819ca10 con 0x7f0798100d80 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f0797fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f0798198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f0797fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f0798198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52092/0 (socket says 172.21.15.2:52092) 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f0797fff640 1 -- 172.21.15.2:0/3768577357 learned_addr learned my addr 172.21.15.2:0/3768577357 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f07977fe640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0798100d80 0x7f07981986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f0797fff640 1 -- 172.21.15.2:0/3768577357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0798100d80 msgr2=0x7f07981986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f0797fff640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0798100d80 0x7f07981986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f0797fff640 1 -- 172.21.15.2:0/3768577357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c009560 con 0x7f07980fe6d0 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.236+0000 7f0797fff640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f0798198180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f078800d6f0 tx=0x7f078800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.237+0000 7f07957fa640 1 -- 172.21.15.2:0/3768577357 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0788009a30 con 0x7f07980fe6d0 2024-03-16T10:56:02.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.237+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079819ccf0 con 0x7f07980fe6d0 2024-03-16T10:56:02.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.237+0000 7f07957fa640 1 -- 172.21.15.2:0/3768577357 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07880054e0 con 0x7f07980fe6d0 2024-03-16T10:56:02.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.237+0000 7f07957fa640 1 -- 172.21.15.2:0/3768577357 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0788010460 con 0x7f07980fe6d0 2024-03-16T10:56:02.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.237+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079819d2d0 con 0x7f07980fe6d0 2024-03-16T10:56:02.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.238+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07980681a0 con 0x7f07980fe6d0 2024-03-16T10:56:02.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.238+0000 7f07957fa640 1 -- 172.21.15.2:0/3768577357 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f0788009b90 con 0x7f07980fe6d0 2024-03-16T10:56:02.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.239+0000 7f07957fa640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f078006df10 0x7f07800703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.239+0000 7f07977fe640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f078006df10 0x7f07800703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.239+0000 7f07957fa640 1 -- 172.21.15.2:0/3768577357 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f078808d990 con 0x7f07980fe6d0 2024-03-16T10:56:02.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.239+0000 7f07977fe640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f078006df10 0x7f07800703d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0798199680 tx=0x7f078c02c800 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:02.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.244+0000 7f07957fa640 1 -- 172.21.15.2:0/3768577357 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0788056d60 con 0x7f07980fe6d0 2024-03-16T10:56:02.314 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:02 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/11566536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:56:02.315 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:02 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3014370543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:56:02.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.430+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07981994f0 con 0x7f07980fe6d0 2024-03-16T10:56:02.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.432+0000 7f07957fa640 1 -- 172.21.15.2:0/3768577357 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16795 (secure 0 0 0) 0x7f078805a420 con 0x7f07980fe6d0 2024-03-16T10:56:02.433 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-03-16T10:56:02.433 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":86,"fsid":"b3aed5e6-e380-11ee-95c9-87774f69a715","created":"2024-03-16T10:34:41.302807+0000","modified":"2024-03-16T10:52:09.467585+0000","last_up_change":"2024-03-16T10:51:06.706637+0000","last_in_change":"2024-03-16T10:40:14.935894+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:39:18.453991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-16T10:41:48.797183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-16T10:41:49.583833+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4befd156-b58c-44e4-bc1e-f16112b3df27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":51,"up_from":55,"up_thru":80,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6803","nonce":1147005518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6805","nonce":1147005518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6809","nonce":1147005518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1147005518},{"type":"v1","addr":"172.21.15.2:6807","nonce":1147005518}]},"public_addr":"172.21.15.2:6803/1147005518","cluster_addr":"172.21.15.2:6805/1147005518","heartbeat_back_addr":"172.21.15.2:6809/1147005518","heartbeat_front_addr":"172.21.15.2:6807/1147005518","state":["exists","up"]},{"osd":1,"uuid":"95448bdc-721b-45c0-8c58-0b416e188aa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":56,"up_from":60,"up_thru":80,"down_at":57,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6811","nonce":648077700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6813","nonce":648077700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6817","nonce":648077700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":648077700},{"type":"v1","addr":"172.21.15.2:6815","nonce":648077700}]},"public_addr":"172.21.15.2:6811/648077700","cluster_addr":"172.21.15.2:6813/648077700","heartbeat_back_addr":"172.21.15.2:6817/648077700","heartbeat_front_addr":"172.21.15.2:6815/648077700","state":["exists","up"]},{"osd":2,"uuid":"ce6a7a3f-db89-426c-bbb4-1339c5adac81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":61,"up_from":65,"up_thru":80,"down_at":62,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6819","nonce":1082105549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6821","nonce":1082105549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6825","nonce":1082105549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":1082105549},{"type":"v1","addr":"172.21.15.2:6823","nonce":1082105549}]},"public_addr":"172.21.15.2:6819/1082105549","cluster_addr":"172.21.15.2:6821/1082105549","heartbeat_back_addr":"172.21.15.2:6825/1082105549","heartbeat_front_addr":"172.21.15.2:6823/1082105549","state":["exists","up"]},{"osd":3,"uuid":"8d760311-5fda-425b-ac3c-5045ecb1cbea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":66,"up_from":70,"up_thru":80,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6801","nonce":3946506757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6803","nonce":3946506757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6807","nonce":3946506757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3946506757},{"type":"v1","addr":"172.21.15.26:6805","nonce":3946506757}]},"public_addr":"172.21.15.26:6801/3946506757","cluster_addr":"172.21.15.26:6803/3946506757","heartbeat_back_addr":"172.21.15.26:6807/3946506757","heartbeat_front_addr":"172.21.15.26:6805/3946506757","state":["exists","up"]},{"osd":4,"uuid":"250a3093-6818-4f4f-85d5-d1f2e4381afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":71,"up_from":75,"up_thru":80,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6809","nonce":2367920652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6811","nonce":2367920652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6815","nonce":2367920652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2367920652},{"type":"v1","addr":"172.21.15.26:6813","nonce":2367920652}]},"public_addr":"172.21.15.26:6809/2367920652","cluster_addr":"172.21.15.26:6811/2367920652","heartbeat_back_addr":"172.21.15.26:6815/2367920652","heartbeat_front_addr":"172.21.15.26:6813/2367920652","state":["exists","up"]},{"osd":5,"uuid":"b16f392a-1d2f-47b5-b356-cab94bc834cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":76,"up_from":80,"up_thru":80,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6817","nonce":2348241985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6819","nonce":2348241985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6823","nonce":2348241985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":2348241985},{"type":"v1","addr":"172.21.15.26:6821","nonce":2348241985}]},"public_addr":"172.21.15.26:6817/2348241985","cluster_addr":"172.21.15.26:6819/2348241985","heartbeat_back_addr":"172.21.15.26:6823/2348241985","heartbeat_front_addr":"172.21.15.26:6821/2348241985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:18.975873+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:38:47.618973+0000","dead_epoch":56},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:15.584615+0000","dead_epoch":61},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:39:39.921213+0000","dead_epoch":66},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:06.286712+0000","dead_epoch":71},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:40:32.567112+0000","dead_epoch":76}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6826/2911421579":"2024-03-17T10:52:09.467533+0000","172.21.15.2:0/3116141534":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/899962313":"2024-03-17T10:46:19.722815+0000","172.21.15.26:0/2161833313":"2024-03-17T10:45:15.531471+0000","172.21.15.26:0/2320376866":"2024-03-17T10:45:15.531471+0000","172.21.15.2:6801/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/486800969":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/1498322259":"2024-03-17T10:37:00.118083+0000","172.21.15.26:0/1307372554":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/4132065053":"2024-03-17T10:46:19.722815+0000","172.21.15.26:6825/2074466615":"2024-03-17T10:51:46.663682+0000","172.21.15.2:0/2565089320":"2024-03-17T10:35:17.551709+0000","172.21.15.26:6824/2074466615":"2024-03-17T10:51:46.663682+0000","172.21.15.2:0/567924185":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/100368483":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/3514263595":"2024-03-17T10:43:58.039282+0000","172.21.15.26:6827/2911421579":"2024-03-17T10:52:09.467533+0000","172.21.15.2:6801/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:0/3421178356":"2024-03-17T10:37:00.118083+0000","172.21.15.2:0/2322904669":"2024-03-17T10:35:17.551709+0000","172.21.15.26:0/2024487458":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/401513378":"2024-03-17T10:43:58.039282+0000","172.21.15.26:0/2419254018":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/1971862470":"2024-03-17T10:37:00.118083+0000","172.21.15.2:6800/730693462":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/2563315045":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/1789736475":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6800/3464791333":"2024-03-17T10:35:17.551709+0000","172.21.15.2:6801/730693462":"2024-03-17T10:46:19.722815+0000","172.21.15.2:0/4201560868":"2024-03-17T10:43:58.039282+0000","172.21.15.2:0/1144469616":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6826/2835000958":"2024-03-17T10:42:06.208262+0000","172.21.15.2:0/3462160730":"2024-03-17T10:35:51.598930+0000","172.21.15.2:6827/2835000958":"2024-03-17T10:42:06.208262+0000","172.21.15.2:6800/3739300318":"2024-03-17T10:35:51.598930+0000","172.21.15.26:6829/274393035":"2024-03-17T10:45:15.531471+0000","172.21.15.2:6800/3195213211":"2024-03-17T10:43:58.039282+0000","172.21.15.2:6801/3195213211":"2024-03-17T10:43:58.039282+0000","172.21.15.2:0/3582566838":"2024-03-17T10:43:58.039282+0000","172.21.15.26:6828/274393035":"2024-03-17T10:45:15.531471+0000","172.21.15.26:0/1025795450":"2024-03-17T10:45:15.531471+0000","172.21.15.2:0/2348229359":"2024-03-17T10:35:51.598930+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f078006df10 msgr2=0x7f07800703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f078006df10 0x7f07800703d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0798199680 tx=0x7f078c02c800 comp rx=0 tx=0).stop 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 msgr2=0x7f0798198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f0798198180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f078800d6f0 tx=0x7f078800dbb0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 shutdown_connections 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f078006df10 0x7f07800703d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0798100d80 0x7f07981986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 --2- 172.21.15.2:0/3768577357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07980fe6d0 0x7f0798198180 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 >> 172.21.15.2:0/3768577357 conn(0x7f0798074050 msgr2=0x7f0798106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 shutdown_connections 2024-03-16T10:56:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.434+0000 7f079e50a640 1 -- 172.21.15.2:0/3768577357 wait complete. 2024-03-16T10:56:02.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:39:18.453991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-16T10:41:48.797183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-16T10:41:49.583833+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:56:02.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:56:02.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 -- 172.21.15.2:0/241046404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 msgr2=0x7f8de01005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 --2- 172.21.15.2:0/241046404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de01005b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009880 tx=0x7f8dd002bce0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 -- 172.21.15.2:0/241046404 shutdown_connections 2024-03-16T10:56:02.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 --2- 172.21.15.2:0/241046404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de0100b80 0x7f8de0104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 --2- 172.21.15.2:0/241046404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de01005b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 -- 172.21.15.2:0/241046404 >> 172.21.15.2:0/241046404 conn(0x7f8de00fba50 msgr2=0x7f8de00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 -- 172.21.15.2:0/241046404 shutdown_connections 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 -- 172.21.15.2:0/241046404 wait complete. 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 Processor -- start 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.551+0000 7f8de7666640 1 -- start start 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de7666640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de0195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de7666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de0196520 0x7f8de019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de7666640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0196b20 con 0x7f8de01001d0 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de7666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de0196c90 con 0x7f8de0196520 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de5402640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de0195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de5402640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de0195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39136/0 (socket says 172.21.15.2:39136) 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de5402640 1 -- 172.21.15.2:0/386331526 learned_addr learned my addr 172.21.15.2:0/386331526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de5402640 1 -- 172.21.15.2:0/386331526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de0196520 msgr2=0x7f8de019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de4c01640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de0196520 0x7f8de019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de5402640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de0196520 0x7f8de019a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de5402640 1 -- 172.21.15.2:0/386331526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd0009560 con 0x7f8de01001d0 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de5402640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de0195fe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0036040 tx=0x7f8dd0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8dd67fc640 1 -- 172.21.15.2:0/386331526 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd00035f0 con 0x7f8de01001d0 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de019ae10 con 0x7f8de01001d0 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8dd67fc640 1 -- 172.21.15.2:0/386331526 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dd0003c10 con 0x7f8de01001d0 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8dd67fc640 1 -- 172.21.15.2:0/386331526 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd0034440 con 0x7f8de01001d0 2024-03-16T10:56:02.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.552+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de019b370 con 0x7f8de01001d0 2024-03-16T10:56:02.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.553+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de01016a0 con 0x7f8de01001d0 2024-03-16T10:56:02.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.554+0000 7f8dd67fc640 1 -- 172.21.15.2:0/386331526 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f8dd0004f60 con 0x7f8de01001d0 2024-03-16T10:56:02.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.554+0000 7f8dd67fc640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8dcc06df20 0x7f8dcc0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.554+0000 7f8dd67fc640 1 -- 172.21.15.2:0/386331526 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f8dd00b0250 con 0x7f8de01001d0 2024-03-16T10:56:02.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.554+0000 7f8de4c01640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8dcc06df20 0x7f8dcc0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.555+0000 7f8de4c01640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8dcc06df20 0x7f8dcc0703e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0058c0 tx=0x7f8ddc005c60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:02.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.559+0000 7f8dd67fc640 1 -- 172.21.15.2:0/386331526 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8dd0079620 con 0x7f8de01001d0 2024-03-16T10:56:02.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.813+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8de01084d0 con 0x7f8de01001d0 2024-03-16T10:56:02.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.813+0000 7f8dd67fc640 1 -- 172.21.15.2:0/386331526 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dd007cce0 con 0x7f8de01001d0 2024-03-16T10:56:02.814 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8dcc06df20 msgr2=0x7f8dcc0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8dcc06df20 0x7f8dcc0703e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0058c0 tx=0x7f8ddc005c60 comp rx=0 tx=0).stop 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 msgr2=0x7f8de0195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de0195fe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0036040 tx=0x7f8dd0004040 comp rx=0 tx=0).stop 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 shutdown_connections 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f8dcc06df20 0x7f8dcc0703e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8de0196520 0x7f8de019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 --2- 172.21.15.2:0/386331526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de01001d0 0x7f8de0195fe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 >> 172.21.15.2:0/386331526 conn(0x7f8de00fba50 msgr2=0x7f8de00fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:02.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 shutdown_connections 2024-03-16T10:56:02.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.815+0000 7f8de7666640 1 -- 172.21.15.2:0/386331526 wait complete. 2024-03-16T10:56:02.826 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-16T10:56:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.931+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/2243823536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 msgr2=0x7f2dd80ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/2243823536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd80ff5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009880 tx=0x7f2dd402bd10 comp rx=0 tx=0).stop 2024-03-16T10:56:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/2243823536 shutdown_connections 2024-03-16T10:56:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/2243823536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2dd80ffb70 0x7f2dd8103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/2243823536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd80ff5a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/2243823536 >> 172.21.15.2:0/2243823536 conn(0x7f2dd8074960 msgr2=0x7f2dd8074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/2243823536 shutdown_connections 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/2243823536 wait complete. 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 Processor -- start 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 -- start start 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.932+0000 7f2ddf7ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd8100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddf7ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2dd8101080 0x7f2dd81014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddf7ae640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8102aa0 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddf7ae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd81a22f0 con 0x7f2dd8101080 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddd54a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd8100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddd54a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd8100b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39152/0 (socket says 172.21.15.2:39152) 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddd54a640 1 -- 172.21.15.2:0/718219045 learned_addr learned my addr 172.21.15.2:0/718219045 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddcd49640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2dd8101080 0x7f2dd81014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddd54a640 1 -- 172.21.15.2:0/718219045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2dd8101080 msgr2=0x7f2dd81014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddd54a640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2dd8101080 0x7f2dd81014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddd54a640 1 -- 172.21.15.2:0/718219045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd4009560 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddcd49640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2dd8101080 0x7f2dd81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:56:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddd54a640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd8100b40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2dd402c250 tx=0x7f2dd4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2dce7fc640 1 -- 172.21.15.2:0/718219045 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dd40044a0 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2dce7fc640 1 -- 172.21.15.2:0/718219045 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2dd4003ab0 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd81a24f0 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2dce7fc640 1 -- 172.21.15.2:0/718219045 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dd4005620 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.933+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd81a2a20 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.934+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd8100580 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.935+0000 7f2dce7fc640 1 -- 172.21.15.2:0/718219045 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f2dd4004e60 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.935+0000 7f2dce7fc640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2dc406df10 0x7f2dc40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.935+0000 7f2dce7fc640 1 -- 172.21.15.2:0/718219045 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f2dd40b0a10 con 0x7f2dd80ff1c0 2024-03-16T10:56:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.935+0000 7f2ddcd49640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2dc406df10 0x7f2dc40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.936+0000 7f2ddcd49640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2dc406df10 0x7f2dc40703d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8006fd0 tx=0x7f2dc8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:02.944+0000 7f2dce7fc640 1 -- 172.21.15.2:0/718219045 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2dd4079d60 con 0x7f2dd80ff1c0 2024-03-16T10:56:03.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.130+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f2dd804e030 con 0x7f2dd80ff1c0 2024-03-16T10:56:03.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.130+0000 7f2dce7fc640 1 -- 172.21.15.2:0/718219045 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2dd407d420 con 0x7f2dd80ff1c0 2024-03-16T10:56:03.131 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2dc406df10 msgr2=0x7f2dc40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2dc406df10 0x7f2dc40703d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8006fd0 tx=0x7f2dc8008040 comp rx=0 tx=0).stop 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 msgr2=0x7f2dd8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd8100b40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2dd402c250 tx=0x7f2dd4005e50 comp rx=0 tx=0).stop 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 shutdown_connections 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f2dc406df10 0x7f2dc40703d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2dd8101080 0x7f2dd81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 --2- 172.21.15.2:0/718219045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2dd80ff1c0 0x7f2dd8100b40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 >> 172.21.15.2:0/718219045 conn(0x7f2dd8074960 msgr2=0x7f2dd8067540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:03.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 shutdown_connections 2024-03-16T10:56:03.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.132+0000 7f2ddf7ae640 1 -- 172.21.15.2:0/718219045 wait complete. 2024-03-16T10:56:03.142 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:03 smithi002 ceph-mon[94239]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:03.143 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:03 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/201011534' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:56:03.143 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:03 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/3768577357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:56:03.143 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:03 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/386331526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:56:03.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-16T10:56:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 -- 172.21.15.2:0/2349850822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55100ffa30 msgr2=0x7f5510103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 --2- 172.21.15.2:0/2349850822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55100ffa30 0x7f5510103a70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5500009880 tx=0x7f550002bce0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 -- 172.21.15.2:0/2349850822 shutdown_connections 2024-03-16T10:56:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 --2- 172.21.15.2:0/2349850822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55100ffa30 0x7f5510103a70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 --2- 172.21.15.2:0/2349850822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100ff080 0x7f55100ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 -- 172.21.15.2:0/2349850822 >> 172.21.15.2:0/2349850822 conn(0x7f55100fa900 msgr2=0x7f55100fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 -- 172.21.15.2:0/2349850822 shutdown_connections 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 -- 172.21.15.2:0/2349850822 wait complete. 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.250+0000 7f5515f77640 1 Processor -- start 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f5515f77640 1 -- start start 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f5515f77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100ff080 0x7f5510100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f5515f77640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5510101050 0x7f55101014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f5515f77640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510102ac0 con 0x7f5510101050 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f5515f77640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55101a2170 con 0x7f55100ff080 2024-03-16T10:56:03.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5510101050 0x7f55101014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5510101050 0x7f55101014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39170/0 (socket says 172.21.15.2:39170) 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550effd640 1 -- 172.21.15.2:0/4104958921 learned_addr learned my addr 172.21.15.2:0/4104958921 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550f7fe640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100ff080 0x7f5510100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550effd640 1 -- 172.21.15.2:0/4104958921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100ff080 msgr2=0x7f5510100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550effd640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100ff080 0x7f5510100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550effd640 1 -- 172.21.15.2:0/4104958921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5500009560 con 0x7f5510101050 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550effd640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5510101050 0x7f55101014b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f550002c220 tx=0x7f550002cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f550cff9640 1 -- 172.21.15.2:0/4104958921 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55000044a0 con 0x7f5510101050 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.251+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55101a2370 con 0x7f5510101050 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.252+0000 7f550cff9640 1 -- 172.21.15.2:0/4104958921 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5500003ab0 con 0x7f5510101050 2024-03-16T10:56:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.252+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55101a2840 con 0x7f5510101050 2024-03-16T10:56:03.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.252+0000 7f550cff9640 1 -- 172.21.15.2:0/4104958921 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5500005560 con 0x7f5510101050 2024-03-16T10:56:03.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.253+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5510100550 con 0x7f5510101050 2024-03-16T10:56:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.254+0000 7f550cff9640 1 -- 172.21.15.2:0/4104958921 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f5500004dd0 con 0x7f5510101050 2024-03-16T10:56:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.254+0000 7f550cff9640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54fc06df10 0x7f54fc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.254+0000 7f550cff9640 1 -- 172.21.15.2:0/4104958921 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f55000b01e0 con 0x7f5510101050 2024-03-16T10:56:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.254+0000 7f550f7fe640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54fc06df10 0x7f54fc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:03.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.255+0000 7f550f7fe640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54fc06df10 0x7f54fc0703d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005fd0 tx=0x7f54f8005850 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:03.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.259+0000 7f550cff9640 1 -- 172.21.15.2:0/4104958921 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55000795b0 con 0x7f5510101050 2024-03-16T10:56:03.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.442+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f551004e030 con 0x7f5510101050 2024-03-16T10:56:03.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.442+0000 7f550cff9640 1 -- 172.21.15.2:0/4104958921 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f550007cc70 con 0x7f5510101050 2024-03-16T10:56:03.443 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-03-16T10:56:03.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54fc06df10 msgr2=0x7f54fc0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54fc06df10 0x7f54fc0703d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005fd0 tx=0x7f54f8005850 comp rx=0 tx=0).stop 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5510101050 msgr2=0x7f55101014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5510101050 0x7f55101014b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f550002c220 tx=0x7f550002cea0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 shutdown_connections 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f54fc06df10 0x7f54fc0703d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5510101050 0x7f55101014b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 --2- 172.21.15.2:0/4104958921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100ff080 0x7f5510100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 >> 172.21.15.2:0/4104958921 conn(0x7f55100fa900 msgr2=0x7f55100687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 shutdown_connections 2024-03-16T10:56:03.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.444+0000 7f5515f77640 1 -- 172.21.15.2:0/4104958921 wait complete. 2024-03-16T10:56:03.456 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:56:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:03 smithi026 ceph-mon[82600]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:03 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/201011534' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:56:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:03 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/3768577357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T10:56:03.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:03 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/386331526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T10:56:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 -- 172.21.15.2:0/155056589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c0ffa50 msgr2=0x7f5d8c103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 --2- 172.21.15.2:0/155056589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c0ffa50 0x7f5d8c103a90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009880 tx=0x7f5d7c02bce0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 -- 172.21.15.2:0/155056589 shutdown_connections 2024-03-16T10:56:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 --2- 172.21.15.2:0/155056589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c0ffa50 0x7f5d8c103a90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 --2- 172.21.15.2:0/155056589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8c0ff0a0 0x7f5d8c0ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 -- 172.21.15.2:0/155056589 >> 172.21.15.2:0/155056589 conn(0x7f5d8c075b10 msgr2=0x7f5d8c0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 -- 172.21.15.2:0/155056589 shutdown_connections 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 -- 172.21.15.2:0/155056589 wait complete. 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 Processor -- start 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.560+0000 7f5d926fc640 1 -- start start 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d926fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8c0ff0a0 0x7f5d8c194bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d926fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c195130 0x7f5d8c1955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d926fc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c068120 con 0x7f5d8c195130 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d926fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c1a2260 con 0x7f5d8c0ff0a0 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c195130 0x7f5d8c1955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c195130 0x7f5d8c1955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39190/0 (socket says 172.21.15.2:39190) 2024-03-16T10:56:03.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8b7fe640 1 -- 172.21.15.2:0/716732597 learned_addr learned my addr 172.21.15.2:0/716732597 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8bfff640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8c0ff0a0 0x7f5d8c194bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8b7fe640 1 -- 172.21.15.2:0/716732597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8c0ff0a0 msgr2=0x7f5d8c194bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8b7fe640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8c0ff0a0 0x7f5d8c194bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8b7fe640 1 -- 172.21.15.2:0/716732597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c009560 con 0x7f5d8c195130 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d8b7fe640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c195130 0x7f5d8c1955b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c02c220 tx=0x7f5d7c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.561+0000 7f5d897fa640 1 -- 172.21.15.2:0/716732597 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d7c0035f0 con 0x7f5d8c195130 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.562+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c1a23a0 con 0x7f5d8c195130 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.562+0000 7f5d897fa640 1 -- 172.21.15.2:0/716732597 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d7c003c10 con 0x7f5d8c195130 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.562+0000 7f5d897fa640 1 -- 172.21.15.2:0/716732597 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d7c034440 con 0x7f5d8c195130 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.562+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1a2870 con 0x7f5d8c195130 2024-03-16T10:56:03.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.563+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8c100570 con 0x7f5d8c195130 2024-03-16T10:56:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.563+0000 7f5d897fa640 1 -- 172.21.15.2:0/716732597 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 38) v1 ==== 91839+0+0 (secure 0 0 0) 0x7f5d7c004f60 con 0x7f5d8c195130 2024-03-16T10:56:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.563+0000 7f5d897fa640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d7806df10 0x7f5d780703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:56:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.564+0000 7f5d897fa640 1 -- 172.21.15.2:0/716732597 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f5d7c0b0110 con 0x7f5d8c195130 2024-03-16T10:56:03.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.564+0000 7f5d8bfff640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d7806df10 0x7f5d780703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:56:03.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.564+0000 7f5d8bfff640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d7806df10 0x7f5d780703d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d740058c0 tx=0x7f5d7400a250 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:56:03.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.568+0000 7f5d897fa640 1 -- 172.21.15.2:0/716732597 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d7c079590 con 0x7f5d8c195130 2024-03-16T10:56:03.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.821+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d8c0677e0 con 0x7f5d8c195130 2024-03-16T10:56:03.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.822+0000 7f5d897fa640 1 -- 172.21.15.2:0/716732597 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v86) v1 ==== 81+0+2507 (secure 0 0 0) 0x7f5d7c077e60 con 0x7f5d8c195130 2024-03-16T10:56:03.823 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6826/2911421579 2024-03-17T10:52:09.467533+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3116141534 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/899962313 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2161833313 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2320376866 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/486800969 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/486800969 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1498322259 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/1307372554 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.824 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4132065053 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6825/2074466615 2024-03-17T10:51:46.663682+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2565089320 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6824/2074466615 2024-03-17T10:51:46.663682+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/567924185 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/100368483 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3514263595 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6827/2911421579 2024-03-17T10:52:09.467533+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3464791333 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3421178356 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2322904669 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2024487458 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/401513378 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/2419254018 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1971862470 2024-03-17T10:37:00.118083+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/730693462 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2563315045 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1789736475 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3739300318 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3464791333 2024-03-17T10:35:17.551709+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/730693462 2024-03-17T10:46:19.722815+0000 2024-03-16T10:56:03.826 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4201560868 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1144469616 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6826/2835000958 2024-03-17T10:42:06.208262+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3462160730 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6827/2835000958 2024-03-17T10:42:06.208262+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3739300318 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6829/274393035 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3195213211 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3195213211 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3582566838 2024-03-17T10:43:58.039282+0000 2024-03-16T10:56:03.827 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:6828/274393035 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.828 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.26:0/1025795450 2024-03-17T10:45:15.531471+0000 2024-03-16T10:56:03.828 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2348229359 2024-03-17T10:35:51.598930+0000 2024-03-16T10:56:03.829 INFO:teuthology.orchestra.run.smithi002.stderr:listed 43 entries 2024-03-16T10:56:03.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d7806df10 msgr2=0x7f5d780703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d7806df10 0x7f5d780703d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d740058c0 tx=0x7f5d7400a250 comp rx=0 tx=0).stop 2024-03-16T10:56:03.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c195130 msgr2=0x7f5d8c1955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:56:03.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c195130 0x7f5d8c1955b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c02c220 tx=0x7f5d7c004040 comp rx=0 tx=0).stop 2024-03-16T10:56:03.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 shutdown_connections 2024-03-16T10:56:03.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:6800/1520670831,v1:172.21.15.2:6801/1520670831] conn(0x7f5d7806df10 0x7f5d780703d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d8c195130 0x7f5d8c1955b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 --2- 172.21.15.2:0/716732597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d8c0ff0a0 0x7f5d8c194bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:56:03.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 >> 172.21.15.2:0/716732597 conn(0x7f5d8c075b10 msgr2=0x7f5d8c0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:56:03.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 shutdown_connections 2024-03-16T10:56:03.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-03-16T10:56:03.824+0000 7f5d926fc640 1 -- 172.21.15.2:0/716732597 wait complete. 2024-03-16T10:56:03.836 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2024-03-16T10:56:03.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:56:03.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-16T10:56:03.863 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:56:04.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:04 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/718219045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:56:04.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:04 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/4104958921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:56:04.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:04 smithi002 ceph-mon[94239]: from='client.? 172.21.15.2:0/716732597' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:56:04.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:04 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/718219045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-16T10:56:04.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:04 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/4104958921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-16T10:56:04.482 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:04 smithi026 ceph-mon[82600]: from='client.? 172.21.15.2:0/716732597' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T10:56:05.472 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:05 smithi002 ceph-mon[94239]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:05.472 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:05 smithi026 ceph-mon[82600]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:07.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:07 smithi026 ceph-mon[82600]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:07 smithi002 ceph-mon[94239]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:09.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:09 smithi026 ceph-mon[82600]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:09.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:09 smithi002 ceph-mon[94239]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:11 smithi026 ceph-mon[82600]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:11.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:11 smithi002 ceph-mon[94239]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:13 smithi026 ceph-mon[82600]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:13 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:56:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:13 smithi002 ceph-mon[94239]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:13 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:56:14.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:56:14.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:56:14.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:14 smithi026 ceph-mon[82600]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:14.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:56:14.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:56:14.732 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:56:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:56:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:56:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:14 smithi002 ceph-mon[94239]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:56:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:56:14.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:56:17.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:16 smithi026 ceph-mon[82600]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:16 smithi002 ceph-mon[94239]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:18 smithi026 ceph-mon[82600]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:18 smithi002 ceph-mon[94239]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:20.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:20 smithi026 ceph-mon[82600]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:20 smithi002 ceph-mon[94239]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:22 smithi026 ceph-mon[82600]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:22 smithi002 ceph-mon[94239]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:25.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:24 smithi026 ceph-mon[82600]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:24 smithi002 ceph-mon[94239]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:27.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:26 smithi026 ceph-mon[82600]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:26 smithi002 ceph-mon[94239]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:28 smithi026 ceph-mon[82600]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:28 smithi002 ceph-mon[94239]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:30 smithi026 ceph-mon[82600]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:30 smithi002 ceph-mon[94239]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:32 smithi026 ceph-mon[82600]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:32 smithi002 ceph-mon[94239]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:34 smithi026 ceph-mon[82600]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:34 smithi002 ceph-mon[94239]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:36 smithi026 ceph-mon[82600]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:36 smithi002 ceph-mon[94239]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:38 smithi026 ceph-mon[82600]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:38 smithi002 ceph-mon[94239]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:41.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:40 smithi026 ceph-mon[82600]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:40 smithi002 ceph-mon[94239]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:42 smithi026 ceph-mon[82600]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:42 smithi002 ceph-mon[94239]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:44 smithi026 ceph-mon[82600]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:44 smithi002 ceph-mon[94239]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:46.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:46 smithi026 ceph-mon[82600]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:47.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:46 smithi002 ceph-mon[94239]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:48 smithi026 ceph-mon[82600]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:48 smithi002 ceph-mon[94239]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:50.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:49 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:49 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:56:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:50 smithi026 ceph-mon[82600]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:50 smithi002 ceph-mon[94239]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:52 smithi026 ceph-mon[82600]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:52 smithi002 ceph-mon[94239]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:55.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:54 smithi026 ceph-mon[82600]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:54 smithi002 ceph-mon[94239]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:57.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:56 smithi026 ceph-mon[82600]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:56 smithi002 ceph-mon[94239]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:59.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:56:58 smithi026 ceph-mon[82600]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:56:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:56:58 smithi002 ceph-mon[94239]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:01.233 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:00 smithi026 ceph-mon[82600]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:00 smithi002 ceph-mon[94239]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:03.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:02 smithi026 ceph-mon[82600]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:02 smithi002 ceph-mon[94239]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:04 smithi026 ceph-mon[82600]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:05.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:04 smithi002 ceph-mon[94239]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:07.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:06 smithi026 ceph-mon[82600]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:06 smithi002 ceph-mon[94239]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:09.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:08 smithi026 ceph-mon[82600]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:08 smithi002 ceph-mon[94239]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:11.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:10 smithi026 ceph-mon[82600]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:10 smithi002 ceph-mon[94239]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:13.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:12 smithi026 ceph-mon[82600]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:12 smithi002 ceph-mon[94239]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:14 smithi026 ceph-mon[82600]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:57:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:57:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:57:15.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:14 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:57:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:14 smithi002 ceph-mon[94239]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:57:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:57:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:57:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:14 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:57:17.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:16 smithi026 ceph-mon[82600]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:16 smithi002 ceph-mon[94239]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:18 smithi026 ceph-mon[82600]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:18 smithi002 ceph-mon[94239]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:20.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:20 smithi026 ceph-mon[82600]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:20 smithi002 ceph-mon[94239]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:23.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:22 smithi026 ceph-mon[82600]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:22 smithi002 ceph-mon[94239]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:25.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:24 smithi026 ceph-mon[82600]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:24 smithi002 ceph-mon[94239]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:27.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:26 smithi026 ceph-mon[82600]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:26 smithi002 ceph-mon[94239]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:28 smithi026 ceph-mon[82600]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:28 smithi002 ceph-mon[94239]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:30 smithi026 ceph-mon[82600]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:30 smithi002 ceph-mon[94239]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:33.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:32 smithi002 ceph-mon[94239]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:32 smithi026 ceph-mon[82600]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:34 smithi026 ceph-mon[82600]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:34 smithi002 ceph-mon[94239]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:36 smithi026 ceph-mon[82600]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:36 smithi002 ceph-mon[94239]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:39.252 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:38 smithi002 ceph-mon[94239]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:39.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:38 smithi026 ceph-mon[82600]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:40 smithi002 ceph-mon[94239]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:41.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:40 smithi026 ceph-mon[82600]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:42 smithi026 ceph-mon[82600]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:42 smithi002 ceph-mon[94239]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:44 smithi002 ceph-mon[94239]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:45.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:44 smithi026 ceph-mon[82600]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:46 smithi002 ceph-mon[94239]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:46 smithi026 ceph-mon[82600]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:48 smithi002 ceph-mon[94239]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:49.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:48 smithi026 ceph-mon[82600]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:57:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:51 smithi002 ceph-mon[94239]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:51.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:51 smithi026 ceph-mon[82600]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:53 smithi026 ceph-mon[82600]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:53 smithi002 ceph-mon[94239]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:55 smithi002 ceph-mon[94239]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:55 smithi026 ceph-mon[82600]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:57 smithi002 ceph-mon[94239]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:57.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:57 smithi026 ceph-mon[82600]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:57:59 smithi002 ceph-mon[94239]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:57:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:57:59 smithi026 ceph-mon[82600]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:01 smithi002 ceph-mon[94239]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:01 smithi026 ceph-mon[82600]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:03.283 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:03 smithi026 ceph-mon[82600]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:03 smithi002 ceph-mon[94239]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:05 smithi002 ceph-mon[94239]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:05 smithi026 ceph-mon[82600]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:07 smithi002 ceph-mon[94239]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:07.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:07 smithi026 ceph-mon[82600]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:09 smithi002 ceph-mon[94239]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:09.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:09 smithi026 ceph-mon[82600]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:11 smithi002 ceph-mon[94239]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:11 smithi026 ceph-mon[82600]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:13 smithi002 ceph-mon[94239]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:13 smithi026 ceph-mon[82600]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:15 smithi002 ceph-mon[94239]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:15 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:58:15.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:15 smithi026 ceph-mon[82600]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:15.525 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:15 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:58:16.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:58:16.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:58:16.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:16 smithi002 ceph-mon[94239]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:16.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:58:16.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:58:16.798 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:16 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:58:16.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:58:16.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:58:16.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:16 smithi026 ceph-mon[82600]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:16.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:58:16.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:58:16.981 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:16 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:58:19.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:18 smithi026 ceph-mon[82600]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:18 smithi002 ceph-mon[94239]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:20.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:19 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:19 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:21.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:20 smithi026 ceph-mon[82600]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:20 smithi002 ceph-mon[94239]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:23.216 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:22 smithi026 ceph-mon[82600]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:22 smithi002 ceph-mon[94239]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:25.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:24 smithi026 ceph-mon[82600]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:24 smithi002 ceph-mon[94239]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:27.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:26 smithi026 ceph-mon[82600]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:26 smithi002 ceph-mon[94239]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:28 smithi026 ceph-mon[82600]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:28 smithi002 ceph-mon[94239]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:31.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:30 smithi026 ceph-mon[82600]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:30 smithi002 ceph-mon[94239]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:33.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:32 smithi026 ceph-mon[82600]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:32 smithi002 ceph-mon[94239]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:35.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:34 smithi026 ceph-mon[82600]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:35.232 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:34 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:34 smithi002 ceph-mon[94239]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:34 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:37.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:36 smithi026 ceph-mon[82600]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:36 smithi002 ceph-mon[94239]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:38 smithi026 ceph-mon[82600]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:38 smithi002 ceph-mon[94239]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:41.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:40 smithi026 ceph-mon[82600]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:40 smithi002 ceph-mon[94239]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:43.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:42 smithi026 ceph-mon[82600]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:42 smithi002 ceph-mon[94239]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:45.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:44 smithi026 ceph-mon[82600]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:44 smithi002 ceph-mon[94239]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:47.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:46 smithi026 ceph-mon[82600]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:46 smithi002 ceph-mon[94239]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:49.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:48 smithi026 ceph-mon[82600]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:49.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:48 smithi002 ceph-mon[94239]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:50.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:49 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:49 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:58:51.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:50 smithi026 ceph-mon[82600]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:51.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:50 smithi002 ceph-mon[94239]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:53.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:52 smithi026 ceph-mon[82600]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:53.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:52 smithi002 ceph-mon[94239]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:54 smithi002 ceph-mon[94239]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:54 smithi026 ceph-mon[82600]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:57.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:56 smithi002 ceph-mon[94239]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:57.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:56 smithi026 ceph-mon[82600]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:59.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:58:58 smithi002 ceph-mon[94239]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:58:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:58:58 smithi026 ceph-mon[82600]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:01.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:00 smithi002 ceph-mon[94239]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:00 smithi026 ceph-mon[82600]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:03.283 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:02 smithi026 ceph-mon[82600]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:02 smithi002 ceph-mon[94239]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:04 smithi002 ceph-mon[94239]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:05.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:04 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:04 smithi026 ceph-mon[82600]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:04 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:07.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:06 smithi002 ceph-mon[94239]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:07.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:06 smithi026 ceph-mon[82600]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:09.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:08 smithi002 ceph-mon[94239]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:09.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:08 smithi026 ceph-mon[82600]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:11.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:10 smithi002 ceph-mon[94239]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:11 smithi026 ceph-mon[82600]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:13.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:13 smithi002 ceph-mon[94239]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:13 smithi026 ceph-mon[82600]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:15.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:15 smithi002 ceph-mon[94239]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:15.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:15 smithi026 ceph-mon[82600]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:17 smithi002 ceph-mon[94239]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:59:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:59:17.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:17 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:59:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:17 smithi026 ceph-mon[82600]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T10:59:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T10:59:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:17 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T10:59:18.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:18 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:59:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:18 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T10:59:19.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:19 smithi002 ceph-mon[94239]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:19.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:19 smithi026 ceph-mon[82600]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:20.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:20.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:21.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:21 smithi002 ceph-mon[94239]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:21.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:21 smithi026 ceph-mon[82600]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:23.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:23 smithi002 ceph-mon[94239]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:23.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:23 smithi026 ceph-mon[82600]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:25.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:25 smithi002 ceph-mon[94239]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:25.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:25 smithi026 ceph-mon[82600]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:27.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:27 smithi002 ceph-mon[94239]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:27.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:27 smithi026 ceph-mon[82600]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:29.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:29 smithi026 ceph-mon[82600]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:29.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:29 smithi002 ceph-mon[94239]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:31.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:31 smithi002 ceph-mon[94239]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:31.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:31 smithi026 ceph-mon[82600]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:33.284 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:33 smithi026 ceph-mon[82600]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:33.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:33 smithi002 ceph-mon[94239]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:35 smithi002 ceph-mon[94239]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:35.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:35.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:35 smithi026 ceph-mon[82600]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:35.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:37.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:37 smithi002 ceph-mon[94239]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:37 smithi026 ceph-mon[82600]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:39.231 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:39 smithi026 ceph-mon[82600]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:39.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:39 smithi002 ceph-mon[94239]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:41.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:41 smithi002 ceph-mon[94239]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:41.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:41 smithi026 ceph-mon[82600]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:43.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:43 smithi002 ceph-mon[94239]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:43.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:43 smithi026 ceph-mon[82600]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:45.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:45 smithi002 ceph-mon[94239]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:45.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:45 smithi026 ceph-mon[82600]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:47.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:47 smithi002 ceph-mon[94239]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:47 smithi026 ceph-mon[82600]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:49.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:49 smithi026 ceph-mon[82600]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:49.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:49 smithi002 ceph-mon[94239]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:50.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T10:59:51.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:51 smithi026 ceph-mon[82600]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:51.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:51 smithi002 ceph-mon[94239]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:53.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:53 smithi026 ceph-mon[82600]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:53.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:53 smithi002 ceph-mon[94239]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:55 smithi026 ceph-mon[82600]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:55.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:55 smithi002 ceph-mon[94239]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:57.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:57 smithi026 ceph-mon[82600]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:57.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:57 smithi002 ceph-mon[94239]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:59.365 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 10:59:59 smithi002 ceph-mon[94239]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T10:59:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 10:59:59 smithi026 ceph-mon[82600]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:00 smithi026 ceph-mon[82600]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T11:00:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:00 smithi026 ceph-mon[82600]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T11:00:00.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:00 smithi026 ceph-mon[82600]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T11:00:00.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:00 smithi002 ceph-mon[94239]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-16T11:00:00.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:00 smithi002 ceph-mon[94239]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-16T11:00:00.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:00 smithi002 ceph-mon[94239]: fs cephfs has deprecated feature inline_data enabled. 2024-03-16T11:00:01.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:01 smithi026 ceph-mon[82600]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:01.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:01 smithi002 ceph-mon[94239]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:03 smithi026 ceph-mon[82600]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:03.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:03 smithi002 ceph-mon[94239]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:05 smithi026 ceph-mon[82600]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:05.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:05 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:05 smithi002 ceph-mon[94239]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:05.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:05 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:07.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:07 smithi026 ceph-mon[82600]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:07.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:07 smithi002 ceph-mon[94239]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:09.474 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:09 smithi002 ceph-mon[94239]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:09.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:09 smithi026 ceph-mon[82600]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:11.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:11 smithi026 ceph-mon[82600]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:11.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:11 smithi002 ceph-mon[94239]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:13.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:13 smithi026 ceph-mon[82600]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:13.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:13 smithi002 ceph-mon[94239]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:15.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:15 smithi026 ceph-mon[82600]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:15.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:15 smithi002 ceph-mon[94239]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:17.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:17 smithi026 ceph-mon[82600]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:17.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:17 smithi002 ceph-mon[94239]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:18 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T11:00:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:18 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T11:00:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:18 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T11:00:18.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:18 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T11:00:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:18 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T11:00:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:18 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T11:00:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:18 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T11:00:18.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:18 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' 2024-03-16T11:00:19.348 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:19 smithi026 ceph-mon[82600]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:19.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:19 smithi002 ceph-mon[94239]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:20.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:20 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:20.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:20 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:21.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:21 smithi026 ceph-mon[82600]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:21.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:21 smithi002 ceph-mon[94239]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:23.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:23 smithi026 ceph-mon[82600]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:23.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:23 smithi002 ceph-mon[94239]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:25.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:25 smithi026 ceph-mon[82600]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:25.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:25 smithi002 ceph-mon[94239]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:27.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:27 smithi026 ceph-mon[82600]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:27.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:27 smithi002 ceph-mon[94239]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:29.446 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:29 smithi026 ceph-mon[82600]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:29.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:29 smithi002 ceph-mon[94239]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:31.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:31 smithi026 ceph-mon[82600]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:31.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:31 smithi002 ceph-mon[94239]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:33.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:33 smithi026 ceph-mon[82600]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:33.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:33 smithi002 ceph-mon[94239]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:35.329 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:35 smithi002 ceph-mon[94239]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:35.330 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:35 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:35.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:35 smithi026 ceph-mon[82600]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:35.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:35 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:37.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:37 smithi026 ceph-mon[82600]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:37.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:37 smithi002 ceph-mon[94239]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:39.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:39 smithi026 ceph-mon[82600]: pgmap v467: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:39.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:39 smithi002 ceph-mon[94239]: pgmap v467: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:41.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:41 smithi026 ceph-mon[82600]: pgmap v468: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:41.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:41 smithi002 ceph-mon[94239]: pgmap v468: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:43.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:43 smithi026 ceph-mon[82600]: pgmap v469: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:43.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:43 smithi002 ceph-mon[94239]: pgmap v469: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:45.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:45 smithi026 ceph-mon[82600]: pgmap v470: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:45.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:45 smithi002 ceph-mon[94239]: pgmap v470: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:47.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:47 smithi026 ceph-mon[82600]: pgmap v471: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:47.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:47 smithi002 ceph-mon[94239]: pgmap v471: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:49.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:49 smithi026 ceph-mon[82600]: pgmap v472: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:49.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:49 smithi002 ceph-mon[94239]: pgmap v472: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:50.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:50 smithi002 ceph-mon[94239]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:50.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:50 smithi026 ceph-mon[82600]: from='mgr.34102 172.21.15.2:0/2216598531' entity='mgr.smithi002.zevrtm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-16T11:00:51.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:51 smithi026 ceph-mon[82600]: pgmap v473: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:51.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:51 smithi002 ceph-mon[94239]: pgmap v473: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:53.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:53 smithi026 ceph-mon[82600]: pgmap v474: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:53.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:53 smithi002 ceph-mon[94239]: pgmap v474: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:55.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:55 smithi002 ceph-mon[94239]: pgmap v475: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:55.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:55 smithi026 ceph-mon[82600]: pgmap v475: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:57.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:57 smithi026 ceph-mon[82600]: pgmap v476: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:57.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:57 smithi002 ceph-mon[94239]: pgmap v476: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:59.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:00:59 smithi026 ceph-mon[82600]: pgmap v477: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:00:59.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:00:59 smithi002 ceph-mon[94239]: pgmap v477: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:01:01.394 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:01 smithi026 ceph-mon[82600]: pgmap v478: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:01:01.548 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:01 smithi002 ceph-mon[94239]: pgmap v478: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:01:02.937 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-16T11:01:02.938 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-16T11:01:02.940 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-16T11:01:02.955 INFO:tasks.cephadm:Teardown begin 2024-03-16T11:01:02.955 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephadm.py", line 1867, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-16T11:01:02.956 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T11:01:02.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T11:01:03.023 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-16T11:01:03.024 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T11:01:03.044 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T11:01:03.080 INFO:tasks.cephadm:Stopping all daemons... 2024-03-16T11:01:03.080 INFO:tasks.cephadm.mon.smithi002:Stopping mon.smithi002... 2024-03-16T11:01:03.081 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002 2024-03-16T11:01:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:03 smithi002 systemd[1]: Stopping Ceph mon.smithi002 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:03.298 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:03 smithi002 ceph-mon[94239]: pgmap v479: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:01:03.481 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:03 smithi026 ceph-mon[82600]: pgmap v479: 65 pgs: 65 active+clean; 259 MiB data, 995 MiB used, 535 GiB / 536 GiB avail 2024-03-16T11:01:03.796 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[94235]: 2024-03-16T11:01:03.395+0000 7fa11bce4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi002 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:03.796 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:03 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002[94235]: 2024-03-16T11:01:03.395+0000 7fa11bce4700 -1 mon.smithi002@0(leader) e3 *** Got Signal Terminated *** 2024-03-16T11:01:03.796 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:03 smithi002 podman[153342]: 2024-03-16 11:01:03.537815972 +0000 UTC m=+0.263737618 container died 89f53efb5b4422894b2c7326c68154d53a7e8b0fe2b50e7744a83f8f6746ff5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-16T11:01:04.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:04 smithi002 podman[153342]: 2024-03-16 11:01:04.004132215 +0000 UTC m=+0.730053867 container cleanup 89f53efb5b4422894b2c7326c68154d53a7e8b0fe2b50e7744a83f8f6746ff5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-16T11:01:04.048 INFO:journalctl@ceph.mon.smithi002.smithi002.stdout:Mar 16 11:01:04 smithi002 bash[153342]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi002 2024-03-16T11:01:04.442 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi002.service' 2024-03-16T11:01:04.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:04.492 INFO:tasks.cephadm.mon.smithi002:Stopped mon.smithi002 2024-03-16T11:01:04.492 INFO:tasks.cephadm.mon.smithi026:Stopping mon.smithi026... 2024-03-16T11:01:04.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi026 2024-03-16T11:01:04.774 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:04 smithi026 systemd[1]: Stopping Ceph mon.smithi026 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:05.176 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:04 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026[82595]: 2024-03-16T11:01:04.817+0000 7f7ddbf9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi026 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:05.177 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:04 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026[82595]: 2024-03-16T11:01:04.817+0000 7f7ddbf9b700 -1 mon.smithi026@1(peon) e3 *** Got Signal Terminated *** 2024-03-16T11:01:05.177 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:04 smithi026 podman[123354]: 2024-03-16 11:01:04.889294104 +0000 UTC m=+0.197953011 container died c4945cc8635f643a5e818e1cd87884b04558ff16aebe346affb21c008694918a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-03-16T11:01:05.727 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:05 smithi026 podman[123354]: 2024-03-16 11:01:05.460014293 +0000 UTC m=+0.768673196 container cleanup c4945cc8635f643a5e818e1cd87884b04558ff16aebe346affb21c008694918a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T11:01:05.728 INFO:journalctl@ceph.mon.smithi026.smithi026.stdout:Mar 16 11:01:05 smithi026 bash[123354]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-mon-smithi026 2024-03-16T11:01:05.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@mon.smithi026.service' 2024-03-16T11:01:05.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:05.939 INFO:tasks.cephadm.mon.smithi026:Stopped mon.smithi026 2024-03-16T11:01:05.939 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-16T11:01:05.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.0 2024-03-16T11:01:06.200 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:05 smithi002 systemd[1]: Stopping Ceph osd.0 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:06.548 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:06 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[102895]: 2024-03-16T11:01:06.200+0000 7f98e5ff9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:06.548 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:06 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[102895]: 2024-03-16T11:01:06.200+0000 7f98e5ff9700 -1 osd.0 86 *** Got signal Terminated *** 2024-03-16T11:01:06.548 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:06 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0[102895]: 2024-03-16T11:01:06.200+0000 7f98e5ff9700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T11:01:12.010 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:11 smithi002 podman[153474]: 2024-03-16 11:01:11.722615858 +0000 UTC m=+5.603101170 container died 986db8fe457d758c30a063a1a747a0217bc515badb6e7f63e6f01dc5a8777055 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-16T11:01:12.298 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:12 smithi002 podman[153474]: 2024-03-16 11:01:12.100589822 +0000 UTC m=+5.981075132 container cleanup 986db8fe457d758c30a063a1a747a0217bc515badb6e7f63e6f01dc5a8777055 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T11:01:12.298 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:12 smithi002 bash[153474]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0 2024-03-16T11:01:12.723 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:12 smithi002 podman[153503]: 2024-03-16 11:01:12.385428419 +0000 UTC m=+0.655700688 container remove 986db8fe457d758c30a063a1a747a0217bc515badb6e7f63e6f01dc5a8777055 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-16T11:01:13.013 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:12 smithi002 podman[153595]: 2024-03-16 11:01:12.629194204 +0000 UTC m=+0.021350123 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T11:01:13.014 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:12 smithi002 podman[153595]: 2024-03-16 11:01:12.744777448 +0000 UTC m=+0.136933347 container create 9c57ef40f708e603f7413701f04f7a6ab04e34b64c83c8ce6537b912db811308 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-03-16T11:01:13.014 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:13 smithi002 podman[153595]: 2024-03-16 11:01:13.014265051 +0000 UTC m=+0.406420964 container init 9c57ef40f708e603f7413701f04f7a6ab04e34b64c83c8ce6537b912db811308 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T11:01:13.298 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:13 smithi002 podman[153595]: 2024-03-16 11:01:13.019983105 +0000 UTC m=+0.412139014 container start 9c57ef40f708e603f7413701f04f7a6ab04e34b64c83c8ce6537b912db811308 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-16T11:01:13.298 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:13 smithi002 podman[153595]: 2024-03-16 11:01:13.131300413 +0000 UTC m=+0.523456332 container attach 9c57ef40f708e603f7413701f04f7a6ab04e34b64c83c8ce6537b912db811308 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-16T11:01:13.626 INFO:journalctl@ceph.osd.0.smithi002.stdout:Mar 16 11:01:13 smithi002 podman[153595]: 2024-03-16 11:01:13.319251635 +0000 UTC m=+0.711407542 container died 9c57ef40f708e603f7413701f04f7a6ab04e34b64c83c8ce6537b912db811308 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T11:01:13.936 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.0.service' 2024-03-16T11:01:13.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:13.977 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-16T11:01:13.978 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-16T11:01:13.978 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.1 2024-03-16T11:01:14.298 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:14 smithi002 systemd[1]: Stopping Ceph osd.1 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:14.798 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[108320]: 2024-03-16T11:01:14.304+0000 7f495ed39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:14.798 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[108320]: 2024-03-16T11:01:14.304+0000 7f495ed39700 -1 osd.1 86 *** Got signal Terminated *** 2024-03-16T11:01:14.798 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:14 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1[108320]: 2024-03-16T11:01:14.304+0000 7f495ed39700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T11:01:20.113 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:19 smithi002 podman[153702]: 2024-03-16 11:01:19.806044818 +0000 UTC m=+5.612823296 container died dfff225aa9a34d4e170d9cd81230b2b00c41736935f0fe1fcef793864aa98c4d (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T11:01:20.473 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:20 smithi002 podman[153702]: 2024-03-16 11:01:20.212865707 +0000 UTC m=+6.019644185 container cleanup dfff225aa9a34d4e170d9cd81230b2b00c41736935f0fe1fcef793864aa98c4d (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-16T11:01:20.473 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:20 smithi002 bash[153702]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1 2024-03-16T11:01:20.798 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:20 smithi002 podman[153719]: 2024-03-16 11:01:20.473948871 +0000 UTC m=+0.670252956 container remove dfff225aa9a34d4e170d9cd81230b2b00c41736935f0fe1fcef793864aa98c4d (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-16T11:01:21.110 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:20 smithi002 podman[153807]: 2024-03-16 11:01:20.764315241 +0000 UTC m=+0.021946620 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T11:01:21.110 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:20 smithi002 podman[153807]: 2024-03-16 11:01:20.865539555 +0000 UTC m=+0.123170898 container create 09bb9a8dd6e09e0ede17db95b24fc2b8256a76aa36f33df90ca5b5147ff4d2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True) 2024-03-16T11:01:21.110 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:21 smithi002 podman[153807]: 2024-03-16 11:01:21.110319266 +0000 UTC m=+0.367950619 container init 09bb9a8dd6e09e0ede17db95b24fc2b8256a76aa36f33df90ca5b5147ff4d2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T11:01:21.401 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:21 smithi002 podman[153807]: 2024-03-16 11:01:21.114175887 +0000 UTC m=+0.371807241 container start 09bb9a8dd6e09e0ede17db95b24fc2b8256a76aa36f33df90ca5b5147ff4d2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T11:01:21.401 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:21 smithi002 podman[153807]: 2024-03-16 11:01:21.193302469 +0000 UTC m=+0.450933822 container attach 09bb9a8dd6e09e0ede17db95b24fc2b8256a76aa36f33df90ca5b5147ff4d2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-16T11:01:21.771 INFO:journalctl@ceph.osd.1.smithi002.stdout:Mar 16 11:01:21 smithi002 podman[153807]: 2024-03-16 11:01:21.402486963 +0000 UTC m=+0.660118308 container died 09bb9a8dd6e09e0ede17db95b24fc2b8256a76aa36f33df90ca5b5147ff4d2f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-16T11:01:22.127 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.1.service' 2024-03-16T11:01:22.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:22.168 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-16T11:01:22.168 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-16T11:01:22.168 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.2 2024-03-16T11:01:22.542 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:22 smithi002 systemd[1]: Stopping Ceph osd.2 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:22.798 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:22 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[113733]: 2024-03-16T11:01:22.542+0000 7f14a2a1d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:22.798 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:22 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[113733]: 2024-03-16T11:01:22.542+0000 7f14a2a1d700 -1 osd.2 86 *** Got signal Terminated *** 2024-03-16T11:01:22.798 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:22 smithi002 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2[113733]: 2024-03-16T11:01:22.542+0000 7f14a2a1d700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T11:01:28.267 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:27 smithi002 podman[153910]: 2024-03-16 11:01:27.964547551 +0000 UTC m=+5.553544780 container died 803c7eeb943b58b37bed2d799a73a87a188b09b73217fa1a1d8f3d1f5f00e066 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-16T11:01:28.548 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:28 smithi002 podman[153910]: 2024-03-16 11:01:28.350534439 +0000 UTC m=+5.939531666 container cleanup 803c7eeb943b58b37bed2d799a73a87a188b09b73217fa1a1d8f3d1f5f00e066 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-16T11:01:28.548 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:28 smithi002 bash[153910]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2 2024-03-16T11:01:28.990 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:28 smithi002 podman[153927]: 2024-03-16 11:01:28.635230452 +0000 UTC m=+0.663625200 container remove 803c7eeb943b58b37bed2d799a73a87a188b09b73217fa1a1d8f3d1f5f00e066 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-16T11:01:29.242 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:28 smithi002 podman[154018]: 2024-03-16 11:01:28.894105117 +0000 UTC m=+0.018054325 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T11:01:29.242 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:28 smithi002 podman[154018]: 2024-03-16 11:01:28.99437934 +0000 UTC m=+0.118328538 container create 880ee0be3f76941666f0d4371d223c5ef86c4ee7d9c9fa5ec32b5d8ac8013ad4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-16T11:01:29.242 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:29 smithi002 podman[154018]: 2024-03-16 11:01:29.239174334 +0000 UTC m=+0.363123546 container init 880ee0be3f76941666f0d4371d223c5ef86c4ee7d9c9fa5ec32b5d8ac8013ad4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T11:01:29.243 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:29 smithi002 podman[154018]: 2024-03-16 11:01:29.242364918 +0000 UTC m=+0.366314121 container start 880ee0be3f76941666f0d4371d223c5ef86c4ee7d9c9fa5ec32b5d8ac8013ad4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T11:01:29.548 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:29 smithi002 podman[154018]: 2024-03-16 11:01:29.339100847 +0000 UTC m=+0.463050061 container attach 880ee0be3f76941666f0d4371d223c5ef86c4ee7d9c9fa5ec32b5d8ac8013ad4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T11:01:29.867 INFO:journalctl@ceph.osd.2.smithi002.stdout:Mar 16 11:01:29 smithi002 podman[154018]: 2024-03-16 11:01:29.577924945 +0000 UTC m=+0.701874145 container died 880ee0be3f76941666f0d4371d223c5ef86c4ee7d9c9fa5ec32b5d8ac8013ad4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-16T11:01:30.202 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.2.service' 2024-03-16T11:01:30.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:30.242 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-16T11:01:30.242 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-16T11:01:30.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.3 2024-03-16T11:01:30.538 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:30 smithi026 systemd[1]: Stopping Ceph osd.3 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:30.981 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:30 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T11:01:30.537+0000 7f3948779700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:30.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:30 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T11:01:30.537+0000 7f3948779700 -1 osd.3 86 *** Got signal Terminated *** 2024-03-16T11:01:30.982 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:30 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T11:01:30.537+0000 7f3948779700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T11:01:32.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:32 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:32.673+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:33.481 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:33 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T11:01:33.100+0000 7f3940d65700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:09.462103+0000 front 2024-03-16T11:01:09.461962+0000 (oldest deadline 2024-03-16T11:01:32.962070+0000) 2024-03-16T11:01:33.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:33 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:33.677+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:34.481 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:34 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T11:01:34.144+0000 7f3940d65700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:09.462103+0000 front 2024-03-16T11:01:09.461962+0000 (oldest deadline 2024-03-16T11:01:32.962070+0000) 2024-03-16T11:01:34.764 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:34 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:34.645+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:35.481 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:35 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3[92180]: 2024-03-16T11:01:35.191+0000 7f3940d65700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:09.462103+0000 front 2024-03-16T11:01:09.461962+0000 (oldest deadline 2024-03-16T11:01:32.962070+0000) 2024-03-16T11:01:35.981 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:35 smithi026 podman[123486]: 2024-03-16 11:01:35.851591681 +0000 UTC m=+5.409401581 container died 7a6319cdca6d5a603694f841bebf83d8d1748cf80013fd8ca16c248446581a7a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T11:01:35.982 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:35 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:35.666+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:36.481 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:36 smithi026 podman[123486]: 2024-03-16 11:01:36.246225545 +0000 UTC m=+5.804035453 container cleanup 7a6319cdca6d5a603694f841bebf83d8d1748cf80013fd8ca16c248446581a7a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307) 2024-03-16T11:01:36.481 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:36 smithi026 bash[123486]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3 2024-03-16T11:01:36.864 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:36 smithi026 podman[123515]: 2024-03-16 11:01:36.505916152 +0000 UTC m=+0.647013076 container remove 7a6319cdca6d5a603694f841bebf83d8d1748cf80013fd8ca16c248446581a7a (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-16T11:01:36.865 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:36 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:36.644+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:37.175 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:36 smithi026 podman[123607]: 2024-03-16 11:01:36.865056872 +0000 UTC m=+0.113718680 container create eba5f671ba004275f93fc81b7230b51805ea5811dcd5c7a83eb96f3bb134fdcd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True) 2024-03-16T11:01:37.175 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:36 smithi026 podman[123607]: 2024-03-16 11:01:36.769113629 +0000 UTC m=+0.017775451 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T11:01:37.175 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:37 smithi026 podman[123607]: 2024-03-16 11:01:37.093188322 +0000 UTC m=+0.341850139 container init eba5f671ba004275f93fc81b7230b51805ea5811dcd5c7a83eb96f3bb134fdcd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-16T11:01:37.176 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:37 smithi026 podman[123607]: 2024-03-16 11:01:37.096689912 +0000 UTC m=+0.345351721 container start eba5f671ba004275f93fc81b7230b51805ea5811dcd5c7a83eb96f3bb134fdcd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-16T11:01:37.176 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:37 smithi026 podman[123607]: 2024-03-16 11:01:37.17581987 +0000 UTC m=+0.424481687 container attach eba5f671ba004275f93fc81b7230b51805ea5811dcd5c7a83eb96f3bb134fdcd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T11:01:37.481 INFO:journalctl@ceph.osd.3.smithi026.stdout:Mar 16 11:01:37 smithi026 podman[123607]: 2024-03-16 11:01:37.406143433 +0000 UTC m=+0.654805235 container died eba5f671ba004275f93fc81b7230b51805ea5811dcd5c7a83eb96f3bb134fdcd (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-16T11:01:37.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:37 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:37.623+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:37.981 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:37 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:37.853+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:38.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.3.service' 2024-03-16T11:01:38.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:38.094 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-16T11:01:38.094 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-16T11:01:38.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.4 2024-03-16T11:01:38.433 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:38 smithi026 systemd[1]: Stopping Ceph osd.4 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:38.731 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:38 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:38.432+0000 7fca78a15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:38.731 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:38 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:38.432+0000 7fca78a15700 -1 osd.4 86 *** Got signal Terminated *** 2024-03-16T11:01:38.731 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:38 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:38.432+0000 7fca78a15700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T11:01:38.731 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:38 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:38.612+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:39.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:38 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:38.805+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:39.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:39 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:39.609+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:39.981 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:39 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:39.778+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:40.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:40 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:40.646+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:40.981 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:40 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:40.807+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:41.981 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:41 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:41.850+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:41.981 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:41 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:41.675+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:41.982 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:41 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:41.675+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.432228+0000 front 2024-03-16T11:01:17.432211+0000 (oldest deadline 2024-03-16T11:01:40.932160+0000) 2024-03-16T11:01:42.981 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:42 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:42.824+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:42.982 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:42 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:42.688+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:08.031159+0000 front 2024-03-16T11:01:08.030995+0000 (oldest deadline 2024-03-16T11:01:32.131151+0000) 2024-03-16T11:01:42.982 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:42 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4[97062]: 2024-03-16T11:01:42.688+0000 7fca75009700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.432228+0000 front 2024-03-16T11:01:17.432211+0000 (oldest deadline 2024-03-16T11:01:40.932160+0000) 2024-03-16T11:01:44.191 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:43 smithi026 podman[123714]: 2024-03-16 11:01:43.79901494 +0000 UTC m=+5.472897472 container died ae5f63e37d7725bd75601956314b75ae0f8bdcb4f044466a0800396375e6b13e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T11:01:44.191 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:43 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:43.860+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:44.192 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:43 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:43.860+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:44.481 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:44 smithi026 podman[123714]: 2024-03-16 11:01:44.27439079 +0000 UTC m=+5.948273320 container cleanup ae5f63e37d7725bd75601956314b75ae0f8bdcb4f044466a0800396375e6b13e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-16T11:01:44.481 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:44 smithi026 bash[123714]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4 2024-03-16T11:01:44.825 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:44 smithi026 podman[123731]: 2024-03-16 11:01:44.542157179 +0000 UTC m=+0.736833988 container remove ae5f63e37d7725bd75601956314b75ae0f8bdcb4f044466a0800396375e6b13e (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-16T11:01:45.076 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:44 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:44.825+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:45.076 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:44 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:44.825+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:45.077 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:44 smithi026 podman[123823]: 2024-03-16 11:01:44.793323324 +0000 UTC m=+0.018074221 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T11:01:45.077 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:44 smithi026 podman[123823]: 2024-03-16 11:01:44.959834688 +0000 UTC m=+0.184585631 container create 02d01e352ff6bd3df424463495bc85d47f6fda6c88b7f70b3d3a7cfeea441145 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T11:01:45.481 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:45 smithi026 podman[123823]: 2024-03-16 11:01:45.204342771 +0000 UTC m=+0.429093667 container init 02d01e352ff6bd3df424463495bc85d47f6fda6c88b7f70b3d3a7cfeea441145 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T11:01:45.481 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:45 smithi026 podman[123823]: 2024-03-16 11:01:45.209155658 +0000 UTC m=+0.433906545 container start 02d01e352ff6bd3df424463495bc85d47f6fda6c88b7f70b3d3a7cfeea441145 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-16T11:01:45.481 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:45 smithi026 podman[123823]: 2024-03-16 11:01:45.295748498 +0000 UTC m=+0.520499394 container attach 02d01e352ff6bd3df424463495bc85d47f6fda6c88b7f70b3d3a7cfeea441145 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5) 2024-03-16T11:01:45.779 INFO:journalctl@ceph.osd.4.smithi026.stdout:Mar 16 11:01:45 smithi026 podman[123823]: 2024-03-16 11:01:45.510794385 +0000 UTC m=+0.735545272 container died 02d01e352ff6bd3df424463495bc85d47f6fda6c88b7f70b3d3a7cfeea441145 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-16T11:01:46.132 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:45 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:45.853+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:46.132 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:45 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:45.853+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:46.153 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.4.service' 2024-03-16T11:01:46.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:46.194 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-16T11:01:46.194 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-16T11:01:46.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.5 2024-03-16T11:01:46.481 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:46 smithi026 systemd[1]: Stopping Ceph osd.5 for b3aed5e6-e380-11ee-95c9-87774f69a715... 2024-03-16T11:01:46.812 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:46.526+0000 7f3ea6eb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T11:01:46.812 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:46.526+0000 7f3ea6eb5700 -1 osd.5 86 *** Got signal Terminated *** 2024-03-16T11:01:46.812 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:46.526+0000 7f3ea6eb5700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T11:01:46.812 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:46.811+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:46.812 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:46 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:46.811+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:48.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:47 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:47.825+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:48.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:47 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:47.825+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:49.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:48 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:48.848+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:49.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:48 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:48.848+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:50.152 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:49 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:49.871+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:50.153 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:49 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:49.871+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:50.153 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:49 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:49.871+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6822 osd.2 since back 2024-03-16T11:01:25.590573+0000 front 2024-03-16T11:01:25.590598+0000 (oldest deadline 2024-03-16T11:01:49.690575+0000) 2024-03-16T11:01:51.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:50.836+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-03-16T11:01:05.688843+0000 front 2024-03-16T11:01:05.688934+0000 (oldest deadline 2024-03-16T11:01:37.489183+0000) 2024-03-16T11:01:51.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:50.836+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6814 osd.1 since back 2024-03-16T11:01:17.989845+0000 front 2024-03-16T11:01:17.989881+0000 (oldest deadline 2024-03-16T11:01:43.289900+0000) 2024-03-16T11:01:51.231 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:50 smithi026 ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5[102255]: 2024-03-16T11:01:50.836+0000 7f3e9f4a1700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.2:6822 osd.2 since back 2024-03-16T11:01:25.590573+0000 front 2024-03-16T11:01:25.590598+0000 (oldest deadline 2024-03-16T11:01:49.690575+0000) 2024-03-16T11:01:52.136 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:51 smithi026 podman[123929]: 2024-03-16 11:01:51.885325086 +0000 UTC m=+5.469659868 container died cf40ac20a9779aed7177f0cc5e3a26750e1dacc971da9236d55c5e8fb6aff802 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-16T11:01:52.419 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:52 smithi026 podman[123929]: 2024-03-16 11:01:52.226943539 +0000 UTC m=+5.811278320 container cleanup cf40ac20a9779aed7177f0cc5e3a26750e1dacc971da9236d55c5e8fb6aff802 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-16T11:01:52.419 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:52 smithi026 bash[123929]: ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5 2024-03-16T11:01:52.731 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:52 smithi026 podman[123946]: 2024-03-16 11:01:52.419738669 +0000 UTC m=+0.528640418 container remove cf40ac20a9779aed7177f0cc5e3a26750e1dacc971da9236d55c5e8fb6aff802 (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T11:01:53.156 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:52 smithi026 podman[124037]: 2024-03-16 11:01:52.751716658 +0000 UTC m=+0.017703890 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T11:01:53.156 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:52 smithi026 podman[124037]: 2024-03-16 11:01:52.86228722 +0000 UTC m=+0.128274444 container create ea4121cf53f53d99c970e194df502ccdb5c7d9f4aed1e39da481bd87253ee59c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-16T11:01:53.156 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:53 smithi026 podman[124037]: 2024-03-16 11:01:53.073213632 +0000 UTC m=+0.339200873 container init ea4121cf53f53d99c970e194df502ccdb5c7d9f4aed1e39da481bd87253ee59c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-16T11:01:53.156 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:53 smithi026 podman[124037]: 2024-03-16 11:01:53.077129411 +0000 UTC m=+0.343116638 container start ea4121cf53f53d99c970e194df502ccdb5c7d9f4aed1e39da481bd87253ee59c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T11:01:53.156 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:53 smithi026 podman[124037]: 2024-03-16 11:01:53.156342057 +0000 UTC m=+0.422329291 container attach ea4121cf53f53d99c970e194df502ccdb5c7d9f4aed1e39da481bd87253ee59c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T11:01:53.481 INFO:journalctl@ceph.osd.5.smithi026.stdout:Mar 16 11:01:53 smithi026 podman[124037]: 2024-03-16 11:01:53.373753783 +0000 UTC m=+0.639741008 container died ea4121cf53f53d99c970e194df502ccdb5c7d9f4aed1e39da481bd87253ee59c (image=quay.ceph.io/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-b3aed5e6-e380-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-16T11:01:54.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3aed5e6-e380-11ee-95c9-87774f69a715@osd.5.service' 2024-03-16T11:01:54.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T11:01:54.075 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-16T11:01:54.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T11:01:54.225 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T11:01:58.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[77951]: fuse finished with error 0 and tester_r 0 2024-03-16T11:02:17.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T11:02:17.957 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T11:02:30.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T11:02:30.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T11:02:30.111 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-16T11:02:30.113 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376/remote/smithi002/crash 2024-03-16T11:02:30.114 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/crash -- . 2024-03-16T11:02:30.152 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T11:02:30.152 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2024-03-16T11:02:30.154 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376/remote/smithi026/crash 2024-03-16T11:02:30.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/crash -- . 2024-03-16T11:02:30.181 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T11:02:30.181 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2024-03-16T11:02:30.182 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-16T11:02:30.183 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T11:02:30.231 INFO:teuthology.orchestra.run.smithi002.stdout:2024-03-16T10:48:02.008054+0000 mon.smithi002 (mon.0) 272 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-16T11:02:30.231 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-16T11:02:30.231 INFO:tasks.cephadm:Compressing logs... 2024-03-16T11:02:30.231 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T11:02:30.273 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T11:02:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T11:02:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:: No such file or directory 2024-03-16T11:02:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mon.smithi026.log 2024-03-16T11:02:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T11:02:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-16T11:02:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mon.smithi026.log: /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:: No such file or directory 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi002.qvbseu.log 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mon.smithi002.log 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mgr.smithi002.zevrtm.log 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi002.qvbseu.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T11:02:30.305 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mon.smithi002.log: /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mgr.smithi002.zevrtm.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi002.ijcmqs.log 2024-03-16T11:02:30.306 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-16T11:02:30.306 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi002.ijcmqs.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-16T11:02:30.309 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.1.log: 91.5% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T11:02:30.309 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T11:02:30.309 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.0.log: 94.3% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi002.ijcmqs.log.gz 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi026.dzzagf.log 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.4.log: /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi026.ojzcsw.log 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi026.dzzagf.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi026.ojzcsw.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mgr.smithi026.omjkpo.log 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr: 92.5%/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.3.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr: 91.7% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mgr.smithi026.omjkpo.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.log 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.cephadm.log: 85.9% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T11:02:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T11:02:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.log: 88.4% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T11:02:30.312 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi026.log 2024-03-16T11:02:30.317 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T11:02:30.317 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.log 2024-03-16T11:02:30.318 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-16T11:02:30.319 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.log: 85.9% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T11:02:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T11:02:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.2.log: 88.4% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T11:02:30.324 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-03-16T11:02:30.326 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi026.log: 94.2% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi026.log.gz 2024-03-16T11:02:30.332 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi002.log 2024-03-16T11:02:30.338 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.1.log: 91.1% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T11:02:30.349 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi002.log: 94.2% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi002.log.gz 2024-03-16T11:02:30.358 INFO:teuthology.orchestra.run.smithi002.stderr: 91.1% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T11:02:30.358 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mgr.smithi026.omjkpo.log.gz 2024-03-16T11:02:30.590 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mgr.smithi002.zevrtm.log.gz 2024-03-16T11:02:30.752 INFO:teuthology.orchestra.run.smithi026.stderr: 92.6% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mon.smithi026.log.gz 2024-03-16T11:02:31.528 INFO:teuthology.orchestra.run.smithi002.stderr: 90.5% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mon.smithi002.log.gz 2024-03-16T11:02:34.050 INFO:teuthology.orchestra.run.smithi002.stderr: 93.9% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-16T11:02:34.535 INFO:teuthology.orchestra.run.smithi002.stderr: 93.7% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-16T11:02:35.474 INFO:teuthology.orchestra.run.smithi026.stderr: 93.8% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-16T11:02:35.619 INFO:teuthology.orchestra.run.smithi026.stderr: 93.9% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-16T11:02:35.914 INFO:teuthology.orchestra.run.smithi002.stderr: 93.8% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-16T11:02:36.527 INFO:teuthology.orchestra.run.smithi026.stderr: 93.9% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-16T11:02:37.821 INFO:teuthology.orchestra.run.smithi002.stderr: 94.7% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi002.qvbseu.log.gz 2024-03-16T11:02:38.364 INFO:teuthology.orchestra.run.smithi026.stderr: 94.8% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi026.ojzcsw.log.gz 2024-03-16T11:02:45.513 INFO:teuthology.orchestra.run.smithi002.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-16T11:02:45.514 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-03-16T11:02:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:real 0m15.228s 2024-03-16T11:02:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:user 0m36.660s 2024-03-16T11:02:45.515 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m1.185s 2024-03-16T11:02:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-16T11:02:46.640 INFO:teuthology.orchestra.run.smithi026.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-03-16T11:04:01.062 INFO:teuthology.orchestra.run.smithi026.stderr: 92.7% -- replaced with /var/log/ceph/b3aed5e6-e380-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi026.dzzagf.log.gz 2024-03-16T11:04:01.063 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-03-16T11:04:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:real 1m30.777s 2024-03-16T11:04:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:user 2m8.696s 2024-03-16T11:04:01.064 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m3.470s 2024-03-16T11:04:01.064 INFO:tasks.cephadm:Archiving logs... 2024-03-16T11:04:01.064 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376/remote/smithi002/log 2024-03-16T11:04:01.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T11:04:06.018 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376/remote/smithi026/log 2024-03-16T11:04:06.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T11:04:25.161 INFO:tasks.cephadm:Removing cluster... 2024-03-16T11:04:25.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 --force 2024-03-16T11:04:25.321 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T11:04:25.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3aed5e6-e380-11ee-95c9-87774f69a715 --force 2024-03-16T11:04:25.763 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: b3aed5e6-e380-11ee-95c9-87774f69a715 2024-03-16T11:04:26.154 INFO:tasks.cephadm:Removing cephadm ... 2024-03-16T11:04:26.155 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T11:04:26.171 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T11:04:26.185 INFO:tasks.cephadm:Teardown complete 2024-03-16T11:04:26.186 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T11:04:26.202 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-16T11:04:26.203 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T11:04:26.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T11:04:26.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T11:04:26.302 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T11:04:26.313 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T11:04:26.313 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T11:04:26.316 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T11:04:26.327 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-03-16T11:04:26.329 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-03-16T11:04:26.332 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T11:04:26.332 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-03-16T11:04:26.333 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 71 +916us[ +938us] +/- 85ms 2024-03-16T11:04:26.333 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -557us[ -545us] +/- 53ms 2024-03-16T11:04:26.333 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +69us[ +69us] +/- 62ms 2024-03-16T11:04:26.333 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T11:04:26.334 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T11:04:26.334 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-03-16T11:04:26.334 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 +846us[ +846us] +/- 86ms 2024-03-16T11:04:26.334 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -570us[ -566us] +/- 53ms 2024-03-16T11:04:26.334 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +50us[ +50us] +/- 62ms 2024-03-16T11:04:26.334 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T11:04:26.334 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T11:04:26.348 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T11:04:26.349 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T11:04:26.364 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T11:04:26.460 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T11:04:26.541 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T11:04:26.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T11:04:26.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T11:04:26.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T11:04:26.601 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T11:04:26.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T11:04:26.631 INFO:teuthology.task.internal:Duration was 2208.289103 seconds 2024-03-16T11:04:26.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T11:04:26.647 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T11:04:26.647 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T11:04:26.650 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T11:04:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T11:04:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T11:04:27.094 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T11:04:27.094 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-03-16T11:04:27.094 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T11:04:27.153 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-03-16T11:04:27.154 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T11:04:27.176 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T11:04:27.176 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T11:04:27.195 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T11:04:27.245 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T11:04:27.246 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T11:04:27.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T11:04:27.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T11:04:27.935 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T11:04:27.935 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T11:04:27.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T11:04:27.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T11:04:28.004 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T11:04:28.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T11:04:28.029 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-03-16T11:04:28.056 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-03-16T11:04:28.090 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T11:04:28.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T11:04:28.108 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T11:04:28.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T11:04:28.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T11:04:28.156 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T11:04:28.157 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376/remote/smithi002 2024-03-16T11:04:28.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T11:04:28.206 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376/remote/smithi026 2024-03-16T11:04:28.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T11:04:28.246 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T11:04:28.246 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T11:04:28.249 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T11:04:28.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T11:04:28.311 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T11:04:28.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T11:04:28.325 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T11:04:28.325 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T11:04:28.328 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T11:04:28.339 INFO:teuthology.orchestra.run.smithi002.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 16 11:04 /home/ubuntu/cephtest 2024-03-16T11:04:28.340 INFO:teuthology.orchestra.run.smithi002.stdout: 554395 4 d--------- 2 ubuntu ubuntu 4096 Mar 16 10:42 /home/ubuntu/cephtest/mnt.0 2024-03-16T11:04:28.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T11:04:28.341 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-16T11:04:28.341 INFO:teuthology.orchestra.run.smithi002.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-16T11:04:28.342 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi002 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-16T11:04:28.343 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T11:04:28.358 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T11:04:28.377 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-16T11:04:28.377 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T11:04:28.405 DEBUG:teuthology.nuke:shortname: smithi002 2024-03-16T11:04:28.405 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T11:04:28.429 DEBUG:teuthology.nuke:shortname: smithi026 2024-03-16T11:04:28.429 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T11:04:28.435 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 10:19:04.656286', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPTP5ffQoRaQWg2M7qz4kbZA1CctnYZjlxe2eMtw0CIrI4WN9rM6sBriyjphVKwGLr5jWsjUJD9OUfFFia5X5qM='} 2024-03-16T11:04:28.454 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602376', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 10:19:04.658306', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBi1Dd9A+cCFtgVisZOc7XJ8tr0N0mScbsEA4uyKj9Wxzsy6nLgHer/SejJjek/uTw+c9/ZL0+vXMj+6gVUJc/Q='} 2024-03-16T11:04:28.486 INFO:teuthology.orchestra.console.smithi002:Power off 2024-03-16T11:04:28.486 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T11:04:28.514 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2024-03-16T11:04:28.514 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T11:04:28.528 INFO:teuthology.orchestra.console.smithi026:Power off 2024-03-16T11:04:28.529 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T11:04:28.538 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-03-16T11:04:28.550 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2024-03-16T11:04:28.551 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T11:04:28.581 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-03-16T11:04:32.539 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T11:04:32.582 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T11:04:32.648 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-03-16T11:04:32.690 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-03-16T11:04:36.650 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T11:04:36.692 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T11:04:36.761 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-03-16T11:04:36.802 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2024-03-16T11:04:36.862 INFO:teuthology.orchestra.console.smithi002:Power off completed 2024-03-16T11:04:36.903 INFO:teuthology.orchestra.console.smithi026:Power off completed 2024-03-16T11:04:37.007 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2208.289103269577 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-03-16T11:04:37.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T11:04:37.094 INFO:teuthology.run:FAIL